Download Slides - School of Informatics

Special Probability Distributions
Special Probability Densities
Formal Modeling in Cognitive Science
Lecture 23: Special Distributions and Densities
Steve Renals (notes by Frank Keller)
School of Informatics
University of Edinburgh
[email protected]
5 March 2007
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
1
Special Probability Distributions
Special Probability Densities
1
Special Probability Distributions
Uniform Distribution
Binominal Distribution
2
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
2
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Binominal Distribution
Uniform Distribution
Definition: Uniform Distribution
A random variable X has a discrete uniform distribution iff its
probability distribution is given by:
f (x) =
1
for x = x1 , x2 , . . . , xk
k
where xi 6= xj when i 6= j.
The mean and variance of the uniform distribution are:
µ=
k
X
i=1
xi ·
1
k
Steve Renals (notes by Frank Keller)
σ2 =
k
X
i=1
(xi − µ)2
1
k
Formal Modeling in Cognitive Science
3
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Binominal Distribution
Binominal Distribution
Often we are interested in experiments with repeated trials:
assume there is a fixed number of trials;
each of the trial can have two outcomes (e.g., success and
failure, head and tail);
the probability of success and failure is the same for each trial:
θ and 1 − θ;
the trials are all independent.
Then the probability of getting x successes
in n trials in a given
n
order is θx (1 − θ)n−x . And
there
are
x different orders, so the
n x
n−x
overall probability is x θ (1 − θ) .
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
4
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Binominal Distribution
Binominal Distribution
Definition: Binomial Distribution
A random variable X has a binominal distribution iff its probability
distribution is given by:
n x
b(x; n, θ) =
θ (1 − θ)n−x for x = 0, 1, 2, . . . , n
x
Example
The probability of getting five heads and seven tail in 12 flips of a
balanced coin is:
1
12 1 5
1
b(5; 12, ) =
( ) (1 − )12−5
2
5 2
2
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
5
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Binominal Distribution
Binominal Distribution
1
0.9
0.8
b(x; 12, 0.5)
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
1
2
3
4
5
Steve Renals (notes by Frank Keller)
6
x
7
8
9
10 11 12
Formal Modeling in Cognitive Science
6
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Binominal Distribution
Binominal Distribution
If we invert successes and failures (or heads and tails), then the
probability stays the same. Therefore:
Theorem: Binomial Distribution
b(x; n, θ) = b(n − x; n, 1 − θ)
Two other important properties of the binomial distribution are:
Theorem: Binomial Distribution
The mean and the variance of the binomial distribution are:
µ = nθ
and σ 2 = nθ(1 − θ)
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
7
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Uniform Distribution
Definition: Uniform Distribution
A random variable X has a continuous uniform distribution iff its
probability density is given by:
1
β−α for α < x < β
u(x; α, β) =
0
elsewhere
The mean and variance of the uniform distribution are:
µ=
α+β
2
Steve Renals (notes by Frank Keller)
σ2 =
1
(α − β)2
12
Formal Modeling in Cognitive Science
8
Uniform Distribution
Exponential Distribution
Normal Distribution
Special Probability Distributions
Special Probability Densities
Uniform Distribution
0.6
0.5
0.4
0.3
0.2
0.1
0
1
1.5
2
2.5
3
3.5
4
x
Uniform distribution for α = 1 and β = 4.
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
9
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Exponential Distribution
Definition: Exponential Distribution
A random variable X has an exponential distribution iff its
probability density is given by:
1 −x/θ
for x > 0
θe
g (x; θ) =
0
elsewhere
The mean and variance of the exponential distribution are:
µ=θ
Steve Renals (notes by Frank Keller)
σ2 = θ2
Formal Modeling in Cognitive Science
10
Uniform Distribution
Exponential Distribution
Normal Distribution
Special Probability Distributions
Special Probability Densities
Exponential Distribution
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
1
2
3
4
5
x
Exponential distribution for θ = {0.5, 1, 2, 3}.
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
11
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Normal Distribution
Definition: Normal Distribution
A random variable X has a normal distribution iff its probability
density is given by:
1 x−µ 2
1
n(x; µ, σ) = √ e − 2 ( σ ) for − ∞ < x < ∞
σ 2π
Normally distributed random variables are ubiquitous in
probability theory;
many measurements of physical, biological, or cognitive
processes yield normally distributed data;
such data can be modeled using a normal distributions
(sometimes using mixtures of several normal distributions);
also called the Gaussian distribution.
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
12
Uniform Distribution
Exponential Distribution
Normal Distribution
Special Probability Distributions
Special Probability Densities
Standard Normal Distribution
0.4
0.3
0.2
0.1
0
-4
-2
0
2
4
x
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
13
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Normal Distribution
Definition: Standard Normal Distribution
The normal distribution with µ = 0 and σ = 1 is referred to as the
standard normal distribution:
1 2
1
n(x; 0, 1) = √ e − 2 x
2π
Theorem: Standard Normal Distribution
If a random variable X has a normal distribution, then:
P(|x − µ| < σ) = 0.6826
P(|x − µ| < 2σ) = 0.9544
This follows from Chebyshev’s Theorem (see previous lecture).
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
14
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Normal Distribution
Theorem: Z -Scores
If a random variable X has a normal distribution with the mean µ
and the standard deviation σ then:
Z=
X −µ
σ
has the standard normal distribution.
This conversion is often used to make results obtained by different
experiments comparable: convert the distributions to Z -scores.
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
15
Special Probability Distributions
Special Probability Densities
Uniform Distribution
Exponential Distribution
Normal Distribution
Summary
The uniform distribution assigns each value the same
probability;
The binomial distributions models an experiment with a fixed
number of independent binary trials, each with the same
probability;
The normal distribution models the data generated by
measurements of physical, biological, or cognitive processes;
Z -scores can be used to convert a normal distribution into the
standard normal distribution.
Steve Renals (notes by Frank Keller)
Formal Modeling in Cognitive Science
16