Download Probability Distributions - University of Illinois Urbana

Carolyn Anderson & Youngshil Paek
(Slide Contributors: Shuai Wang, Yi Zhang,
Michael Culbertson, & Haiyan Li)
Department of Educational Psychology
University of Illinois at Urbana-Champaign
1
2
Key Points
1.
2.
3.
4.
5.
Random variable
Probability distributions for discrete random
variables
Mean of a probability distribution
Summarizing the spread of a probability distribution
Probability distribution for continuous random
variables
3
Random Variable
 A random variable is a numerical
measurement of the outcome of a random
phenomenon, such as
 Selecting a random sample from a population
e.g., the height of a student randomly sampled
from UIUC students
 Performing a randomized experiment
 e.g., the number of heads when a coin is
flipped 3 times

4
Random Variable
 Use a capital letter, such as X, to refer to the random
variable itself.
 Use a lower-case letter, such as x, to symbolize a
particular value of the random variable.
Example: Flip a coin three times
 X = number of heads in the 3 flips; defines the random
variable
 x = 2; represents a possible value of the random variable
5
Probability Distribution
 Each possible outcome of a random variable has
a specific probability of occurring.
 The probability distribution of a random
variable specifies its possible values and their
probabilities.
 Note: It is the randomness of the variable that
allows us to specify probabilities for the
outcomes
6
Probability Distribution of a Discrete
Random Variable
 A discrete random variable X has separate
values (such as 0,1,2,…) as its possible outcomes.
 Its probability distribution assigns a probability
P(x) to each possible value x:
 For each x, the probability P(x) falls between 0 and 1
 The sum of the probabilities for all the possible x
values equals 1
7
Example: Number of HRs in a game
 What is the estimated probability of at least three
home runs in a game for Boston Red Sox in 2004?
P(3)+P(4)+P(5)=0.13+0.03+0.01=0.17
8
The Mean of a Discrete Probability Distribution
 The mean of a probability distribution for a discrete
random variable is
μ=
𝑛
𝑥𝑖 𝑝( 𝑥𝑖 )
𝑖=1
where the sum is taken over all possible values of x.
 The mean of a probability distribution is denoted
by the parameter, µ.
 The mean is a weighted average; values of x that
are more likely receive greater weight P(x).
9
The Mean is the Expected Value of X
 The mean of a probability distribution of a random
variable X is also called the expected value of X.
 The expected value reflects not what we’ll observe
in a single observation, but rather that we expect
for the average in a long run of observations.
 It is not unusual for the expected value of a random
variable to equal a number that is NOT a possible
outcome.
μ=𝑬𝑿 =
𝒏
𝒊=𝟏 𝒙𝒊 p(𝒙𝒊 )
10
Example
 Find the mean of this probability distribution.
11
Example
 Find the mean of this probability distribution.
m = å x × p(x)
= 0(0.23) +1(0.38) + 2(0.22) + 3(0.13) + 4(0.03) + 5(0.01)
= 1.38
12
Suppose that a random number generator can
generate integers 1 through 10 with any value being
equally likely to be chosen. What would be the mean
of this distribution?
a)
b)
c)
d)
e)
4.5
5
5.5
6
Cannot be determined
Copyright © 2009 Pearson Education
The Standard Deviation of a Probability
Distribution
 The standard deviation of a probability
distribution, denoted by the parameter, σ,
measures its spread.
 Larger values of σ correspond to greater spread.
 Roughly, σ describes how far the random variable
falls, on the average, from the mean of its
distribution

  E[( X  E[ X ]) ]
2
2
  [( x   ) 2  p( x)]   [ x 2  p( x)]   2
14
Example
 Find the standard deviation of this probability
distribution.
s=
2
2
[x
×
p(x)]m
å
= 0 2 (0.23) +12 (0.38) + 2 2 (0.22) +
= 1.12
+ 52 (0.01) -1.382
15
Continuous Random Variable
 A continuous random variable has an infinite
continum of possible values in an interval.
 Examples are: time, age and size measures such
as height and weight.
 Continuous variables are measured in a discrete
manner because of rounding (precision of
measurement).
16
Probability Distribution of a Continuous Random
Variable
 A continuous random variable has possible values
that form an interval.
 Its probability distribution is specified by a curve.
 Each interval has probability between 0 and 1.
 The interval containing all possible values has
probability equal to 1.
17
Key Points Revisited
1.
2.
3.
4.
5.
Random variable
Probability distributions for discrete random
variables
Mean of a probability distribution
Summarizing the spread of a probability distribution
Probability distribution for continuous random
variables
18
19
Key Points
Normal Distribution (parameters are mean &
variance)
2. 68-95-99.7 Rule for normal distributions
3. Z-Scores and the Standard Normal Distribution
4. The Standard Normal Table: Finding Probabilities
5. Using the Standard Normal Table in Reverse
6. Use R rather than tables
7. Probabilities for Normally Distributed Random
Variables
8. Percentiles for Normally Distributed Random
Variables
9. Using Z-scores to Compare Distributions
1.
20
Normal Distribution
 The normal distribution is symmetric, bell-shaped,
and characterized by its mean µ and standard
deviation .
 The normal distribution is the most important
distribution in statistics
 Many distributions have an approximate normal
distribution.
 Approximates many discrete distributions well when
there are a large number of possible outcomes.
 Many statistical methods use it even when the data are
not bell shaped.
21
Normal Distribution
 Normal distributions are
 Bell shaped
 Symmetric around the mean
 The mean () and the standard deviation ()
completely describe the density curve
 Increasing/decreasing  moves the curve along the
horizontal axis
 Increasing/decreasing  controls the spread of the curve
22
68-95-99.7 Rule for Any Normal Curve
 68% of the observations fall within one standard
deviation of the mean
 95% fall within two standard deviations of the
mean
 99.7% fall within three standard deviations of the
mean
23
Normal Distribution
 Within what interval do almost all of the men’s
heights fall? Women’s height?
24
PDF of Normal Distribution
 The Probability Density Function (PDF) is
𝑓 𝑥 =
1
2πσ2
exp( −
(𝑥−μ)2
)
σ2
for any value of x.
 Parameters are μ and σ2 .
 When x=μ, curve is at it’s highest point
 When x>μ or x<μ, value is smaller
 (𝑥 − μ) this is like a distance
 Because difference is squared, we get symmetric shape of
curve.
25
Example : 68-95-99.7% Rule
 Heights of adult women
 can be approximated by a normal distribution
 = 65 inches; =3.5 inches
 68-95-99.7 Rule for women’s heights
 Approximately, 68% are between 61.5 & 68.5 inches

µ   = 65  3.5
 Approximately, 95% are between 58 & 72 inches

µ  2 = 65  2(3.5) = 65  7
 Approximately, 99.7% are between 54.5 & 75.5 inches

µ  3 = 65  3(3.5) = 65  10.5
26
Example : 68-95-99.7% Rule
 What proportion of women are less than 68.5
inches tall?
16%
68%
(by 68-95-99.7 Rule)
?
-1
+1
65
68.5
(height values)
27
Beyond the 68-95-99.7% Rule
 Now, what proportion of women are less than
69.9 inches tall?
?
65
69.9
(height values)
28
Z-Scores and the Standard Normal Distribution
 The z-score for a value x of a random variable is
the number of standard deviations that x falls
from the mean
z=
x -m
s
 A negative (positive) z-score indicates that the
value is below (above) the mean
 z-scores can be used to calculate the
probabilities of a normal random variable using
the normal tables in the back of the book
29
Z-Scores and the Standard Normal
Distribution
 When a random variable has a normal
distribution and its values are converted to zscores by subtracting the mean and dividing by
the standard deviation, the z-scores have the
standard normal distribution.
 A standard normal distribution has mean µ=0
and standard deviation σ=1
30
Table A: Standard Normal Probabilities
The z-table enables us to find normal probabilities
 It tabulates the normal cumulative probabilities
falling below the point  + z
To use the table:
 Find the corresponding z-score
 Look up the closest standardized score (z) in the
table.


First column gives z to the first decimal place
First row gives the second decimal place of z
 The corresponding probability found in the body of
the table gives the probability of falling below the zscore
31
Example: Using Z-Table
 Find the probability that a normal random variable
takes a value less than 1.43 standard deviations above
µ; P(z<1.43) = .9236
32
Example: Using Z-Table
 Find the probability that a normal random variable takes
a value greater than 1.43 standard deviations above µ:
P(z > 1.43) = 1−.9236 = .0764
33
Example: Using Z-Table
 Find the probability that a normal random variable
assumes a value within 1.43 standard deviations of µ
 Probability below 1.43σ = .9236
 Probability below −1.43σ = .0764 =(1−.9236)
 P(−1.43 < z < 1.43) =.9236−.0764 = .8472
34
TI Calculator = Normcdf(-1.43,1.43,0,1)= .8472
Very few use Tables anymore
# In R: Proportion women < 68.5
meanWomen <- 65
stdWomen <- 3.5
p_less68.5 <- pnorm(68.5,
mean=meanWomen,
sd=stdWomen)
p_less68.5
35
More Accurate than Rules
# More accurate than mean +/- 2(std) for 95%
meanWomen <- 65
stdWomen <- 3.5
quantiles <- c(.0275,.975)
p95women <- qnorm(quantiles,
mean=meanWomen,
sd=stdWomen)
p95women
36
Finding Probabilities for Normally Distributed
Random Variables
State the problem in terms of the observed
random variable X, i.e., P(X < x)
2. Standardize X to restate the problem in terms
of a standard normal variable Z
1.
æ
x - mö
P(X < x) = Pç Z < z =
÷
è
s ø
3. Draw a picture to show the desired probability
under the standard normal curve
4. Find the area under the standard normal
curve using the z-table.
37
Example
 Adult systolic blood pressure is normally distributed
with µ = 120 and σ = 20. What percentage of adults
have systolic blood pressure less than 100?
æ
100 -120) ö
(
P(X <100) = P ç Z <
÷ = P(z < -1.00) =.1587
20
è
ø
 15.9% of adults have systolic blood pressure less than 100
 What percentage of adults have systolic blood pressure
greater than or equal to 100?
P(X ³100) =1- P(X <100) =1-.1587 =.8413
38
Example: P(a<X<b)
 Adult systolic blood pressure is normally distributed with µ = 120
and σ = 20. What percentage of adults have systolic blood
pressure between 100 and 133?
P(100 < X <133) = P(X <133) - P(X £100)
æ (133-120) ö æ (100 -120) ö
= PçZ <
÷ - PçZ £
÷
20
20
è
ø è
ø
= P(Z <.65)- P(Z < -1.00) = .7422 -.1587 = .5835
 58% of adults have systolic blood pressure between 100 and 133
39
How Can We Find the Value of z for a
Certain Cumulative Probability?
 To solve some of our problems, we will need to
find the value of z that corresponds to a certain
normal cumulative probability
 To do so, we use the z-table in reverse
 Rather than finding z using the first column (value
of z up to one decimal) and the first row (second
decimal of z)


Find the probability in the body of the table
The z-score is given by the corresponding values in the
first column and row
40
How Can We Find the Value of z for a Certain
Cumulative Probability?
 Example: Find the value of z for a cumulative probability of
0.025.
 Look up the cumulative probability of 0.025 in the body of the
z-table.
 A cumulative probability of 0.025 corresponds to z = −1.96.
 Thus, the probability that a
normal random variable falls
at least 1.96 standard
deviations below the mean
is 0.025.
41
How Can We Find the Value of z for a
Certain Cumulative Probability?
 Example: Find the value of z for a cumulative probability of
0.975.
 Look up the cumulative probability of 0.975 in the body of
the z-table.
 A cumulative probability of 0.975 corresponds to z = 1.96.
 Thus, the probability that a
normal random variable takes
a value no more than 1.96
standard deviations above
the mean is 0.975.
42
Find X Value Given Area to Left
 Adult systolic blood pressure is normally distributed
with µ = 120 and σ = 20. What is the 1st quartile?
 P(X<x) = .25, find x:
 Look up .25 in the body of z-table to find z= −0.67
 Solve equation to find x:
x = m + zs =120 + (-0.67)*20 =106.6
 Check:
 P(X<106.6) = P(Z<−0.67)=0.25
43
Using Z-scores to Compare Distributions
 Z-scores can be used to compare observations from
different normal distributions
 Example:
 You score 650 on the SAT, which has =500 and
=100, and you score 30 on the ACT, which has =21.0 and
=4.7. On which test did you perform better?
44
Using Z-scores to Compare Distributions
 Z-scores can be used to compare observations from
different normal distributions
 Example:
 You score 650 on the SAT, which has =500 and
=100, and you score 30 on the ACT, which has =21.0 and
=4.7. On which test did you perform better?
 Compare z-scores
SAT:
650 - 500
z=
= 1.5
100
ACT:
30 - 21
z=
= 1.91
4.7
 Since your z-score is greater for the ACT, you performed
better on this exam
45
Key Points Revisited
Normal Distribution
2. 68-95-99.7 Rule for normal distributions
3. Z-Scores and the Standard Normal Distribution
4. The Standard Normal Table: Finding Probabilities
5. Using the Standard Normal Table in Reverse
6. Probabilities for Normally Distributed Random
Variables
7. Percentiles for Normally Distributed Random
Variables
8. Using Z-scores to Compare Distributions
1.
46
47
Key Points
1.
2.
3.
4.
5.
6.
7.
8.
The Binomial Distribution
Conditions for a Binomial Distribution
Probabilities for a Binomial Distribution
Factorials
Examples using Binomial Distribution
Do the Binomial Conditions Apply?
Mean and Standard Deviation of the Binomial
Distribution
Normal Approximation to the Binomial
48
The Binomial Distribution
 Each observation is binary: it has one of two
possible outcomes.
 Examples:
 Accept, or decline an offer from a bank for a credit
card.
 Have, or do not have, health insurance.
 Vote yes or no on a referendum.
49
Conditions for the Binomial Distribution
 Each of n trials has two possible outcomes:
“success” or “failure”.
 Each trial has the same probability of success,
denoted by p.
 The n trials are independent.
 The binomial random variable X is the number of
successes in the n trials.
50
Quick Question
 If the probability of getting a tail is p = 0.4 when you
flip coins (for some reason, the flip is not fair. Maybe
the head side is heavier). You flip the coins for 3 times.
 What’s the probability that you get only 1 tail, and
that’s your first flip?
 What’s the probability that you get only 1 tail?
51
Example 1: Does John really posses ESP?
 John claims to possess ESP.
 An experiment is conducted:
 A person in one room picks one of the integers 1, 2, 3,
4, 5 at random.
 In another room, John Doe identifies the number he
believes was picked.
 Three trials are performed for the experiment.
 John got the correct answer twice.
52
Example 1: Does John really posses ESP?
 If John Doe does not actually have ESP and is actually
guessing the number, each time the probability that he
makes a correct guess is 1/5 = 0.2.
 The outcome of his guess each time is independent
from other times. For two correct guesses and one
incorrect guess, the probability is (0.2)(0.2)(0.8) =
0.032.
 There are three ways John Doe could make two correct
guesses in three trials: SSF, SFS, and FSS.
 The total probability of two correct guesses is
3(0.032)=0.096.
53
Example 1: Does John really posses ESP?
The probability of exactly 2 correct guesses is the binomial probability
with n = 3 trials, x = 2 correct guesses and p = 0.2 probability of a
correct guess.
3!
P(2) =
(0.2) 2 (0.8)1 = 3(0.04)(0.8) = 0.096
2!1!
54
Formula:
Probabilities for a Binomial Distribution
 Denote the probability of success on a trial by
p.
 For n independent trials, the probability of x
successes equals:
n!
x
n-x
P(x) =
p (1- p) , x = 0,1,2,...,n
x!(n - x)!
55
Factorials
Rules for factorials:
 n! = n * (n1) * (n−2) … 2 * 1
 1! = 1
 0! = 1
For example,
 4! = 4 * 3 * 2 * 1 = 24
In R, the command is
factorial(n)
56
Example
57
58
59
60
61
Example2: Are Women Passed Over for
Managerial Training?
 A large supermarket chain occasionally selects
employees to receive management training. A
group of women there claimed that female
employees were passed over for this training in
favor of their male colleagues. But the company
denied this claim.
 How can we evaluate whether there was a gender
bias?
62
Example2: Are Women Passed Over for
Managerial Training?
 Suppose there were 1000 employees and 50% were
female. 10 employees were chosen for management
training, but none of them were female.
 If the selection was fair between the two genders, we
would expect there to be 5 males and 5 females in
the training group. But due to sampling variation, it
need not always happen that exactly 50% of those
selected are female (like flipping a fair coin for 10
times).
 To evaluate whether there was gender bias, now our
question becomes: “how unlikely is the selection
result if the selection were in fact fair?”
63
Example 2: Is the Selection Fair?
 1000 employees, 50% Female
 None of the 10 employees chosen for management
training were female.
 If the selection is fair between the two genders, p =
0.5. And the probability that no females are chosen
among the 10 is:
10!
P(0) =
(0.50) 0 (0.50)10 = 0.001
0!10!
 It is very unlikely (one chance in a thousand) that
none of the 10 selected for management training
would be female if the selection is fair between the
64
two genders.
Do the Binomial Conditions Apply?
 Before using the binomial distribution,
check that its three conditions apply:
 Binary data (success or failure).
 The same probability of success for each trial
(denoted by p).
 Independent trials.
65
The Mean and Standard Deviation of
Binomial Distribution
 The binomial probability distribution for n
trials with probability p of success on each
trial has mean µ and standard deviation σ
given by:
m = np, s = np(1 - p)
 Run R simulation of μ versus σ2 .
66
Approximating the Binomial Distribution with the
Normal Distribution
 The binomial distribution can be well
approximated by the normal distribution when
the expected number of successes, np, and the
expected number of failures, n(1– p) are both at
least 15.
 Run R simulation of Sampling Distribution of
p.
67
Example: Racial Profiling?
 262 police car stops in Philadelphia in 1997.
 207 of the drivers stopped were African-American.
 In 1997, Philadelphia’s population was 42.2%
African-American.
 Does the number of African-Americans stopped
suggest possible bias, being higher than we would
expect (other things being equal, such as the rate of
violating traffic laws)?
68
Example: Racial Profiling?
 Assume:
 262 car stops represent n = 262 trials.
 Successive police car stops are independent.
 P(driver is African-American) is p = 0.422.
 Calculate the mean and standard deviation of this
binomial distribution:
m = 262(0.422) = 111
s =
262(0.422)(0.578) = 8
69
Example: Racial Profiling?
 Recall: Empirical Rule
 When a distribution is bell-shaped, close to 100% of
the observations fall within 3 standard deviations of
the mean.
u - 3  111 - 3(8)  87
  3  111  3(8)  135
70
Example: Racial Profiling?
 If there is no racial profiling, we would not be surprised if
between about 87 and 135 of the 262 drivers stopped were
African-American.
 The actual number stopped (207) is well above these values.
 The number of African-Americans stopped is too high, even
taking into account random variation.
71
2000 Presidential Election
 The 2000 US presidential election came down to votes





in Florida. The official results from the Florida
Deparment of State, Division of Elections for the two
top candidates on Sunday November 28, 2000 were
George W. Bush = 2,912,790
Al Gore = 2,912,253
Total = 5,825,043
Bush had only a 537 vote lead
Was this election statistical a tie? (i.e. p=.5?)
72
2000 Presidential Election
 Observed proportion for Bush
𝑝=
2,912,790
=
5,825,043
.500046094
.500046094 − .5
𝑧=
= .0222
.00207166
Bush was .0222 standard deviations ahead of Gore.
If mean was .5, the probability of making a mistake
declaring a winner = P(|z| > .0222) = .98. Bush would
have had to have won by 6,217,208 votes for this to be a
decisive win.
73
Key Points Revisited
1.
2.
3.
4.
5.
6.
7.
8.
The Binomial Distribution
Conditions for a Binomial Distribution
Probabilities for a Binomial Distribution
Factorials
Examples using Binomial Distribution
Do the Binomial Conditions Apply?
Mean and Standard Deviation of the Binomial
Distribution
Normal Approximation to the Binomial
74