Download Math 141 - Lecture 3: The Binomial Distribution

Coin Tossing
The General Case
Math 141
Lecture 3: The Binomial Distribution
Albyn Jones1
1 Library 304
[email protected]
www.people.reed.edu/∼jones/courses/141
Albyn Jones
Math 141
Coin Tossing
The General Case
Outline
Coin Tosses
The General Case
Albyn Jones
Math 141
Coin Tossing
The General Case
Independent Coin Tosses
Crucial Features
Dichotomous Trials: Each toss results in either Heads, H,
or Tails T .
Independence: Successive tosses are independent;
knowing we got H on the first toss does not help us predict
the outcome of the second toss.
Constant probability: Each trial has the same probability
P(H) = 1/2 = P(T ) (a ‘fair coin’).
Albyn Jones
Math 141
Coin Tossing
The General Case
Examples of Different Experiments
Binomial: Count the number of Heads in a fixed number of
tosses.
Geometric: Count the number of Tails before the first Head.
Negative Binomial: Count the number of Tails before
before the k-th Head.
Albyn Jones
Math 141
Coin Tossing
The General Case
Random Variables
Random Variable: A random variable is a function mapping
points in the sample space to R, the real numbers.
Example: Let X be the number of Heads in 3 independent
tosses of a fair coin.
Albyn Jones
Math 141
Coin Tossing
The General Case
A Binomial Random Variable
Toss a fair coin 3 times, count the number of Heads. Let X be
the number of Heads. What are the possible outcomes?
{HHH} 7→ X = 3
{HHT } ∪ {HTH} ∪ {THH} 7→ X = 2
{HTT } ∪ {THT } ∪ {TTH} 7→ X = 1
{TTT } 7→ X = 0
Albyn Jones
Math 141
Coin Tossing
The General Case
Probabilities
Thus we can compute probabilities for the random variable (RV)
X is we can compute probabilities of events in the original
sample space.
P(X = 3) = P({HHH})
P(X = 2) = P({HHT } ∪ {HTH} ∪ {THH})
P(X = 1) = P({HTT } ∪ {THT } ∪ {TTH})
P(X = 0) = P({TTT })
Albyn Jones
Math 141
Coin Tossing
The General Case
Probabilities
Successive tosses are independent, so
3
1
P(X = 3) = P({HHH}) = P(H)P(H)P(H) =
2
and
3
1
P(X = 0) = P({TTT }) = P(T )P(T )P(T ) =
2
In fact, the probability of any sequence of 3 tosses is the same,
for example
3
1
P({HHT }) = P(H)P(H)P(T ) =
2
Albyn Jones
Math 141
Coin Tossing
The General Case
More Probabilities
Since probabilities of unions of disjoint events add, we just have
to count the number of sequences of three tosses with 2 Heads
to get P(X = 2):
P(X = 2) = P({HHT } ∪ {HTH} ∪ {THH})
= P({HHT }) + P({HTH}) + P({THH})
Again, due to independence, the order of getting the two Heads
and one Tail doesn’t matter, so
3
1
P({HHT }) = P({HTH}) = P({THH}) =
2
Thus
3
3
1
=
P(X = 2) = 3
2
8
Albyn Jones
Math 141
Coin Tossing
The General Case
Finally: Probabilities for 3 tosses
3
1
2
3
1
P(X = 2) = 3
2
3
1
P(X = 1) = 3
2
3
1
P(X = 0) = 1
2
P(X = 3) = 1
=
1
8
=
3
8
=
3
8
=
1
8
Note:
P(X = 0) + P(X = 1) + P(X = 2) + P(X = 3) = 1
Albyn Jones
Math 141
Coin Tossing
The General Case
X ∼ Binomial(n, 1/2)
Count Heads in n independent tosses of a fair coin
As with 3 independent tosses, any sequence of n
independent tosses of a fair coin has the same probability.
Example: for 8 tosses,
P(HHHHTTTT ) = P(HTHTHTHT ) = P(TTTTTTHH) =
8
1
2
The point: to compute P(X = k) we have to count the
number of sequences of n symbols {H, T } with k H’s, and
thus (n − k ) T ’s, then multiply by the probability of any
sequence of n tosses.
Albyn Jones
Math 141
Coin Tossing
The General Case
Counting Things: Permutations
How many ways are there to rearrange the numbers 1, 2, 3, 4?
We could list them all, with some effort:
{1, 2, 3, 4}, {2, 1, 3, 4}, {3, 1, 2, 4}, {4, 1, 2, 3}, . . .
We could be clever:
There are 4 choices for the first position.
After choosing the first, there remain 3 choices for the
second position.
After choosing the first and second positions, there are 2
possibilities for the third, and only 1 for the fourth.
4 · 3 · 2 · 1 = 4! = 24
Albyn Jones
Math 141
Coin Tossing
The General Case
Permutations of n objects
How many ways are there to rearrange the numbers
1, 2, 3, . . . , n?
There are n choices for the first position.
After choosing the first, there remain (n − 1) choices for
the second position.
After choosing the first and second positions, there are
(n − 2) possibilities for the third, and so forth.
n · (n − 1) · (n − 2) . . . · 1 = n!
n! gets big quickly: 10! = 3628800.
Albyn Jones
Math 141
Coin Tossing
The General Case
Example: Permutations of 4 objects
Each row lists permuations beginning with the same object,
grouped by the choices for the second object:
1234
1243
1324
1342
1423
1432
2134
2134
2314
2341
2413
2431
3124
3142
3214
3241
3412
3421
4123
4123
4213
4231
4312
4321
Albyn Jones
Math 141
Coin Tossing
The General Case
Practice with Factorials
What is 1! ?
What is 2! ?
What is 3! ?
What is 0! ?
Albyn Jones
Math 141
Coin Tossing
The General Case
A Formal definition for Factorials
The standard recursive definition is
0! = 1
For n > 0 ∈ Z,
n! = n · (n − 1)!
It is useful to remember this recursive definition! For
example it makes obvious the relation
n!
= (n − 1)!
n
Albyn Jones
Math 141
Coin Tossing
The General Case
Combinations
How many ways are there to rearrange the 5 symbols
H, H, H, T , T , that is three Heads and two Tails?
Label them uniquely: H1 , H2 , H3 , T4 , T5
We have 5 objects, there are 5! orders.
Some of those orders are redundant: {H1 , H2 , H3 , T4 , T5 } is
indistinguishable from {H3 , H2 , H1 , T5 , T4 } or
{H3 , H1 , H2 , T4 , T5 }.
Divide out the number of indistinguishable (or equivalent)
patterns of 3 Heads, i.e. 3!, and 2 Tails, i.e. 2!.
Answer:
5!
5 · 4 · 3!
5·4
=
=
= 10
3! · 2!
3! · 2!
2·1
Albyn Jones
Math 141
Coin Tossing
The General Case
Example: 4 Coin Tosses
4
HHHH
4
4
3
HHHT , HHTH, HTHH, THHH
4
3
2 HHTT , HTHT , THHT , TTHH, THTH, HTTH
4
2
1
TTTH, TTHT , THTT , HTTT
4
1
0
TTTT
4
0
Albyn Jones
Math 141
Coin Tossing
The General Case
Sequences of k Heads in n tosses
For 3 H and 2 T the number of possible orders was
5!
3! · 2!
For k H and (n − k ) T in n tosses the number of possible
orders will be ‘n choose k’: the number of ways to select k
objects out of a set of size n.
n
n!
=
k! · (n − k)!
k
Note:
n
n−k
n!
=
=
(n − k )! · k!
Albyn Jones
Math 141
n
k
Coin Tossing
The General Case
Practice with Binomial Coefficients
What is each of the following?
n
0
n
n
n
1
Albyn Jones
n
n−1
n
2
Math 141
Coin Tossing
The General Case
The General Case
In general, we might be working with dichotomous trials where
the event of interest has probability p, possibly not 1/2. Most of
what we have learned about coin-tossing carries over to the
general case.
Albyn Jones
Math 141
Coin Tossing
The General Case
The Binomial Distribution
X ∼ Binomial(n, p)
Dichotomous Trials: Each trial results in either a ‘Success’,
S, or a ‘Failure’ F .
Independence: Trials are mutually independent; knowing
we got S on one trial does not help us predict the outcome
of any other trial.
Constant probability: Each trial has the same probability
P(S) = p, and P(F ) = 1 − p.
X counts the number of S’s.
n the number of trials is fixed.
Albyn Jones
Math 141
Coin Tossing
The General Case
Binomial(n, p) Probabilities
Independence: Every sequence of n trials with k
successes has the same probability!
P(SSF ) = P(S)P(S)P(F ) = P(F )P(S)P(S) = P(FSS)
Let p = P(S) and q = 1 − p = P(F ), then for a RV
X ∼ Binomial(n, p)
n
P(X = k ) =
pk q n−k
k
Again, kn counts the number of sequences of length n
with k successes.
Albyn Jones
Math 141
Coin Tossing
The General Case
Example
Suppose we roll a fair die 5 times. What is the probability we
get no ones?
What Binomial distribution should we use?
Let S be the event we roll a 1. If the die is fair,
p = P(S) = 1/6. n = 5, as there are 5 trials. Let X be the
number of 1’s we get in the 5 rolls. Then
1
X ∼ Binomial(5, )
6
Thus
0 5
5
1
5
3125
P(X = 0) =
∼ 0.402
=1·1·
0
6
6
7776
Albyn Jones
Math 141
Coin Tossing
The General Case
0.00
0.05
0.10
0.15
0.20
Binomial(10,1/2)
0
1
2
3
4
Albyn Jones
5
6
7
Math 141
8
9
10
Coin Tossing
The General Case
0.00
0.05
0.10
0.15
0.20
0.25
0.30
Binomial(10,1/5)
0
1
2
3
4
Albyn Jones
5
6
7
Math 141
8
9
10
Coin Tossing
The General Case
Sums of Binomial RV’s
Suppose that X ∼ Binomial(n, p) and Y ∼ Binomial(m, p)
are independent Binomial RV’s with the same probability p.
What is the distribution of X + Y ?
X + Y ∼ Binomial(n + m, p)!
Albyn Jones
Math 141
Coin Tossing
The General Case
R functions
Density Function: P(X = k ) = dbinom(k , n, p)
(Cumulative) Distribution Function:
P(X ≤ k ) = pbinom(k, n, p)
Random numbers: rbinom(N, n, p)
Last Example: X ∼ Binomial(5, 16 ),
P(X = 0) = dbinom(0, 5, 1/6) = 0.4018776
Albyn Jones
Math 141
Coin Tossing
The General Case
Binomial Random Walk
Random Walk functions on 141 website
RW = function(n)
{
x = sample(c(-1,1),size=n,replace=T)
rw = cumsum(x)
plot(1:n,rw,xlab="N",ylim=c(-3.1*sqrt(n),3.1*sqr
abline(h=0,lty=2)
}
RW1= function(n,color="blue")
{
x = sample(c(-1,1),size=n,replace=T)
rw = cumsum(x)
lines(1:n,rw,col=color)
}
Albyn Jones
Math 141
Coin Tossing
The General Case
Summary
There are n! permutations of n objects.
Binomial coefficients, the number of subsets of size k from
a set of n objects:
n
n!
=
k! (n − k)!
k
The Binomial Distribution:
n k n−k
P(X = k ) =
p q
k
Albyn Jones
Math 141