Download Discrete Probability The Sample Space and Events R. Inkulu http

Discrete Probability
The Sample Space and Events
R. Inkulu
http://www.iitg.ac.in/rinkulu/
(The Sample Space and Events)
1 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
• rolling two dice
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
• rolling two dice
• classifying smokers/non-smoker couples
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
• rolling two dice
• classifying smokers/non-smoker couples
• distributing r labeled balls into n labeled bins
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
• rolling two dice
• classifying smokers/non-smoker couples
• distributing r labeled balls into n labeled bins
• distributing r unlabeled balls into n labeled bins
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Sample space
An experiment is a procedure that yields one of a given set of possible
outcomes. The sample space of an experiment is the set of possible outcomes.
Each outcome in a sample space is also known as a sample point.
Ex.:
• tossing a coin three times
• rolling two dice
• classifying smokers/non-smoker couples
• distributing r labeled balls into n labeled bins
• distributing r unlabeled balls into n labeled bins
A sample space is called discrete if it contains finite or countably infinite
sample points.1
1
The discrete probability is the field which studies the probability theory when the sample
space is discrete.
(The Sample Space and Events)
2 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
(The Sample Space and Events)
3 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
• S: tossing a coin three times; E: at least one head
(The Sample Space and Events)
3 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
• S: tossing a coin three times; E: at least one head
• S: rolling two dice; E: the sum of the dice equals 7
(The Sample Space and Events)
3 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
• S: tossing a coin three times; E: at least one head
• S: rolling two dice; E: the sum of the dice equals 7
• S: classifying smokers/non-smoker couples; E: smoker female
(The Sample Space and Events)
3 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
• S: tossing a coin three times; E: at least one head
• S: rolling two dice; E: the sum of the dice equals 7
• S: classifying smokers/non-smoker couples; E: smoker female
• S: distributing r labeled balls into n labeled bins; E: one cell is multiply
occupied
(The Sample Space and Events)
3 / 19
Events
Any subset of the sample space is known as an event. Each sample point is a
simple (indecomposable) event; otherwise, it is a compound (decomposable)
event. Further, an event that comprise no sample point is denoted with φ.
Ex.:
• S: tossing a coin three times; E: at least one head
• S: rolling two dice; E: the sum of the dice equals 7
• S: classifying smokers/non-smoker couples; E: smoker female
• S: distributing r labeled balls into n labeled bins; E: one cell is multiply
occupied
• S: distributing r unlabeled balls into n labeled bins; E: first cell is not
empty
(The Sample Space and Events)
3 / 19
Probability: definition
For
P a discrete sample space S, consider any function p : S → [0, 1] such that
s∈S p(s) = 1. Then p(s) is said to be the probability
P of sample point s.
Further, the probability of any event E is defined as sample point s∈E p(s). Note
that p(φ) = 0.
Ex.:
(The Sample Space and Events)
4 / 19
Probability: definition
For
P a discrete sample space S, consider any function p : S → [0, 1] such that
s∈S p(s) = 1. Then p(s) is said to be the probability
P of sample point s.
Further, the probability of any event E is defined as sample point s∈E p(s). Note
that p(φ) = 0.
Ex.:
• S: tossing an unfair coin
• S: rolling a fair dice
(The Sample Space and Events)
4 / 19
Probability: definition
For
P a discrete sample space S, consider any function p : S → [0, 1] such that
s∈S p(s) = 1. Then p(s) is said to be the probability
P of sample point s.
Further, the probability of any event E is defined as sample point s∈E p(s). Note
that p(φ) = 0.
Ex.:
• S: tossing an unfair coin
• S: rolling a fair dice
The function p is called a probability distribution over the sample spce S.
(The Sample Space and Events)
4 / 19
Combination of events
For every collection E, F of events of a sample space S2 ,
• event E ∪ F consists of all the sample points that are either in E or in F
• event E ∩ F consists of all the sample points that are both in E and in F
• event E consists of all sample points that are part of S but not in E
• E ⊆ F denotes that all sample points of E are included in F
2
obvious generalizations are possible when there are more than two base events
(The Sample Space and Events)
5 / 19
Few propositions
• p(E) = 1 − p(E)
The odds ratio of an event A is
p(A)
p(A)
i.e.,
p(A)
1−p(A)
• if E ⊆ F then p(E) ≤ p(F)
• Union bound (a.k.a. Boole’s inequality):
p(
S
i≥1 Ei )
≤
P
i≥1 p(Ei )
• Inclusion-exclusion principle:
p(E
. ∪ En ) =
P
Pn 1 ∪ E2 ∪ . .P
r+1
p(E
)
−
i
i1 <i2 <...<ir p(Ei1 ∩
i1 <i2 p(Ei1 ∩ Ei2 ) + . . . + (−1)
i=1
n+1
Ei2 ∩ . . . ∩ Eir ) + . . . + (−1) p(E1 ∩ E2 ∩ . . . ∩ En )
(The Sample Space and Events)
6 / 19
Mutually exclusive events
• Two events E1 , E2 are mutually exclusive (a.k.a. disjoint) if they cannot
occur at the same time i.e, p(E1 ∩ E2 ) = 0.
Hence, for any finite or countably infinite sequence of pairwise mutually
exclusive
events
PE1 , E2 , E3 , . . ., Boole’s inequality reduces to
S
p( i≥1 Ei ) = i≥1 p(Ei ).
(The Sample Space and Events)
7 / 19
Uniform probability
• If S is a finite nonempty sample space of equally likely sample points,
and E is an event, that is a subset of S, then the probability of E is
1
p(E) = |E|
|S| . The uniform distribution assigns the probability |S| to each
outcome in the sample space S.
• When the probability of choosing any element from a sample space S is
equal to the probability of choosing any other element of S, then the
elements are said to be chosen (in uniformly) at random.
(The Sample Space and Events)
8 / 19
Random sampling
Let S be the set of balls in a bin and
P for every b ∈ S let p(b) be the probability
of selecting ball b from S so that b∈S p(b) = 1. A random sample S′ is a
subset of S, wherein each ball in S′ is chosen according to the probability
distribution function p.
(The Sample Space and Events)
9 / 19
Typical pipeline for problem solving
• find the sample space
tree diagrams help in understanding the sample space
• define events of interest
• associate probabilities to sample points
• compute event probabilities
(The Sample Space and Events)
10 / 19
Outline
1
Examples
(The Sample Space and Events)
11 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
are white and the other are black:
(The Sample Space and Events)
12 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
( m )(k−kn 1 )
.
are white and the other are black: k1 m+n
( k )
(The Sample Space and Events)
12 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
( m )(k−kn 1 )
.
are white and the other are black: k1 m+n
( k )
• Suppose that n red and m blue unlabeled balls are arranged in a row such
that all possible orderings are equally likely. Then every ordering of the
colors has probability
(The Sample Space and Events)
12 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
( m )(k−kn 1 )
.
are white and the other are black: k1 m+n
( k )
• Suppose that n red and m blue unlabeled balls are arranged in a row such
that all possible orderings are equally likely. Then every ordering of the
n!m!
colors has probability (n+m)!
.
(The Sample Space and Events)
12 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
( m )(k−kn 1 )
.
are white and the other are black: k1 m+n
( k )
• Suppose that n red and m blue unlabeled balls are arranged in a row such
that all possible orderings are equally likely. Then every ordering of the
n!m!
colors has probability (n+m)!
.
• A sequence of n bits is randomly generated. Then the probability that at
least one of these bits zero is
(The Sample Space and Events)
12 / 19
• If k balls are randomly drawn (without replacement) from a bowl
containing m white and n black balls, the probability that k1 of the drawn
( m )(k−kn 1 )
.
are white and the other are black: k1 m+n
( k )
• Suppose that n red and m blue unlabeled balls are arranged in a row such
that all possible orderings are equally likely. Then every ordering of the
n!m!
colors has probability (n+m)!
.
• A sequence of n bits is randomly generated. Then the probability that at
least one of these bits zero is 1 −
(The Sample Space and Events)
1
2n .
12 / 19
Monty Hall three-door puzzle
• Suppose you are a game show contestant. You have a chance to win a
large prize. You are asked to select one of three doors to open; the large
prize is behind one of the three doors and the other two doors are losers.
Once you select a door, the game show host, who knows what is behind
each door, does the following. First, whether or not you selected the
winning door, he opens one of the other two doors that he knows is a
losing door. Then he asks you whether you would like to switch doors.
The probability of winning if you change the door is:
(The Sample Space and Events)
13 / 19
Monty Hall three-door puzzle
• Suppose you are a game show contestant. You have a chance to win a
large prize. You are asked to select one of three doors to open; the large
prize is behind one of the three doors and the other two doors are losers.
Once you select a door, the game show host, who knows what is behind
each door, does the following. First, whether or not you selected the
winning door, he opens one of the other two doors that he knows is a
losing door. Then he asks you whether you would like to switch doors.
The probability of winning if you change the door is: 23
(The Sample Space and Events)
13 / 19
Birthday problem
• The minimum number of people that need be in a room so that the
probability that at least two of them have the same birthday is greater
than 12 :
(The Sample Space and Events)
14 / 19
Birthday problem
• The minimum number of people that need be in a room so that the
probability that at least two of them have the same birthday is greater
than 12 :
minimum value of n that satisfies
2
3
1
)(1 − 365
)(1 − 365
) . . . (1 −
1 − ((1 − 365
(The Sample Space and Events)
n−1
365 ))
>
1
2
is just 23
14 / 19
Collision in hashing
• Assuming that the probability that a randomly selected key from n keys
is mapped to a location is uniformly at random and let there be m
locations, the probability that at least two keys in n keys get mapped to
the same location is
(The Sample Space and Events)
15 / 19
Collision in hashing
• Assuming that the probability that a randomly selected key from n keys
is mapped to a location is uniformly at random and let there be m
locations, the probability that at least two keys in n keys get mapped to
m−2
m−n+1
the same location is 1 − m−1
m
m ...
m .
(The Sample Space and Events)
15 / 19
Runs
• Assuming that every permutation of n αs and m βs are equally likely, the
probability that a permutation has exactly r ≥ 1 runs of αs is
(The Sample Space and Events)
16 / 19
Runs
• Assuming that every permutation of n αs and m βs are equally likely, the
probability that a permutation has exactly r ≥ 1 runs of αs is
(The Sample Space and Events)
m+1
(n−1
r−1 )( r )
(m+n
n )
16 / 19
An occupancy problem
Assuming that any arrangement of the placement of r labeled balls into n
labeled bins has probability n1r ,
• probability that all bins are occupied p0 (r, n) is
(The Sample Space and Events)
17 / 19
An occupancy problem
Assuming that any arrangement of the placement of r labeled balls into n
labeled bins has probability n1r ,
• probability that all bins are occupied p0 (r, n) is
1−
Pn
k=1 (−1)
k+1 n
k
(The Sample Space and Events)
(1 − nk )r
17 / 19
An occupancy problem
Assuming that any arrangement of the placement of r labeled balls into n
labeled bins has probability n1r ,
• probability that all bins are occupied p0 (r, n) is
1−
Pn
k=1 (−1)
k+1 n
k
(1 − nk )r
• probability that exactly m bins remain empty is
(The Sample Space and Events)
17 / 19
An occupancy problem
Assuming that any arrangement of the placement of r labeled balls into n
labeled bins has probability n1r ,
• probability that all bins are occupied p0 (r, n) is
1−
Pn
k=1 (−1)
k+1 n
k
(1 − nk )r
• probability that exactly m bins remain empty is
1 n
nr m
(n − m)r p0 (r, n − m) — homework
(The Sample Space and Events)
17 / 19
Yet another occupancy problem
• If n married couples are seated (assume unlabeled seats) at random
around a round table, compute the probability that no wife sits next to
her husband is
(The Sample Space and Events)
18 / 19
Yet another occupancy problem
• If n married couples are seated (assume unlabeled seats) at random
around a round table, compute
that no wife sits next to
Pn the probability
1
k+1 n 2k (2n − k − 1)!.
her husband is 1 − (2n−1)!
(−1)
k=1
k
(The Sample Space and Events)
18 / 19
Hat-check probem
• Suppose that each of n men at a party throws his hat into the center of the
room and then each man selects a hat randomly. The probability that
none of the men selects his own hat is
(The Sample Space and Events)
19 / 19
Hat-check probem
• Suppose that each of n men at a party throws his hat into the center of the
room and then each man selects a hat randomly. The probability that
n
none of the men selects his own hat is 1 − 1 + 2!1 − 3!1 + . . . + (−1)
n! .
(The Sample Space and Events)
19 / 19
Hat-check probem
• Suppose that each of n men at a party throws his hat into the center of the
room and then each man selects a hat randomly. The probability that
n
none of the men selects his own hat is 1 − 1 + 2!1 − 3!1 + . . . + (−1)
n! .
• Further, the probability exactly k of the men select their own hats is
(The Sample Space and Events)
19 / 19
Hat-check probem
• Suppose that each of n men at a party throws his hat into the center of the
room and then each man selects a hat randomly. The probability that
n
none of the men selects his own hat is 1 − 1 + 2!1 − 3!1 + . . . + (−1)
n! .
• Further, the probability exactly k of the men select their own hats is
1 n
n! k
(n − k)!(1 − 1 +
(The Sample Space and Events)
1
2!
−
1
3!
+ ... +
(−1)n−k
(n−k)! ).
19 / 19