Download Conditional Probability and Independence

Conditional Probability and Independence
Conditional Probability and Independence
Conditional Probabilities (§ 3.2)
Bayes’ Formula (§ 3.3)
Independent Events (§ 3.4)
P(·|F ) is a Probability (§ 3.5)
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Suppose we have partial information concerning the result of an
experiment.
We assume that the event F occurred on the experiment.
But, we do not know which outcome in F occurred, only that the outcome is
a member of the event F .
Suppose x is a member of the event F , and the event E occurred
⇐⇒ x is a member of E as well.
E occurred if x is a member of F ∩ E, and E did not occur if x is a
member of F ∩ E c .
Since we cannot say whether E occurred or not, we need an
alternative description of probability for the situation.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Since we cannot say whether E occurred or not, we need an
alternative description of probability for the situation.
Suppose that P(E) and P(E c ) should be updated in terms of the
partial information that the event F occurred.
In generally, P(E) is called the unconditional or prior probability of
the event E. The updated probability of E is called the conditional
probability of E given the event F occurred, and is denoted by
P(E|F ).
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Definition
Suppose E and F are two events in a sample space S. Then, the conditional
probability of E given that F has occurred is defined to be
P(E|F ) =
P(E ∩ F )
,
P(F )
where P(F ) > 0.
Note 2.1: In order for the event E occurred by given the event F
occurred, we need the actual outcome occurred both in E and F .
Note 2.2: The event F is known as the new (or reduced) sample
space.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Note 2.3: Given the event E in the sample space S, we have
P(E|S) = P(E).
Proof:
Example 2.1
Two fair dice are rolled, given that the same number was obtained on both dice.
What is the probability that the sum of two dice is 2 or 3 or 4?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Note 2.4: When all outcomes are equally likely, it is often easier
to compute a conditional probability by considering the reduced
sample space rather than using
P(E|F ) =
P(E ∩ F )
.
P(F )
Example 2.2
All 52 cards of a standard desk of cards are dealt out randomly and equally to 4
player, say North, South, East, and West. If North and South have 2 aces among
them, what is the probability that West has the other 2 aces?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Multiplication Rule
Given n events of E1 , . . . , En in a sample space S, we have
P(E1 E2 · · · En ) = P(E1 )P(E2 |E1 )P(E3 |E1 E2 ) · · · P(En |E1 · · · En−1 ).
Note 2.5: From the definition of conditional probability, we have
P(EF ) = P(E)P(E|F ),
which is the multiplication rule for n = 2.
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Example 2.3
A bowl contains 12 Red and 8 Black balls. Four balls are selected at random and
at a time without replacement. What is the probability that the balls are are Red?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Conditional probability can often be used to compute the desired
probabilities more easily.
Suppose we want to find the probability of an event E from a
sample space S.
Another approach is that dividing E into some mutually exclusive
events, and sum the probabilities.
Let F be any event such that
E = E ∩F ∪ E ∩Fc .
Then,
P(E) = P E ∩ F + P E ∩ F c .
Using the Multiplication Rule, we can write
P(E) = P(E|F )P(F ) + P(E|F c )P(F c ).
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Law of Total Probability
Suppose E is an event in a sample space S, and let F1 , F2 , . . . , Fn be mutually
exclusive events such that ∪ni=1 Fi = S, and that P(Fi ) > 0, i = 1, . . . , n. Then,
n
P(E) =
∑ P(E|Fi )P(Fi ).
i=1
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Bayes’ Formula
♦ Bayes’ Formula
Bayes’ Formula
Suppose that F1 , . . . , Fn are n mutually exclusive events in a sample space S such
that ∪ni=1 Fi = S. Then, for any event E in S
P(Fr |E) =
P(E|Fr ) · P(Fr )
.
∑ni=1 P(E|Fi ) · P(Fi )
Proof: Using the definition of Conditional Probability and the Law of Total
Probability.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Bayes’ Formula
♦ Bayes’ Formula
Note 2.5: For the case when n = 2, Bayes’ Formula can be
written as
P(F |E) =
P(E|F )P(F )
.
P(E|F )P(F ) + P(E|F c ) 1 − P(F )
Proof:
Example 2.4
A box contains 4, 5, or 6 balls with each possibility equally likely, and one of the
ball is marked. A ball is chosen at random from box and it is the marked ball. What
is the probability that the urn contain 4 balls?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
In the special case where P(E|F ) does in fact equal to P(E), we
say that the event E is independent of F .
Definition
Suppose E and F are two events in a sample space S with positive probabilities.
Then, E and F are said to be independent if
P(E ∩ F ) = P(E) · P(F ).
That is
P(E|F ) = P(E) and P(F |E) = P(F ).
Two events E and F which are not independent are said to be
dependent.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Note 2.6: Independent events cannot be mutually exclusive, and
vice versa.
P(E ∩ F ) = P(E)P(F ) 6= 0 for independent event ⇒ E and F cannot be
mutually exclusive.
P(E ∩ F ) = 0 6= P(E)P(F ) for mutually exclusive events ⇒ E and F cannot
be independent.
Example 2.5
Suppose we toss a fair die once, and let A = {1, 2}, B = {2, 4, 6}, and
C = {1, 3, 5}. Then, A and B are independent events, but B and C are dependent
events.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Proposition 2.1
If E and F are independent, then so are E and F c , E c and F , and E c and F c .
Proof:
Note 2.7: If the events E and F are independent, and the events
E and G are independent, then E is not necessarily independent
of F ∩ G.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Example 2.6
Suppose two fair dice are tossed, and let
E =
2 dice have same number ;
st
F =
1 die is 1 ;
nd
G =
2 die is 6 .
Then, E and F are independent, and E and G are independent, but E and F ∩ G
are not independent.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Independent of 3 Events
The 3 events E, F , and G in a sample space S with positive probabilities are said
to be independent if
P(EFG)
=
P(E)P(F )P(G);
P(EF )
=
P(E)P(F );
P(EG)
=
P(E)P(G);
P(FG)
=
P(F )P(G).
Note 2.8: In general, pairwise independent of three events does
not imply independence.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Example 2.7
Suppose two fair dice are tossed, and let
st
E =
1 die is even ;
nd
F =
2 die is odd ;
G =
2 dice are both even or both odd .
Then, E, F and G are pairwise independent, but E, F and G are not independent.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Independent of n Events
The n events E1 , E2 , . . . , En in a sample space S with positive probabilities are said
to be independent if, for every subset E10 , E20 , . . . , Er 0 , r ≤ n of these events,
P E10 E20 · · · Er 0 = P E10 P E20 · · · P Er 0 .
Note 2.8: In general, pairwise independent of three events does
not imply independence.
Independent of Infinite Set of Events
The events E1 , E2 , . . . are independent if every finite subset of these events is
independent.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
P(·|F ) is a Probability
♦ P(·|F ) is a Probability
Proposition 2.2
The conditional probability P(E|F ) satisfies the three axioms of a probability. That
is,
(1) 0 ≤ P(E|F ) ≤ 1.
(2) P(S|F ) = 1.
(3) If E1 , E2 , . . ., are mutually exclusive events, then
∞
∞
[
P
Ei |F = ∑ P(Ei |F ).
i=1
i=1
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference