* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Conditional Probability and Independence
Survey
Document related concepts
Transcript
Conditional Probability and Independence
Conditional Probability and Independence
Conditional Probabilities (§ 3.2)
Bayes’ Formula (§ 3.3)
Independent Events (§ 3.4)
P(·|F ) is a Probability (§ 3.5)
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Suppose we have partial information concerning the result of an
experiment.
We assume that the event F occurred on the experiment.
But, we do not know which outcome in F occurred, only that the outcome is
a member of the event F .
Suppose x is a member of the event F , and the event E occurred
⇐⇒ x is a member of E as well.
E occurred if x is a member of F ∩ E, and E did not occur if x is a
member of F ∩ E c .
Since we cannot say whether E occurred or not, we need an
alternative description of probability for the situation.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Since we cannot say whether E occurred or not, we need an
alternative description of probability for the situation.
Suppose that P(E) and P(E c ) should be updated in terms of the
partial information that the event F occurred.
In generally, P(E) is called the unconditional or prior probability of
the event E. The updated probability of E is called the conditional
probability of E given the event F occurred, and is denoted by
P(E|F ).
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Definition
Suppose E and F are two events in a sample space S. Then, the conditional
probability of E given that F has occurred is defined to be
P(E|F ) =
P(E ∩ F )
,
P(F )
where P(F ) > 0.
Note 2.1: In order for the event E occurred by given the event F
occurred, we need the actual outcome occurred both in E and F .
Note 2.2: The event F is known as the new (or reduced) sample
space.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Note 2.3: Given the event E in the sample space S, we have
P(E|S) = P(E).
Proof:
Example 2.1
Two fair dice are rolled, given that the same number was obtained on both dice.
What is the probability that the sum of two dice is 2 or 3 or 4?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Note 2.4: When all outcomes are equally likely, it is often easier
to compute a conditional probability by considering the reduced
sample space rather than using
P(E|F ) =
P(E ∩ F )
.
P(F )
Example 2.2
All 52 cards of a standard desk of cards are dealt out randomly and equally to 4
player, say North, South, East, and West. If North and South have 2 aces among
them, what is the probability that West has the other 2 aces?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Multiplication Rule
Given n events of E1 , . . . , En in a sample space S, we have
P(E1 E2 · · · En ) = P(E1 )P(E2 |E1 )P(E3 |E1 E2 ) · · · P(En |E1 · · · En−1 ).
Note 2.5: From the definition of conditional probability, we have
P(EF ) = P(E)P(E|F ),
which is the multiplication rule for n = 2.
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Example 2.3
A bowl contains 12 Red and 8 Black balls. Four balls are selected at random and
at a time without replacement. What is the probability that the balls are are Red?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Conditional probability can often be used to compute the desired
probabilities more easily.
Suppose we want to find the probability of an event E from a
sample space S.
Another approach is that dividing E into some mutually exclusive
events, and sum the probabilities.
Let F be any event such that
E = E ∩F ∪ E ∩Fc .
Then,
P(E) = P E ∩ F + P E ∩ F c .
Using the Multiplication Rule, we can write
P(E) = P(E|F )P(F ) + P(E|F c )P(F c ).
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Conditional Probabilities
♦ Conditional Probabilities
Law of Total Probability
Suppose E is an event in a sample space S, and let F1 , F2 , . . . , Fn be mutually
exclusive events such that ∪ni=1 Fi = S, and that P(Fi ) > 0, i = 1, . . . , n. Then,
n
P(E) =
∑ P(E|Fi )P(Fi ).
i=1
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Bayes’ Formula
♦ Bayes’ Formula
Bayes’ Formula
Suppose that F1 , . . . , Fn are n mutually exclusive events in a sample space S such
that ∪ni=1 Fi = S. Then, for any event E in S
P(Fr |E) =
P(E|Fr ) · P(Fr )
.
∑ni=1 P(E|Fi ) · P(Fi )
Proof: Using the definition of Conditional Probability and the Law of Total
Probability.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Bayes’ Formula
♦ Bayes’ Formula
Note 2.5: For the case when n = 2, Bayes’ Formula can be
written as
P(F |E) =
P(E|F )P(F )
.
P(E|F )P(F ) + P(E|F c ) 1 − P(F )
Proof:
Example 2.4
A box contains 4, 5, or 6 balls with each possibility equally likely, and one of the
ball is marked. A ball is chosen at random from box and it is the marked ball. What
is the probability that the urn contain 4 balls?
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
In the special case where P(E|F ) does in fact equal to P(E), we
say that the event E is independent of F .
Definition
Suppose E and F are two events in a sample space S with positive probabilities.
Then, E and F are said to be independent if
P(E ∩ F ) = P(E) · P(F ).
That is
P(E|F ) = P(E) and P(F |E) = P(F ).
Two events E and F which are not independent are said to be
dependent.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Note 2.6: Independent events cannot be mutually exclusive, and
vice versa.
P(E ∩ F ) = P(E)P(F ) 6= 0 for independent event ⇒ E and F cannot be
mutually exclusive.
P(E ∩ F ) = 0 6= P(E)P(F ) for mutually exclusive events ⇒ E and F cannot
be independent.
Example 2.5
Suppose we toss a fair die once, and let A = {1, 2}, B = {2, 4, 6}, and
C = {1, 3, 5}. Then, A and B are independent events, but B and C are dependent
events.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Proposition 2.1
If E and F are independent, then so are E and F c , E c and F , and E c and F c .
Proof:
Note 2.7: If the events E and F are independent, and the events
E and G are independent, then E is not necessarily independent
of F ∩ G.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Example 2.6
Suppose two fair dice are tossed, and let
E =
2 dice have same number ;
st
F =
1 die is 1 ;
nd
G =
2 die is 6 .
Then, E and F are independent, and E and G are independent, but E and F ∩ G
are not independent.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Independent of 3 Events
The 3 events E, F , and G in a sample space S with positive probabilities are said
to be independent if
P(EFG)
=
P(E)P(F )P(G);
P(EF )
=
P(E)P(F );
P(EG)
=
P(E)P(G);
P(FG)
=
P(F )P(G).
Note 2.8: In general, pairwise independent of three events does
not imply independence.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Example 2.7
Suppose two fair dice are tossed, and let
st
E =
1 die is even ;
nd
F =
2 die is odd ;
G =
2 dice are both even or both odd .
Then, E, F and G are pairwise independent, but E, F and G are not independent.
Solution:
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
Independent Events
♦ Independent Events
Independent of n Events
The n events E1 , E2 , . . . , En in a sample space S with positive probabilities are said
to be independent if, for every subset E10 , E20 , . . . , Er 0 , r ≤ n of these events,
P E10 E20 · · · Er 0 = P E10 P E20 · · · P Er 0 .
Note 2.8: In general, pairwise independent of three events does
not imply independence.
Independent of Infinite Set of Events
The events E1 , E2 , . . . are independent if every finite subset of these events is
independent.
Qihao Xie
Introduction to Probability and Basic Statistical Inference
Conditional Probability and Independence
⇒
P(·|F ) is a Probability
♦ P(·|F ) is a Probability
Proposition 2.2
The conditional probability P(E|F ) satisfies the three axioms of a probability. That
is,
(1) 0 ≤ P(E|F ) ≤ 1.
(2) P(S|F ) = 1.
(3) If E1 , E2 , . . ., are mutually exclusive events, then
∞
∞
[
P
Ei |F = ∑ P(Ei |F ).
i=1
i=1
Proof:
Qihao Xie
Introduction to Probability and Basic Statistical Inference