Download condition

STATISTICAL INDEPENDENCE
AND
AN INVITATION TO THE Art OF CONDITIONING
Tutorial 2
STAT1301 Fall 2010
28SEP2010, MB103@HKU
By Joseph Dong
“Look, imagine a remote village where there has been a long drought. One day, local peasants in desperation go to the church, and the priest says a prayer for rain. And the next day the rain arrives! These are independent events.”
Legendary answer made by Kolmogorov when he was questioned by a senior Soviet Minister
about whether the concept of independent events of his probability theory is consistent with
materialistic determinism.
What is the intuitive background of Independence? How is the concept of
independence related from that of mutual exclusiveness?
2
MOTIVATING CONCEPT INDEPENDENCE
•
The event of today’s raining is by no means affected by the prayer said at the
church yesterday.
•
How do you represent this wording in symbolic language so that you can analyze it in a
more precise manner?
•
Using the language of event (set):
• The realization of the event A={it rains today} is by no means affected by the
realization of event B={villagers prayed yesterday}.
• If B has already realized, A will have an unaffected probability of realizing.
• For convenience we use a notation Pr | to denote the probability of A’s
realization given B’s realization. Then the foregoing motivation for the concept
independence will require us to establish the precise meaning of Pr |
Pr
in a symbolic way.
•
A and B independent ↭ Pr
Pr
|
??
3
INDEPENDENCE & MUTUAL EXCLUSIVENESS
•
Two events are mutually exclusive if there is no possibility that they can happen together
(simultaneously).
•
Mutual exclusiveness is a concept/phenomenon on the set theory layer.
•
Independence is a concept/phenomenon on the probability(measure) theory layer.
•
Two events are independent if … (see next slide)
4
Independence:
•
A formal, less intuitive, precise, and deep definition: Two events are independent if
Pr
Pr
A = {The villagers prayed yesterday}, Pr
,
B = {It rains today}, Pr
Now we just need to specify one more probability to establish independence!
If events (A, B) are independent, so are events (A, Bc), (Ac, B), and (Ac, Bc).
Practice: handout problem 3.
5
NOW SHOW ALGEBRAICALLY…
,
•
If the events , , and are completely independent (mutually independent), then the
pairs ,
, ,
, ,
also consist of independent events.
•
⇒
,
,
If the events , are independent, show that the pairs
consist of independent events.
•
,
,
•
also
But not the reverse:
• R.E. = Tossing a coin twice.
• A = {1st toss shows H}, B = {2nd toss shows H}, C = {both tosses show the same side}
6
MORE ON
MUTUAL INDEPENDENCE
•
Intrinsic additive relations:
• the probabilities of the 4 vertices of a face add to the probability of the face.
• the probabilities of the 2 vertices of an edge add to the probability of the edge.
•
Mutual independence require us to establish all multiplicative relations among
vertices, edges, and faces.
7
MORE ON
MUTUAL INDEPENDENCE (CONT’D)
Pr
Pr
⟹ Pr
Pr
Pr
Pr
Pr
Pr
•
Pr
•
Any two of the following three can derive the third.
• Pr
• Pr
Pr
Pr
• Pr
•
Pr
Pr
Pr
Pr
Pr
Pr
Pr
Pr
Pr
How to specify the multiplicative relations of the probabilities of a minimal set of
vertices, edges, and faces?
• Solution 1: 4 vertices
• Solution 2: ?
8
ANOTHER RELATED SOURCE OF PERSPECTIVE
Are the first two columns of digits
independent?
A
B
C
D
1
1
0
0
•
What about A and Bc? Ac and Bc?
0
1
1
0
•
What about second and third?
0
0
0
1
•
What about columns A & B together
and column C? If you are allowed to
add more rows, how could you make
them independent?
1
0
1
0
•
What about the first and last columns?
•
What about the first and last columns
together and column C?
•
9
A CONCRETE EXAMPLE
•
R.E. = Tossing a die once.
•
Event
•
Event
•
What’s Pr
•
If you think A and B are not independent, then what can you say about Pr
•
Actually Pr
•
Are you surprised?
•
Can you give 3 more pairs of events that are independent?
and Pr
2
?
?
?
10
CONDITIONAL PROBABILITY
Pr
Pr
Pr
Pr
?
?
?
?
⋯⋯
Pr
(Total: 4
2
?
8 conditional probabilities.)
Given an event (and of course also its sample space whereby it’s
contained), the probability conditional on event , Pr ⋅ | satisfies
Kolmogorov’s Axioms of Probability.
Pr Ω|
?
Pr |
Pr
?
11
⋅∩
FORMAL DEFINITION:
• Conditional Probability is a valid probability:
• Pr ⋅
0?
• Pr Ω
1? (“conditional sample space”)
• If
’s are disjoint events(subsets of Ω), then
∑ Pr
Pr ⋃
|
?
• Exercise: Prove the identity
• Exercise: Prove the identity Pr
• Exercise: Prove Pr
|
Pr
Pr
Pr
|
|
Pr
Pr
|
|
• Practice: Handout Problems 1 and 2.
12
USING CONDITIONAL PROBABILITY ON ITS OWN
•
In English, we use “if”, “when”, and the alike to indicate a condition.
•
In Probabilistic analysis, we use the vertical bar “|” to indicate a condition.
•
If she says yes, we’ll adopt plan A; if she says no or is indifferent, we’ll go with plan B.
•
If the coin turns a head, I’ll go basketball tonight; if tail, 50% chance I’ll read a book
chapter on Bayes Theory, and 50% chance I’ll go to bed early.
•
If the airplane crashes in the prairie, then there is a high probability to find it; if in the
mountain, then medium probability to find it; if in the sea, then very low probability to find
it.
•
If tomorrow the announcement of last season’s financial result is above public
expectation, then the stock price will go up with probability 0.89; if equals the expectation,
the stock price will go up with probability 0.5; if below expectation, the stock price will go
up with probability 0.08.
13
PARTITION (OF SAMPLE SPACE)
AND CONDITION (ON THE PARTITION)
•
Formal Def. of a Partition of a Set.
• Any disjoint and exhaustive collection of subsets of a given set forms a partition of that set.
,
A trivial partition of Ω: For each subset
•
Throwing a die 10 times and we are interested in the sequence of the ten outcomes. Then
Ω
1111111111,
1111111112,
⋯,
6666666666
If
Each
⊂ Ω,
is a partition of Ω.
•
,
1, 2, ⋯ , 6 , then
is a subset of Ω
The six
s are disjoint
The six
s together are exhaustive.
The six
s together is a partition of Ω.
14
EXERCISE
•
•
We work in the sample space Ω. Let the collection of events
, ,⋯,
be a partition of Ω. Also we have another event
Prove the following identity:
⊂ Ω.
Further, prove the following additional identity:
15
PARTITION AND CONDITION
REVISITED – LAYMAN EDITION
•
You have made your first choice at box A, hoping that it contains the key to unlock a brand new
BMW. Then the host opens box B and it’s empty. Now it is your decision to make on whether
you will switch to choose C or stay with A.
•
What particular partition to form, and hence what conditions to make?
•
In a probability veteran’s language, it is said that we would “condition on A”
.
•
Meaning we first identify the event A as
•
Then form the trivial partition
•
Then you proceed the analysis by making a condition “if event X happens, then” for each
event X in the partition.
•
Here…
• if event
empty.
• if event
the key.
•
,
of the sample space.
happens (ie, …), and with the knowledge
happens (ie, …), and with the knowledge
is empty, then
is empty, then
must also be
must contain
Now it is your turn to reach the end of this thread …
16
QUESTION TO THE LAYMAN EDITION
• What subtlety will be involved if, unfortunately,
you “conditioned on C”, instead of A?!
17
PARTITION AND CONDITION
REVISITED – EXPERT EDITION
•
Brief setup: Let
where ∈ , ,
•
Recall an observation made in tutorial 1: What
are the two events involved when the host,
seeing you have chosen box A, shows you the
emptiness of box B?
•
Let
•
Known:
•
Pr
|
•
Pr
|
,
•
Pr
|
,
•
Pr
0
1
Pr
Pr
,
| ,
•
Want:
•
Pr
1
,
|
•
Pr
,
|
•
Want =
??
Pr
Pr
|
|
,
,
Pr
Pr
|
|
Pr , |
Pr , |
Pr
Pr , | Pr
1 1
1
2 3
1 2 1 1 3
2 3 2 3
Now it’s your turn: mimic an expert solution for the
“Three Prisoners Problem”.
18
EXERCISE
Handout Problem 4
•
What events are involved in the
statement:
• “The initial search was in the
mountains, and the plane was not
found.”
Handout Problem 5
•
Require a clever conditioning
(although the hint has effectively
eliminated this requirement) and some
mathematical maturity
19