Download AXIOMATIC PROBABILITY THEORY

AXIOMATIC PROBABILITY
THEORY
• Our
intuition about
collections of subsets of
large sets is bad
• Our
intuition about
uncountable sets is bad
• Our
intuition about
probabilities is bad
• Our
intuition about large
numbers is bad
PROBABILISTIC PARADOXES
• St. Petersburg
• Bertrands
• Monty
paradox
Box Paradox
Hall paradox
• Borel-Kolmogorov
paradox
EXERCISE
Resolve the Borel-Kolmogorov paradox
BASIC OBJECTS
Probability space:
P = (⌦, , µ)
measure
set of outcomes
set of events
BASIC OBJECTS
Probability space:
P = (⌦, , µ)
The probability space P consists of three objects:
1. The sample set ⌦
2. A collection of subsets of ⌦ called the -algebra
3. and a measure µ that maps elements of
to the closed interval [0, 1]
THE OUTCOME SPACE
!2⌦
For instance in the experiment “flipping two coins”
there are four such elementary events or samples
⌦ = {HH, T T, HT, T H}
THE σ-ALGEBRA
This is a collection of subsets of Ω with certain
proper ties that are necessary for a consistent
development of axiomatic probability theory:
1. The set ⌦ must be in it, i.e. ⌦ 2
2. For any subset A ⇢ ⌦ that is and element of A 2
also be in , i.e. Ā 2
3. TheSunion of countably many sets An 2
i.e. n An 2
its complement must
must also be an element of ,
THE σ-ALGEBRA
1. The set ⌦ must be in it, i.e. ⌦ 2
2. For any subset A ⇢ ⌦ that is and element of A 2
also be in , i.e. Ā 2
3. TheSunion of countably many sets An 2
i.e. n An 2
• The elements in
its complement must
must also be an element of ,
are called events.
• when an experimental outcome ! 2 ⌦ occurs and ! 2 A 2
the event A occurs, too, see below.
we say that
THE σ-ALGEBRA
• The elements in
are called events.
• when an experimental outcome ! 2 ⌦ occurs and ! 2 A 2
the event A occurs, too, see below.
In a way, the elements of this
collection of sets is the
information available to an
observer.
we say that
THE POWER SET
Ω
2
The powerset 2⌦ of a set ⌦ is the collection of all possible subsets of ⌦. And
clearly all the above requirements are fulfilled if ⌦ is finite. In the example of
tossing two coins the powerset has 16 elements
=
2⌦ = [⌦, ;, {HH}, {T T }, {HT }, {T H},
{HH, T T }, {HH, HT }, {HH, T H}, {T T, HT },
{T T, T H}, {T H, HT }, {HH, T T, HT },
{HH, T T, T H}, {HH, T H, HT }, {T T, HT, T H}]
THE POWER SET
=
Ω
2
2⌦ = [⌦, ;, {HH}, {T T }, {HT }, {T H},
{HH, T T }, {HH, HT }, {HH, T H}, {T T, HT },
{T T, T H}, {T H, HT }, {HH, T T, HT },
{HH, T T, T H}, {HH, T H, HT }, {T T, HT, T H}]
• {HH} both coins show heads
• the complement is {T T, T H, HT } one coin shows tails
• At least one coin showed heads {HT, T H, HH} the complement being
• no coin showed heads: {T T }
• the experiment occured: ⌦
• it did not: ;
THE RELATION OF EXPERIMENTAL
OUTCOME AND EVENTS
The elements ! 2 ⌦ can be considered experimental outcomes of elementary
events. They induce the occurance of events contained in the -algebra. That
means if
!2A2
then A occured.
1. A1 = ⌦, i.e. something happened
2. A2 = {T T }, i.e the elementary event itself
3. A3 = {HH, T T }, i.e. both coins show the same face
4. A4 = {T T, HT }, i.e. the second coin has tails
5. A5 = {T T, T H}, i.e. the forst coin has tails
6. A6 = {T T, HT, T H}, i.e at least one coin has tails
7. A7 = {HH, T T, HT }, i.e. the first coin with head, the second coin with
tails did not happen.
OTHER σ-ALGEBRAS
= {⌦, ;}
= [;, ⌦, {HH, HT, T H}, {T T }]
= [;, ⌦, {T T, HH}, {T H, HT }]
BASIC OBJECTS
Probability space:
P = (⌦, , µ)
measure
set of outcomes
set of events
THE PROBABILITY MEASURE
µ :
! [0, 1]
A 7! p = µ(A)
THE PROBABILITY MEASURE
µ :
A
1. µ(A)
!
7!
0
[0, 1]
p = µ(A)
8 A2
2. µ(⌦) = 1
3. if AnTwith (n = 1, 2, ....) is a countable collection of nonoverlapping sets
(An Am = ; 8 n 6= m) then
!
X
[
µ
An =
µ (An )
n
n
THE PROBABILITY MEASURE
1. µ(A)
0
8 A2
2. µ(⌦) = 1
3. if AnTwith (n = 1, 2, ....) is a countable collection of nonoverlapping sets
(An Am = ; 8 n 6= m) then
!
X
[
µ
An =
µ (An )
n
µ(Ā) = 1
µ(A)
n
µ(;) = 0
THE PROBABILITY MEASURE
1. µ(A)
0
8 A2
2. µ(⌦) = 1
3. if AnTwith (n = 1, 2, ....) is a countable collection of nonoverlapping sets
(An Am = ; 8 n 6= m) then
!
X
[
µ
An =
µ (An )
n
n
The most important of theTaxioms above is the third one. It means that
if we have two disjoint sets
S A B = ; and the experimental outcome ! 2 ⌦
occurs the event C = A B can only occur when either ! 2 A or ! 2 B and
thus intuitively
[
µ(A B) = µ(A) + µ(B).
JOINT PROBABILITIES
µ :
A
!
7!
[0, 1]
p = µ(A)
A
!
B
If the outcome ! of an experiment induces two events A and B, i.e if ! 2 A
and ! 2 B that means the joint probability of events A and B is given by
µ(A \ B)
because this is the measure of all the single individual outcomes ! that are
elements of both sets.
CONDITIONAL PROBABILITIES
!
A
B
given that event B occured
!2B
how likely is that A
occured, too
A
!
B
!2A
µ(A \ B)
µ(A|B) =
µ(B)
CONDITIONAL PROBABILITIES
A
!
B
because
µ(A \ B)
µ(A|B) =
µ(B)
A\B =B\A
we obtain
µ(A|B) µ(B) = µ(B|A) µ(A).
SPANNING SETS
Now let’s assume that the entire set of outcomes, i.e. the sample set ⌦, can
be split into a countable set of mutually exclusive events Bn with n = 1, 2, ....
[
Bn = ⌦.
Bn \ Bm = ; and
n
[
n
(A \ Bn ) = A \
[
n
Bn
!
=A\⌦=A
SPANNING SETS
[
n
(A \ Bn ) = A \
X
n
X
n
[
n
Bn
!
=A\⌦=A
µ(A \ Bn ) = µ(A)
µ(A|Bn ) µ(Bn ) = µ(A)
SPANNING SETS
[
n
(A \ Bn ) = A \
X
n
[
n
Bn
!
=A\⌦=A
µ(A|Bn ) µ(Bn ) = µ(A)
X
n
µ(A \ Bn ) = µ(A)
INDEPENDENCE
If an event A is a subset of B, i.e. A ⇢ B then whenever A occurs then so
does B and A \ B = A. Hence
µ(A \ B) = µ(A).
µ(B|A)µ(A) = µ(A)
B
A
µ(B|A) = 1
!
this is full dependence
INDEPENDENCE
µ(A|B) = µ(A) and µ(B|A) = µ(B)
µ(A \ B)
µ(A|B) =
µ(B)
µ(A \ B) = µ(A)µ(B)
B
A
MEASURE PROPERTIES
µ(A \ B)
µ(A|B) =
µ(B)
µ(A|B) µ(B) = µ(B|A) µ(A).
µ(A \ B) = µ(A)µ(B)
µ(Ā) = 1
µ(A)
X
n
µ(A|Bn ) µ(Bn ) = µ(A)