Download PИШω b Ω вω1 1ЭНа 1

P ROBABILITY T HEORY
098416
L ECTURE #1
O CT 10, 2015
L ECTURER : O REN L OUIDOR
1
1.1
S CRIBE : S EGEV S HLOMOV
Motivation
Random Walk
An ant filliping a coin infinitely ,at each iteration if heads comes then ant moves one step right and
if tails, moves one step left. A possible probability space: Ω
˜ω
ˆω1 , ω1 , ω1 , ...
ωi > ˜0, 1
which describe all the infinite series of zeros and ones (note that the cardinality of this set is
¯
, same cardinality of the continuum). Let’s define some probability measure (marked as P ):
P ˆ˜ω > Ω ω1
1
2
1
which equivalent to:
P ˆAt iteration ik got ak , f or k
1.2
1, 2, .., m
1m
f or i1 @ i2 @ ... @ im and a1 , a2 , ..., am > ˜0, 1
2
(1)
Probability measure
We are looking for a function P defined on a set of events in our probability space Ω to 0, 1
satisfies:
(A1) P ˆΩ
1
(A2) For each disjoint set A, B > Ω , P ˆA < B 
P ˆA P ˆB 
This property called finite additivity
(A3) P ˆ"i 1 Ai 
ª
Pi 1 P ˆAi
ª
where " means disjoint union.
This property called countable additivity, note that this property includes the previous one.
(A4) P ˆA B P ˆB  for any A b B (deduced from A3).
1
We are going to ask several question:
Question 1
Does such P , which satisfies A1-A4 exist over all subsets of Ω? (In the random walk problem: no)
Question 2
In case the answer to Q1 is no, can we choose a smaller subsets of Ω, which contains enough
information, that P exist over him? (In the random walk problem: yes)
Question 3
Is such P , in case the answer to Q2 is yes, which satisfies A1-A4 and (1), unique? (In the random
walk problem: yes)
Let’s try to build P explicitly based on (1):
P ˆˆω1 , ω2 , ... B P ˆ˜ω̃ > Ω ω̃i
ˆA4
which true for every m, thus P ˆω 
since A
1, 2, .., m
ωi , i
ˆ1
1m
2
0. Note that if A > Ω countable then P ˆA
"ω>A˜ω, but what about a non-countable A?
(2)
Pω>A P ˆ˜ω
0
Question 4
Is there a non-countable event A > Ω with probability 0? Note that P y 0 since P ˆ˜1, ω2 , ...
1.3
1
2
Law of large numbers
Assuming the answer to Q2 is yes, define our probability measure on a limited subsets of Ω, let’s
see a much complicated event:
Define Xn - the nth step of the ant. We know that Xn , n
identically distributed) and for all n
thus, E Xn Let Sn
0 and Var Xn 1, 2, ... , P ˆXn
1
1, 2, ... are i.i.d (independent and
P ˆ Xn
1.
Pnk 1 Xk - ant position at time n. Consider Sn
n
2
as n
ª
1
1
2.
1.3.1
Week law of large numbers
For each @ 0
Sn
P ˆS
n
Var Snn Sn
E
S C 
B
ˆChebyshev 
n
2
11
n 2
´¹¹ ¹ ¹ ¸¹¹ ¹ ¹ ¹¶
n
ÐÐ
ª
0
(3)
0
Which means that
1.3.2
Sn
n
converges to 0 in probability.
Strong law of large numbers
P ˆ˜ lim
n
ª
Sn
n
0
1
(4)
which equivalent to:
P ˆ˜ω > Ω Which means that
Let A
˜ω
>Ω
Sn
n
1 n
Q
nk 1
n
ÐÐ
ª
0
1
(5)
converges to 0 a.s (almost surly).
ÐÐ
number of +1 in first n bytes n
n
ª
1
2 ,
note that P ˆA
1 and thus, P ˆΩƒA
0 , surely
this is a non-countable event.
Let’s change the story and think about Sn as gambler’s money, at each iteration the gambler earns
one dollar or lose one. An interesting question: is there any strategy (stopping criteria) that ensures
profit? define a random variable τ which depends on X1 , X2 , ... such that E Sτ A 0.
For example: τ
min˜n C 1 Sn
1, note that τ
1. In addition, one can prove that P ˆτ
since Sτ
ª

ª
if Sn @ 1 ¦n
1, 2, 3, ..., and ”E Sτ ”
1
0.
Notice that τ can get big value, lets say 1010 ,this is a problem since we probably can’t play 1010
rounds (it’s can also be a money problem).
Define: τm
min˜τ, m and then:
E Sτ1 0
E Sτ2 1 ‡ 0.5 2 ‡ 0.25 0 ‡ 0.25
0 and so on.
As expected, one can prove that each stopping strategy which deepens on finite money/time gives
zero expectation. In particular E Sτm Note that since P ˆτ @ ª
1 then τm
So we got interesting result: Sτm
Ða.s
0 ¦m C 1
ÐmÐÐ
ª
τ a.s
τ but E Sτm 3
0
ÐmÐÐ
ª
E Sτ 1
which means that if a series of random variables converges to some random variable then not
necessarily the expectations converge to each other.
1.4
Length measure
One example for a measure is length, we would like to define a length ˆλ on subsets of R.
This function (λ) satisfies:
1. λˆ a, b
λˆ a, b
λˆ a, ª
눈ª, b
ª
@ a @ b @ ª and
ª
Pi 1 λˆAi - countable additivity.
2. λˆ"i 1 Ai 
ª
ª
3. λˆg
b a for
0
Again, we can ask: existence over all subsets of R (NO). Existence over a particular, rich enough,
subsets? (YES). Uniqueness? (YES)
2
Measure
Let Ω be a non empty set.
Definition 2.1. A collection P of subsets of Ω called π-system if:
A, B > P
A
9
B>P
Definition 2.2. A collection A of subsets of Ω called algebra if:
1.
g
>A
2. A > A
ΩƒA Ac Ā > A
3. A, B > A
A
8
B>A
Note that from (1)+(2)+(3) A 9 B > A and each algebra is in particular a π-system, and (3) holds
only for finite union (does not include countable union).
4
Definition 2.3. A collection Σ of subsets of Ω called σ-algebra if:
1.
g
>A
2. A > A
Ac > A
3. A1 , A2 , ... > Σ
"i 1 Ai > Σ
ª
Of course that each σ-algebra is algebra. Note that from (3)+(De Morgan) A1 , A2 , ... > Σ
ª
i 1 Ai
>Σ
Definition 2.4. Let C be a collection of subsets of Ω then, αˆC  is the smallest algebra containing
C and σ ˆC  is the smallest σ-algebra containing C. For these definitions to be well defined one
shall prove that intersection between two algebras is also algebra and intersection between two
σ-algebras is also σ-algebra
2.1
Let Ω
Examples
R, a π-system will be:
P
˜Iab
ª
Define our algebra by: A
B a B b B ª , where Iab
¢̈
¨
¨ ˆa, b
¦
¨
¨ ˆa, ª
¤̈
if
ª
if b
BaBb@ª
(6)
ª
αˆP  , meaning smallest algebra containing P
Claim 2.5.
A
n
˜
# Ia b
i i
,n
1, 2, ... ,
ª
B a1 B b1 B a2 B b2 B ... B an B bn B ª
i 1
Note that this collection is not ”rich” enough since, for example, we can not represent the rational
numbers with intersection and union of these sets.
Define our σ-algebra by: Σ
σ ˆA 
Claim 2.6.
σ ˆall open subsets of R
5
σ ˆP 
σ ˆA 
σ ˆP 
Proof. Let A be σ ˆall open subsets of R, we are going to prove that A
First direction
Notice that P b A since we can write each interval in the form ˆa, b as
n 1 ˆa, b
ª
1
n
which of
course in A since each element in the intersection is in A (and from property A2 of σ-algebra). In
case of b
ª
it’s already an open set so of course it’s in A
Second direction
˜ˆall
open subsets of R b σ ˆP  since any open set can be written as countable union of half-
open intervals.
Proof. Let N
1, 2, 3, ... and consider ˜ˆ nk , kn1 Gn
k > R. Let G be some open set and define
ˆ
k k 1
k ˆ n
, n bG
k k1
,

n n
(7)
n 1 Gn holds since for every x > G and open set G, exist N such that ˆx n4 , x n4  b G,
but then, exist k such that x b ˆ nk , kn1  thus,x > n 1 Gn and G b n 1 Gn . The other direction is similar. Note that n 1 Gn is countable union of intervals from P and so we proved that
then, G
ª
ª
ª
ª
˜ˆall
open subsets of R b σ ˆP 
Thus, σ ˆall open subsets of R b σ ˆP  and overall, we got σ ˆall open subsets of R
Similarly we can prove that A
σ ˆP 
σ ˆA  which conclude our proof
A σ-algebra forms from all open subsets of Ω is called Borel σ-algebra and marked as B ˆΩ, in
our case we found that Σ
B ˆR
6