Download 1.2. Probability Measure and Probability Space

1.2. Probability Measure and Probability Space
1.2.1. Setup: Statistical Experiment
Definition 1.1. A statistical experiment is a chance mechanism satisfying:
(a) all possible distinct outcomes are known a priori
(b) in any particular trial, the outcome is not known a priori, but some regularity is
associated with the outcomes
The experiment is random if
(c) it can be repeated under identical conditions
Examples
Tossing a coin twice: S1 = {(HH), (TH), (HT), (TT)}
Tossing a coin until ”heads”: S2 = {(H), (TH), (TTH), . . .}
Closing daily prices of a stock: S3 = {x : x ∈ R, 0 ≤ x ≤ ∞} or [0, ∞)
Outcome Set / Sample Space: S
• Set which includes all possible distinct outcomes of a statistical experiment
• S can be finite, countable infinite or uncountable.
An event A is a subset of S: A ⊂ S (and if we can assign a probability to A).
1.2.2. σ-fields / σ-algebra
Aim: Assign probabilities to events of interest
Problem: It is impossible to assign probabilities to all subsets A ⊂ S if S is uncountable,
e.g. S = R or S = Rk y power set P(S) cannot always be used as set of relevant events
Solution: we define a probability measure on a collection F of subsets of S.
Definition 1.2. A collection F of subsets of S that satisfies:
(i) S ∈ F
(ii) if A ∈ F , then A ∈ F
(iii) if Ai ∈ F , i = 1, 2, 3, . . ., then
S∞
i=1
Ai ∈ F
is called a σ−field or σ−algebra.
1
Remarks:
(1) σ-field is non-empty and closed under countable unions and intersections.
(2) F allows to focus on relevant events and avoid use of power sets of S.
(3) Extension of the definition of the field.
(4) From a given collection C of subsets of S it is possible to construct a smallest
σ-field containing C: FC = σ(C) by adding complements and countable unions.
FC is the σ-field generated by C.
(5) For uncountable outcomes set it is usually not possible to give an explicit description of F .
1.2.3. Borel σ-field
Most important σ-field defined on real line R −→ B(R) How to define B(R) given R
has infinite number of elements?
Definition 1.3. The smallest σ-field B that contains all open intervals (a, b), (−∞ ≤ a <
b ≤ ∞), is called Borel σ-field. A set A ∈ B is called a Borel set.
Construction: it turns out that a number of different intervals such as [a, b), (a, b] ,
[a, b], (a, b), (−∞, b] can be used to generate the same Borel field.
Preferred B5 = {(−∞, b] : b ∈ R}; generate σ-field B(R) = σ(B5 ) using set theoretical
operations; B(R)⊂P(R)! (Rule out certain strange subsets of R.)
It is difficult to understand which sets are in B. But it is not really necessary. We
are able to define a probability measure w.r.t. B5 and B is just the collection of sets for
which a probability measure is automatically defined.
Theorem 1.4. Put
B1 = {(a, b] : −∞ ≤ a < b < ∞},
B2 = {[a, b) : −∞ < a < b ≤ ∞},
B3 = {[a, b] : −∞ < a < b < ∞},
B4 = {(a, b) : −∞ ≤ a < b ≤ ∞},
B5 = {(−∞, b] : −∞ ≤ b < ∞}.
Then it holds for j = 1, . . . , 5 that B is the smallest σ-field that contains B j .
⇒ Borel fields generated by B j are all the same.
1.2.4. Probability Measure
Story so far: Sample space S, σ-field F containing relevant events, now define Pmeasure for assigning probability to events in F .
2
Definition 1.5. A mapping P(·): F −→ [0, 1] from a σ-field F of subsets of a set S into
the unit interval is a probability measure (or probability set function) if it satisfies the
following three axioms (of probability):
(1) P(s) = 1 for any sample space S
(2) P(A) ≥ 0 for any event A ∈ F
(3) Countable (or σ-) additivity: For a countable sequence of pairwise disjoint events,
T
S
)=
i.e. Ai ∈ F , i = 1, 2, . . . such that Ai A j = ∅ for all i , j, i, j = 1, 2, . . ., P( ∞
i=1
P∞
P(Ai ).
i=1
Countable additivity provides a way to attach probability to disjoint events. For
use of Axiom (iii) see Spanos, pp. 63-65.
Example: P-measure in case of inappropriate collection of events
For an uncountable sample space S: S = {x : 0 ≤ x ≤ 1, x ∈ R} = [0, 1] one can use
axiom (3) since interval [0, 1] can be expressed as countable union of disjoint sets Ai ,
S
T
i = 1, 2, 3, . . . : [0, 1] = ∞
A j , 0, i , j, with P[Ai ] the same for all.
i=1 Ai , where Ai
S∞
P∞
By Axiom (3): P([0, 1]) = P( i=1 ) = i=1 P(Ai ) ⇒ P([0, 1]) = 0 if P(Ai ) = 0 or P([0, 1]) = ∞
if P(Ai ) > 0 ⇒ Contradiction!
Reason: Disjoint sets are members of power set P([0, 1]) but they are not necessary
members of an appropriate σ-field (Borel field) associated with unit interval.
How to assign probability in case of uncountable sample spaces?
Solution: Define P(·) on a smaller treatable set D ⊂ F . We hope this defines P(·) on F .
This works as follows.
(i) Choose D as a field
(a) S ∈ D
(b) A ∈ D ⇒ A ∈ D
(c) A1 , A2 ∈ D ⇒ (A1
S
A2 ) ∈ D
(ii) Define F = σ(D) as the smallest σ-field with D ⊂ F .
(iii) Choose PD : D → [0, 1] with
(a) PD (S) = 1
(b) PD (A) ≥ 0 for any event A ∈ D
(c) Ai ∈ D, i = 1, 2, . . . such that Ai
S
P∞ D
PD ( ∞
i=1 ) =
i=1 P (A j )
T
A j = ∅ for all i , j
3
(iv) Apply the following Theorem 1.6.
Theorem 1.6 (Caratheodory’s extension theorem)
For D, F , PD as above in (i)-(iii) there exists a unique probability measure P on F with
P(A) = PD (A) for A ∈ D.
Example: Uniform probability measure w.r.t. S = [0, 1] (Bierens, Section 1.6)
We can define P-measure PD on field D of subsets of [0, 1], namely
D = {(a, b), [a, b], (a, b], [a, b), ∀a, b ∈ [0, 1], a ≤ b, and their finite unions},
where [a, a] = {a}, (a, a) = [a, a) = (a, a] = ∅ such that PD ([a, b]) = PD ((a, b]) = PD ([a, b)) =
PD ((a, b)) = a − b for 0 ≤ a ≤ b ≤ 1.
Note:
• Any finite union of intervals can be written as a finite union of disjoint intervals
by cutting off the overlap.
• Add up lengths of disjoint intervals to obtain P.
S
S∞
Problem: PD ( ∞
i=1 Ai ) is not always defined since
i=1 Ai ∈ D is not always guaranteed.
D
y Need to extend P to assign probability to any Borel set in [0, 1]. Done by
Theorem 1.6, this gives uniform probability measure.
1.2.5. Probability Space
• Now we can describe the statistical experiment by the triple {S, F , P(·)} which is
called a probability space.
• The tuple {S, F } is called a measurable space.
o Review properties of σ-field and P-measure in Bierens, Section 1.4-1.5 and Spanos,
Section 2.6.7: continuity property
4