Download Probability Theory Measure Spaces Definition 1. Sigma

Probability Theory
Measure Spaces
Definition 1. Sigma Algebra
Let E be a set. A non-empty collection E of subsets of E is called a sigma-algebra on E if
(1) A ∈ E ⇒ E − A S
∈E
(2) {Ai }i∈N ⊆ E ⇒ i∈N Ai ∈ E
Corollary 1. Let E be a set and E be a sigma-algebra on E. Then
(1) E, Ø ∈ E
T
(2) {Ai }i∈N ⊆ E ⇒ i∈N Ai ∈ E
Definition 2. Measurable Space
A pair (E, E) where E is a set and E is a σ-algebra on E is called a measurable space.
Throughout, fix a measurable space (E, E).
Definition 3. Measurable Function
Let (E, E) and (F, F) be measurable spaces. A function f : E → F is called measurable (with respect to
F and E) if
f −1 (A) = {x ∈ E|f (x) ∈ A} ∈ E f or each A ∈ F
Corollary 2. Let (E, E), (F, F) and (G, G) be measurable spaces and f : E → F, g : F → G be measurable.
Then g ◦ f : E → G is measurable.
Definition 4. Measure
A function µ : E → R+ is called a measure on (E, E) if
(1) µ(Ø)
F =0
P
(2) µ( i∈N Ai ) = i∈N µ(Ai ) for each disjoint {Ai }i∈N ⊆ E
Definition 5. Measure Space
A triple (E, E, µ) is called a measure space if (E, E) is a measurable space and µ is a measure on it.
Integration
Throughout, fix a measure space (E, E, µ) and let R be equipped with its Borel sigma algebra B(R), so
that the pair form a measurable space.
Definition 6. Simple
A function φ : E → R is called simple if for some disjoint {Ai }ni=1 ⊆ E and {ai ∈ R}ni=1
φ=
n
X
ai · 1Ai
i=1
Corollary 3. Simple functions are measurable.
Theorem 1. A function f : E → R+ is measurable if and only if it can be expressed as the monotone limit
of a sequence of simple functions.
Indeed, if f is measurable then the sequence {φi }i∈N defined by let φn : E → R+ be defined by
n if f (x) ≥ n
φn (x) =
i−1
i
n
if i−1
2n
2n ≤ f (x) < 2n for i = 1, ..., n2
is a monotone sequence of simple functions converging to f.
Theorem 2. Let f : E → R be measurable. Then the functions f + , f − : E → R+ defined by
f + (x) = max{f (x), 0} =
1
1
(|f (x)| + f (x)) and f − (x) = max{−f (x), 0} = (|f (x)| − f (x))
2
2
are measurable, and clearly satisfy
f = f+ − f−
and
1
f +, f − ≥ 0
Definition 7. Integral
Pn
Let φ : E → R+ be simple so that φ = i=1 ai · 1Ai . Then the integral of f with respect to µ is
Z
φ(x) µ(dx) =
E
n
X
ai · µ(Ai )
i=1
R
and in particular note that E 1A µ(dx) = µ(A).
More generally, let f : E → R+ be measurable and let {φi }i∈N be a monotone sequence of simple functions
converging to f. Then the integral of f with respect to µ is
Z
Z
f (x) µ(dx) = lim
φn (x) µ(dx)
n→∞
E
E
R
R
Finally, let f : E → R be measurable and such that at least one of E f + (x) µ(dx), E f − (x) µ(dx) is
finite. Then the integral of f with respect to µ is
Z
Z
Z
f (x) µ(dx) =
f + (x) µ(dx) −
f − (x) µ(dx)
E
E
E
Definition 8. Integral over a set
Let A ∈ E and f : E → R be measurable. Then the integral over A of f is defined as
Z
Z
f (x) µ(dx) =
f (x) · 1A (x) µ(dx)
A
E
Definition 9. Monotone Class
A monotone class of functions is a collection M of measurable functions f : E → R such that
(1) 1 ∈ M
(2) Bounded f, g ∈ M; a, b ∈ R ⇒ af + bg ∈ M
(3) {fn }n∈N ⊆ M; fn ↑ f ⇒ f ∈ M
Theorem 3. Monotone Class Theorem for functions
Let M be a monotone class of functions on E such that 1A ∈ M for each A ∈ E. Then M contains all
positive (or negative) measurable functions and all bounded measurable functions f : E → R.
Probability Spaces and Random Variables
Definition 10. Probability Space
A probability space is a measure space (Ω, H, P) such that P(Ω) = 1.
Throughout, fix a probability space (Ω, H, P).
Definition 11. Random Variable
A random variable is a measurable function X : Ω → E.
Definition 12. Image Measure
Let (E, E, µ) be a measure space, (F, F) be a measurable space and f : E → F be a measurable function.
The image measure of µ under f is the measure µf : F → R+ on F defined by
µf = µ ◦ h−1
so that
µf (A) = µ ◦ f −1 (A) = µ({x ∈ E|f (x) ∈ A})
Corollary 4. The image measure can only be defined for a measurable function.
Definition 13. Distribution Measure of a Random Variable
Let X : Ω → E be a random variable. The distribution measure of X is the image measure of P under X.
Corollary 5. Let X : Ω → E be a random variable and PX denote the distribution measure of X. Then
P(X ∈ A) = P({ω|X(ω) ∈ A}) = PX (A)
2
f or each A ∈ E
Definition 14. Expected Value
Let X : Ω → R be a real-valued random variable. Then the expected value of X is defined as
Z
E(X) =
X P(dω)
E
Corollary 6. E(1A ) = P(A) for each A ∈ H
Example 1. ’Discrete’ Random Variables
Let X be a real-valued random variable that takes on at most countably many values, say {ai }i∈N , and
let Ai = X −1 (ai ) = {ω ∈ Ω|X(ω) = ai } for each i ∈ N. Then X can be expressed
Z
X
X
X
X=
ai · 1Ai
and thus
E(X) =
X(ω) P(dω) =
ai · P(Ai ) =
ai · P(X = ai )
Ω
i∈N
i∈N
i∈N
More generally, let g : R → R be a measurable function. Then g ◦ X : Ω → R is again a random variable
which takes only the countably many values {g(ai )}i∈N . Thus, arguing as before, we have
Z
X
X
E(g(X)) =
g(X(ω)) P(dω) =
g(ai ) · P(g(X) = g(ai )) =
g(ai ) · P(X = ai )
Ω
i∈N
i∈N
Definition 15. Absolutely Continuous
Let µ, ν : E → R+ be measures. µ is called absolutely continuous with respect to ν, denoted µ ν if
ν(A) = 0 ⇒ µ(A) = 0
f or each A ∈ E
Theorem 4. Radon-Nikodym Theorem and Derivative
Let µ, ν : E → R+ be measures. Then µ ν if and only if there exists a measurable f : E → R+ such
that for each B ∈ E we have
Z
Z
Z
µ(B) =
f (x) ν(dx)
or equivalently
g(y) µ(dy) =
g(x) · f (x) ν(dx)
B
E
E
for each measurable g : E → R. In this case, f is called the Radon-Nikodym derivative, or density, of µ with
respect to ν.
Example 2. ’Continuous’ Random Variables
Let (R, B(R), Leb) denote the measure space of the real numbers equipped with their Borel sigma algebra
and Lebesgue measure. Let X be a real-valued random variable such that P (X ∈ Z) = 0 for each measure
zero set Z ∈ B(R), or equivalently, such that PX Leb. Then by the Radon-Nikodym theorem there exists
f such that
Z
P(X ∈ A) = PX (A) =
f (x) dx
A
Theorem 5. The Change of Variables Theorem
Let (E, E, µ) be a measure space, (F, F) be a measurable space and f : E → F, g : F → R be measurable
functions. Then
Z
Z
g ◦ f (x) µ(dx) =
E
g(y) µf (dy)
F
Example 3. ’Continuous’ Random Variables Continued
As before, let X be a real valued random variable such that PX Leb and further let g : R → R be
measurable. Then
Z
Z
Z
E(g(X)) =
g(X(ω)) P(dω) =
g(x) PX (dx) =
g(x) · f (x) dx
Ω
R
and in particular, letting g be the identity function g(x) = x we obtain
Z
E(X) =
x · f (x) dt
R
3
R
Example 4. Cummulative Distribution Functions
Again, let X be a real valued random variable. Then the cummulative distribution function is defined as
F (x) = PX ((−∞, x)) = P(X ≤ x)
Now if as above PX Leb, then we may express this as
Z
x
F (x) = PX ((−∞, x)) =
f (y) dy
−∞
and when this is the case, Lebesgue’s fundamental theorem of calculus yeilds that F is differentiable almost
everywhere and F 0 (x) = f (x).
Sigma Algebras and Determinability
Recall we have a fixed probability space (Ω, H, P) and measure spaces (E, E, µ), (F, F, ν), (R, B(R), Leb).
Definition 16. Sigma Algebra Generated by a Random Variable
Let X : Ω → E be a random variable (that is, measurable). The sigma algebra σX generated by X is the
smallest sigma algebra on Ω such that X is measurable, and equivalently,
σf = σ{X −1 A|A ∈ E}
Theorem 6. Let X : Ω → E be H-measurable. Another random variable Y : Ω → F is σX-measurable if
and only if there exists a measurable f : E → F such that
Y =f ◦X
Corollary 7. Let {Xi : Ω → E} be H-measurable. Another random variable Y : Ω → F is σ{Xi }-measurable
if and only if there exists a measurable f : E → F such that
Y = f ◦ (Xt1 , Xt2 , ...)
for some sequence {Xti }ti ∈N ⊆ {Xi }
Example 5. Let F ⊆ H be a sub-sigma algebra on Ω. Then
F = σ{1A |A ∈ F}
so that measurability with respect to an arbitrary sigma algebra may be understood in terms of the above
theorem.
Conditional Expectations and Probabilities
Definition 17. Conditional (on a sigma algebra) Expectation
Let X : Ω → R+ be a random variable and F ⊆ H be a sub-sigma algebra on Ω. The conditional
expectation of X given F is a random variable EF X such that
(1) EF X is F measurable.
(2) EEF X · 1A = EX · 1A for each A ∈ F
or equivalently (to 2)
(2’) EEF X · Y = EX · Y for each F-measurable random variable Y .
More generally, for (not necesarilly positive) random variables X : Ω → R such that E(X) exists, we
define
EF X = EF X + − EF X −
Definition 18. Conditional (on a random variable) Expectation
Let X, Y : Ω → R+ be random variables. The conditional expectation of X given Y is
EY X = EσY X
Definition 19. Conditional Probability
Let A ∈ H and F ⊆ H be a sub-sigma algebra on Ω. The conditional probability of A given F is
PF (A) = EF (1A )
4