Download Spaces of Random Variables

Spaces of Random Variables
Peter Ouwehand
Department of Mathematical Sciences
University of Stellenbosch
November 2010
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
1 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
P. Ouwehand (Stellenbosch Univ.)
for all x ∈ V ;
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
P. Ouwehand (Stellenbosch Univ.)
for all x ∈ V and α ∈ R;
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
for all x ∈ V and α ∈ R;
(iv) ||x + y || ≤ ||x|| + ||y ||
P. Ouwehand (Stellenbosch Univ.)
for all x, y ∈ V
Spaces of Random Variables
(∆–Inequality);
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
for all x ∈ V and α ∈ R;
(iv) ||x + y || ≤ ||x|| + ||y ||
for all x, y ∈ V
(∆–Inequality);
Think of ||v || as the length of v .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
for all x ∈ V and α ∈ R;
(iv) ||x + y || ≤ ||x|| + ||y ||
for all x, y ∈ V
(∆–Inequality);
Think of ||v || as the length of v .
Think of ||v − w || as the distance between v and w .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
for all x ∈ V and α ∈ R;
(iv) ||x + y || ≤ ||x|| + ||y ||
for all x, y ∈ V
(∆–Inequality);
Think of ||v || as the length of v .
Think of ||v − w || as the distance between v and w .
We say vn → v iff ||vn − v || → 0.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
I
Definition: A normed space is a pair (V , || · ||), where V is a vector
space and || · || is a norm on V , i.e. a function || · || : V → R with
the following properties:
(i) ||x|| ≥ 0
for all x ∈ V ;
(ii) ||x|| = 0 if and only if x = 0;
(iii) ||αx|| = |α| ||x||
for all x ∈ V and α ∈ R;
(iv) ||x + y || ≤ ||x|| + ||y ||
for all x, y ∈ V
(∆–Inequality);
Think of ||v || as the length of v .
Think of ||v − w || as the distance between v and w .
We say vn → v iff ||vn − v || → 0.
A normed space (V , || · ||) is called a Banach space if is complete,
i.e. if every Cauchy sequence in V converges.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
2 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
P. Ouwehand (Stellenbosch Univ.)
for all x, y ∈ V ;
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
P. Ouwehand (Stellenbosch Univ.)
for all x, y ∈ V ;
for all x ∈ V ;
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
for all x, y ∈ V ;
for all x ∈ V ;
(iii) hx, xi = 0 if and only if x = 0;
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
for all x, y ∈ V ;
for all x ∈ V ;
(iii) hx, xi = 0 if and only if x = 0;
(iv) hx, y + zi = hx, y i + hx, zi
P. Ouwehand (Stellenbosch Univ.)
for all x, y , z ∈ V ;
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
for all x, y ∈ V ;
for all x ∈ V ;
(iii) hx, xi = 0 if and only if x = 0;
(iv) hx, y + zi = hx, y i + hx, zi
(v) hαx, y i = αhx, y i
P. Ouwehand (Stellenbosch Univ.)
for all x, y , z ∈ V ;
for all x, y ∈ V and α ∈ R.
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
for all x, y ∈ V ;
for all x ∈ V ;
(iii) hx, xi = 0 if and only if x = 0;
(iv) hx, y + zi = hx, y i + hx, zi
(v) hαx, y i = αhx, y i
for all x, y , z ∈ V ;
for all x, y ∈ V and α ∈ R.
Think of hv , w i as a dot product.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
II
Definition: An inner product space is a pair (V , h·, ·i), where V
is a vector space over R ( and h·, ·i is an inner product on V , i.e. a
function h·, ·i : V × V → R with the following properties:
(i) hx, y i = hy , xi
(ii) hx, xi ≥ 0
for all x, y ∈ V ;
for all x ∈ V ;
(iii) hx, xi = 0 if and only if x = 0;
(iv) hx, y + zi = hx, y i + hx, zi
(v) hαx, y i = αhx, y i
for all x, y , z ∈ V ;
for all x, y ∈ V and α ∈ R.
Think of hv , w i as a dot product.
On Rn , the product induces both length and angle:
√
|x| = x · x
x · y = |x| |y| cos θ
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
3 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
p
Propn: ||x|| := hx, xi defines a norm on V .
Thus every inner product space is a normed space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
p
Propn: ||x|| := hx, xi defines a norm on V .
Thus every inner product space is a normed space.
For the induced norm || · ||:
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
p
Propn: ||x|| := hx, xi defines a norm on V .
Thus every inner product space is a normed space.
For the induced norm || · ||:
I
(Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then
||v + w ||2 = ||v ||2 + ||w ||2
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
p
Propn: ||x|| := hx, xi defines a norm on V .
Thus every inner product space is a normed space.
For the induced norm || · ||:
I
I
(Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then
||v + w ||2 = ||v ||2 + ||w ||2
(Parallelogram Law) If v , w ∈ V , then
||v + w ||2 + ||v − w ||2 = 2||v ||2 + 2||w ||2
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Topological Vector Spaces
III
Let (V , h·, ·i) be an inner product space.
p
p
Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i.
Equality holds iff y is a scalar multiple of x.
p
Propn: ||x|| := hx, xi defines a norm on V .
Thus every inner product space is a normed space.
For the induced norm || · ||:
I
I
(Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then
||v + w ||2 = ||v ||2 + ||w ||2
(Parallelogram Law) If v , w ∈ V , then
||v + w ||2 + ||v − w ||2 = 2||v ||2 + 2||w ||2
An inner product space is called a Hilbert space it is complete.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
4 / 11
Geometry in Hilbert Space
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
I
November 2010
5 / 11
Geometry in Hilbert Space
I
In Rn , the angle θ between two vectors x, y is given by
cos θ =
P. Ouwehand (Stellenbosch Univ.)
x·y
|x| |y|
Spaces of Random Variables
November 2010
5 / 11
Geometry in Hilbert Space
I
In Rn , the angle θ between two vectors x, y is given by
cos θ =
x·y
|x| |y|
In an inner product space V , we therefore define the “angle”
between x, y ∈ V by
cos θ :=
P. Ouwehand (Stellenbosch Univ.)
hx, y i
||x|| ||y ||
where ||x|| :=
Spaces of Random Variables
p
hx, xi
November 2010
5 / 11
Geometry in Hilbert Space
I
In Rn , the angle θ between two vectors x, y is given by
cos θ =
x·y
|x| |y|
In an inner product space V , we therefore define the “angle”
between x, y ∈ V by
cos θ :=
hx, y i
||x|| ||y ||
where ||x|| :=
p
hx, xi
(By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that
this definition makes sense. It also follows that | cos θ| = 1 if and only
if x is a scalar multiple of y , i.e. iff x, y are parallel.)
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
5 / 11
Geometry in Hilbert Space
I
In Rn , the angle θ between two vectors x, y is given by
cos θ =
x·y
|x| |y|
In an inner product space V , we therefore define the “angle”
between x, y ∈ V by
cos θ :=
hx, y i
||x|| ||y ||
where ||x|| :=
p
hx, xi
(By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that
this definition makes sense. It also follows that | cos θ| = 1 if and only
if x is a scalar multiple of y , i.e. iff x, y are parallel.)
We say that x, y ∈ V are orthogonal, and write x ⊥ y , if and only if
hx, y i = 0.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
5 / 11
Geometry in Hilbert Space
I
In Rn , the angle θ between two vectors x, y is given by
cos θ =
x·y
|x| |y|
In an inner product space V , we therefore define the “angle”
between x, y ∈ V by
cos θ :=
hx, y i
||x|| ||y ||
where ||x|| :=
p
hx, xi
(By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that
this definition makes sense. It also follows that | cos θ| = 1 if and only
if x is a scalar multiple of y , i.e. iff x, y are parallel.)
We say that x, y ∈ V are orthogonal, and write x ⊥ y , if and only if
hx, y i = 0.
If G ⊆ V , we say that x ⊥ G iff ∀g ∈ G (x ⊥ g ).
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
5 / 11
Geometry in Hilbert Space
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
II
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
Think of x|| as the best approximation to x in W : It is the vector in
W which lies closest to x.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
Think of x|| as the best approximation to x in W : It is the vector in
W which lies closest to x.
Suppose that V is a Hilbert space, and that W is a linear subspace of
V . For v0 ∈ V , we would like to find the best approximation of v0
in W , i.e. the unique vector w0 such that
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
Think of x|| as the best approximation to x in W : It is the vector in
W which lies closest to x.
Suppose that V is a Hilbert space, and that W is a linear subspace of
V . For v0 ∈ V , we would like to find the best approximation of v0
in W , i.e. the unique vector w0 such that
I
w0 ∈ W , and
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
Think of x|| as the best approximation to x in W : It is the vector in
W which lies closest to x.
Suppose that V is a Hilbert space, and that W is a linear subspace of
V . For v0 ∈ V , we would like to find the best approximation of v0
in W , i.e. the unique vector w0 such that
I
I
w0 ∈ W , and
||v0 − w0 || = inf{||v0 − w || : w ∈ W }, i.e. w0 is the vector in W that
lies closest to v0 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
II
If W is a linear subspace of Rn , then we can project any x ∈ Rn onto
W:
x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W
We call x|| the orthogonal projection of x onto W .
Think of x|| as the best approximation to x in W : It is the vector in
W which lies closest to x.
Suppose that V is a Hilbert space, and that W is a linear subspace of
V . For v0 ∈ V , we would like to find the best approximation of v0
in W , i.e. the unique vector w0 such that
I
I
I
w0 ∈ W , and
||v0 − w0 || = inf{||v0 − w || : w ∈ W }, i.e. w0 is the vector in W that
lies closest to v0 .
Moreover, (v0 − w0 ) ⊥ W .
The vector w0 is called the orthogonal projection of v0 onto W .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
6 / 11
Geometry in Hilbert Space
III
Proposition: Let V be a Hilbert space, and let W be a closed linear
subspace of V . Then any v0 in V has a unique decomposition
||
v0 = v0 + v0⊥
||
where v0 ∈ W ,
v0⊥ ⊥ W
||
v0 is called the orthogonal projection of v0 onto W .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
7 / 11
The Banach Space L1 (Ω, F, P)
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
I
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
For such X , define
Z
||X ||1 :=
P. Ouwehand (Stellenbosch Univ.)
|X | dP < ∞ = E|X |
Spaces of Random Variables
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
For such X , define
Z
||X ||1 :=
|X | dP < ∞ = E|X |
Proposition: || · ||1 is almost a norm on L1 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
For such X , define
Z
||X ||1 :=
|X | dP < ∞ = E|X |
Proposition: || · ||1 is almost a norm on L1 .
Problem: ||X ||1 = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
For such X , define
Z
||X ||1 :=
|X | dP < ∞ = E|X |
Proposition: || · ||1 is almost a norm on L1 .
Problem: ||X ||1 = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
Solution: Form the (quotient) space L1 (Ω, F, P) by regarding as
equal any two RV’s which are equal P–a.s.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
8 / 11
The Banach Space L1 (Ω, F, P)
I
Let L1 (Ω, F, P) be the set of all integrable random variables X . This
is a vector space.
For such X , define
Z
||X ||1 :=
|X | dP < ∞ = E|X |
Proposition: || · ||1 is almost a norm on L1 .
Problem: ||X ||1 = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
Solution: Form the (quotient) space L1 (Ω, F, P) by regarding as
equal any two RV’s which are equal P–a.s.
Theorem: (Riesz–Fischer) L1 (Ω, F, P) is a Banach Space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
8 / 11
The Hilbert Space L2 (Ω, F, P)
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
I
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
Proposition: h·, ·i is almost an inner product on L2 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
Proposition: h·, ·i is almost an inner product on L2 .
Problem: hX , X i = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
Proposition: h·, ·i is almost an inner product on L2 .
Problem: hX , X i = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as
equal any two RV’s which are equal P–a.s.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
Proposition: h·, ·i is almost an inner product on L2 .
Problem: hX , X i = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as
equal any two RV’s which are equal P–a.s.
Theorem: (Riesz–Fischer) L2 (Ω, F, P) is a Hilbert Space.
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
The Hilbert Space L2 (Ω, F, P)
I
Let L2 (Ω, F, P) be the set of all square–integrable random variables
X (i.e for which X 2 is integrable). This is a vector space.
For such X , Y , define
Z
hX , Y i := XY dP = EXY
Proposition: h·, ·i is almost an inner product on L2 .
Problem: hX , X i = 0 does not imply that X = 0, but merely that
X = 0 P–a.s.
Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as
equal any two RV’s which are equal P–a.s.
Theorem: (Riesz–Fischer) L2 (Ω, F, P) is a Hilbert Space.
The induced norm on L2 is defined by
p
1
||X ||2 := hX , X i = E[X 2 ] 2
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
9 / 11
Statistics and Geometry
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
I
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
Now if X ∈ L2 , then
√
p
p
E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2
i.e. ||X ||1 ≤ ||X ||2 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
Now if X ∈ L2 , then
√
p
p
E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2
i.e. ||X ||1 ≤ ||X ||2 .
Hence for probability spaces L2 ⊆ L1 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
Now if X ∈ L2 , then
√
p
p
E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2
i.e. ||X ||1 ≤ ||X ||2 .
Hence for probability spaces L2 ⊆ L1 .
In statistics the variance Var(X ) and standard deviation σX of a
random variable X are defined by
p
Var(X ) := E(X − E[X ])2
σX := Var(X )
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
Now if X ∈ L2 , then
√
p
p
E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2
i.e. ||X ||1 ≤ ||X ||2 .
Hence for probability spaces L2 ⊆ L1 .
In statistics the variance Var(X ) and standard deviation σX of a
random variable X are defined by
p
Var(X ) := E(X − E[X ])2
σX := Var(X )
The covariance Cov(X , Y ) and correlation ρX ,Y of two random
variables X , Y are defined by
Cov(X , Y ) := E[(X − EX )(Y − EY )]
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
ρX ,Y :=
Cov(X , Y )
σX σY
November 2010
10 / 11
Statistics and Geometry
I
Note that if (Ω, F, P) is a probability space, then X has a mean EX
precisely if X ∈ L1 .
Now if X ∈ L2 , then
√
p
p
E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2
i.e. ||X ||1 ≤ ||X ||2 .
Hence for probability spaces L2 ⊆ L1 .
In statistics the variance Var(X ) and standard deviation σX of a
random variable X are defined by
p
Var(X ) := E(X − E[X ])2
σX := Var(X )
The covariance Cov(X , Y ) and correlation ρX ,Y of two random
variables X , Y are defined by
Cov(X , Y ) := E[(X − EX )(Y − EY )]
ρX ,Y :=
Cov(X , Y )
σX σY
These quantities exist and are finite precisely for X , Y ∈ L2 .
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
10 / 11
Statistics and Geometry
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
II
November 2010
11 / 11
Statistics and Geometry
II
Consider now the space L20 := {X ∈ L2 : EX = 0}
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
11 / 11
Statistics and Geometry
II
Consider now the space L20 := {X ∈ L2 : EX = 0}
For X ∈ L20 , we have
√
p
p
σX := Var(X ) = EX 2 = hX , X i = ||X ||2
P. Ouwehand (Stellenbosch Univ.)
Spaces of Random Variables
November 2010
11 / 11
Statistics and Geometry
II
Consider now the space L20 := {X ∈ L2 : EX = 0}
For X ∈ L20 , we have
√
p
p
σX := Var(X ) = EX 2 = hX , X i = ||X ||2
For X , Y ∈ L20 we have
ρX ,Y :=
P. Ouwehand (Stellenbosch Univ.)
Cov(X , Y )
EXY
hX , Y i
=
=
= cos θ
σX σY
σX σY
||X ||2 ||Y |||2
Spaces of Random Variables
November 2010
11 / 11