Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Spaces of Random Variables Peter Ouwehand Department of Mathematical Sciences University of Stellenbosch November 2010 P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 1 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 P. Ouwehand (Stellenbosch Univ.) for all x ∈ V ; Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| P. Ouwehand (Stellenbosch Univ.) for all x ∈ V and α ∈ R; Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| for all x ∈ V and α ∈ R; (iv) ||x + y || ≤ ||x|| + ||y || P. Ouwehand (Stellenbosch Univ.) for all x, y ∈ V Spaces of Random Variables (∆–Inequality); November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| for all x ∈ V and α ∈ R; (iv) ||x + y || ≤ ||x|| + ||y || for all x, y ∈ V (∆–Inequality); Think of ||v || as the length of v . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| for all x ∈ V and α ∈ R; (iv) ||x + y || ≤ ||x|| + ||y || for all x, y ∈ V (∆–Inequality); Think of ||v || as the length of v . Think of ||v − w || as the distance between v and w . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| for all x ∈ V and α ∈ R; (iv) ||x + y || ≤ ||x|| + ||y || for all x, y ∈ V (∆–Inequality); Think of ||v || as the length of v . Think of ||v − w || as the distance between v and w . We say vn → v iff ||vn − v || → 0. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces I Definition: A normed space is a pair (V , || · ||), where V is a vector space and || · || is a norm on V , i.e. a function || · || : V → R with the following properties: (i) ||x|| ≥ 0 for all x ∈ V ; (ii) ||x|| = 0 if and only if x = 0; (iii) ||αx|| = |α| ||x|| for all x ∈ V and α ∈ R; (iv) ||x + y || ≤ ||x|| + ||y || for all x, y ∈ V (∆–Inequality); Think of ||v || as the length of v . Think of ||v − w || as the distance between v and w . We say vn → v iff ||vn − v || → 0. A normed space (V , || · ||) is called a Banach space if is complete, i.e. if every Cauchy sequence in V converges. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 2 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi P. Ouwehand (Stellenbosch Univ.) for all x, y ∈ V ; Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 P. Ouwehand (Stellenbosch Univ.) for all x, y ∈ V ; for all x ∈ V ; Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 for all x, y ∈ V ; for all x ∈ V ; (iii) hx, xi = 0 if and only if x = 0; P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 for all x, y ∈ V ; for all x ∈ V ; (iii) hx, xi = 0 if and only if x = 0; (iv) hx, y + zi = hx, y i + hx, zi P. Ouwehand (Stellenbosch Univ.) for all x, y , z ∈ V ; Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 for all x, y ∈ V ; for all x ∈ V ; (iii) hx, xi = 0 if and only if x = 0; (iv) hx, y + zi = hx, y i + hx, zi (v) hαx, y i = αhx, y i P. Ouwehand (Stellenbosch Univ.) for all x, y , z ∈ V ; for all x, y ∈ V and α ∈ R. Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 for all x, y ∈ V ; for all x ∈ V ; (iii) hx, xi = 0 if and only if x = 0; (iv) hx, y + zi = hx, y i + hx, zi (v) hαx, y i = αhx, y i for all x, y , z ∈ V ; for all x, y ∈ V and α ∈ R. Think of hv , w i as a dot product. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces II Definition: An inner product space is a pair (V , h·, ·i), where V is a vector space over R ( and h·, ·i is an inner product on V , i.e. a function h·, ·i : V × V → R with the following properties: (i) hx, y i = hy , xi (ii) hx, xi ≥ 0 for all x, y ∈ V ; for all x ∈ V ; (iii) hx, xi = 0 if and only if x = 0; (iv) hx, y + zi = hx, y i + hx, zi (v) hαx, y i = αhx, y i for all x, y , z ∈ V ; for all x, y ∈ V and α ∈ R. Think of hv , w i as a dot product. On Rn , the product induces both length and angle: √ |x| = x · x x · y = |x| |y| cos θ P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 3 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. p Propn: ||x|| := hx, xi defines a norm on V . Thus every inner product space is a normed space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. p Propn: ||x|| := hx, xi defines a norm on V . Thus every inner product space is a normed space. For the induced norm || · ||: P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. p Propn: ||x|| := hx, xi defines a norm on V . Thus every inner product space is a normed space. For the induced norm || · ||: I (Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then ||v + w ||2 = ||v ||2 + ||w ||2 P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. p Propn: ||x|| := hx, xi defines a norm on V . Thus every inner product space is a normed space. For the induced norm || · ||: I I (Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then ||v + w ||2 = ||v ||2 + ||w ||2 (Parallelogram Law) If v , w ∈ V , then ||v + w ||2 + ||v − w ||2 = 2||v ||2 + 2||w ||2 P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Topological Vector Spaces III Let (V , h·, ·i) be an inner product space. p p Cauchy–Schwarz Inequality: |hx, y i| ≤ hx, xi hy , y i. Equality holds iff y is a scalar multiple of x. p Propn: ||x|| := hx, xi defines a norm on V . Thus every inner product space is a normed space. For the induced norm || · ||: I I (Pythagoras’ Law) If v , w ∈ V , with v ⊥ w , then ||v + w ||2 = ||v ||2 + ||w ||2 (Parallelogram Law) If v , w ∈ V , then ||v + w ||2 + ||v − w ||2 = 2||v ||2 + 2||w ||2 An inner product space is called a Hilbert space it is complete. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 4 / 11 Geometry in Hilbert Space P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables I November 2010 5 / 11 Geometry in Hilbert Space I In Rn , the angle θ between two vectors x, y is given by cos θ = P. Ouwehand (Stellenbosch Univ.) x·y |x| |y| Spaces of Random Variables November 2010 5 / 11 Geometry in Hilbert Space I In Rn , the angle θ between two vectors x, y is given by cos θ = x·y |x| |y| In an inner product space V , we therefore define the “angle” between x, y ∈ V by cos θ := P. Ouwehand (Stellenbosch Univ.) hx, y i ||x|| ||y || where ||x|| := Spaces of Random Variables p hx, xi November 2010 5 / 11 Geometry in Hilbert Space I In Rn , the angle θ between two vectors x, y is given by cos θ = x·y |x| |y| In an inner product space V , we therefore define the “angle” between x, y ∈ V by cos θ := hx, y i ||x|| ||y || where ||x|| := p hx, xi (By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that this definition makes sense. It also follows that | cos θ| = 1 if and only if x is a scalar multiple of y , i.e. iff x, y are parallel.) P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 5 / 11 Geometry in Hilbert Space I In Rn , the angle θ between two vectors x, y is given by cos θ = x·y |x| |y| In an inner product space V , we therefore define the “angle” between x, y ∈ V by cos θ := hx, y i ||x|| ||y || where ||x|| := p hx, xi (By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that this definition makes sense. It also follows that | cos θ| = 1 if and only if x is a scalar multiple of y , i.e. iff x, y are parallel.) We say that x, y ∈ V are orthogonal, and write x ⊥ y , if and only if hx, y i = 0. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 5 / 11 Geometry in Hilbert Space I In Rn , the angle θ between two vectors x, y is given by cos θ = x·y |x| |y| In an inner product space V , we therefore define the “angle” between x, y ∈ V by cos θ := hx, y i ||x|| ||y || where ||x|| := p hx, xi (By the Cauchy–Schwarz inequality it follows that | cos θ| ≤ 1, so that this definition makes sense. It also follows that | cos θ| = 1 if and only if x is a scalar multiple of y , i.e. iff x, y are parallel.) We say that x, y ∈ V are orthogonal, and write x ⊥ y , if and only if hx, y i = 0. If G ⊆ V , we say that x ⊥ G iff ∀g ∈ G (x ⊥ g ). P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 5 / 11 Geometry in Hilbert Space P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables II November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . Think of x|| as the best approximation to x in W : It is the vector in W which lies closest to x. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . Think of x|| as the best approximation to x in W : It is the vector in W which lies closest to x. Suppose that V is a Hilbert space, and that W is a linear subspace of V . For v0 ∈ V , we would like to find the best approximation of v0 in W , i.e. the unique vector w0 such that P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . Think of x|| as the best approximation to x in W : It is the vector in W which lies closest to x. Suppose that V is a Hilbert space, and that W is a linear subspace of V . For v0 ∈ V , we would like to find the best approximation of v0 in W , i.e. the unique vector w0 such that I w0 ∈ W , and P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . Think of x|| as the best approximation to x in W : It is the vector in W which lies closest to x. Suppose that V is a Hilbert space, and that W is a linear subspace of V . For v0 ∈ V , we would like to find the best approximation of v0 in W , i.e. the unique vector w0 such that I I w0 ∈ W , and ||v0 − w0 || = inf{||v0 − w || : w ∈ W }, i.e. w0 is the vector in W that lies closest to v0 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space II If W is a linear subspace of Rn , then we can project any x ∈ Rn onto W: x = x|| + x⊥ where x|| ∈ W , x⊥ ⊥ W We call x|| the orthogonal projection of x onto W . Think of x|| as the best approximation to x in W : It is the vector in W which lies closest to x. Suppose that V is a Hilbert space, and that W is a linear subspace of V . For v0 ∈ V , we would like to find the best approximation of v0 in W , i.e. the unique vector w0 such that I I I w0 ∈ W , and ||v0 − w0 || = inf{||v0 − w || : w ∈ W }, i.e. w0 is the vector in W that lies closest to v0 . Moreover, (v0 − w0 ) ⊥ W . The vector w0 is called the orthogonal projection of v0 onto W . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 6 / 11 Geometry in Hilbert Space III Proposition: Let V be a Hilbert space, and let W be a closed linear subspace of V . Then any v0 in V has a unique decomposition || v0 = v0 + v0⊥ || where v0 ∈ W , v0⊥ ⊥ W || v0 is called the orthogonal projection of v0 onto W . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 7 / 11 The Banach Space L1 (Ω, F, P) P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables I November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. For such X , define Z ||X ||1 := P. Ouwehand (Stellenbosch Univ.) |X | dP < ∞ = E|X | Spaces of Random Variables November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. For such X , define Z ||X ||1 := |X | dP < ∞ = E|X | Proposition: || · ||1 is almost a norm on L1 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. For such X , define Z ||X ||1 := |X | dP < ∞ = E|X | Proposition: || · ||1 is almost a norm on L1 . Problem: ||X ||1 = 0 does not imply that X = 0, but merely that X = 0 P–a.s. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. For such X , define Z ||X ||1 := |X | dP < ∞ = E|X | Proposition: || · ||1 is almost a norm on L1 . Problem: ||X ||1 = 0 does not imply that X = 0, but merely that X = 0 P–a.s. Solution: Form the (quotient) space L1 (Ω, F, P) by regarding as equal any two RV’s which are equal P–a.s. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 8 / 11 The Banach Space L1 (Ω, F, P) I Let L1 (Ω, F, P) be the set of all integrable random variables X . This is a vector space. For such X , define Z ||X ||1 := |X | dP < ∞ = E|X | Proposition: || · ||1 is almost a norm on L1 . Problem: ||X ||1 = 0 does not imply that X = 0, but merely that X = 0 P–a.s. Solution: Form the (quotient) space L1 (Ω, F, P) by regarding as equal any two RV’s which are equal P–a.s. Theorem: (Riesz–Fischer) L1 (Ω, F, P) is a Banach Space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 8 / 11 The Hilbert Space L2 (Ω, F, P) P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables I November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY Proposition: h·, ·i is almost an inner product on L2 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY Proposition: h·, ·i is almost an inner product on L2 . Problem: hX , X i = 0 does not imply that X = 0, but merely that X = 0 P–a.s. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY Proposition: h·, ·i is almost an inner product on L2 . Problem: hX , X i = 0 does not imply that X = 0, but merely that X = 0 P–a.s. Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as equal any two RV’s which are equal P–a.s. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY Proposition: h·, ·i is almost an inner product on L2 . Problem: hX , X i = 0 does not imply that X = 0, but merely that X = 0 P–a.s. Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as equal any two RV’s which are equal P–a.s. Theorem: (Riesz–Fischer) L2 (Ω, F, P) is a Hilbert Space. P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 The Hilbert Space L2 (Ω, F, P) I Let L2 (Ω, F, P) be the set of all square–integrable random variables X (i.e for which X 2 is integrable). This is a vector space. For such X , Y , define Z hX , Y i := XY dP = EXY Proposition: h·, ·i is almost an inner product on L2 . Problem: hX , X i = 0 does not imply that X = 0, but merely that X = 0 P–a.s. Solution: Form the (quotient) space L2 (Ω, F, P) by regarding as equal any two RV’s which are equal P–a.s. Theorem: (Riesz–Fischer) L2 (Ω, F, P) is a Hilbert Space. The induced norm on L2 is defined by p 1 ||X ||2 := hX , X i = E[X 2 ] 2 P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 9 / 11 Statistics and Geometry P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables I November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . Now if X ∈ L2 , then √ p p E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2 i.e. ||X ||1 ≤ ||X ||2 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . Now if X ∈ L2 , then √ p p E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2 i.e. ||X ||1 ≤ ||X ||2 . Hence for probability spaces L2 ⊆ L1 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . Now if X ∈ L2 , then √ p p E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2 i.e. ||X ||1 ≤ ||X ||2 . Hence for probability spaces L2 ⊆ L1 . In statistics the variance Var(X ) and standard deviation σX of a random variable X are defined by p Var(X ) := E(X − E[X ])2 σX := Var(X ) P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . Now if X ∈ L2 , then √ p p E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2 i.e. ||X ||1 ≤ ||X ||2 . Hence for probability spaces L2 ⊆ L1 . In statistics the variance Var(X ) and standard deviation σX of a random variable X are defined by p Var(X ) := E(X − E[X ])2 σX := Var(X ) The covariance Cov(X , Y ) and correlation ρX ,Y of two random variables X , Y are defined by Cov(X , Y ) := E[(X − EX )(Y − EY )] P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables ρX ,Y := Cov(X , Y ) σX σY November 2010 10 / 11 Statistics and Geometry I Note that if (Ω, F, P) is a probability space, then X has a mean EX precisely if X ∈ L1 . Now if X ∈ L2 , then √ p p E|X | = h|X |, 1i ≤ h|X |, |X |i h1, 1i = EX 2 i.e. ||X ||1 ≤ ||X ||2 . Hence for probability spaces L2 ⊆ L1 . In statistics the variance Var(X ) and standard deviation σX of a random variable X are defined by p Var(X ) := E(X − E[X ])2 σX := Var(X ) The covariance Cov(X , Y ) and correlation ρX ,Y of two random variables X , Y are defined by Cov(X , Y ) := E[(X − EX )(Y − EY )] ρX ,Y := Cov(X , Y ) σX σY These quantities exist and are finite precisely for X , Y ∈ L2 . P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 10 / 11 Statistics and Geometry P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables II November 2010 11 / 11 Statistics and Geometry II Consider now the space L20 := {X ∈ L2 : EX = 0} P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 11 / 11 Statistics and Geometry II Consider now the space L20 := {X ∈ L2 : EX = 0} For X ∈ L20 , we have √ p p σX := Var(X ) = EX 2 = hX , X i = ||X ||2 P. Ouwehand (Stellenbosch Univ.) Spaces of Random Variables November 2010 11 / 11 Statistics and Geometry II Consider now the space L20 := {X ∈ L2 : EX = 0} For X ∈ L20 , we have √ p p σX := Var(X ) = EX 2 = hX , X i = ||X ||2 For X , Y ∈ L20 we have ρX ,Y := P. Ouwehand (Stellenbosch Univ.) Cov(X , Y ) EXY hX , Y i = = = cos θ σX σY σX σY ||X ||2 ||Y |||2 Spaces of Random Variables November 2010 11 / 11