Download Mean and Variance of Discrete Random Variables

Mean and Variance of Discrete Random Variables
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected Value of Discrete Random Variable
Suppose you and I play a betting game: we flip a coin and if it lands heads, I
give you a dollar, and if it lands tails, you give me a dollar.
On average, how much am I expected to win or lose?
expected winnings =
1
(−1)
2
| {z }
win -$1 half the time
Arthur Berg
+
1
(1)
2
| {z }
=0
win $1 half the time
Mean and Variance of Discrete Random Variables
2/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected Value of Discrete Random Variable
Suppose you and I play a betting game: we flip a coin and if it lands heads, I
give you a dollar, and if it lands tails, you give me a dollar.
On average, how much am I expected to win or lose?
expected winnings =
1
(−1)
2
| {z }
win -$1 half the time
Arthur Berg
+
1
(1)
2
| {z }
=0
win $1 half the time
Mean and Variance of Discrete Random Variables
2/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected Value of Discrete Random Variable
Suppose you and I play a betting game: we flip a coin and if it lands heads, I
give you a dollar, and if it lands tails, you give me a dollar.
On average, how much am I expected to win or lose?
expected winnings =
1
(−1)
2
| {z }
win -$1 half the time
Arthur Berg
+
1
(1)
2
| {z }
=0
win $1 half the time
Mean and Variance of Discrete Random Variables
2/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected Value of Discrete Random Variable
Suppose you and I play a betting game: we flip a coin and if it lands heads, I
give you a dollar, and if it lands tails, you give me a dollar.
On average, how much am I expected to win or lose?
expected winnings =
1
(−1)
2
| {z }
win -$1 half the time
Arthur Berg
+
1
(1)
2
| {z }
=0
win $1 half the time
Mean and Variance of Discrete Random Variables
2/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected Value of Discrete Random Variable
Suppose you and I play a betting game: we flip a coin and if it lands heads, I
give you a dollar, and if it lands tails, you give me a dollar.
On average, how much am I expected to win or lose?
expected winnings =
1
(−1)
2
| {z }
win -$1 half the time
Arthur Berg
+
1
(1)
2
| {z }
=0
win $1 half the time
Mean and Variance of Discrete Random Variables
2/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Definition of Expected Value of a Discrete Random Variable
Definition
The expected value of a discrete random variable X with probability
distribution p(x) is given by
X
E(X) , µ =
x pX (x)
(?)
x
where the sum is over all values of x for which pX (x) > 0.
Note that in order for (?) to exist, the sum must converge absolutely; that is
X
|x| pX (x) < ∞
(??)
x
If (??) does not hold, we say the expected value of X does not exist.
Arthur Berg
Mean and Variance of Discrete Random Variables
3/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Definition of Expected Value of a Discrete Random Variable
Definition
The expected value of a discrete random variable X with probability
distribution p(x) is given by
X
E(X) , µ =
x pX (x)
(?)
x
where the sum is over all values of x for which pX (x) > 0.
Note that in order for (?) to exist, the sum must converge absolutely; that is
X
|x| pX (x) < ∞
(??)
x
If (??) does not hold, we say the expected value of X does not exist.
Arthur Berg
Mean and Variance of Discrete Random Variables
3/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Definition of Expected Value of a Discrete Random Variable
Definition
The expected value of a discrete random variable X with probability
distribution p(x) is given by
X
E(X) , µ =
x pX (x)
(?)
x
where the sum is over all values of x for which pX (x) > 0.
Note that in order for (?) to exist, the sum must converge absolutely; that is
X
|x| pX (x) < ∞
(??)
x
If (??) does not hold, we say the expected value of X does not exist.
Arthur Berg
Mean and Variance of Discrete Random Variables
3/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected value of functions of random variables
The expected value of g(X) where g is any real-valued function is naturally
X
E(g(X)) =
g(x)p(x)
x
Example
Consider the Bernoulli random variable
(
0, w.p. 1/2
X=
1, w.p. 1/2
Compute E X 2 − 1 .
1
1
−1
2
E X − 1 = (0 − 1)
+ ((1) − 1)
=
2
2
2
2
Arthur Berg
2
Mean and Variance of Discrete Random Variables
4/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Expected value of functions of random variables
The expected value of g(X) where g is any real-valued function is naturally
X
E(g(X)) =
g(x)p(x)
x
Example
Consider the Bernoulli random variable
(
0, w.p. 1/2
X=
1, w.p. 1/2
Compute E X 2 − 1 .
1
1
−1
2
E X − 1 = (0 − 1)
+ ((1) − 1)
=
2
2
2
2
Arthur Berg
2
Mean and Variance of Discrete Random Variables
4/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Birthday Problem Revisited
65 people participated in the birthday game a few weeks back.
I claimed that if no two birthdays matched, then I would pay everyone 30
monopoly dollars, but otherwise each person would pay me one monopoly
dollar.
Therefore either I would lose 65*30=1950 monopoly dollars, or I would win 65
monopoly dollars.
My expected earnings should be somewhere between -1950 and +65. Let’s
compute it.
My total earnings are represented by the following random variable
(
30,
w.p. p
X=
−1950, w.p. 1 − p
where
p≈
365(364)(363) · · · (301)
≈ .9977
36565
Therefore
E(X) = 30(.9977) − 1950(.0023) = 25.446
Arthur Berg
Mean and Variance of Discrete Random Variables
5/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Variance
Definition (variance)
The variance of a random variable X with expected value µ is given by
var(X) , σ 2 = E (X−µ )2
Definition
The standard deviation of a random variable X is, σ, the square root of the
variance, i.e.
q
p
sd(X) , σ = E [(X − µ)2 ] = var(X)
Arthur Berg
Mean and Variance of Discrete Random Variables
6/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Variance
Definition (variance)
The variance of a random variable X with expected value µ is given by
var(X) , σ 2 = E (X−µ )2
Definition
The standard deviation of a random variable X is, σ, the square root of the
variance, i.e.
q
p
sd(X) , σ = E [(X − µ)2 ] = var(X)
Arthur Berg
Mean and Variance of Discrete Random Variables
6/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Some Distributions
(
(
(
−1, w.p. 1/2
−2, w.p. 1/2
−3, w.p. 1/2
X1 =
X2 =
X3 =
1,
w.p. 1/2
2,
w.p. 1/2
3,
w.p. 1/2
Hence E(X1 ) = E(X2 ) = E(X3 ) = 0 and
1
1
var(X1 ) = (1 − 0)2
+ (−1 − 0)2
=1
2
2
1
1
var(X2 ) = (2 − 0)2
+ (−2 − 0)2
=4
2
2
2 1
2 1
var(X3 ) = (3 − 0)
+ (−3 − 0)
=9
2
2
Therefore sd(X1 ) = 1, sd(X2 ) = 2, and sd(X3 ) = 3.
Arthur Berg
Mean and Variance of Discrete Random Variables
7/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Some Distributions
(
(
(
−1, w.p. 1/2
−2, w.p. 1/2
−3, w.p. 1/2
X1 =
X2 =
X3 =
1,
w.p. 1/2
2,
w.p. 1/2
3,
w.p. 1/2
Hence E(X1 ) = E(X2 ) = E(X3 ) = 0 and
1
1
var(X1 ) = (1 − 0)2
+ (−1 − 0)2
=1
2
2
1
1
var(X2 ) = (2 − 0)2
+ (−2 − 0)2
=4
2
2
2 1
2 1
var(X3 ) = (3 − 0)
+ (−3 − 0)
=9
2
2
Therefore sd(X1 ) = 1, sd(X2 ) = 2, and sd(X3 ) = 3.
Arthur Berg
Mean and Variance of Discrete Random Variables
7/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Some Distributions
(
(
(
−1, w.p. 1/2
−2, w.p. 1/2
−3, w.p. 1/2
X1 =
X2 =
X3 =
1,
w.p. 1/2
2,
w.p. 1/2
3,
w.p. 1/2
Hence E(X1 ) = E(X2 ) = E(X3 ) = 0 and
1
1
var(X1 ) = (1 − 0)2
+ (−1 − 0)2
=1
2
2
1
1
var(X2 ) = (2 − 0)2
+ (−2 − 0)2
=4
2
2
2 1
2 1
var(X3 ) = (3 − 0)
+ (−3 − 0)
=9
2
2
Therefore sd(X1 ) = 1, sd(X2 ) = 2, and sd(X3 ) = 3.
Arthur Berg
Mean and Variance of Discrete Random Variables
7/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Theorem 4.2
Theorem (mean and variance of aX + b)
For any random variable X (discrete or not) and constants a and b,
1
E(aX + b) = a E(X) + b
2
var(aX + b) = a2 var(X)
It follows that if X has mean µ and standard deviation σ, then
Y=
x−µ
σ
has mean 0 and standard deviation 1. This is the standardized form of X.
Theorem
If X and Y are random variables, then E(X + Y) = E(X) + E(Y).
Arthur Berg
Mean and Variance of Discrete Random Variables
8/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Theorem 4.2
Theorem (mean and variance of aX + b)
For any random variable X (discrete or not) and constants a and b,
1
E(aX + b) = a E(X) + b
2
var(aX + b) = a2 var(X)
It follows that if X has mean µ and standard deviation σ, then
Y=
x−µ
σ
has mean 0 and standard deviation 1. This is the standardized form of X.
Theorem
If X and Y are random variables, then E(X + Y) = E(X) + E(Y).
Arthur Berg
Mean and Variance of Discrete Random Variables
8/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Theorem 4.2
Theorem (mean and variance of aX + b)
For any random variable X (discrete or not) and constants a and b,
1
E(aX + b) = a E(X) + b
2
var(aX + b) = a2 var(X)
It follows that if X has mean µ and standard deviation σ, then
Y=
x−µ
σ
has mean 0 and standard deviation 1. This is the standardized form of X.
Theorem
If X and Y are random variables, then E(X + Y) = E(X) + E(Y).
Arthur Berg
Mean and Variance of Discrete Random Variables
8/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Theorem 4.3
Theorem (var(X) = E(X 2 ) − µ2 )
If X is a random variable with mean µ, then
var(X) = E(X 2 ) − µ2
Proof.
var(X) = E (X − µ)2
= E(X 2 − 2Xµ + µ2 )
= E(X 2 ) − E(2Xµ) + E(µ2 )
= E(X 2 ) − 2µ2 + µ2
= E(X 2 ) − µ2
Arthur Berg
Mean and Variance of Discrete Random Variables
9/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Theorem 4.3
Theorem (var(X) = E(X 2 ) − µ2 )
If X is a random variable with mean µ, then
var(X) = E(X 2 ) − µ2
Proof.
var(X) = E (X − µ)2
= E(X 2 − 2Xµ + µ2 )
= E(X 2 ) − E(2Xµ) + E(µ2 )
= E(X 2 ) − 2µ2 + µ2
= E(X 2 ) − µ2
Arthur Berg
Mean and Variance of Discrete Random Variables
9/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Chebyshev’s inequality
There are many random variables X with a given mean µ and a given variance
σ 2 , but they all must satisfy the following inequality.
Theorem
Chebyshev’s inequality Let X be a random variable with mean µ and variance
σ 2 . Then for any positive k,
P(|X − µ| < kσ) ≥ 1 −
1
k2
Letting k = 2 in Chebyshev’s inequality gives
P(µ − 2σ < X < µ + 2σ) ≥ 1 −
1
3
=
4
4
That is, the interval from µ − 2σ to µ + 2σ must contain at least 3/4 of the
probability mass.
Arthur Berg
Mean and Variance of Discrete Random Variables
10/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Chebyshev’s inequality
There are many random variables X with a given mean µ and a given variance
σ 2 , but they all must satisfy the following inequality.
Theorem
Chebyshev’s inequality Let X be a random variable with mean µ and variance
σ 2 . Then for any positive k,
P(|X − µ| < kσ) ≥ 1 −
1
k2
Letting k = 2 in Chebyshev’s inequality gives
P(µ − 2σ < X < µ + 2σ) ≥ 1 −
1
3
=
4
4
That is, the interval from µ − 2σ to µ + 2σ must contain at least 3/4 of the
probability mass.
Arthur Berg
Mean and Variance of Discrete Random Variables
10/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Chebyshev’s inequality
There are many random variables X with a given mean µ and a given variance
σ 2 , but they all must satisfy the following inequality.
Theorem
Chebyshev’s inequality Let X be a random variable with mean µ and variance
σ 2 . Then for any positive k,
P(|X − µ| < kσ) ≥ 1 −
1
k2
Letting k = 2 in Chebyshev’s inequality gives
P(µ − 2σ < X < µ + 2σ) ≥ 1 −
1
3
=
4
4
That is, the interval from µ − 2σ to µ + 2σ must contain at least 3/4 of the
probability mass.
Arthur Berg
Mean and Variance of Discrete Random Variables
10/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Exercise 4.17
Arthur Berg
Mean and Variance of Discrete Random Variables
11/ 12
Expected Value
Variance and Standard Deviation
Practice Exercises
Exercise 4.37
Arthur Berg
Mean and Variance of Discrete Random Variables
12/ 12