Download Random Variables

Random Variables
Supplementary Notes
Prepared by Raymond Wong
Presented by Raymond Wong
1
e.g.1 (Page 3)


Suppose that we flip the coin 5 times.
The following shows the sample space
of flipping the coin 5 times.
X(HHHHH) = 5 X(HTHHT) = 3
HTHHT
HHHHH
…
Sample Space
THTHT
TTTTT
…
X(THTHT) = 2
X(TTTTT) = 0
…
….
Suppose that we are interested in the total number of heads when
we flip the coin 5 times.
We define a random variable X to denote the total number of heads when
we flip the coin 5 times.
2
e.g.2 (Page 6)


Consider the example of flipping the coin 1
time.
Let X be the random variable where
X=


1 if the flip is successful (i.e., showing a head)
0 if the flip is unsuccessful (i.e., not showing a head)
X is called a Bernoulli random variable.
The flip is called a Bernoulli trial.
3
e.g.3 (Page 8)


Consider the example of flipping the coin 5 times.
Let Xi be the random variable where
Xi =





1 if the i-th flip is successful (i.e., showing a head)
0 if the i-th flip is unsuccessful (i.e., not showing a head)
Xi is called a Bernoulli random variable.
Each flip is called a Bernoulli trial.
Flipping it 5 times is a Bernoulli trial process.
Suppose that we are interested in the number of
heads.
We have X1 + X2 + X3 + X4 + X5.
(i.e., the sum of Bernoulli Random Variables)
4
e.g.4 (Page 9)



We have 5 Bernoulli trials
with probability p success on each trial
Let S denote success and F denote failure.
What is the probability of the following?
 (a) SSSFF
 (b) FFSSS
 (c) SFSFS
 (d) any particular ordering on three S’s and any particular ordering
on two F’s (e.g., FSFSS)
(a) Since each trial is independent, we have
P(SSSFF) = P(S) x P(S) x P(S) x P(F) x P(F)
= p x p x p x (1-p) x (1-p)
= p3(1-p)2
5
e.g.4



We have 5 Bernoulli trials
with probability p success on each trial
Let S denote success and F denote failure.
What is the probability of the following?
 (a) SSSFF
 (b) FFSSS
 (c) SFSFS
 (d) any particular ordering on three S’s and any particular ordering
on two F’s (e.g., FSFSS)
(b) Since each trial is independent, we have
P(FFSSS) = P(F) x P(F) x P(S) x P(S) x P(S)
= (1-p) x (1-p) x p x p x p
= p3(1-p)2
6
e.g.4



We have 5 Bernoulli trials
with probability p success on each trial
Let S denote success and F denote failure.
What is the probability of the following?
 (a) SSSFF
 (b) FFSSS
 (c) SFSFS
 (d) any particular ordering on three S’s and any particular ordering
on two F’s (e.g., FSFSS)
(c) Since each trial is independent, we have
P(SFSFS) = P(S) x P(F) x P(S) x P(F) x P(S)
= p x (1-p) x p x (1-p) x p
= p3(1-p)2
7
P(any particular ordering on three S’s and any particular ordering on two F’s) = p3(1-p)2
e.g.4



We have 5 Bernoulli trials
with probability p success on each trial
Let S denote success and F denote failure.
What is the probability of the following?
 (a) SSSFF
 (b) FFSSS
 (c) SFSFS
 (d) any particular ordering on three S’s and any particular ordering
on two F’s (e.g., FSFSS)
(d) Since each trial is independent, we have
P(any particular ordering on three S’s and any particular ordering on two F’s)
= P(S) x P(S) x P(S) x P(F) x P(F)
= p x p x p x (1-p) x (1-p)
= p3(1-p)2
8
P(any particular ordering on three S’s and any particular ordering on two F’s) = p3(1-p)2
e.g.5 (Page 10)



We have 5 Bernoulli trials
with probability p success on each trial
Let S denote success and F denote failure.
What is the probability that the 5 trials
contain exactly 3 successes?
Is it equal to p3(1-p)2?
No.
The total number of trials containing 3 successes and 2 failures =
P(5 trails contain exactly 3 successes)
= P(SSSFF) + P(SSFSF) + …+ P(FFSSS)
= p3(1-p)2 + p3(1-p)2 + …+ p3(1-p)2
= 5 p3(1-p)2
3
5
3
9
e.g.6 (Page 12)

The sample space for the Binomial
Random Variable X is:
n p0(1-p)n-0
0
n p1(1-p)n-1
1
n p2(1-p)n-2
2
X=0
X=1
X=2
…
…
…
…
X=n
Sample Space
n pn(1-p)n-n
n
10
e.g.7 (Page 12)
The binomial theorem is
n
 n  k nk
n
( x  y )     x y
k 0  k 
 If x = p and y = 1-p, we have
n
n k
n
( p  [1  p])     p (1  p) nk
k 0  k 
n
n k
n
1     p (1  p) nk
k 0  k 
n
n k
nk


p
(
1

p
)
1

k 
k 0  

11
10 questions
P(he answers correctly) = 0.8
P(he answers incorrectly) = 0.2
(a) What is P(answer exactly 8 questions correctly)?
(b) What is P(answer exactly 9 questions correctly)?
(c) What is P(answer exactly 10 questions correctly)?
e.g.8 (Page 13)






There are 10 questions in a test.
A student takes this test.
Suppose that he who knows 80% of the course material
has probability 0.8 of success on any question,
independent of how he did on another question.
(a) What is the probability that he answers exactly 8
questions correctly?
(b) What is the probability that he answers exactly 9
questions correctly?
(c) What is the probability that he answers exactly 10
questions correctly?
12
10 questions
P(he answers correctly) = 0.8
P(he answers incorrectly) = 0.2
(a) What is P(answer exactly 8 questions correctly)?
(b) What is P(answer exactly 9 questions correctly)?
(c) What is P(answer exactly 10 questions correctly)?
e.g.8
This example is similar to the Bernoulli trial process.
A trial in this example is answering a question.
A success in this example is that he answers the question correctly.
A failure in this example is that he answers the question incorrectly.
Thus, we can use the formula of the Bernoulli trail process
(or Binomial Random Variable X)
10 0.8kx0.210-k
if 0  k  10
k
P(X = k) =
otherwise
0
(a)
(b)
(c)
Let X be the total number of questions answered correctly.
10 0.88x0.210-8 = 10 0.88x0.22 =0.302
P(X = 8) =
8
8
10 0.89x0.210-9 = 10 0.89x0.21 =0.268
P(X = 9) =
9
9
10 0.810x0.210-10 = 10 0.810x0.20=0.107
P(X = 10)=
10
10
13
e.g.9 (Page 15)


Suppose that we flip a fair coin TWICE.
The sample space is
0 head
1 head
TT
HT
TH
HH
Sample Space
1 head
2 heads
14
e.g.10 (Page 15)


Suppose that we flip a fair coin THREE
times.
The sample space is
1 head
0 head
1 head
1 head
TTT
TTH
THT
HTT
THH
HTH
HHT
HHH
Sample Space
2 heads
2 heads
2 heads
3 heads
15
e.g.11 (Page 16)

Illustration 1 for Page 16
Step 1
$ ???
You
I
Step 2
Flip 3 coins
You
I
Step 3
$2
I
You
The outcome is HHT
16
e.g.12 (Page 16)

Illustration 2 for Page 16
Step 1
$ ???
You
I
Step 2
Flip 3 coins
You
I
Step 3
$0
I
You
The outcome is TTT
17
e.g.13 (Page 17)
Let X be a random variable denoting a number equal to 0, 1, 2, or 3.
The sample space where we consider random variable X is
P(X = xi)
1/4
Sample Space
What is E(X)?
X=0
xi
1/4
X=1
1/4
X=2
1/4
X=3
E(X) = 0 x ¼ + 1 x ¼ + 2 x ¼ + 3 x ¼
= 3/2
18
e.g.14 (Page 17)


Suppose that we flip a fair coin THREE times.
The sample space where we flip a fair coin THREE times is
2 tails
3 tails
1/8
TTT
2 tails
2 tails
1/8
TTH
1/8
THT
1/8
HTT
HTH
HHT
1/8
HHH
1/8
Sample Space
THH
1/8
1 tail
1/8
1 tail
1 tail
0 tail
Let X be the random variable denoting the number of tails.
What is E(X)?
E(X) = 3x1/8 + 2x1/8 + 2x1/8 + 2x1/8
+ 1x1/8 + 1x1/8 + 1x1/8 + 0x1/8
= 1.5
19
e.g.15 (Page 17)
 Suppose that we flip a biased coin THREE times
where P(tail) = 2/3 and P(head) = 1/3
 The sample space where we flip a biased coin THREE times is
2 tails
3 tails
8/27
TTT
2 tails
2 tails
4/27
TTH
4/27
THT
4/27
HTT
HTH
HHT
2/27
HHH
1/27
Sample Space
THH
2/27
1 tail
2/27
1 tail
1 tail
0 tail
Let X be the random variable denoting the number of tails.
What is E(X)?
E(X) = 3x8/27 + 2x4/27 + 2x4/27 + 2x4/27
+ 1x2/27 + 1x2/27 + 1x2/27 + 0x1/27
=2
20
e.g.16 (Page 17)
 Suppose that we flip a biased coin THREE times
where P(tail) = 2/3 and P(head) = 1/3
 Let X be the random variable denoting the number of tails.
 The sample space where we consider random variable X is
3 (2/3)0(1/3)3-0
0
Sample Space
3 (2/3)1(1/3)3-1
1
X=0
X=1
3 (2/3)2(1/3)3-2
2
X=2
3 (2/3)3(1/3)3-3
3
X=3
What is E(X)?
E(X) = 0 x
=2
3 (2/3)0(1/3)3-0
3 (2/3)0(1/3)3-0
+ 1x
0
0
3 (2/3)0(1/3)3-0
3 (2/3)0(1/3)3-0
+ 3x
+ 2x
0
0
21
e.g.17 (Page 18)

Suppose that I want to throw one 6-sided dice.
Sample space
={
,
1/6
1 spot


,
1/6
2 spots
,
,
1/6
1/6
3 spots
4 spots
,
1/6
5 spots
}
1/6
6 spots
Let X be the number of spots shown.
What is E(X)?
E(X) = 1x1/6 + 2x1/6 + 3x1/6
+ 4x1/6 + 5x1/6 + 6x1/6
= 7/2
22
e.g.18 (Page 18)


Suppose that I want to throw two fair dice.
Let Y be the random variable denoting the number of spots shown.
Dice 1
Dice 2
1
1
1
Sum
Dice 1
Dice 2
2
3
1
2
3
3
1
3
4
1
4
1
Sum
Dice 1
Dice 2
Sum
4
5
1
6
2
5
5
2
7
3
3
6
5
3
8
5
3
4
7
5
4
9
5
6
3
5
8
5
5
10
1
6
7
3
6
9
5
6
11
2
1
3
4
1
5
6
1
7
2
2
4
4
2
6
6
2
8
2
3
5
4
3
7
6
3
9
2
4
6
4
4
8
6
4
10
2
5
7
4
5
9
6
5
11
2
6
8
4
6
10
6
6
12
i
P(Y=i)
2
1/36
3
2/36
4
3/36
5
4/36
6
5/36
7
6/36
8
5/36
9
4/36
10
3/36
11
2/36
12
1/36
23
e.g.19 (Page 18)


Suppose that I want to throw two fair dice.
Let Y be the random variable denoting the number of spots shown.
E(Y) = 2 x 2/36 + 3 x 2/36 + 4 x 3/36 + 5 x 4/36
+ 6 x 5/36 + 7 x 6/36 + 8 x 5/36 + 9 x4/36
+ 10 x 3/36 + 11 x 2/36 + 12 x 1/36
=7
i
P(Y=i)
2
1/36
3
2/36
4
3/36
5
4/36
6
5/36
7
6/36
8
5/36
9
4/36
10
3/36
11
2/36
12
1/36
24
e.g.20 (Page 20)


Suppose that we flip a fair coin THREE times.
The sample space where we flip a fair coin THREE times is
X(s)
3 tails
2 tails
2 tails
P(s) 1/8
1/8
TTH
TTT s
Sample Space
HTH
THH
1/8
1 tail
1/8
1 tail
2 tails
1/8
THT
1/8
HTT
HHT
1/8
HHH
1/8
1 tail
0 tail
Let X be the random variable denoting the number of tails.
What is E(X)?
E(X) = 0x1/8 + 1x1/8 + 1x1/8 + 1x1/8
+ 2x1/8 + 2x1/8 + 2x1/8 + 3x1/8
= 1.5
25
e.g.21 (Page 25)
IMPORTANT:
X and Y can be independent
X and Y can be dependent.
Theorem 5.10
Suppose X and Y are random variables on the (finite) sample space S. Then
E(X + Y) = E(X) + E(Y)
Let Z = X+Y.
Why is it correct?
That is, given an outcome s in S,
Z(s) = X(s) + Y(s)
According to Lemma 5.9,
Lemma 5.9
If a random variable X is defined on a (finite) sample space S, then its
expected value is given by
E(X) =

X(s) P(s)
s:sS
we have E(X+Y) = E(Z)

=
=
Z(s) P(s)
s:sS
s:sS
[X(s) + Y(s)] P(s)
 [X(s)P(s) + Y(s)P(s) ]
=  X(s)P(s) +  Y(s)P(s)
=
s:sS
s:sS
= E(X) + E(Y)
s:sS
26
X=
1 if head
0 if tail
Y=
0 if head
1 if tail
e.g.22 (Page 26)


Suppose that we flip a fair coin.
We have two random variables X and Y.
X=




(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(X + Y)
(without using Theorem 5.10)?
(d) What is E(X+Y)
(by using Theorem 5.10)?
1 if head
0 if tail
Y=
0 if head
1 if tail
(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(X + Y)
(without using Theorem 5.10)?
(d) What is E(X+Y)
(by using Theorem 5.10)?
27
X=
1 if head
0 if tail
Y=
0 if head
1 if tail
e.g.22
(a)
E(X) = 1 x ½ + 0 x ½
=½
(b)
E(Y) = 0 x ½ + 1 x ½
=½
(c)
Consider two cases.
Case 1: head
(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(X + Y)
(without using Theorem 5.10)?
(d) What is E(X+Y)
(by using Theorem 5.10)?
X = 1 and Y = 0
X+Y = 1
Case 2: tail
X = 0 and Y = 1
X+Y = 1
E(X+Y) = 1 x ½ + 1 x ½
=1
(d) By using the theorem, we have
E(X + Y) = E(X) + E(Y)
=½+½
=1
28
X=
1 if head
0 if tail
Y=
0 if head
1 if tail
e.g.23 (Page 26)


Suppose that we flip a fair coin.
We have two random variables X and Y.
X=




(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(XY)?
(d) Is “E(XY) = E(X)E(Y)”?
1 if head
0 if tail
Y=
0 if head
1 if tail
(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(XY)?
(d) Is “E(XY) = E(X)E(Y)”?
29
X=
1 if head
0 if tail
Y=
0 if head
1 if tail
e.g.23
(a)
(b)
(c)
(a) What is E(X)?
(b) What is E(Y)?
(c) What is E(XY)?
(d) Is “E(XY) = E(X)E(Y)”?
E(X) = ½
E(Y) = ½
Consider two cases.
Case 1: head
X = 1 and Y = 0
XY = 0
Case 2: tail
X = 0 and Y = 1
XY = 0
E(XY) = 0 x ½ + 0 x ½
=0
(d)
Consider E(X)E(Y)
=½x½
=¼
We know that E(XY) = 0 (from part (c))
Thus, E(XY)  E(X)E(Y)
30
e.g.24 (Page 26)



In any cases (or in general),
E(X + Y) = E(X) + E(Y)
In some cases,
E(XY)  E(X)E(Y)
In some other cases,
E(XY) = E(X)E(Y)
31
e.g.24 (Page 27)


Illustration of Theorem 5.11
E.g. E(2X) = 2E(X)
The reason is
E(2X) = E(X + X)
= E(X) + E(X)
= 2E(X)
32
5 students (or n students)
e.g.25 (Page 36)




Consider Derangement Problem (or
Dearrangement Problem)
Suppose that there are 5 (or n) students.
They put their backpacks along the wall.
Someone mixed up the backpacks so students
get back “random” backpacks.
33
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
(a) Are E1 and E2 independent when n = 2?
(b) What is E(X) when n = 5?
e.g.25




Let X be the total number of students who get
their backpacks back correctly
Let Xi be an indicator random variable denoting
the event Ei that student i gets his backpack
correctly
(a) Are E1 and E2 independent when n =2?
(b) What is E(X) when n = 5?
34
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
(a) Are E1 and E2 independent when n = 2?
(b) What is E(X) when n = 5?
e.g.23
(a) Suppose that student 1 is “Raymond” and student 2 is “Peter”.
E1: the event that “Raymond” gets his backpack correctly.
E2: the event that “Peter” gets his backpack correctly.
There are only two cases.
Case 1:
Ray
Peter
Case 2:
Peter
Peter
Raymond
Raymond
P(E1) = P(“Raymond” gets his backpack correctly) = ½
P(E2) = P(“Peter” gets his backpack correctly)
=½
Ray
Peter
P(E1  E2) = P(“Raymond and “Peter” get their backpack correctly) = ½
Note that P(E1) x P(E2) = ½ x ½ = ¼
Thus, P(E1) x P(E2)  P(E1  E2)
Thus, E1 are E2 are not independent.
35
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
Note that events Ei (or the
correspondence random
(a) Are E1 and E2 independent when n = 2?
variables Xi) are not
(b) What is E(X) when n = 5?
independent.
(b) Note that X = X1 + X2 + X3 + X4 + X5
We can still use this linearity
By linearity of expectation,
of expectation.
E(X) = E(X1 + X2 + X3 + X4 + X5)
e.g.23
= E(X1) + E(X2) + E(X3) + E(X4) + E(X5)
The next question is : What is E(Xi)?
1 if student i takes his backpack correctly
Note that Xi =
0 if student i takes his backpack incorrectly
E(Xi) = 1 x P(student i takes his backpack correctly)
+ 0 x P(student i takes his backpack incorrectly)
= P(student i takes his backpack correctly)
36
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
(a) Are E1 and E2 independent when n = 2?
(b) What is E(X) when n = 5?
e.g.23
(b) Note that X = X1 + X2 + X3 + X4 + X5
By linearity of expectation,
E(X) = E(X1 + X2 + X3 + X4 + X5)
= E(X1) + E(X2) + E(X3) + E(X4) + E(X5)
The next question is : What is E(Xi)?
E(Xi)
= P(student i takes his backpack correctly)
37
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
(a) Are E1 and E2 independent when n = 2?
(b) What is E(X) when n = 5?
e.g.23
(b) Note that X = X1 + X2 + X3 + X4 + X5
By linearity of expectation,
E(X) = E(X1 + X2 + X3 + X4 + X5)
= E(X1) + E(X2) + E(X3) + E(X4) + E(X5)
The next question is : What is E(Xi)?
E(Xi) = P(student i takes his backpack correctly)
= (5-1)!/5!
= 4!/5!
= 1/5
There are (5-1)! cases that Raymond gets his OWN backpack back.
There are totally 5! cases
Raymond
Peter
38
5 students (or n students)
X: total number of students who get their backpacks back correctly
Xi be an indicator random variable denoting the event Ei
that student i gets his backpack correctly
(a) Are E1 and E2 independent when n = 2?
(b) What is E(X) when n = 5?
e.g.23
(b) Note that X = X1 + X2 + X3 + X4 + X5
By linearity of expectation,
E(X) = E(X1 + X2 + X3 + X4 + X5)
= E(X1) + E(X2) + E(X3) + E(X4) + E(X5)
The next question is : What is E(Xi)?
E(Xi) = P(student i takes his backpack correctly)
= (5-1)!/5!
= 4!/5!
= 1/5
Thus, E(X) = E(X1) + E(X2) + E(X3) + E(X4) + E(X5)
= 1/5 + 1/5 + 1/5 + 1/5 + 1/5
=1
Additional Question: If n can be any number, what is E(X)?
E(X) = 1
Note that it is independent of n. E.g., If n = 1000, we expect that there is
only one student who gets his backpack correctly.
39
e.g.26 (Page 40)


Suppose that we flip a coin.
The sample space of flipping a coin is
Event
H
T
P(H) = ½
P(T) = ½
Suppose that I flip a coin repeatedly.
We want to see a head.
Do you think that we “expect” to see a
head within TWO flips?
40
e.g.27
(Page 40)


Suppose that I throw two dice repeatedly.
We want to see the sum = 7.
Do you think that we “expect” to see “sum
= 7” within SIX times of throwing?
Suppose that we throw two dice.
The sample space of throwing two dice is
Dice 1
Dice 2
1
1
1
Sum
Dice 1
Dice 2
2
3
1
2
3
3
1
3
4
1
4
1
Sum
Dice 1
Dice 2
Sum
4
5
1
6
2
5
5
2
7
3
3
6
5
3
8
5
3
4
7
5
4
9
5
6
3
5
8
5
5
10
1
6
7
3
6
9
5
6
11
2
1
3
4
1
5
6
1
7
2
2
4
4
2
6
6
2
8
2
3
5
4
3
7
6
3
9
2
4
6
4
4
8
6
4
10
2
5
7
4
5
9
6
5
11
2
6
8
4
6
10
6
6
12
P(sum=7) = 6/36
= 1/6
41
e.g.28 (Page 43)





Suppose that the trial process is
“FFFS”
where F corresponds to a failure and S
corresponds to a success.
Let X be a random variable denoting the trial
number where the first success occurs.
Let p be the probability of success.
X(FFFS) = 4
(a) What is X(FFFS)?
P(FFFS) = (1-p)3p
(b) What is P(FFFS)?
42
e.g.29 (Page 44)


We know the following known fact.
Theorem 4.6: For any real number x  1,
(1)
n
i
(2)
xi
=
nxn+2 – (n+1)xn+1 + x
(1-x)2
For any real number -1 < x < 1,

x
.
i
i x =

(1-x)2
i 1
i 1

.
You don’t need to recite (2). If we have (1), we can derive (2)
Why?
This is because nxn is equal to 0 when n is very large.
If n is very large and -1 < x < 1, then what is the value of nxn?
1
. xn
Consider limnnxn
= limn
(ln x)(-1)
n
= limn
x-n
=0
(This is because
1
= limn -n
n = 0)
lim
x
n
x (ln x)(-1)
(By L’Hospital’s Rule)
43
e.g.29


We know the following known fact.
Theorem 4.6: For any real number x  1,
(1)
n
i
(2)
xi
=
nxn+2 – (n+1)xn+1 + x
(1-x)2
For any real number -1 < x < 1,

x
.
i
i x =

(1-x)2
i 1
i 1

.
You don’t need to recite (2). If we have (1), we can derive (2)
This is because nxn is equal to 0 when n is very large.
Consider nxn+2
= nxn.x2
= 0.x2 if n is very large
=0
Similarly, (n+1)xn+1 = 0 if n is very large
Thus, from Theorem 4.6
n
nxn+2 – (n+1)xn+1 + x
.
i
i x =

i 1
(1-x)2
If n is large, we have

x
.
i
i x =

(1-x)2
i 1
44