Download COMP 170 L2

COMP 170 L2
L18: Random Variables: Independence and Variance
Page 1
COMP 170 L2
Page 2
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Page 3
Distribution Functions of RVs

P(X=k) viewed as a function of k: The distribution function of X.

In general
COMP 170 L2
Probability Weight and Distribution Function
Page 4
COMP 170 L2
Distribution Functions of RVs
Page 5
COMP 170 L2
Distribution Functions of RVs

For coin example

D(1, 9) = P(1 <= X <=9 ) ~= 1
Page 6
COMP 170 L2
Page 7
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Independence Between Events

E is independent of F
If P(E|F) = P(E)

The information that “Event F occurred” does not change the
probability of E
COMP 170 L2
Page 9
Random Variable and Event

Given a rand variable, we can define many difference events
COMP 170 L2
Page 10
Independence between RVs
COMP 170 L2
Independence between RVs
Page 11
COMP 170 L2
Page 12
Independence between RVs

No need to further check

P(X=0, Y=1) ?= P(X=0)P(Y=1)

P(X=1, Y=0) ? = P(X=1)P(Y=0)

P(X=1, Y=1) ?= P(X=1)P(Y=1)
COMP 170 L2
Page 13
Independence between RVs

Draw two cards from a deck of 52 cards

X: number on first card

Y: number on second card

X and Y are independent when drawing with replacement

X and Y are not independent when drawing without replacement
COMP 170 L2
Page 14
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Page 15
Independence and Expectation
COMP 170 L2
Page 16
Independence and Expectation
COMP 170 L2
Page 17
COMP 170 L2
Page 18
COMP 170 L2
Page 19
Illustrating Proof via Example
COMP 170 L2
Page 20
Illustrating Proof via Example
COMP 170 L2
Page 21
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Sum of independent RVs

The Trend

Central limit theorem
COMP 170 L2
Page 22
Variance of RVs

Probability starts with a process whose outcome is uncertain

Sample space: the set of all possible outcomes

A random variable (RV) is a function defined on the sample space

Different runs of the process might yield different outcomes

The RV might take different values in different runs


In other words, the value of RV varies across different runs
Some RVs vary more and some vary less

Number of heads in 1 coin flip

Number of heads in 10 coin flips

Number of heads in 100 coin flips
COMP 170 L2
Page 23
Variance of RVs

Variance of an RV X

Measures how much it varies (across different runs of process)

Relative to the mean value
COMP 170 L2
Page 24
COMP 170 L2
Page 25
COMP 170 L2
Page 26
Which RV vary the most?
Flip fair coin

Variance

Number of heads in 1 flip
1/4

Number of heads in 10 flips
10/4

Number of heads in 100 flips
100/4
COMP 170 L2
Page 27
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Page 28
Calculating Variance
COMP 170 L2
Page 29
Example 1: We have already seen
Number of heads in n flips


Variance

n=1
1/4

n=10
10/4 = 10 * 1/4
(x1+X2+…+X10)

n=100
(X1+X2+…+X100)

Additivity is true here.
100/4 = 100 * 1/4
COMP 170 L2

Additivity is true here.
Page 30
COMP 170 L2
Page 31
Example 3 (Counter Example)
COMP 170 L2
Page 32
Example 3 (Counter Example)
COMP 170 L2
Page 33
COMP 170 L2
Page 34
Two Lemmas
COMP 170 L2
Page 35
COMP 170 L2
Page 36
COMP 170 L2
Page 37
An Application of Theorem 5.29
COMP 170 L2
Page 38
A Corollary
COMP 170 L2
Page 39
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Page 40
Standard Deviation

Standard deviation is another measure of how much a rv varies or
how much a distribution spread out

It is the square root of variance.

Next page

Shows several distributions, variances and standard deviations

Highlights the differences between variance and standard
deviation
COMP 170 L2
Distributions, Variances, and Standard Deviations
Page 41
COMP 170 L2
Page 42
Distributions, Variances, and Standard Deviations

The examples on the previous page show


Standard deviation is a natural measure of “spread” of distribution
Variance is easier to manipulate mathematically. Will see this in
further study of probability theory and statistics.
COMP 170 L2
Page 43
Outline



Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs
Variance of RV

Definition and Examples

Additivity

Standard deviation
Central limit theorem
COMP 170 L2
Page 44
A Pattern

If we flip of a coin a large number of times,


“The number of heads” has bell-shaped distribution.
This phenomenon is not unique to coin flipping
COMP 170 L2
Test with .8 probability of getting correct answer
Page 45
COMP 170 L2
Page 46
COMP 170 L2
Page 47
COMP 170 L2
Page 48
Normal Distribution
COMP 170 L2
Page 49
COMP 170 L2
Page 50
COMP 170 L2
Recap: 13-05-2010
COMP 170 L2
Recap: 13-05-2010

E is independent of F
If P(E|F) = P(E)

The information that “Event F occurred” does not change the
probability of E
COMP 170 L2
Recap: 13-05-2010
Flip fair coin

Number of heads in 1 flip: Variance = 1/4

Number of heads in 10 flips: Variance = 10/4

Number of heads in 100 flips: Variance = 100/4
COMP 170 L2
Page 54
Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until
you see a 1?

P(getting 1 at each throw) = 1/6

Answer: 1/(1/6) = 6
COMP 170 L2
Page 55
Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until
you see a 2 and 3 in that order?

Expected number of throws to see 2: 6

After that, expected number of throws to see 3: 6

Answer: 6+6 = 12
COMP 170 L2
Page 56
Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until
you see a 2 and 3 , where the order does not matter?

P( get 2 or 3 in one throw ) = 1/3

Expected number throws until you see one of 2 or 3: 3

After that, P( get the other number in each throw) = 1/6

Expected number of throws until you see the other number: 6

Answer: 3+6 = 9
COMP 170 L2
Old Exam Question 1
P( at least one copy of T1 in 20 weeks)
= 1 – P( no copy of T1 in 20 weeks)
= 1 – P ( no T1 in week1 AND no T1 in week 2 AND …)
= 1- P(no T1 in week1) P(no T1 in week 2) …
=
Page 57
COMP 170 L2
COMP 170 L2

Page 59
Can we do this?
P(all 10 types of toys in 20 weeks)
= P(copy of T1 in 20 weeks AND copy of T2 in 20 weeks AND …)
= P(copy of T1 in 20 weeks) P(copy of T2 in 20 weeks)…
NO, events not independent

Correct way
P(all 10 types of toys in 20 weeks) = 1 – P(A)
Use inclusion-exclusion to calculate P(A)
COMP 170 L2
COMP 170 L2
Old Exam Question 2
Page 61
COMP 170 L2
COMP 170 L2