Download Continuous random variables

TDC 369 / TDC 432
April 2, 2003
Greg Brewster
Topics
• Math Review
• Probability
– Distributions
– Random Variables
– Expected Values
Math Review
• Simple integrals and differentials
• Sums
• Permutations
• Combinations
• Probability
Math Review: Sums
n ( n  1)
k

2
k 0
n
n 1
1

q
k
q


1 q
k 0
n

1
q 

1 q
k 0
k
( q  1)
(| q | 1)
Math Review:
Permutations
• Given N objects, there are N! = N(N-1)…1
different ways to arrange them
• Example: Given 3 balls, colored Red, White and
Blue, there are 3! = 6 ways to order them
– RWB, RBW, BWR, BRW, WBR, WRB
Math Review:
Combinations
• The number of ways to select K unique objects
from a set of N objects without replacement is
C(N,K) =
N 
N!
  
 K  K! ( N  K )!
• Example: Given 3 balls, RBW, there are C(3,2) =
3 ways to uniquely choose 2 balls
– RB, RW, BW
Probability
• Probability theory is concerned with the
likelihood of observable outcomes (“events”) of
some experiment.
• Let  be the set of all outcomes and let E   be
some event in , then the probability of E
occurring = Pr[E] is the fraction of times E will
occur if the experiment is repeated infinitely often.
Probability
• Example:
– Experiment = tossing a 6-sided die
– Observable outcomes = {1, 2, 3, 4, 5, 6}
– For fair die,
• Pr{die = 1} =
1
6
• Pr{die = 2} =
1
6
• Pr{die = 3} =
1
6
1
6
• Pr{die = 4} =
• Pr{die = 5} =
• Pr{die = 6} =
1
6
1
6
Probability Pie
Die=6
Die=5
Die=4
Die=1
Die=2
Die=3
Valid Probability Measure
• A probability measure, Pr, on an event space
{Ei} must satisfy the following:
– For all Ei , 0 <= Pr[Ei ] <= 1
– Each pair of events, Ei and Ek, are mutually exclusive,
that is,
Ei  Ek  , i  k
– All event probabilities sum to 1, that is,

 
Pr  Ek    Pr[ Ek ]  1
 k 1  k 1
Probability Mass Function
1
0.8
0.6
0.4
0.2
0
1
2
3
4
Pr(Die = x)
5
6
Mass Function = Histogram
• If you are starting with some repeatable events,
then the Probability Mass function is like a
histogram of outcomes for those events.
• The difference is a histogram indicates how
many times an event happened (out of some
total number of attempts), while a mass
function shows the fraction of time an event
happens (number of times / total attempts).
Dice Roll Histogram
1200 attempts
250
200
150
100
50
0
1
2
3
4
5
6
Number of times Die = x
Probability Distribution Function
(Cumulative Distribution Function)
1
0.8
0.6
0.4
0.2
0
1
2
3
4
5
Pr(Die <= x)
6
Combining Events
• Probability of event not happening:
–

Pr E  1  Pr[ E ]
• Probability of both E and F happening:
– IF events E and F are independent
•
PrE  F   Pr[ E] Pr[ F ]
• Probability of either E or F happening:
–
PrE  F   Pr[ E]  Pr[ F ]  Pr[ E  F ]
Conditional Probabilities
• The conditional probability that E occurs, given
that F occurs, written Pr[E | F], is defined as
Pr[ E  F ]
Pr[ E | F ] 
Pr[ F ]
Conditional Probabilities
• Example: The conditional probability that the
value of a die is 6, given that the value is greater
than 3, is Pr[die=6 | die>3] =
Pr[ die  6  die  3]
Pr[ die  6 | die  3] 
Pr[ die  3]
Pr[ die  6] 1 / 6


 1/ 3
Pr[ die  3] 1 / 2
Probability Pie
Die=6
Die=1
Die=5
Die=2
Die=4
Die=3
Conditional Probability Pie
Die=6
Die=4
Die=5
Independence
• Two events E and F are independent if the
probability of E conditioned on F is equal to the
unconditional probability of E. That is, Pr[E | F] =
Pr[E].
• In other words, the occurrence of F has no effect on
the occurrence of E.
Random Variables
• A random variable, R, represents the outcome of
some random event. Example: R = the roll of a die.
• The probability distribution of a random
variable, Pr[R], is a probability measure mapping
each possible value of R into its associated
probability.
Sum of Two Dice
• Example:
– R = the sum of the values of 2 dice
• Probability Distribution: due to independence:
 Pr[die  j ] Pr[die  k ]
– Pr[ R  i ] 
j , k : j  k i
1
  I{ j  k i} 
36
j 1 k 1
6
6
( where I{Q}  1 if Q is true, 0 otherwise)
Sum of Two Dice
1
Pr[ R  2]  Pr[ die1  1] Pr[ die2  1] 
36
Pr[ R  3]  Pr[ die1  1] Pr[ die2  2]
2
 Pr[ die1  2] Pr[ die2  1] 
36
Pr[ R  4]  Pr[ die1  1] Pr[ die2  3]
 Pr[ die1  2] Pr[ die2  2]
3
 Pr[ die1  3] Pr[ die2  1] 
36
etc...
Probability Mass Function:
R = Sum of 2 dice
0.5
0.4
0.3
0.2
0.1
0
2
3
4
5
6
7
8
Pr(R = x)
9
10 11 12
Continuous Random Variables
• So far, we have only considered discrete random
variables, which can take on a countable number
of distinct values.
• Continuous random variables and take on any
real value over some (possibly infinite) range.
– Example: R = Inter-packet-arrival times at a router.
Continuous Density Functions
• There is no probability mass function for a continuous
random variable, since, typically, Pr[R = x] = 0 for
any fixed value of x because there are infinitely many
possible values for R.
• Instead, we can generate density functions by starting
with histograms split into small intervals and
smoothing them (letting interval size go to zero).
Example: Bus Waiting Time
• Example: I arrive at a bus stop at a random time. I
know that buses arrive exactly once every 10
minutes. How long do I have to wait?
• Answer: My waiting time is uniformly
distributed between 0 and 10 minutes. That is, I
am equally likely to wait for any time between 0
and 10 minutes
Bus Wait Histogram
2000 attempts (histogram interval = 2 min)
600
400
200
0
0--2
2--4
4--6
6--8
8--10
Waiting Times (using 2-minute ‘buckets’)
Bus Wait Histogram
2000 attempts (histogram interval = 1 min)
600
400
200
0
0--1 1--2 2--3 3--4 4--5 5--6 6--7 7--8 8--9 9--10
Waiting Times (using 1-minute ‘buckets’)
Bus Waiting Time
Uniform Density Function
0.4
0.3
0.2
0.1
0
0 min.
5 min.
10
1
0 10dx  1
10 min.
Value for Density Function
• The histograms show the shape that the
density function should have, but what are the
values for the density function?
• Answer: Density function must be set so that the
function integrates to 1.

f

R
( x)dx  1
Continuous Density Functions
• To determine the probability that the random value lies
in any interval (a, b), we integrate the function on that
interval.
b
Pr[ a  R  b]   f R ( x)dx
a
• So, the probability that you wait between 3 and 5
minutes for the bus is 20%:
5
1
Pr[3  R  5]   dx  0.2
10
3
Cumulative Distribution Function
• For every probability density function, fR(x), there
is a corresponding cumulative distribution function,
FR(x), which gives the probability that the random
value is less than or equal to a fixed value, x.
x
FR ( x)  Pr[ R  x] 
f

R
( y )dy
Example: Bus Waiting Time
• For the bus waiting time described earlier,
the cumulative distribution function is
x
1
x
FR ( x)   dy 
10
10
0
Bus Waiting Time
Cumulative Distribution Function
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 min.
5 min.
Pr(R <= x)
10 min.
Cumulative Distribution Functions
• The probability that the random value lies in any interval
(a, b) can also easily be calculated using the cumulative
distribution function
Pr[a  R  b]  FR (b)  FR (a)
• So, the probability that you wait between 3 and 5
minutes for the bus is 20%:
5 3
Pr[3  R  5]    0.2
10 10
Expectation
• The expected value of a random variable, E[R], is
the mean value of that random variable. This may
also be called the average value of the random
variable.
Calculating E[R]
• Discrete R.V.
E[ R ] 
• Continuous R.V.

 x Pr[ R  x]
x  

E[ R ] 
 xf

R
( x )dx
E[R] examples
• Expected sum of 2 dice
12
E[ R]   x Pr[ R  x]  7
x 2
• Expected bus waiting time
10
1
100
E[ R]   x dx 
 5 min .
10
20
0
Moments
• The nth moment of R is defined to be the expected
value of Rn
– Discrete:

E[ R n ] 
n
x
 Pr[ R  x]
x  

– Continuous:
E[ R ] 
n
x

n
f R ( x )dx
Standard Deviation
• The standard deviation of R, (R), can be defined
using the 2nd moment of R:
 ( R)  Var( R)
 E[ R ]  ( E[ R])
2
2
Coefficient of Variation
• The coefficient of variation, CV(R), is a common
measure of the variability of R which is
independent of the mean value of R:
CV [ R ] 
 ( R)
E[ R ]
Coefficient of Variation
• The coefficient of variation for the exponential
random variable is always equal to 1.
• Random variables with CV greater than 1 are
sometimes called hyperexponential variables.
• Random variables with CV less than 1 are
sometimes called hypoexponential variables.
Common Discrete R.V.s
Bernouli random variable
• A Bernouli random variable w/ parameter p
reflects a 2-valued experiment with results of
success (R=1) w/ probability p
Pr[ R  1]  p
Pr[ R  0]  1  p
E[ R ]  p
1 p
CV [ R ] 
p
Common Discrete R.V.s
Geometric random variable
• A Geometric random variable reflects the number
of Bernouli trials required up to and including the
first success
Pr[ R  i ]  p(1  p)i 1
1
E[ R] 
p
CV [ R]  1  p
Geometric Mass Function
# Die Rolls until a 6 is rolled
0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
5
6
7
8
Pr(R = x)
9 10 11 12
Geometric Cumulative Function
# Die Rolls until a 6 is rolled
1
0.8
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
Pr(R <= x)
9 10 11 12
Common Discrete R.V.s
Binomial random variable
• A Binomial random variable w/ parameters (n,p) is
the number of successes found in a sequence of n
Bernoulli trials w/ parameter p
n i
n i
Pr[ R  i ]    p (1  p)
i
E[ R]  np
1 p
CV [ R ] 
np
Binomial Mass Function
# 6’s rolled in 12 die rolls
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
1
2 3
4
5
6
7
Pr(R = x)
8
9 10 11 12
Common Discrete R.V.s
Poisson random variable
• A Poisson random variable w/ parameter  models
the number of arrivals during 1 time unit for a
random system whose mean arrival rate is 
arrivals per time unit
Pr[ R  i ]  e
E[R ]  


i
i!
CV [ R ] 
1

Poisson Mass Function
Number of Arrivals per second given an
average of 4 arrivals per second ( = 4)
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0
1
2 3
4
5
6
7
Pr(R = x)
8
9 10 11 12
Continuous R.V.s
Continuous Uniform random variable
• A Continuous Uniform random variable is one
whose density function is constant over some
interval (a,b):
1
f R ( x) 
, a xb
ba
xa
FR ( x ) 
, a xb
ba
ba
E[ R] 
2
Exponential random variable
• A (Negative) Exponential random variable with
parameter  represents the inter-arrival time
between arrivals to a Poisson system:
f R ( x)  e  x , x  0
FR ( x )  1  e x , x  0
Exponential random variable
• Mean (expected value) and coefficient of variation
for Exponential random variable:
E[ R] 
1

CV [ R]  1
Exponential Delay
Poisson 4 arrivals/unit (E[R] = 0.25)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
Pr(R <= x)
1
1.2