Download ppt

Review of Probability Theory
[Source: Stanford University]
1
Random Variable
A random experiment with set of outcomes
 Random variable is a function from set of
outcomes to real numbers

2
Example

Indicator random variable:

A : A subset of
is called an event
3
CDF and PDF

Discrete random variable:


The possible values are discrete (countable)
Continuous random variable:

The rv can take a range of values in R

Cumulative Distribution Function (CDF):

PDF and PMF:
4
Expectation and higher
moments

Expectation (mean):


if X>0 :
Variance:
5
Two or more random variables

Joint CDF:

Covariance:
6
Independence

For two events A and B:

Two random variables

IID : Independent and Identically Distributed
7
Useful Distributions
8
Bernoulli Distribution


The same as indicator rv:
IID Bernoulli rvs (e.g. sequence of coin
flips)
9
Binomial Distribution

Repeated Trials:


Number of times an event A happens among n
trials has Binomial distribution


Repeat the same random experiment n times. (Experiments are
independent of each other)
(e.g., number of heads in n coin tosses, number of arrivals in n
time slots,…)
Binomial is sum of n IID Bernoulli rvs
10
Mean of Binomial

Note that:
11
Binomial - Example
0.45
n=4
0.4
0.35
p=0.2
n=10
0.3
0.25
n=20
0.2
n=40
0.15
0.1
0.05
0
1
2
3
4
5
6
7
8
9
10
11
12
12
Binomial – Example (ball-bin)

There are B bins, n balls are randomly
dropped into bins.

: Probability that a ball goes to bin i

: Number of balls in bin i after n drops
13
Multinomial Distribution
Generalization of Binomial
 Repeated Trails (we are interested in more
than just one event A)


A partition of W into A1,A2,…,Al
 Xi shows the number of times
among n trials.
Ai occurs
14
Poisson Distribution

Used to model number of arrivals
15
Poisson Graphs
0.5
l=.5
0.45
0.4
l=1
0.35
0.3
0.25
l=4
0.2
l=10
0.15
0.1
0.05
0
0
5
10
15
16
Poisson as limit of Binomial

Poisson is the limit of Binomial(n,p) as

Let
17
Poisson and Binomial
0.4
n=5,p=4/5
0.35
Poisson(4)
0.3
0.25
n=10,p=.4
0.2
n=20, p=.2
0.15
0.1
n=50,p=.08
0.05
0
0
1
2
3
4
5
6
7
8
9
10
18
Geometric Distribution

Repeated Trials: Number of trials till some
event occurs
19
Exponential Distribution
Continuous random variable
 Models lifetime, inter-arrivals,…

20
Minimum of Independent
Exponential rvs

: Independent Exponentials
21
Memoryless property

True for Geometric and Exponential Dist.:


The coin does not remember that it came up tails l times
Root cause of Markov Property.
22
Proof for Geometric
23
Characteristic Function
Moment Generating Function (MGF)
 For continuous rvs (similar to Laplace
transform)


For Discrete rvs (similar to Z-transform):
24
Characteristic Function


Can be used to compute mean or higher
moments:
If X and Y are independent and T=X+Y
25
Useful CFs

Bernoulli(p) :

Binomial(n,p) :

Multinomial:

Poisson:
26