Download Random variables - ece.virginia.edu

Random variables
M. Veeraraghavan
A random variable is a rule that assigns a numerical value to each possible outcome of an experiment. Outcomes of an experiment form the sample space S .
Definition: A random variable X on a sample space S is a function X : S → ℜ that assigns a
real number X ( s ) to each sample point s ∈ S .
We define the event space A x to be the subset of S to which the random variable X assigns the
value x .
Ax = { s ∈ S X ( s ) = x }
(1)
Image of a rv X is the set of all values taken by X . Inverse image is A x .
Example: Take 3 Bernoulli trials. Sample space has 8 possible outcomes: 000, 001, 010, 100, 011,
110, 101, 111. But if we only interested in the number of successes, then 001, 100, 010 sample
points map to the value 1 (for number of 1s within the three trials). Similarly, 011, 110, 101 map
to 2. The event space has only 4 outcomes 0, 1, 2, 3 instead of the original sample space which has
8 points.
Typically, we are only interested in the event space A x rather than in the whole sample space S if
our interest lies only in the experimental values of the rv X .
Discrete random variables:
A random variable is discrete if it can take on values from a discrete set of numbers.
Probability mass function (pmf):
P ( Ax ) =
∑
P ( s ) = P [ X = x ] = pX ( x )
X(s) = x
Probabiltiy distribution function or Cumulative distribution function (cdf):
1
(2)
FX ( t ) = P ( X ≤ t ) =
∑ pX ( x )
(3)
x≤t
Often we will see the statement, “X is a discrete rv with pmf P X ( x ) “ with no hint or mention of
the sample space.
Know the pmf, cdf, mean (expected value) and variance for the following types of discrete random variables:
1. Bernoulli - one parameter p (of success)
2. Binomial - count number of successes in n independent trials with p as probability of success
of each trial - two parameters n and p .
3. Geometric - count number of trials upto 1st success (while binomial counts number of successes) - one parameter p
pX ( i ) = p ( 1 – p )
i–1
FX ( t ) = 1 – ( 1 – p )
for i = 1, 2, … and
t
for ( t ≥ 0 )
(4)
(5)
4. Negative binomial - count number of trials until r th success - two parameters p and r .
5. Poisson - approximation of binomial if n is large and p is small. One parameter α = np .
–α k
e α
p X ( k ) = -------------k!
for k = 0, 1, 2, …
(6)
6. Hypergeometric - sampling without replacement while binomial is sampling with replacement; N components of which d are defective. In a sample of n components what is the probability that k are defective.
7. Uniform - one parameter - range N .
8. Constant - one parameter c
9. Indicator - one parameter p .
Continuous random variables:
Probability density function (pdf):
dF X ( x )
f X ( x ) = ----------------dx
2
(7)
Probabiltiy distribution function or Cumulative distribution function (cdf):
t
∫ fX ( r ) dr
FX ( t ) = P ( X ≤ t ) =
(8)
–∞
Know the pdf, cdf, mean (expected value) and variance for the following types of random variables:
1. Exponential - 1 parameter λ
 1 – e – λx
F(x) = 
0

if ( 0 ≤ x < ∞ )
otherwise
 λe –λx
f( x) = 
 0
if ( x > 0 )
otherwise
(9)
(10)
1
E [ X ] = --λ
(11)
1
Var [ X ] = ----2λ
(12)
2. Gamma, chi-square, Student-t distributions
3. Erlang distribution
4. Hypoexponential distribution
5. Hyperexponential distribution
6. Normal (Gaussian) distribution
7. Pareto distribution
8. Weibull distribution
Relationships between random variables
1. Mutually exclusive events
P(A ∪ B) = P(A) + P(B ) ; P(A ∩ B) = 0 .
(13)
In general P ( A ∪ B ) = P ( A ) + P ( B ) – P ( A ∩ B )
(14)
2. Independent events
P ( A ∩ B ) = P ( A )P ( B )
3. Bayes rule
3
(15)
P ( A B j )P ( B j )
P ( B j A ) = ----------------------------------------∑ P ( A Bi )P ( Bi )
(16)
i
4. Theorem of total probability
n
P(A) =
∑ P ( A Bi )P ( Bi )
(17)
i=1
5. Correlation of two random variables
r XY = E [ XY ]
E [ XY ] =
∑ ∑
(18)
xyP X, Y ( x, y ) for a discrete r.v.
(19)
x ∈ SX y ∈ SY
If X and Y are independent random variables then because P X, Y ( x, y ) = P X ( x )P Y ( y ) ,
2
E [ XY ] = E [ X ]E [ Y ] (see [2, page 107]). Second moment of a single random variable is E [ X ] .
6. Covariance of two random variables
cov ( X, Y ) = E [ X – µ X ] [ Y – µ Y ] = E [ XY ] – µ X µ Y , where µ X = E [ X ] and µ Y = E [ Y ] . Relate
this to variance of a single random variable
2
2
Var [ X ] = E [ ( X – µ X ) ] = E [ X 2 ] – µ X
(20)
cov ( X, Y )
ρ XY = --------------------------------------var ( X )var ( Y )
(21)
7. Correlation coefficient:
Memoryless property:
Two distributions, the exponential continuous r.v. distribution and geometric discrete r.v. distribution enjoy this property.
Let X be the lifetime of a component. Suppose we have observed that the system has been operational for time t . We would like to know the probability that it will be operational for y more
hours. Say Y = X – t . We will show that
4
P ( Y ≤ y X > t ) = P ( Y ≤ y ) , in other words, how long it lasts beyond some time t is independent
of how long the component has been operational so far, i.e. t .
P(Y ≤ y X > t) = P(X – t ≤ y X > t) = P(X ≤ y + t X > t)
(22)
P ( X ≤ y + t and X > t )
P(t < X ≤ y + t)
= ----------------------------------------------------- = ------------------------------------P(X > t)
P(X > t)
Exponential distribution:
(y + t)
∫
P( Y ≤ y X > t) =
λe
– λx
dx
t
------------------------------∞
– λx
∫ λe
dx
– λt
– λy
e ( 1 – e -)
– λy
= -------------------------------= (1 – e ) = P(Y ≤ y)
– λt
e
(23)
t
Geometric distribution:
Let X be a rv with a geometric distribution and Y = X – t . We will show that
P( Y ≤ y X > t) = P( Y ≤ y ) .
y+t
t
t
y
1 – (1 – p)
– ( 1 – ( 1 – p ) )( 1 – p ) ( 1 – ( 1 – p ) )P ( Y ≤ y X > t ) = --------------------------------------------------------------------------------= -----------------------------------------------------------= P(Y ≤ y)
t
t
(1 – p)
(1 – p)
Note: Y = X – t has the same distribution as X because t is a constant. See the stat lecture to see
distributions of functions of random variables.
Relation between exponential distribution (for continuous rv) and geometric distribution
(for discrete rv): [2]
If X is an exponential random variable with parameter a , then K =
X is a geometric random
–a
variable with parameter p = 1 – e .
References
[1]
[2]
K. S. Trivedi, “Probability, Statistics with Reliability, Queueing and Computer Science Applications,” Second
Edition, Wiley, 2002, ISBN 0-471-33341-7.
R. Yates and D. Goodman, “Probability and Stochastic Processes,” Wiley, ISBN 0-471-17837-3.
5