Download random variables

2. Random variables




Introduction
Distribution of a random variable
Distribution function properties
Discrete random variables
 Point mass
 Discrete uniform
 Bernoulli
 Binomial
 Geometric
 Poisson
1
2. Random variables







Continuous random variables
 Uniform
 Exponential
 Normal
Transformations of random variables
Bivariate random variables
Independent random variables
Conditional distributions
Expectation of a random variable
kth moment
2
2. Random variables






Variance
Covariance
Correlation
Expectation of transformed variables
Sample mean and sample variance
Conditional expectation
3
Introduction
Random variables assign a real number to each
outcome:
X : 
  X ( )
Random variables can be:
 Discrete: if it takes at most countably many
values (integers).
 Continuous: if it can take any real number.
RANDOM VARIABLES
4
Distribution of a random variable
Distribution function
F ( x)  FX ( x)  P( X  x)
RANDOM VARIABLES
5
Distribution function properties
(i)
F ( x)  0 when
x  
(ii)
F ( x)  1 when
x  
(iii) F (x )
is nondecreasing.
x1  x2  F ( x1 )  F ( x2 )
(iv) F (x )
is right-continuous.
F ( x)  F ( x0 ) when
x  x0
x  x0
RANDOM VARIABLES
6
Distribution of a random variable
For a random variable, we define
Probability function
Density function,
depending on wether is either discrete or continuous
RANDOM VARIABLES
7
Distribution of a random variable
Probability function
p( x)  pX ( x)  P( X  x)
verifies
(i ) p ( x)  0
(ii )  p ( x)  1
x
RANDOM VARIABLES
8
Distribution of a random variable
Probability density function
f (x )
verifies
(i )
(ii )
f ( x)  0



f ( x)dx  1
We have
x
F ( x)   f (t )dt and f ( x)  F ' ( x).

RANDOM VARIABLES
9
Distribution of a random variable
F completely determines the distribution
of a random variable.
  p ( x)
a  x  b
P(a  X  b)  F (b)  F (a)   b
 f (t )dt
a
RANDOM VARIABLES
10
Discrete random variables
Point mass
X  a
P( X  a)  1
0 if
F ( x)  
1 if
1--
0
xa
xa
a
RANDOM VARIABLES
11
Discrete random variables
Discrete uniform
X  U (1,2,..., k )
1
P( X  i ) 
i  1,2,..., k
k
1
2
3
k
1
2
3
k-1
k
RANDOM VARIABLES
12
Discrete random variables
Bernoulli
X  B(1, p)
P( X  1)  p
P( X  0)  1  p
p
p
1-p
1-p
0
1
0
1
RANDOM VARIABLES
13
Discrete random variables
Binomial
Successes in n independent Bernoulli trials with
success probability p
X  B(n, p)
n x
P( X  x)    p (1  p) n  x
 x
n
n!
with   
 x  x!(n  x)!
x  0,1,2,..., n
RANDOM VARIABLES
14
Discrete random variables
Geometric
Time of first success in a sequence of independent
Bernoulli trials with success probability p
X  G( p)
P( X  x)  (1  p) x 1  p
x  1,2,3,...
RANDOM VARIABLES
15
Discrete random variables
Poisson
X expresses the number of “ rare events”
X  P( ),   0
e   x
P( X  x) 
x!
x  0, 1, 2,...
RANDOM VARIABLES
16
Continuous random variables
Uniform
X  U [ a, b]
 1

for a  x  b
f ( x)   b  a
0 otherwise
0 for x  a
 x  a
F ( x)  
for a  x  b
b  a
1 for x  b
F(x)
f(x)
a
b
a
b
RANDOM VARIABLES
17
Continuous random variables
Exponential
 1 x
for x  0
 e
f ( x)   
0 for x  0
0 for x  0
x
F ( x)  

1

e
for x  0

X  exp(  )
1/
1
F(x)
f(x)
0

RANDOM VARIABLES
18
Continuous random variables
Normal
X  N ( ,  2)
 ( x   )2 
1

f ( x) 
exp  
2
2
 2


x

 2 0
f(x)
F(x)

RANDOM VARIABLES
19
Continuous random variables
Properties of normal distribution
(i)
X 
 N (0,1)
standard normal
(ii)

Z  N (0,1)   Z    N ( ,  2 )
(iii)
X i  N (  i ,  i2 ) independent i=1,2,...,n
n
n
i
i
  X i  N (  i ,   i2 )
i
RANDOM VARIABLES
20
Transformations of random variables
X random variable with
FX
;
Y = r(x); distribution of Y ?
r(•) is one-to-one; r -1(•).
1
1
FY ( y )  P(Y  y )  P(r ( X )  y )  P( X  r ( y ))  FX (r ( y ))
pY ( y )  P(Y  y )  P(r ( X )  y )  P( X  r 1 ( y ))  p X (r 1 ( y ))
fY ( y ) 
d
dy
1
1
FX (r ( y ))  f X (r ( y )) 
d r 1 ( y )
dy
RANDOM VARIABLES
21
Bivariate random variables
(X,Y) random variables;

If (X,Y) is a discrete random variable
p ( x, y )  probability joint function
verifies : p ( x, y )  0
 p( x, y)  1
x, y

If (X,Y) is continuous random variable
f ( x, y )  probability density joint function
verifies : f ( x, y )  0
 f ( x, y)dxdy  1
RANDOM VARIABLES
22
Bivariate random variables
The marginal probability functions for X and Y are:
p X ( x )   p ( x, y )
y
pY ( y )   p( x, y )
x
For continuous random variables, the marginal
densities for X and Y are:
f X ( x)   f ( x, y )dy
fY ( y)   f ( x, y)dx
RANDOM VARIABLES
23
Independent random variables
Two random variables X and Y are independent if
and only if:
p( x, y )  p X ( x) pY ( y )
f ( x, y )  f X ( x) fY ( y ),
for all values x and y.
RANDOM VARIABLES
24
Conditional distributions
Discrete variables
p ( x, y )
p( x | y )  P( X  x | Y  y ) 
p( y )
Continuous variables
f ( x, y )
f ( x | y) 
f ( y)
If X and Y are independent:
p( x | y )  p ( x)
f ( x | y )  f ( x)
RANDOM VARIABLES
25
Expectation of a random variable
EX   X   xp( x)
x
EX   X   xf ( x)dx
Properties:
(i) E

i
i
X i   i E X i
i  1,..., n
i
(ii) If X i , i  1,..., n are independent then:
E X i   EX i
i
i
RANDOM VARIABLES
26
Moment of order k
EX   x p ( x)
k
k
x
EX   x f ( x)dx
k
k
RANDOM VARIABLES
27
Variance
Given X with
  EX
:
VX    E ( X   )
2
X
2
 X  VX  ( E ( X   ) )
2 1/ 2
standard deviation
RANDOM VARIABLES
28
Variance
Properties:
(i)
V (aX  b)  a V ( X )
2
(ii) If X i are independent then
V ( ai X i )   ai V ( X i )
2
i
i
(iii)
VX  EX  (EX )
(iv)
VX  0
2
2
VX  0  P( X  a)  1
RANDOM VARIABLES
29
Covariance
X and Y random variables;
Cov( X , Y )  E ( X  EX )(Y  EY )
Properties
(i) If X, Y are independent then cov( X , Y )  0
(ii)
Cov( X , Y )  EXY  EXEY
(iii) V(X + Y) = V(X) + V(Y) + 2cov(X,Y)
V(X - Y) = V(X) + V(Y) - 2cov(X,Y)
RANDOM VARIABLES
30
Correlation
X and Y random variables;
Cov( X , Y )
 ( X ,Y ) 
VX VY
RANDOM VARIABLES
31
Correlation
Properties
(i)  1   ( X , Y )  1
(ii) If X and Y are independent then  ( X , Y )  0
(iii)
 ( X , Y )  1   a  0 : Y  aX  b
 ( X , Y )  1   a  0 : Y  aX  b
RANDOM VARIABLES
32
Expectation of transformed variables
Y  r ( X );
Er ( X )   r ( x) p X ( x)
x
Er ( X )   r ( x) f X ( x)dx
RANDOM VARIABLES
33
Sample mean and sample variance
Sample mean
1
EX  X   X i
n i
Sample variance
1
2
V (X )  S 
(Xi  X )

n 1 i
2
RANDOM VARIABLES
34
Sample mean and sample variance
Properties
X random variable; EX   , VX   ;
X 1 ,..., X n i. i. d. sample,
2
Then:
(i)
EX  
(ii) VX 
2
n
(iii) ES 2   2
RANDOM VARIABLES
35
Conditional expectation
X and Y are random variables; X | Y  y.
Then:
E ( X | Y  y )   x  p( x | Y  y )
x
E ( X | Y  y)   x  f ( x | y)dx
Properties:
EE ( X | Y )  EX
RANDOM VARIABLES
36