Download Special random variables

Special random variables
Chapter 5
Some discrete or continuous
probability distributions
Some special random variables






Bernoulli
Binomial
Poisson
Hypergeometric
Uniform
Normal and its derivatives





Chi-square
T-distribution
F-distribution
Exponential
Gamma
Bernoulli random variable

A random variable X is said to be a Bernoulli
random variable if its probability mass function
is given as the following:



P{X=0}=1-p, P{X=1}=p; (you may assume X=1
when the experimental outcome is successful and
X=0 when it is failed.)
E[X]=1×P{X=1}+0×P{X=0}=p
Var[X]=E[X2]-E[X]2 =p(1-p)
Binomial random variable


Suppose there are n independent Bernoulli trials,
each of which results in a “success” with the
probability p.
If X represents the number of successes that occur
in the n trials, then X is said to be a binomial
random variable with parameters (n,p).

P{X=i}= n!/[(n-i)! ×i!] × pn(1-p)(n-i) , i=0, 1, 2,…n
Cin
n
n
 p( X  i)   C ( p) (1  p)
i 0
i 0
n
i
i
n i
 [ p  (1  p)]  1
n
The expectation of binomial
random variable




The Binomial random X is composed of n
independent Bernoulli trials
∴X=Σ1~nxi, xi =1 when the ith trial is a
success or xi =0 otherwise
E[X]=Σ1~nE[xi]=n×p
Var[X]=Σ1~nVar[xi]=np(1-p)
Patterns of binomial
distribution




If p=0.5, then X will distribute
symmetrically
If p>0.5, then X will be a left-skewed
distribution
If p<0.5, then X will be a right-skewed
distribution
If n∞, then X will distribute as a
symmetric bell/normal pattern
Poisson random variable

The random variable X is a Poisson
distribution if its prob. mass function is
given by
P[ X  i]  e


i
i!
, i  0,1,2,...


i
i 0
i 0
i!

P
(
X

i
)

e


(By the Taylor series)
 e   e   e 0  1
Expectation and variance of
Poisson distribution

By using the moment generating function


i 0
i 0
 (t )  E[etX ]   e ti (e  i / i!) e   (e t  ) i / i!)

e e
e t
 exp{ (e t  1)}
 ' (t )  e t exp{ (e t  1)};
 " (t )  (e t ) 2 exp{ (e t  1)}  e t exp{ (e t  1)}
E[ X ]   ' (0)   ,
Var[ X ]   " (0)   ' (0) 2  2    2  
Poisson vs. Binomial


A binomial distribution, (n,p),  a Poisson with
meanλ=np, when n is large and p is small.
In other words, if the successful probability of trial is
very small, then the accumulative many trials
distribute as a Poisson.


The probability of one person living over 100 years of age
is 0.01, then the mean number of over-100-year-old
persons 10 may occur within 1000 elders.
λ means the average occurrence among a large number of
experiments, in contrast to, p means the occurring chance
of every trial
Comparisons
A binomial random variable with p near to 0.5
A Poisson random variable
approximates to a binomial
distribution when n becomes
large.
Hypergeometric random
variable


There are N+M objects, of which N are desirable and the
other M are defective. A sample size n is randomly
chosen from N+M without replacements.
Let X be the number of desirable within n chosen objects.
Its probability mass function as the following.
N M
i
n i
N M
n
C C
P[ X  i] 
C
, i  0,1,2,... min( N , n)
We said X is a hypergeometric distribution with
parameters (N,M,n)
Expectation & variance of
hypergeometric distribution
1, if the ith selection is desirable
Xi  
0, otherwise
n
X   X i , X is the number of desired outcome in the sample of size n
i 1
n
n
i 1
i 1
E[X]   E[ X i ]   P( X i  1)  nN
n
Var ( X )   Var ( X i )  2
i 1
(N  M )
n
 Cov( X , X
1i  j n
i
j
)
Expectation & variance of
hypergeometric distribution (cont.)
N 1
N

E[ X i X j ]  P( X i X j  1)  P( X i  1, X j  1) 
N  M N  M 1
N
N ( N  1)
) 2 , for i  j
(
Cov( X i , X j )  E[ X i X j ]  E[ X i ]E[ X j ] 
N M
( N  M )( N  M  1)
 NM

( N  M ) 2 ( N  M  1)
n 1
nMN
 NM
nMN
n
)

1
(

C
2

Var ( X ) 
2
2
2
2
N  M 1
( N  M ) ( N  M  1) ( N  M )
(N  M )
n 1 

if let p  N/(N  M), then E(X)  np, Var(X)  np(1 - p) 1 
 N  M  1
Moreover, if N+M increases to ∞, then Var(X) converges to np(1-p), which
is the variance of a binomial random variable with parameters (n,p).
hypergeometric vs. binomial

Let X, and Y be independent binomial
random variables having respective
parameters (n,p) and (m,p). The
conditional p.m.f. of X given that X+Y=k is
a hypergeometric distribution with
parameters (n,m,k).
The Uniform random variable

A random variable X is said to be uniformly
distributed over the interval [α,β] if its p.d.f.
is given by
 1
, if α  X  

f ( x)     
0,
otherwise

1

(    ) 
dx  1
Pattern of uniform distribution
Expectation and variance of
uniform distribution
E[X]=∫α~β x[1/(β-α)]dx=(α+β)/2

Var(X)=E[X2]-E[X]2
=(β-α)2/12

P.161, Example 5.4b

Normal random variable

A continuous r.v X has a normal distribution
with parameter μ and σ2 if its probability
density function is given by:
We write X~N(μ,σ2)
By using the M.G.F., we obtain E[X]=ψ’(0)=μ, and
Var[X]=ψ”(0)-ψ’(0)2=σ2
Standard normal distribution
The Cumulative Distribution function of Z
(a) 
1
2
P{ X  b}  P{

P{a  X  b}  P{
e
z2 / 2

X 

a

a

b
a
 (
)  (
)


b


dz, -   a  
}  (
X 


b

)

}
b
 (a )  P{Z  a}  P{Z  a}  1   (a ), (by symmetry)
Percentiles of the normal
distribution

Suppose that a test score distributes as a normal
distribution with mean 500 and standard
deviation of 100. What is the 75th percentile
score of this test?
Characteristics of Normal
distribution
P{∣X-μ∣<σ}=ψ(-1<X-μ/σ<1)=ψ(-1<Z<1)= ψ(1)-ψ(-1)=2ψ(1) -1=2× 0.8413-1
P{∣X-μ∣<2σ}=ψ(-2<X-μ/σ<2)=ψ(-2<Z<2)= ψ(2)-ψ(-2)=2ψ(2) -1=2× 0.9772-1
P{∣X-μ∣<3σ}=ψ(-3<X-μ/σ<3)=ψ(-3<Z<3)= ψ(3)-ψ(-3)=2ψ(3) -1=2× 0.9987-1
The pattern of normal
distribution
Exponential random variables
A nonnegative random variable X with a parameter λ obeying
the following pdf and cdf is called an exponential distribution.
x
e x , if x  0
f ( x)  
, F ( x)  P{ X  x}   e y dy  1  e x , x  0
0
0
,
if
x

0

• The exponential distribution is often used to describe the
distribution of the amount of time until some specific
event occurs.
• The amount of time until an earthquake occurs
• The amount of time until a telephone call you receive
turns to be the wrong number
Pattern of exponential
distribution
Expectation and variance of
exponential distribution

E[X]=ψ’(0)=1/λ



E[X] means the average cycle time, λ presents the occurring
frequency per time interval
Var(X)=ψ”(0)-ψ’(0)2=1/λ2
The memoryless property of X

P{X>s+t/X>t}=P{X>s}, if s, t≧0; P{X>s+t}=P{X>s} ×P{X>t}
P{ X  s  t , X  t} P{ X  s  t}
P{ X  s  t / X  t} 

P{ X  t}
P{ X  t}
1  F ( s  t ) 1  [1  e  ( s t ) ]
 s



1

[
1

e
]  1  F ( s)  P{ X  s}
 t
1  F (t )
1  [1  e ]
Poisson vs. exponential

Suppose that independent events are occurring at
random time points, and let N(t) denote the number
of event that occurs in the time interval [0,t]. These
events are said to constitute a Poisson process
having rate λ, λ>0, if




N(0)=0;
The distribution of number of events occurring within an
interval depends on the time length and not on the time
point.
lim h0 P{N(h)=1}/h=λ
lim h0 P{N(h)≧2}/h=0
Poisson vs. exponential (cont.)

P{N(t)=k}=P{k of the n subintervals contain exactly 1
event and the other n-k contain 0 events}
P{ecactly 1 event in a subinterval t/n}≒λ(t/n)
 P{0 events in a subinterval t/n}≒1-λ(t/n)
k
nk
n
P{N(t)=k} ≒   t  1  t  , a binomial
 
n  with p=λ(t/n)
distribution  k  n  



A binomial distribution approximates to a Poisson
distribution with k=n(λt/n) when n is large and p is
k
small.
(

t
)
 t
P[ N (t )  k ]  e
k!
, k  0,1,2,...
Poisson vs. exponential (cont.)

Let X1 is the time of first event.
P{X1>t}=P{N(t)=0} (0 events in the first t time
length)=exp(-λt)
 ∵F(t)=P{X1≦t}=1- exp(-λt) with mean 1/λ


∴X1 is an exponential random variable
Let Xn is the time elapsed between (n-1)st and
nth event. P{Xn>t/Xn-1=s}=P{N(t)=0} =exp(-λt)
 ∵F(t)=P{Xn≦t}=1- exp(-λt) with mean 1/λ

∴ Xn is also an exponential random variable
Gamma distribution

See the gamma definition and proof in p.182-183


If X1 and X2 are independent gamma random variables
having respective parameters (α1,λ) and (α2,λ), then
X1+X2 is a gamma random variable with (α1+α2,λ)
The gamma distribution with (1,λ) reduces to the
exponential with the rate λ.

If X1, X2, …Xn are independent exponential random
variables, each having rate λ, then X1+ X2+ …+Xn is a
gamma random variable with parameters (n,λ)
Expectation and variance of
Gamma distribution

The gamma distribution with (α,1) reduces
to the normal distribution when α becomes
large.


The patters of gamma distribution move from
right-skewed toward symmetric as α increases.
See p.183, using the G.M.F. to compute


E[X]=α/λ
Var(X)=α/(λ2 )
Patterns of gamma distribution
Derivatives from the normal
distribution



The chi-square distribution
The t distribution
The F distribution
The Chi-square distribution



If Z1, Z2,…Zn are independent standard normal
random variables, the X, defined by
X=Z12+Z22+…Zn2, is called chi-square distribution
with n degrees of freedom, and denoted by X~Xn2
If X1 and X2 are independent chi-square random
variables with n1 and n2 degrees of freedom,
respectively, then X1+X2 is chi-square with n1+n2
degrees of freedom.
p.d.f. of Chi-square

The probability density function for the
distribution with r degrees of freedom is
given by
The relation between chi-square
and gamma


A chi-square random variable with n degrees of
freedom is identical to a gamma random variable
with (n/2,1/2), i.e., α=n/2,λ=1/2
The expectation of chi-square distribution


The variance of chi-square distribution


E[X] is the same as the expectation of gamma
withα=n/2,λ=1/2, so E[X]=α/λ=n (degrees of
freedom)
Var(X)=α/(λ2 )=2n
The chi-square distribution moves from rightskewed toward symmetric as the degree of
freedom n increases.
Patterns of chi-square
distribution
The t distribution


If Z and Xn2 are independent random variables, with Z
having a std. normal dist. And Xn2 having a chi-square
dist. with n degrees of freedom,
2
2
2
2
Z

Z

...
Z
Z
X
then Tn defined by
2
n
Tn 
, n  1
n
X n2 / n n
• For large n, the t distribution approximates to the standard
normal distribution
• The t distribution move from flatter and having thicker tails
toward steeper and having thinner tails as n increases.
p.d.f. of t-distribution
B(p,q) is the beta function
Patterns of t distribution
The F distribution


If Xn2 and Xm2 are independent chi-square random
variables with n and m degrees of freedom, respectively,
2
then the random variable Fn,m defined by
Xn / n
Fn,m 
2
m
X /m
• Fn,m is said an F-distribution with n and m degrees of freedom
• F1,m is the same as the square of t-distribution, (Tm)²
p.d.f of F-distribution
where
is the gamma function, B(p,q) is the beta function,
Patterns of F-distribution
Relationship between different
random variables
n∞
p0
λ=np
Binomial (n, p)
Repeated
trials
Bernoulli (p)
The very small time partition
Poisson(λ)
Gamma(α, λ)
n∞,
p0.5
λ=1,
α∞
α=1
α=n/2
λ=1/2
Normal (μ,σ2)
Standardization
Z=(X-μ)/σ
Exponential(λ)
F distribution
Chi-square (n)
Z (0, 1)
t distribution
Homework #4

Problem 5,12,27,31,39,44