Download STAT 3610: Review of Probability Distributions

STAT 3610: Review of Probability Distributions
Mark Carpenter
Professor of Statistics
Department of Mathematics and Statistics
August 25, 2015
Support of a Random Variable
Definition
The support of a random variable, say X , denoted as X , is
defined to be the set of all points on the real line for which the
pdf/pmf is non-zero. That is,
X = {x ∈ R : fX (x) > 0} ,
where the braces indicates a set.
Support of a Random Variable
I
The support of a random variable is usually denoted by the
script form of the letter corresponding to random variable
I
I
I
I
the random variable X has support X
the random variable Y has support Y
the random variable Z has support Z
The support of a random variable is one of the first
characteristics we can use to help identify the distribution of a
random variable.
Continuous versus Discrete Random Variables
Definition
Continuous Random Variable: A random variable is said to be
continuous if its support is a continuous set (made up of unions
and intersections of real intervals). The CDF* of a continuous
random variable must be a continuous function on the real line
Definition
Discrete Random Variable: A random variable is said to be
discrete if its support is a discrete set. The CDF* of a discrete
random variable is not continuous, but it is right continuous.
*CDF stands for Cumulative Distribution Function
Difference between Discrete and Continuous Random
Variables
So, whether the random variable, X , is a continuous or discrete
random variable depends on whether its support is continuous or
discrete.
I
In the discrete case, the pmf* is a discontinuous function with
a positive mass (probability) at each point in the support.
I
In the continuous case, the pdf** itself does not have to be a
continuous function everywhere, but it usually a continuous
function on intervals in the support.
*pmf stands for probability mass function
**pdf stands for probability density function
Exponential Random Variable and Exponential Distribution
Example 1: (continuous support) Suppose X ∼ exponential (θ).
From Section 3.2 (pp. 95-113) of the textbook, the exponential
distribution, indexed by the scale parameter θ (θ > 0) is
1 −x/θ
1 −x/θ
x ≥0
θe
I[0,∞) (x) =
f (x; θ) = e
0
otherwise.
θ
which means {x ∈ R : f (x) > 0} = [0, ∞) and the support of X is
X = [0, ∞), a continuous set.
We see that the pdf for the exponential is zero for all points below
zero, then jumps to λe 0 = λ at x = 0 and is continuous on [0, ∞).
pdf and cdf for the Standard Exponential
1
f (x)
0.5
F (a) = 1 − e −a
1
a
x
1
F (x)
0.5
F (a) = 1 − e −a
1
a
x
Exponential is special case of Gamma and Weibull
You can verify that the non-truncated gamma and Weibull
distributions, from which the exponential is a special case, share
this same support.
If X is a normal random variable, then the support is
X = (−∞, ∞) = R.
Mean or Expected Value of a Random Variable
Recall, for any random variable, X , with pdf/pmf f (x), a measure
of central tendency of the population is the population mean µ, or
the expected value/long run average for X . More formally,
Population Mean: For any random variable, X , with pdf/pmf
f (x), the population mean µ = E (X ) where
 Z ∞


xf (x)dx if X is continuous


 −∞
µ = E (X ) =
X



xf (x)
if X is discrete


x∈X
Population Variance or Variance of a Random Variable
Population Variance: For any random variable, X , with pdf/pmf
f (x), the population variance is σ 2 = E (X − µ)2 , where
 Z ∞


(x − µ)2 f (x)dx if X is continuous


 −∞
σ 2 = E (X − µ)2 =
X



(x − µ)2 f (x)
if X is discrete


x∈X
Sometimes Easier Way to Compute Population Variance
Note that it is often easier to compute the variance by noting that ,
σ 2 = E (X −µ)2 = E (X 2 −2X µ+µ2 ) = EX 2 −2µEX +µ2 = EX 2 −(EX )2 .
So, rather than going through the original express, one need only
compute E (X 2 ) and µ = E (X ) and plug the results in to the
following expression
σ 2 = E (X 2 ) − µ2 .
Expectations of Functions of a Random Variable
You might notice that each of E (X ), E (X 2 ), and E (X − µ)2 are
the expected value of different functions, g1 (x) = x, g2 (x) = X 2
and g3 (x) = (x − µ)2 . The expected value for any function is
defined below.
Moment Generating Functions
Whenever it exists, the moment-generating function for a
random variable X , denoted MX (t), is the continuous function of
t ∈ (−∞, ∞) given as
h i
MX (t) = E e tX , t ∈ (−h, h), h > 0.
The interval (−h, h) is referred to as the radius of convergence.
Properties of a Moment Generating Function (mgf)
This function is called the moment-generating function because
you can find the nth moment for the random variable, X, by
computed its nth derivative with respect to t then setting t = 0, as
follows
d
(n)
n
.
E (X ) = MX (0) = MX (t)
dt
t=0
Notice that the moment generating function is a continuous and
differentiable function of |t| < h, whether or not X is continuous.
In fact, the moment generating function is mathematically
independent of the original variable (since it was integrated or
summed over the support) and only relates to the variable X
through the moments of the distribution and any related
parameters.
Properties of Exponential
We will show on chalkboard that if X ∼ Exp(θ) then
Z ∞
Z ∞
1 −x/θ
I
f (x)dx =
e
dx = 1.
θ
−∞
0
Z ∞
Z ∞
x −x/θ
I µ = E (X ) =
e
dx = λ.
x · f (x)dx =
θ
−∞
0
Z ∞
I σ 2 = E (X − µ)2 =
(x − µ)2 f (x)dx = θ2
0
I
Cumulative Distribution Function (cdf) for any w ≥ 0 is
F (w ) = P(X ≤ w ) = 1 − e −w /θ
I
The moment generating function (mgf), denoted M(t) exists
and
1
1
M(t) =
, t<
(1 − θt)
θ
Discrete Example (Binomial)
Example: (discrete support) Suppose Y is a Binomial random
variable with parameters n and p (see page 117 of textbook) ,then
the probability mass function (pmf) is
 n

p y (1 − p)n−y y = 0, 1, . . . , n
y
fY (y ) =

0
otherwise
which means {y : f (y ) > 0} = {0, 1, . . . , n} and the support of Y
is Y = {0, 1, . . . , n}, a discrete (and finite) set of points.
Recall that,
n
y
=
n!
.
y !(n − y )!
cdf for Binomial Random Variable
Example 2 (binomial): Suppose X is a Binomial (4, 0.5), then
the pmf is
1
4
f (x; n = 4, p = 1/2) =
, x ∈ X = {0, 1, 2, 3, 4}
x
2n
Table : CDF for Binomial (n=4, p=1/2)
(−∞, 0)
P(X < 0) =
0
0
=
0
[0, 1)
P(X ≤ 0) =
P(0)
1
16
=
1
16
[1, 2)
P(X ≤ 1) =
P(0) + P(1)
1
16
4
+ 16
=
5
16
+
4
16
=
11
16
[2, 3)
P(X ≤ 2) =
P(0) + P(1) + P(2)
1
16
[3, 4)
P(X ≤ 3) =
P(0) + P(1) + P(2) + P(3)
1
16
4 + 6 + 4
+ 16
16
16
=
15
16
[4, ∞)
P(X ≤ 4) =
P(0) + P(1) + P(2) + P(3) + P(4)
1
16
4 + 6 + 4 + 1
+ 16
16
16
16
=
1
+
6
16
cdf for Binomial Random Variable
1
15/16
11/16
5/16
1/16
0
1
2
3
4
5
Binomial, hypergeometric, geometric
One can verify that the supports for the negative binomial (r , p),
the Geometric(p), and the Poisson random variables (see page
126) are the same countably infinite set {0, 1, 2, . . .}. The support
of the hypergeomtric(n, M, N) is discrete/finite set
{max(0, n − N + M), . . . , min(n, M).}
Poisson Random Variable
Suppose X is a discrete random variable with a Poisson (λ)
distribution, then the probability mass function (pmf) is


λx e −λ


x = 0, 1, 2, ...,
x!
f (x) =



0
otherwise
where λ > 0.
Recall that the MacLaurin series of an Exponential function, e y , is
∞
X
yi
ey =
, where i! represents the factorial function for an
i!
i=0
integer i.
Some Properties of a Poisson Random Variable
I
X
f (x; λ) =
x!
x=0
x∈X
I
∞
X
λx e −λ
µ = E (X ) =
X
=1
x · f (x) = λ
x∈X
I
σ 2 = E (X − µ)2 =
X
(x − µ)2 · f (x) = λ
x∈X
I
M(t) = E (e
tX
)=
X
x∈X
e tx · f (x) = e λ(e
t −1)
,
−∞ < t < ∞.