Download Probability Distributions - Duke People

Probability Distributions
CEE 201L. Uncertainty, Design, and Optimization
Department of Civil and Environmental Engineering
Duke University
Philip Scott Harvey, Henri P. Gavin and Jeffrey T. Scruggs
Spring, 2016
1 Probability Distributions
Consider a continuous, random variable (rv) X with support over the domain X . The probability
density function (PDF) of X is the function fX (x) such that for any two numbers a and b in the
domain X , with a < b,
P [a < X ≤ b] =
Z b
fX (x) dx
a
For fX (x) to be a proper distribution, it must satisfy the following two conditions:
1. The PDF fX (x) is positive-valued; fX (x) ≥ 0 for all values of x ∈ X .
2. The rule of total probability holds; the total area under fX (x) is 1;
R
X
fX (x) dx = 1.
Alternately, X may be described by its cumulative distribution function (CDF). The CDF
of X is the function FX (x) that gives, for any specified number x ∈ X , the probability that the
random variable X is less than or equal to the number x is written as P [X ≤ x]. For real values of
x, the CDF is defined by
FX (x) = P [X ≤ b] =
Z b
−∞
fX (x) dx ,
so,
P [a < X ≤ b] = FX (b) − FX (a)
By the first fundamental theorem of calculus, the functions fX (x) and FX (x) are related as
fX (x) =
d
FX (x)
dx
2 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
A few important characteristics of CDF’s of X are:
1. CDF’s, FX (x), are monotonic non-decreasing functions of x.
2. For any number a, P [X > a] = 1 − P [X ≤ a] = 1 − FX (a)
3. For any two numbers a and b with a < b, P [a < X ≤ b] = FX (b) − FX (a) =
Rb
a
fX (x)dx
2 Descriptors of random variables
The expected or mean value of a continuous random variable X with PDF fX (x) is the centroid
of the probability density.
Z
∞
µX = E[X] =
−∞
x fX (x) dx
The expected value of an arbitrary function of X, g(X), with respect to the PDF fX (x) is
Z ∞
µg(X) = E[g(X)] =
−∞
g(x) fX (x) dx
The variance of a continuous rv X with PDF fX (x) and mean µX gives a quantitative measure of
how much spread or dispersion there is in the distribution of x values. The variance is calculated
as
2
σX
= V[X] =
Z ∞
−∞
(x − µX )2 fX (x) dx
=
=
=
=
p
The standard deviation (s.d.) of X is σX = V[X]. The coefficient of variation (c.o.v.) of
X is defined as the ratio of the standard deviation σX to the mean µX :
σX cX = µX for non-zero mean. The c.o.v. is a normalized measure of dispersion (dimensionless).
A mode of a probability density function, fX (x), is a value of x such that the PDF is maximized;
d
fX (x)
=0.
dx
x=xmode
The median value, xm , is is the value of x such that
P [X ≤ xm ] = P [X > xm ] = FX (xm ) = 1 − FX (xm ) = 0.5 .
CC BY-NC-ND PSH, HPG, JTS
3
Probability Distributions
3 Some common distributions
A few commonly-used probability distributions are described at the end of this document: the
uniform, triangular, exponential, normal, and log-normal distributions. For each of these distributions, this document provides figures and equations for the PDF and CDF, equations for the mean
and variance, the names of Matlab functions to generate samples, and empirical distributions of
such samples.
3.1
The Normal distribution
The Normal (or Gaussian) distribution is perhaps the most commonly used distribution function.
2 ) denotes that X is a normal random variable with mean µ
The notation X ∼ N (µX , σX
X and
2
variance σX . The standard normal random variable, Z, or “z-statistic”, is distributed as N (0, 1).
The probability density function of a standard normal random variable is so widely used it has its
own special symbol, φ(z),
!
1
z2
φ(z) = √ exp −
2
2π
Any normally distributed random variable can be defined in terms of the standard normal random
variable, through the change of variables
X = µX + σX Z.
If X is normally distributed, it has the PDF
x − µX
fX (x) = φ
σX
1
=q
2
2πσX
(x − µX )2
exp −
2
2σX
!
There is no closed-form equation for the CDF of a normal random variable. Solving the integral
1
Φ(z) = √
2π
Z z
2 /2
e−u
du
−∞
would make you famous. Try it. The CDF of a normal random variable is expressed in terms of the
error function, erf(z). If X is normally distributed, P [X ≤ x] can be found from the standard
normal CDF
x − µX
P [X ≤ x] = FX (x) = Φ
.
σX
Values for Φ(z) are tabulated and can be computed, e.g., the Matlab command . . .
Prob_X_le_x = normcdf(x,muX,sigX). The standard normal PDF is symmetric about z = 0,
so φ(−z) = φ(z), Φ(−z) = 1 − Φ(z), and P [X > x] = 1 − FX (x) = 1 − Φ ((x − µX )/σX ) =
Φ ((µX − x)/σX ).
The linear combination of two independent normal rv’s X1 and X2 (with means µ1 and µ2 and
variances σ12 and σ22 ) is also normally distributed,
aX1 + bX2 ∼ N aµ1 + bµ2 , a2 σ12 + b2 σ22 ,
2 .
and more specifically, aX − b ∼ N aµX − b, a2 σX
CC BY-NC-ND PSH, HPG, JTS
4 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
Given the probability of a normal rv, i.e., given P [X ≤ x], the associated value of x can be found
from the inverse standard normal CDF,
x − µX
= z = Φ−1 (P [X ≤ x]) .
σX
Values of the inverse standard normal CDF are tabulated, and can be computed, e.g., the Matlab
command . . . x = norminv(Prob_X_le_x,muX,sigX).
3.2
The Log-Normal distribution
The Normal distribution is symmetric and can be used to describe random variables that can take
positive as well as negative values, regardless of the value of the mean and standard deviation. For
many random quantities a negative value makes no sense (e.g., modulus of elasticity, air pressure,
and distance). Using a distribution which admits only positive values for such quantities eliminates
any possibility of non-sensical negative values. The log-normal distribution is such a distribution.
If ln X is normally distributed (i.e., ln X ∼ N (µln X , σln X )) then X is called a log-normal random
variable. In other words, if Y (= ln X) is normally distributed, eY (= X) is log-normally distributed.
P [Y ≤ y]
FY (y)
Y
Φ y−µ
σY
P [ln X ≤ ln x] = Fln X (ln x) =
ln x−µln X
Φ
σln X
P [X ≤ x]
FX (x)
µY = µln X ,
2
σY2 = σln
X,
The mean and standard deviation of a log-normal variable X are related to the mean and standard
deviation of ln X.
1 2
µln X = ln µX − σln
2 X
2
2
σln
X = ln 1 + (σX /µX )
If (σX /µX ) < 0.30, σln X ≈ (σX /µX ) = cX
The median, xm , is a useful parameter of log-normal rv’s. By definition of the median value, half
of the population lies above the median, and half lies below, so
Φ
ln xm − µln X
σln X
ln xm − µln X
σln X
= 0.5
= Φ−1 (0.5) = 0
q
and, ln xm = µln X ↔ xm = exp(µln X ) ↔ µX = xm 1 + c2X
For the log-normal distribution xmode < xmedian < xmean . If cX < 0.15, xmedian ≈ xmean .
If ln X is normally distributed (X is log-normal) then (for cX < 0.3)
ln x − ln xm
P [X ≤ x] ≈ Φ
cX
2 ), and ln Y ∼ N (µ
2
n
m then
If ln X ∼ N (µln X , σln
ln Y , σln Y ), and Z = aX /Y
X
2
ln Z = ln a + n ln X − m ln Y ∼ N (µln Z , σln
Z)
where
and
µln Z = ln a + nµln X − mµln Y = ln a + n ln xm − m ln ym
2
2
2
2
2
2
2
2
σln
Z = (nσln X ) + (mσln Y ) = n ln(1 + cX ) + m ln(1 + cY ) = ln(1 + cZ )
CC BY-NC-ND PSH, HPG, JTS
5
Probability Distributions
Uniform X ∼ U[a, b]
Triangular X ∼ T (a, b, c)
X = R, a ≤ c ≤ b
X = R, b > a
2/(b-a)
p.d.f., f(x)
p.d.f., f(x)
1/(b-a)
0
0
a
µ−σ
µ
µ+σ
b
µ−σ c
µ
a
µ−σ c
µ
0
µ−σ
µ
x
(
f (x) =
1
b−a ,
µ+σ
b
µ+σ
µ+2σ
b
x
x ∈ [a, b]
otherwise
0,
F (x) =


 0,
x−a
b−a ,

 1,
f (x) =

2(x−a)


 (b−a)(c−a) , x ∈ [a, c]
2(b−x)
(b−a)(b−c) ,


 0,
x≤a
x ∈ [a, b]
x≥b
µX = 12 (a + b)
2 =
σX
1
12 (b
x ∈ [c, b]
otherwise

0,
x≤a



2

 (x−a) ,
x ∈ [a, c]
(b−a)(c−a)
F (x) =
2
(b−x)

 1 − (b−a)(b−c) , x ∈ [c, b]



x≥b
1,
µX = 13 (a + b + c)
− a)2
1
2
18 (a
2 =
σX
+ b2 + c2 − ab − ac − bc)
x = triangular rnd(a,b,c,1,N);
x = a + (b-a).*rand(1,N);
0.3
0.5
0.25
empirical p.d.f.
empirical p.d.f.
b
0
a
0.2
0.15
0.1
0.05
0
1
2
3
4
5
0.4
0.3
0.2
0.1
0
6
0
1
2
3
4
5
6
5
6
1
empirical c.d.f.
1
empirical c.d.f.
µ+2σ
1
1/2
0
µ+σ
c.d.f., F(x)
c.d.f., F(x)
1
a
0.8
0.6
0.4
0.2
µ=3.0
σ=1.2
0
0.8
0.6
0.4
0.2
µ=2.7
σ=0.8
0
0
1
2
3
x
4
5
6
0
1
2
3
x
4
CC BY-NC-ND PSH, HPG, JTS
6 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
Normal X ∼ N (µ, σ 2 )
2 )
Log-Normal ln X ∼ N (µln X , σln
X
+
+
X = R , µln X ∈ R , σln X > 0
p.d.f., f(x)
p.d.f., f(x)
X = R, µ ∈ R, σ > 0
0
0
µ−2σ
µ−σ
µ
µ+σ
0
µ+2σ
0.159
0.023
µ
µ+2σ
µ+σ
µ+2σ
0
µ−2σ
f (x) =
µ−σ
µ
x
√ 1
2πσ 2
F (x) =
h
1
2
exp
µ+σ
µ+2σ
x
1 + erf
2
− (x−µ)
2σ 2
x−µ
√
2σ 2
f (x) =
i
-3
-2
1
2
exp
1 + erf
2
ln X )
− (ln x−µ
2
2σln
X
ln√
x−µln X
2
2σln
X
µX = xm 1 + c2X
2 = σ2
σX
2 = x2 c2 1 + c2
σX
m X
X
-1
0
q
µX = µ
1
2
3
x = logn rnd(Xm,Cx,1,N);
empirical p.d.f.
-4
√ 1 2
x 2πσln
X
F (x) =
x = muX + sigmaX*randn(1,N);
empirical p.d.f.
µ−σ
µ+σ
c.d.f., F(x)
c.d.f., F(x)
0.5
4
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
1
2
3
µ=2.0
σ=1.0
4
5
4
5
1
empirical c.d.f.
1
empirical c.d.f.
µ
1
0.977
0.841
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
µ−σ
0.8
0.6
0.4
0.2
µ=0.0
σ=1.0
0.8
0.6
0.4
0.2
0
0
-4
-3
-2
-1
0
x
1
2
3
4
0
1
2
3
x
CC BY-NC-ND PSH, HPG, JTS
7
Probability Distributions
Exponential X ∼ E(µ)
X = R+ , µ > 0
Laplace X ∼ L(µ, σ 2 )
Rayleigh X ∼ R(m)
X = R+ , m > 0
X = R, µ ∈ R, σ > 0
1/(e µ)
0
0
0
0
µ
2µ
µ−σ m µ
3µ
µ+σ
µ+2σ
1
1-1/e
0.6
0.4
0.2
0
µ
2µ
x
f (x) =
1
µ
µ−σ m µ
3µ
f (x) =
exp(−x/µ)
µ−2σ
µ−σ
µ
x
µ+σ
µ+2σ
0.5
µ+2σ
exp − 21 (x/m)2
√
f (x) =
2
2σ


1
2
√
exp − 2 |x−µ|
σ
√
exp
2 |x−µ|
σ
x<µ
µX = m π/2
µX = µ
2 = µ2
σX
2 = m2 (4 − π)/2
σX
2 = σ2
σX
x = rayleigh rnd(modeX,1,N);
x = laplace rnd(muX,sigmaX,1,N);
0.4
0.2
1
1.5
2
2.5
3
3.5
4
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.5
1
1.5
2
2.5
3
3.5
0.4
µ=1.0
0.8
0.6
0.4
0.2
m=1.0
0
0
0.5
1
1.5
2
x
2.5
3
3.5
4
x≥µ
0.4
0.3
0.2
0.1
-4
-3
-2
-1
0
1
2
3
4
3
4
1
empirical c.d.f.
empirical c.d.f.
0.6
0.2
 1 − 1 exp − 2
2
σ
0
4
1
0.8
√
|x−µ|
0.5
empirical p.d.f.
0.6
0.5
F (x) =
p
empirical p.d.f.
empirical p.d.f.
µ+2σ
µX = µ
1
empirical c.d.f.
µ+σ
F (x) = 1 − exp − 12 (x/m)2
0.8
0
µ
F (x) = 1 − exp(−x/µ)
1
0
x
m2
µ+σ
x
x = exp rnd(muX,1,N);
0
µ−σ
0.123
0.03
0
0
µ−2σ
0.97
0.878
0.8
c.d.f., F(x)
c.d.f., F(x)
1
c.d.f., F(x)
p.d.f., f(x)
p.d.f., f(x)
p.d.f., f(x)
1
0.8
0.6
0.4
0.2
µ=0.0
σ=1.4
0
0
0.5
1
1.5
2
x
2.5
3
3.5
4
-4
-3
-2
-1
0
x
1
2
CC BY-NC-ND PSH, HPG, JTS
8 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
4 Sums and Differences of Independent Normal Random Variables
2 ) and Y ∼ N (µ , σ 2 ). Any
Consider two normally-distributed random variables, X ∼ N (µX , σX
Y
Y
weighted sum of normal random variables is also normally-distributed.
Z = aX − bY
Z ∼ N aµX − bµY , (aσX )2 + (bσY )2
σZ2 = (aσX )2 + (bσY )2
µZ = aµX − bµY
5 Products and Quotients of Independent LogNormal Random Variables
2 ) and ln Y ∼ N (µ
2
Consider two log-normally-distributed random variables, ln X ∼ N (µln X , σln
ln Y , σln Y ).
X
Any product or quotient of lognormal random variables is also lognormally-distributed.
Z = X/Y
2
2
ln Z ∼ N µln X − µln Y , σln
X + σln Y
2
2
2
σln
Z = σln X + σln Y
µln Z = µln X − µln Y
c2Z = c2X + c2Y + c2X c2Y
6 Examples
1. The strength, S, of a particular grade of steel is log-normally distributed with median 36 ksi
and c.o.v. of 0.15. What is the probability that the strength of a particular sample is greater
than 40 ksi?
ln 40 − ln 36
3.69 − 3.58
P [S > 40] = 1 − P [S ≤ 40] = 1 − Φ
=1−Φ
0.15
0.15
= 1 − Φ(0.702) = 1 − 0.759 = 0.241
0.08
smode = 35.20 ksi
smedian = 36.00 ksi
0.07
smean = 36.40 ksi
0.06
p.d.f.
0.05
0.04
0.03
0.02
0.01
P[S>40]
0
20
25
30
35
40
45
50
55
60
strength of steel, s, ksi
CC BY-NC-ND PSH, HPG, JTS
9
Probability Distributions
2. Highway truck weights in Michigan, W , are assumed to be normally distributed with mean
100 k and standard deviation 40 k. The load capacity of bridges in Michigan, R, are also
assumed to be normally distributed with mean 200 k and standard devation 30 k. What is
the probability of a truck exceeding a bridge load rating?
E = W − R. If E > 0 the truck weight eceededs the bridge capacity.
µE = √
µW − µR = 100 − 200 = −100 k.
σE = 402 + 302 = 50 k.
0 − (−100)
P [E > 0] = 1 − P [E ≤ 0] = 1 − Φ
50
= 1 − Φ(2) = 1 − 0.977 = 0.023
R
W
E=W−R
50
−100
30
40
000000000000000
111111111111111
000000000000000
111111111111111
0
100
200 k
3. Windows in the Cape Hattaras Lighthouse can withstand wind pressures of R. R is lognormal with median of 40 psf and coefficient of variation of 0.25. The peak wind pressure
during a hurricane P in psf is given by the equation P = 1.165 × 10−3 CV 2 where C is a
log-normal coefficient with median of 1.8 and coefficient of variation of 0.20 and V is the
wind speed with median 100 fps and coefficient of variation of 0.30. What is the probability
of the wind pressure exceeding the strength of the window?
The peak wind pressure is also log-normal.
ln P
= ln(1.165 × 10−3 ) + ln C + 2 ln V
µln P
= ln(1.165 × 10−3 ) + µln C + 2µln V
µln P
= ln(1.165 × 10−3 ) + ln(1.8) + 2 ln(100) = 3.0431
2
σln
P
= ln(1 + 0.202 ) + 2 ln(1 + 0.302 ) = 0.2116 . . .
σln P = 0.4600
The wind pressure exceeds the resistance if P/R > 1 (that is, if ln P − ln R > 0)
ln E = ln P − ln R
µln E = µln P − µln R = 3.0431 − ln(40) = −0.646
2
2
σln
E = 0.2116 + ln(1 + 0.25 ) = 0.27220 . . .
σln E = 0.5217
The probability of the wind load load exceeding the resistance of the glass is,
0 + 0.646
P [E > 1] = 1 − P [E ≤ 1] = 1 − P [ln E ≤ 0] = 1 − Φ
0.5217
= 1 − Φ(1.2383) = 0.11
4. Earthquakes with M > 6 earthquake shake the ground at a building site randomly. The peak
ground acceleration (PGA) is log-normally distributed with median of 0.2 g and a coefficient
of variation of 0.25. Assume that the building will sustain no damage for ground motion
shaking up to 0.3 g. What is the probability of damage from an earthquake of M > 6?
ln(0.3) − ln(0.2)
P [D|M > 6] = P [P GA > 0.3] = 1−P [P GA ≤ 0.3] = 1−Φ
0.25
= 1−0.947 = 0.053.
CC BY-NC-ND PSH, HPG, JTS
10 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
There have been two earthquakes with M > 6 in the last 50 years. What is the probability
of no damage from earthquakes with M > 6 in the next 20 years?
P [D0 |M > 6] = 1 − 0.053 = 0.947
From the law of total probability,
P [D0 ] in 20 yr
= P [D0 |0 EQ M > 6 in 20yr] · P [0 EQ M > 6 in 20yr] +
P [D0 |1 EQ M > 6 in 20yr] · P [1 EQ M > 6 in 20yr] +
P [D0 |2 EQ M > 6 in 20yr] · P [2 EQ M > 6 in 20yr] +
P [D0 |3 EQ M > 6 in 20yr] · P [3 EQ M > 6 in 20yr] + · · ·
where P [D0 |n EQ M > 6] = (P [D0 |1 EQ M > 6])n (assuming damage from an earthquake
does not weaken the building . . . ) So,
P [D0 ] in 20 yr
=
∞
X
(0.947)n
n=0
(20/25)n
exp(−20/25)
n!
"
0.8
0.82
0.83
= exp(−0.8) 1 + 0.947
+ (0.947)2
+ (0.947)3
+ ···
1!
2!
3!
#
= exp(−0.8) · exp(0.947 · 0.8) = 0.958
The probability of damage from earthquakes in the next 20 years (given the assumptions in
this example) is close to 4%. Would that be an acceptable level of risk for you?
7 Empirical PDFs, CDFs, and exceedence rates (nonparametric statistics)
The PDF and CDF of a sample of random data can be computed directly from the sample, without
assuming any particular probability distribution . . . (such as a normal, exponential, or other kind
of distribution).
A random sample of N data points can be sorted into increasing numerical order, so that
x1 ≤ x2 ≤ · · · ≤ xi−1 ≤ xi ≤ xi+1 ≤ · · · ≤ xN −1 ≤ xN .
In the sorted sample there are i data points less than or equal to xi . So, if the sample is representative of the population, and the sample is “big enough” the probability that a random X is less
than or equal to the ith sorted value is i/N . In other words, P [X ≤ xi ] = i/N . Unless we know
that no value of X can exceed xN , we must accept some probability that X > xN . So, P [X ≤ xN ]
should be less than 1 and in such cases we can write P [X ≤ xi ] = (i − 1/2)/N .
The empirical CDF computed from a sorted sample of N values is
F̂X (xi ) =
i
N
. . . or . . .
F̂X (xi ) =
i − 1/2
N
The empirical PDF is basically a histogram of the data. The following Matlab lines plot empirical
CDFs and PDFs from a vector of random data, x.
CC BY-NC-ND PSH, HPG, JTS
11
Probability Distributions
1
2
3
4
5
6
7
8
N = length ( x );
nBins = f l o o r ( N /50);
[ fx , xx ] = h i s t (x , nBins );
fx = fx / N * nBins /(max( x ) -min( x )));
F_x = ([1: N ] -0.5)/ N ;
subplot (211); bar( xx , fx );
subplot (212); plot ( sort ( x ) , F_x );
p r o b a b i l i t y _ o f _ f a i l u r e = sum(x >0) / N
%
%
%
%
%
%
%
%
number o f v a l u e s i n t h e sample
number o f b i n s i n t h e h i s t o g r a m
compute t h e h i s t o g r a m
s c a l e t h e h i s t o g r a m t o a PDF
e m p i r i c a l CDF
p l o t e m p i r i c a l PDF
p l o t e m p i r i c a l CDF
probability that X > 0
The number of values in the sample greater than xi is (N − i). If the sample is representative, the
probability of a value exceeding xi is Prob[X > xi ] = 1 − FX (xi ) ≈ 1 − i/N . If the N samples were
collected over a period of time T , the average exceedence rate (number of events greater than xi
per unit time) is ν(xi ) = N (1 − FX (xi ))/T ≈ N (1 − i/N )/T = (N − i)/T .
8 Random variable generation using the Inverse CDF method
A sample of a random variable having virtually any type of CDF, P [X ≤ x] = P = FX (x) can
be generated from a sample of a uniformly-distributed random variable, U , (0 < U < 1), as
long as the inverse CDF, x = FX−1 (P ) can be computed. There are many numerical methods for
generating a sample of uniformly-distributed random numbers. It is important to be aware that
samples from some methods are “more random” than samples from others. The Matlab command
u = rand(1,N) computes a (row) vector sample of N uniformly-distributed random numbers with
0 < u < 1.
If X is a continuous rv with CDF FX (x) and U has a uniform distribution on (0, 1), then the
random variable FX−1 (U ) has the distribution FX . Thus, in order to generate a sample of data
distributed according to the CDF FX , it suffices to generate a sample, u, of the rv U ∼ U[0, 1] and
then make the transformation x = FX−1 (u).
For example, if X is exponentially-distributed, the CDF of X is given by
FX (x) = 1 − e−x/µ ,
so
FX−1 (u) = −µ ln(1 − FX (x)).
Therefore if u is a value from a uniformly-distributed rv in [0, 1], then
x = −µ ln(u)
is a value from an exponentially distributed random variable. (If U is uniformly distributed in [0,1]
then so is 1 − U .)
As another example, if X is log-normally distributed, the CDF of X is
ln x − ln xm
.
σln X
If u is a sample from a standard uniform distribution, then
FX (x) = Φ
h
x = exp ln xm + Φ−1 (u)σln X
i
is a sample from a lognormal distribution.
Note that since expressions for Φ(z) and Φ−1 (P ) do not exist, the generation of normally-distributed
random variables requires other numerical methods. x = muX + sigX*randn(1,N) computes a
(row) vector sample of N normally-distributed random numbers.
CC BY-NC-ND PSH, HPG, JTS
12 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
cdf
1
1
cdf
F(x)
F(x)
X
u
u
X
u = F (x)
u = F (x)
U
x = F (u)
−1
x = F (u)
X
0
0
X
U
pdf
−1
f (u)
X
f (u)
pdf
X
x
x=b
x=a
1
1
x=a
pdf
pdf
f (x) =
X
x
x=b
dFX
f (x) =
dx
X
x=a
x=b
dFX
dx
x
x=a
x=b
x
1
u
cdf
u = FX (x)
1
u
Figure 1. Examples of the generation of uniform random variables from the inverse CDF method.
cdf
1
u = FX(x)
F(x)
X
1
x
pdf
f (x) =
X
−1
U
x = FX (u)
0
0
X
f (u)
pdf
U
x = F (u)
1
−1
f (u)
pdf
F(x)
X
x
pdf
dFX
f (x) =
dx
X
dFX
dx
x
x
Figure 2. Examples of the generation of random variables from the inverse CDF method. The density of
−1
the horizontal arrows u is uniform, whereas the density of the vertical arrows, x = FX
(u), is proportional
0
to FX
(x), that is, proportional to fX (x).
CC BY-NC-ND PSH, HPG, JTS
13
Probability Distributions
9 Functions of Random Variables and Monte Carlo Simulation
The probability distributions of virtually any function of random variables can be computed using
the powerful method of Monte Carlo Simulation (MCS). MCS involves computing values of functions
with large samples of random variables.
For example, consider a function of three random variables, X1 , X2 , and X3 , where X1 is normally
distributed with mean of 6 and standard deviation of 2, X2 is log-normally distributed with median
of 2 and coefficient of variation of 0.3, and X3 is Rayleigh distributed with mode of 1. The function
Y = sin(X1 ) +
p
X2 − exp(−X3 ) − 2
is a function of these three random variables and is therefore also random. The distribution function
and statistics of Y may be difficult to derive analytically, especially if the function Y = g(X) is
complicated. This is where MCS is powerful. Given samples of N values of X1 , X2 and X3 , a
sample of N values of Y can also be computed. The statistics of Y (mean, variance, PDF, and
CDF) can be estimated by computing the average value, sample variance, histogram, and emperical
CDF of the sample of Y . The probability P [Y > 0] can be estimated by counting the number of
positive values in the sample and dividing by N . The Matlab command P_Y_gt_0 = sum(y>0)/N
may be used to estimate this probability.
10 Monte Carlo Simulation in Matlab
1
2
3
4
5
6
%
%
%
%
%
%
MCS intro .m
Monte Car lo S i m u l a t i o n . . . an i n t r o d u c t o r y example
Y = g (X1 , X2 , X3) = s i n (X1) + s q r t (X2) − exp(−X3) − 2 ;
H. P . Gavin , Dept . C i v i l and Environmental E n g i n e e r i n g , Duke Univ , Jan . 2012
7
8
9
%
%
10
11
X1
normal
mu1 = 6;
sd1 = 2;
X2
lognormal
med2 = 2;
cv2 = 0.3;
X3
Rayleigh
mod3 = 2;
12
13
N_MCS = 1000;
% number o f random v a l u e s i n t h e sample
14
15
% ( 1 ) g e n e r a t e a l a r g e sample f o r each random v a r i a b l e i n t h e problem . . .
16
17
18
19
X1 = mu1 + sd1 *randn(1 , N_MCS );
X2 = logn_rnd ( med2 , cv2 ,1 , N_MCS );
X3 = Rayleigh_rnd ( mod3 ,1 , N_MCS );
20
21
% ( 2 ) e v a l u a t e t h e f u n c t i o n f o r each random v a r i a b l e t o compute a new sample
22
23
Y = s i n ( X1 ) + sqrt ( X2 ) - exp( - X3 ) - 2;
24
25
% s u p p o s e ” p r o b a b i l i t y o f f a i l u r e ” i s Prob [ g (X1 , X2 , X3) > 0 ]
...
26
27
P r o b a b i l i t y _ o f _ f a i l u r e = sum(Y >0) / N_MCS
28
29
% ( 3 ) p l o t h i s t o g r a m s o f t h e random v a r i a b l e s
30
31
32
33
34
sort_X1
sort_X2
sort_X3
CDF
=
=
=
=
sort ( X1 );
sort ( X2 );
sort ( X3 );
([1: N_MCS ] -0.5) / N_MCS ;
% e m p i r i c a l CDF o f a l l
quantities
CC BY-NC-ND PSH, HPG, JTS
P.D.F.
14 CEE 201L. Uncertainty, Design, and Optimization – Duke University – Spring 2016 – P.S.H., H.P.G. and J.T.S.
0.25
1
0.2
0.8
0.15
0.6
0.1
0.4
0.05
0.2
0
0
C.D.F.
0 2 4 6 8 101214
P.D.F.
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
0 1 2 3 4 5 6
0 1 2 3 4 5 6 7 8
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0
0
0
0 2 4 6 8 101214
0 1 2 3 4 5 6
0 1 2 3 4 5 6 7 8
X1 : normal
X2 : log-normal
X3 : Rayleigh
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-2.5
PF
-2
-1.5
-1
-0.5
0
0.5
1
1
C.D.F.
0.8
0.6
Y>0
0.4
0.2
0
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
Y = g(X1,X2,X3)
Figure 3. Analytical and empirical PDF’s and CDF’s for X1 , X2 , and X3 , and the Empirical PDF and
CDF for Y = g(X1 , X2 , X3 )
CC BY-NC-ND PSH, HPG, JTS
15
Probability Distributions
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
nBins = f l o o r ( N_MCS /20);
f i g u r e (1)
clf
subplot (231)
[ fx , xx ] = h i s t ( X1 , nBins );
% h i s t o g r a m o f X1
fx = fx / N_MCS * nBins /(max( X1 ) -min( X1 ));
% s c a l e histogram
hold on
plot ( sort_X1 , normpdf ( sort_X1 , mu1 , sd1 ) , ’ -k ’ , ’ LineWidth ’ ,3);
s t a i r s ( xx , fx , ’ LineWidth ’ ,1)
hold off
ylabel ( ’P . D . F . ’)
subplot (234)
hold on
plot ( sort_X1 , normcdf ( sort_X1 , mu1 , sd1 ) , ’ -k ’ , ’ LineWidth ’ ,3);
s t a i r s ( sort_X1 , CDF , ’ LineWidth ’ ,1)
hold off
ylabel ( ’C . D . F . ’)
xlabel ( ’ X_1 : normal ’)
subplot (232)
[ fx , xx ] = h i s t ( X2 , nBins );
% h i s t o g r a m o f X2
fx = fx / N_MCS * nBins /(max( X2 ) -min( X2 ));
% s c a l e histogram
hold on
plot ( sort_X2 , logn_pdf ( sort_X2 , med2 , cv2 ) , ’ -k ’ , ’ LineWidth ’ ,3);
s t a i r s ( xx , fx , ’ LineWidth ’ ,1)
hold off
subplot (235)
hold on
plot ( sort_X2 , logn_cdf ( sort_X2 ,[ med2 , cv2 ]) , ’ -k ’ , ’ LineWidth ’ ,3);
s t a i r s ( sort_X2 , CDF , ’ LineWidth ’ ,1)
hold off
xlabel ( ’ X_2 : log - normal ’)
subplot (233)
[ fx , xx ] = h i s t ( X3 , nBins );
% h i s t o g r a m o f X3
fx = fx / N_MCS * nBins /(max( X3 ) -min( X3 ));
% s c a l e histogram
hold on
plot ( sort_X3 , Rayleigh_pdf ( sort_X3 , mod3 ) , ’ -k ’ , ’ LineWidth ’ ,3);
s t a i r s ( xx , fx , ’ LineWidth ’ ,1)
hold off
subplot (236)
hold on
plot ( sort_X3 , Rayleigh_cdf ( sort_X3 , mod3 ) , ’ -k ’ , ’ LineWidth ’ ,2);
s t a i r s ( sort_X3 , CDF , ’ LineWidth ’ ,1)
hold off
xlabel ( ’ X_3 : Rayleigh ’)
f i g u r e (2)
clf
subplot (211)
[ fx , xx ] = h i s t ( Y , nBins );
% histogram of Y
fx = fx / N_MCS * nBins /(max( Y ) -min( Y ));
% s c a l e histogram
hold on
s t a i r s ( xx , fx , ’ LineWidth ’ ,2)
plot ([0 0] ,[0 0.5] , ’ -k ’ );
hold off
text (0.1 ,0.1 , ’ P_F ’)
ylabel ( ’P . D . F . ’)
subplot (212)
hold on
s t a i r s ( sort ( Y ) , CDF , ’ LineWidth ’ ,2)
plot ([0 0] ,[0 1] , ’ -k ’ );
hold off
text (0.5 ,0.5 , ’Y >0 ’)
ylabel ( ’C . D . F . ’)
xlabel ( ’Y = g ( X_1 , X_2 , X_3 ) ’ );
t o PDF
t o PDF
t o PDF
t o PDF
CC BY-NC-ND PSH, HPG, JTS