Download Theory of Probability - Brett Bernstein Lecture 14

Theory of Probability - Brett Bernstein
Lecture 14
Uniform and Exponential Review Exercises
1. Let X be a continuous random variable with CDF FX and assume that FX is strictly
increasing. Let g(x) = FX (x). What is the distribution of g(X)?
2. The median of a random variable is a value x for which
P (X ≥ x) ≥ 1/2 and P (X ≤ x) ≥ 1/2.
For absolutely continuous random variables, this simplifies to a value x for which
P (X ≤ x) = 1/2. Compute the median of X when
(a) X ∼ Unif(a, b),
(b) X ∼ Exp(λ).
3. Prove that the exponential distribution has the memoryless property:
Theorem 1 (Memoryless). If X ∼ Exp(λ) then
P (X > s + t|X > s) = P (X > t)
for any s, t ≥ 0.
4. A random variable X has a Weibull distribution with parameters a > 0 and b > 0 if
X has PDF fX with
b b−1 −(x/a)b
x e
for x > 0,
ab
fX (x) =
0
for x ≤ 0.
Show that if X b has an exponential distribution, and determine the parameter.
Solutions
1. Let Y = g(X) so that, for y ∈ (0, 1),
FY (y) = P (Y ≤ y) = P (FX (X) ≤ y) = P (X ≤ FX−1 (y)) = FX (FX−1 (y)) = y.
Thus Y ∼ Unif(0, 1). This is an important result in simulation, since it shows that if
Y ∼ Unif(0, 1) and F is some CDF then F −1 (Y ) is distributed according to F . We
emphasize this in a theorem:
Theorem 2. Let F be a strictly increasing CDF of some distribution, and let X ∼
Unif(0, 1). Then Y = F −1 (X) has CDF F .
1
This is true more generally than strictly increasing CDFs, but we wont cover that.
2. (a) We must find m such that
1
=
2
Z
m
fX (x) dx.
−∞
Looking for m in (a, b) we have
Z m
1
m−a
b+a
fX (x) dx =
=
⇐⇒ m =
.
2
b−a
2
a
(b) Looking for m ≥ 0 we compute
Z m
1
log(2)
1
λe−λx dx = 1−e−λm ⇐⇒ e−λm =
=
⇐⇒ λm = log(2) ⇐⇒ m =
.
2
2
λ
0
3. We compute
P (X > s + t|X > s)
=
=
P (X > s + t)
P (X > s)
−λ(s+t)
e
e−λ(s)
=
e−λt
=
P (X > t),
where we used that FX (x) = 1 − e−λx for x ≥ 0 above several times.
4. Let Y = X b and g(y) = y b giving, for y > 0,
b
by (b−1)/b e−(y1/b /a)b
1
1
e−y/a
1/b fY (y) = fX (g (y)) 0 −1
= fX (y ) (b−1)/b =
=
,
g (g (y)) by
ab by (b−1)/b
ab
−1
the PDF for a Exp(a−b ) random variable.
Normal Distribution
A variable X has a normal (or Gaussian) distribution with parameters µ ∈ R and σ 2 > 0 if
it has PDF fX given by
(x−µ)2
1
e− 2σ2 .
fX (x) = √
2πσ 2
We write X ∼ N (µ, σ 2 ). We will see that
E[X] = µ and
2
Var[X] = σ 2 .
When µ = 0 and σ 2 = 1 we call it the standard normal distribution, with PDF
1
2
fX (x) = √ e−(x−µ) /2 .
2π
The normal distribution is the most important distribution for reasons we will see later. For
now
√ I want to highlight some of the interesting features of the PDF. Firstly, where does the
2π come from? Kind of amazing that π would play a role here. To see this we employ a
famous trick that oddly involves squaring the quantity we wish to compute:
Z ∞
2
Z ∞
Z ∞
2
−x2 /2
−x2 /2
e
dx
=
e
dx
e−y /2 dy
−∞
−∞
Z−∞
∞ Z ∞
2
2
e−(x +y )/2 dx dy
=
−∞
Z−∞
2π Z ∞
2
=
e−r /2 r dr dθ
0 Z 0
∞
2
e−r /2 r dr
= 2π
0
=
2π[−e−u ]∞
0
=
2π.
Taking square-roots we get our result for the standard normal. The second point is to
understand how the standard normal relates to the general normal. As you will prove, we
have the following result.
Theorem 3. Let X ∼ N (0, 1) and let Y = aX + b where a 6= 0. Then Y ∼ N (b, a2 ).
The standard normal PDF has a bell shape centered at 0 with rapidly decaying tails. To
get the general normal PDF we simply shift and scale it appropriately. We can also use this
backwards:
Corollary 4 (Standardization). If Y ∼ N (µ, σ 2 ) then
Y −µ
σ
∼ N (0, 1).
Due to this corollary, if you only know the standard normal CDF (often denoted Φ), you
can figure out the general normal CDF.
Example 5. Suppose X ∼ N (µ, σ 2 ). What is P (X ≤ x)? Note that
x−µ
x−µ
X −µ
≤
=Φ
.
P (X ≤ x) = P
σ
σ
σ
The reason the normal distribution is so important is the central limit theorem, which we
will prove later. It basically says that sums of i.i.d. random variables look normal, regardless
of the distribution (as long as it has a mean and variance).
Here are some other properties of the normal distribution. Assume X ∼ N (µ, σ 2 ).
3
1. P (|X − µ| < σ) ≈ .68, P (|X − µ| < 2σ) ≈ .95, and P (|X − µ| < 3σ) ≈ .997.
2. The normal distribution is symmetric about its mean (i.e., the PDF has fX (µ − x) =
fX (µ + x)).
Normal Exercises
1. An expert witness in a paternity suit testifies that gestation time (in days) is approximately normally distributed with µ = 270 and σ 2 = 100.
(a) If the defendant was outside of the country starting 290 days and ending 240 days
before the child’s birth, what is the probability he is the father?
(b) What is the probability a child will gestate for more than 300 days?
2. Show that if X ∼ N (0, 1) and Y = aX + b with a 6= 0 then Y ∼ N (b, a2 ).
3. Compute E[X] and Var[X] for a standard normal random variable. What about for a
general normal random variable (use previous part)?
4. Let f (x) = ax + b where a 6= 0.
(a) If X ∼ Unif(c, d) is f (X) uniformly disributed? If yes, what are its parameters?
(b) If X ∼ Exp(λ) what are E[f (X)] and Var[f (X)]? Is f (X) exponentially distributed? If yes, what is λ?
Solutions
1. (a) The probability is approximately 1 − .95 − .0235 = 1 − .9735 = .0265.
(b) The probability is approximately .0015.
2. We have
fY (y) = fX
2
1
= √ 1 e− 12 ( y−b
a ) .
a
2πa2
y−b
a
3. Let X ∼ N (0, 1). Then we have
1
E[X] = √
2π
Z
∞
xe−x
2 /2
dx = 0
−∞
since the integrand is odd. Continuing, we have
Z ∞
1
2
2
E[X ] = √
x2 e−x /2 dx
2π −∞
∞
Z ∞
1
1
2
−x2 /2
=
− √ xe
−√
−e−x /2 dx
2π
2π −∞
−∞
=
1.
4
Since E[X] = 0 we see Var[X] = E[X 2 ] = 1 and we are done. By the previous part,
we see that if Y ∼ N (µ, σ 2 ) then E[Y ] = µ and Var[Y ] = σ 2 .
4. (a) As usual, we compute the PDF. Letting Y = f (X) we have
y−b
y − b 1 1[c,d] a
fY (y) = fX
a = |a|(d − c) .
a
If a > 0 we have
c≤
y−b
≤ d ⇐⇒ ac + b ≤ y ≤ ad + b.
a
c≤
y−b
≤ d ⇐⇒ ad + b ≤ y ≤ ac + b.
a
If a < 0 we have
In either case, it is a uniform distribution.
(b) Firstly, we have
E[aX + b] = aE[X] + b =
a
+ b and
λ
Var(aX + b) = a2 Var(X) =
a2
.
λ2
Note that if b 6= 0 or a ≤ 0 the distribution cannot be exponential (since an
exponential must be positive, and has positive probability of being close to zero).
If b = 0 and a > 0 we have, for y > 0,
y−b 1
λ
fY (y) = fX
= e−λy/a ,
a
a
a
which is the PDF of an Exp(λ/a) distribution.
Cauchy and Lognormal Distributions
We say that X has a Cauchy distribution if its PDF has the form
fX (x) =
C
.
1 + x2
We will learn more about it in the exercises. We say that Y has a Lognormal distribution
when Y = eX and X is normal. This is an important distribution in finance, as it is used to
model stock fluctuations. The idea is to model percentage moves in a stock with a normal
distribution, since the size of a move should depend on the price of the stock.
5