Download 14. Transformation of Variables Let X be a continuous random

PROBABILITY MODELS
63
14. Transformation of Variables
Let X be a continuous random variable with density f and suppose g is an increasing,
continuously differentiable function. Put Y = g(X). What is the distribution of Y? We let
FX , fX be the distribution function and density of X, respectively, y ∈ R, and we define
h(y) := g−1 (y). Then
FY (y) = P{Y ≤ y} = P{g(X) ≤ y} = P{X ≤ h(y)} = FX (h(y))
and
fY (y) =
d
d
FY (y) =
FX (h(y)) = h0 (y) fX (h(y)).
dy
dy
14.1. Change of variables formula for densities. Suppose g is monotone and differentiable
and X is a continuous random variable with density fX . Then Y = g(X) has density
fY (y) = fX (h(y))|h0 (y)|,
y ∈ R.
Some examples are given below.
• Let X has density fX and Y = aX + b, a, b constant, a , 0. Then g(y) = ay + b and
h(y) = (y − b)/a, h0 (y) = 1/a. Then Y has density
!
y−b 1
fY (y) = fX
.
a
|a|
Therefore, suppose X ∼ Gamma(r, 1) and λ > 0, then what is the density of X/λ?
fX/λ (y) = λ(λy)r−1 e−λy /Γ(r);
X/λ has the Gamma(r, λ) density.
• Suppose U ∼ Unif(0, 1) and Y = − log U. Then X, Y ≥ 0 and g(u) = − log u, h(y) = e−y ,
and h0 (y) = −e−y . It follows that
(
1 × e−y ,
y≥0
0
fY (y) = fX (h(y))|h (y)| =
0,
otherwise.
Therefore, Y ∼ Exp(1).
√
√
• Suppose X ∼ Exp(λ) and Y = X. Then g(x) = x, h(y) = y2 and h0 (y) = 2y. Thus,
(
2
2λye−λy ,
y≥0
fY (y) = fX (x)|dx/dy| =
0,
otherwise
and Y has the Rayleigh distribution with parameter 2λ.
Example 14.1 (Cauchy distribution). Suppose a laser pointer is mounted to a rotating disk whose
center is 1 unit away from a wall of infinite length. The disk spins repeatedly and (when pointed at
the wall) it turns on. What is the distribution of marked spots on the wall?
Let X be the angle of the laser with respect to the line perpendicular to the wall so that Y = tan(X)
and X ∼ Unif(−π/2, π/2). Since Y is a smooth and increasing function, the change of variables
formula applies. We have fX (x) = 1/π on −π/2 ≤ x ≤ π/2 and g(x) = tan x, h(y) = arctan(y),
and h0 (y) = 1/(1 + y)2 . Y has density
fY (y) = fX (h(y))|h0 (y)| =
This is the (standard) Cauchy distribution.
1 1
,
π 1 + y2
−∞ < y < ∞.
64
HARRY CRANE
The Cauchy distribution has very heavy tails:
Z ∞
Z ∞
dy
1
1
P{Y ≥ x} =
dy ≈
=
,
2
2
πx
x π(1 + y )
x πy
for large values of x. Moreover, EY does not exist:
Z ∞
y
+
dy
EY =
2
0 π(1 + y )
Z ∞
du
=
1 2πu
= ∞
= EY− .
14.2. Many-to-one transformations. Suppose X ∼ N(0, 1) and we put Y = X2 . The distribution of Y appears throughout statistics and is called the χ2 -distribution (pronounced
chi-square distribution). The distribution function of Y is
FY (y) = P{Y ≤ y}
= P{X2 ≤ y}
√
√
= P{− y ≤ X ≤ y}
√
√
= FX ( y) − FX (− y)
√
= 2Φ( y) − 1.
The density
√
2
1
e−y/2 I(0,∞) (y)
√ φ( y) = p
2 y
2πy
coincides with density of the Gamma distribution with parameter (1/2, 1/2).
fY (y) = F0Y (y) =
14.3. Simulation. Suppose F is a monotonically increasing distribution function of X. Then
Z := F(X) has the standard Uniform distribution:
P{Z ≤ u} = P{F(X) ≤ u} = P{X ≤ F−1 (u)} = F(F−1 (u)) = u.
Also, if U ∼ Unif(0, 1), then X = F−1 (U) has distribution function F:
P{X ≤ x} = P{F−1 (U) ≤ x} = P{U ≤ F(x)} = F(x).
These facts are important in statistical applications to hypothesis testing and simulation.
For example, suppose X is a discrete random variable with mass function
x
1
2
3
4
pX (x) 0.1 0.3 0.3 0.4
Given U ∼ Unif(0, 1), we simulate X from U by defining G : [0, 1] → {1, 2, 3, 4} as


1, 0 < u ≤ 0, 1




 2, 0.1 < u ≤ 0.4
G(u) := 

3, 0.4 < u ≤ 0.6



 4, 0.6 < u ≤ 1.
For example, P{G(U) = 3} = P{0.4 < U ≤ 0.6} = 0.2 = pX (3).
Exercise 14.2. Describe the general procedure for simulating discrete random variables and convince
yourself that it gives the right distribution.