Download An Intuitive Interpretation of Ratios of Moment Generating Functions

An Intuitive Interpretation of Ratios of Moment Generating Functions Via One-Step Conditioning
Donghui Chen AND Kenneth S. Berenhaut, Department of Mathematics, Wake Forest University
Motivation
Motivation
This paper studies a measure of the quality of local approximations to distributions for sums of independent and identically distributed (i.i.d.) random
variables. In particular, suppose X is a random variable satisfying E(X) = µ
and V ar(X) = σ 2 < ∞ and suppose that X1, X2, . . . is a sequence of i.i.d.
random variables each with the same distribution as X. In addition, suppose that
X has probability function p, where p is either a density or a probability mass
function, and denote the support set of X by SX = {x : p(x) > 0}. Let p(i)
be the i-fold convolution of p ( i.e. the probability function for the partial sum
Si = X1 + X2 + · · · + Xi).
⋆ One step conditioning and independence lead to
p(n)(x) = EX (p(n−1)(x − X)).
1
φnµ,nσ2 (x) = √
√ e
2πσ n
, −∞ < x < ∞
(1)
Our present consideration is motivated by the following two well-known facts:
⋆ There are many well-known local limit theorems for random variables where the
associated attracting distribution is the normal. Two such results are the following
two theorems.
Theorem 1. Suppose X takes values of the form b + N h, N = 0, ±1, ±2, ..., with
probability 1, where b is a constant and h > 0 is the maximal span of the distribution. Then
√ (n)
sup σ n p (nb + N h) − hφnµ,nσ2 (nb + N h) → 0.
N
Theorem 2. Suppose p is a density, and that there exists an N such that pN (x) is
bounded. Then
√ (n)
sup σ n p (x) − φnµ,nσ2 (x) → 0.
X
Definition 1. For any two functions f and g, define the ”convolution ratio”, Hf,g
of f and g with respect to X via
for all x ∈ R such that g(x) 6= 0.
LX,N (t) =
mX
mN
= e
(t2 −p2 )
2p(p−1)
= RX,N
− pe
(t2 −p2 )
2p(p−1)
t−p
p(1 − p)
+ pe
(t−p)(t+p−2)
2p(p−1)
For simplicity, we will write L(t, p) to represent LX,N (t). We have the following
theorem:
Theorem 5. Suppose X is a Bernoulli random variable with parameter p. Then,
for all 0 < t < 1,
1 1
− 32
2
LX,N (t) = L(t, p) ≥ L(t, 1/2) ≥ L(0, 1/2) = (e + e ) ≈ 0.9359.
2
5
0.5
4
0.4
3
0.3
2
0.2
1
0.1
0
0.75
0.5
p
Figure 2:
0.25
0.5
0.25
0.0
t
1.0
0.75
For n > 1, set
where p̃ = (p̃i)i≥1 is a sequence of real valued functions such that p̃i(x) approximates p(i)(x) for each x.
For any random variable Y , let mY be the moment generating function of Y defined
via
mY (t) = E(etY )
for −∞ < t < ∞. As well, for random variables Y1 and Y2, define RY1,Y2 via
P∞
mY1 (t)
ti
k
(i)
RY1,Y2 (t) = mY (t) . Also, let KY = log(mY (t)) =
i=1 Y
i! be the cumulant
2
generating function for Y .
Note that for a normal random variable, N , with density as in (1),
0.75
0.5
p
0.25
1.0
0.75
The function LX,N (left) and | log(L(X, N ))| (right) for X Bernoulli(p) with 0 < p < 1.
Theorem 3. Suppose p̃i = φnµ,nσ2 , if X is continuous (or p̃i = hφnµ,nσ2 if X is
defined on a lattice with maximal spread h). Then
2 2
tµ+ t 2σ
mN (t) = e
.
Table 2: Optimal bounds for LX,N (t) for t ∈ UX for some common
distributions
Lower bound
0.9359257154 . . .
0.9359257154 . . .
0.7450966791 . . .
0.7450966791 . . .
0.8243606355 . . .
0.3032653298 . . .
e−λ(0.1321205588... )
Upper bound
1
Unbounded
1
1
Unbounded
Unbounded
Unbounded
• Discrete Uniformly Distributed Random Variables
Suppose X takes value on B = {0, 1, . . . T −1} with constant probability P (X =
i) = T1 for i ∈ B (where we assume T > 3). The mean and the variance of X are
T 2−1
2
and
σ
=
µ = T −1
2
12 , respectively and TX = [0, T − 1]. In light of Theorem
3, we have
mX
mN
t−µ
σ2 t−µ
σ2

−1
−6T −2t+T
2

T −1 − 1
e
e


=
−1
−6 −2t+T
2
e T −1 − 1
2
2
3 (T −1) −4t
2
T 2 −1
T
Theorem 6. Suppose X is uniformly distributed on B. Then LX,N (t) is maximized at t = T −1
2 and minimized at t = 0. In addition, for all t ∈ [0, T − 1],
1 > L(t, T ) ≥ L(0, T ) =
− T6T
+1
e
6
− T +1
e
−1
−1
!
e
lim C (tn, n) =
n→∞ X,p̃
mN
(3)
provided that the numerator on the right hand side exists. We will refer to the right
hand side of (3) as LX,N (t).
t−µ
Proof. Suppose that t is fixed and mX σ2 < ∞. Then,
E(p̃n−1(tn − X))
=E
p̃n(tn)
CX,p̃(tn, n) = E
√
p̃n−1(tn − X)
.
p̃n(tn)
3 T −1
2 T +1
T
.
Note that limT →∞ L(0, T ) = (−1/6 e−6 + 1/6)e3/2 ≈ 0.7451.
• Continuous Uniformly Distributed Random Variables
Suppose that X possesses a uniform distribution on the interval [a, b]. The mean
(b−a)2
b−a
2
and the variance of X are µ = 2 and σ = 12 , respectively and TX = [a, b].
tb −eta
The moment generating function of X is given by mX (t) = et(b−a)
. Note that
Theorem 4 reduces our consideration through scaling to (a, b) = (0, 1). In light
2 2
(t−X)2
− t −µ2 − 2(n−1)σ2
2σ
n (t−µ)X
√
e σ2 e
n−1
(4)
e
!
2
√
(t−µ)X
t2 −µ2 − (t−X)
−
√ n e σ2 e 2σ2 e 2(n−1)σ2 .
n−1
for n > 1. Now, set Yn =
Since for any n > 1, we have
0 < Yn < 2e
(t−µ)X
σ2
2 2
− t −µ2
e
P
Yn → e
and
2σ
(t−µ)X
σ2
2 2
− t −µ2
e
2σ
,
Lebesgue’s Dominated Convergence Theorem, implies that
lim E e
(t−µ)X
σ2
2 2
− t −µ2
2σ
e
where we have used the fact that mN
X
Bernoulli(1/2)
Bernoulli(p)
Uniform on {0, 1, . . . , T − 1}
Uniform on [a, b]
Exponential(λ)
Geometric(p)
Poisson(λ)
t−µ
σ2 t−µ ,
σ2
mX
n→∞
For simplicity, here we will write L(t, T ) to represent LX,N (t).
0.5
0.25
0.0
t
Corollary 1. Suppose that µ = 0 and σ 2 = 1 and p̃i is as in the statement of
Theorem 3. Then
Simplifying (4) gives
E(p̃n−1(x − X))
CX,p̃(x, n) = Hp̃ ,p̃ (x) =
n−1 n
p̃n(x)
X
LX,N (t) =
0.0
Among our observations is the following theorem for when p̃ is taken as suggested
by Theorems 1 and 2.
CX,p̃(tn, n) =
Example of Uniformly Distributed Random Variables
Suppose X is a Bernoulli random variable with parameter p, i.e. P (X = 1) = p
and P (X = 0) = 1 − p with 0 < p < 1. Then µ = p, σ 2 = p(1 − p) and TX = [0, 1].
2
2
Thus,
we
are
interested
in
bounding
the
ratio
R(v)
for
v
∈
[(0−µ)/σ
,
(1−µ)/σ
]=
i
h
−1 1
1−p , p .
In light of Theorem 3, we have
t−µ
σ2 t−µ
σ2
E(f (x − X))
g(x)
X
(x) =
Hf,g
x
Example for Bernoulli Random Variables
Corollary of Theorem 3 and Conclusion
Essentially, of interest here is the nature of the relationship in (2) when p(n) and
p(n−1) are replaced by approximating functions.
Let φnµ,nσ2 be the density for a normal random variable, N , with mean nµ and
variance nσ 2, i.e.
2
x−nµ
1
√
−2
nσ
(2)
Main Theorem and Its Proof
t−µ
σ2
=
=e
mX
mN
t−µ
σ2 t−µ ,
σ2
(t−µ)2
t−µ
µ+
σ2
2σ 2
=e
t2 −µ2
2σ 2
.
Examples of Distributions with Unbounded Support
• Exponential random variables
Suppose X is an exponential random variable with parameter λ > 0, i.e. P (X =
x) = λe−λx for xT> 0. Then µ = 1/λ, σ 2 = 1/λ2, mX (t) = λ/(λ − t) for t < λ
and TX = [0, ∞] {t : (t − λ−1)/λ−2} = {t : 0 ≤ t < 2/λ}.
In light of Theorem 3, we have
LX,N (t) =
mX
mN
= −
t−1/λ
1/λ2
t−1/λ
1/λ2
− 12 (tλ−1)(tλ+1)
e
tλ − 2
,
Theorem 8. Suppose X is an exponential random variable with parameter λ.
Then, for all 0 < t < 2/λ,
1 1
LX,N (t) = L(t, λ) ≥ L(0, λ) = e 2 ≈ 0.8244.
2
• Poisson Random Variables
Suppose X is a Poisson random variable with parameter λ > 0, i.e. P (X =
t
i) = e−λλi/i! for i = 0, 1, 2, . . . . Then µ = λ, σ 2 = λ, mX (t) = eλ(e −1) and
TX = [0, ∞).
In light of Theorem 3, we have
mX t−λ
λ
t−λ
mN λ
−t+λ
−
λ −t2 λ−1
1/2 −λ2 +2λ2 e
= e
,
Theorem 9. Suppose X is an Poisson random variable with parameter λ. Then,
for all 0 < t < ∞,
LX,N (t) = L(t, λ) ≥ L(0, λ) = e
t2
2
where Z possesses a standard normal distribution, and hence mZ (t) = e .
The next theorem regarding changes of scale and origin follows directly from (3).
Theorem 4. Suppose a, b ∈ R and a 6= 0, then
t−b
LaX+b,N (t) = LX,N
.
a
Remark. Note that ratios as on the right side of (5) occur when considering
Cornish-Fisher expansions.
Remark. The value limn→∞ CX,p̃(tn, n) in (5) and (3) addresses the behavior of
the approximation to the probability function for X̄n = Sn/n, in the limit, near
the value t. Employing (3), we have that RX,N (t) gives information on the local
behavior of the approximation near the value µ + tσ 2. In light of Theorem 3, and
since the probability function of X̄n is zero outside the range of X, we will be
primarily concerned with values of LX,N (t) for t ∈ TX , where
\
t−µ
TX = [m, M ]
<∞ ,
t : mX
σ2
where m = inf(SX ) and M = sup(SX ).
As per the title for this paper, via Theorem 3, we can interpret the ratio of moment
mX (t)
as giving information on the quality of the
generating functions RX,N (t) = m
N (t)
normal approximation for X n locally near the value µ + σ 2t. We will be interested
def
in values of RX,N (t) for t ∈ Q = ((m − µ)/σ 2 , (M − µ)/σ 2). Note that Q is an
interval containing zero.
References
1. Chow, Y. S. AND Teicher, H. Probability Theory, Second Edition, SpringerVerlag, New York, 1988.
2. Billingsley, P. Probability and Measure, Third edition, John Wiley Inc., New
York, 1995.
3. Petrov, V. V. Sums of Independent Random Variables, Springer, Berlin, 1975.
5. Petrov, V. V. A Local Theorem for Densities of Sums of Independent Random
Variables, Theory of Probability and Its Application, Vol 1 Issue 3, pp.316-322,
1956.
6. Sirazhdinov, S. KH. AND Mamatov, M. Short Communication on Convergence
in the Mean for Densities, Theory of Probability and Its Application, Vol 7, Issue
4, pp.424-428, 1962.
7. Gnedenko, B. V. A Local Limit Theorem for Densities, Dokl. Akad. Nauk
SSSR.,95 pp.5-7, 1954.
8. Gnedenko, B. V. AND Kolmogorov A. N. Limit Distributions for Sums of Independent Variables, Addison-Wesley, Boston, 1954.
9. Prokhorov, Y. V. A Local Theorem for Densities, Dokl. Akad. Nauk SSSR., 83
pp.797-800,1952.
10. Feller, W. Probability Theory and Its Applications, Vol. 2, Wiley, New York,
1966.
11. Wallace, D. L. Asymptotic Approximatio to Distributions, The Annals of Mathematics Statistics, Vol.29, No.3, pp.635-654, 1958.
for t > 0. For simplicity, we will write L(t, λ) to represent LX,N (t).
−λ e−2
2e
(5)
4. Petrov, V. V. On Local Limit Theorems for Sums of Independent Random Variables, Theory of Probability and Its Application, Vol. 9, Issue 2, pp.312-320,
1962.
for t < 2/λ. For simplicity, we will write L(t, λ) to represent LX,N (t).
LX,N (t) =
mX (t)
lim C (tn, n) =
n→∞ X,p̃
mZ (t)
≈ e−.1321λ.
12. Cornish, E. A. AND Fisher, R. A. Moments and Cumulants in the Specification
of Distributions, Review of the International Statistical Institute, Vol.5, No.4,
pp.307-320, 1938.
13. Mitrinovic, D. S. Analytic Inequalities, Springer-Verlag, New York, 1970.