Download ON THE CONVERGENCE OF SERIES OF INDEPENDENT RANDOM VARIABLES Introduction

Publicationes Mathematicae 4 (1956), pp. 410–417.
ON THE CONVERGENCE OF SERIES
OF INDEPENDENT RANDOM VARIABLES
To the memory of Professor Tibor Szele
András Prékopa (Budapest)
Received: October 24, 1955
Introduction
Let ξ1 , ξ2 , . . . denote a sequence of independent random variables. In the present paper
two theorems concerning the convergence (regardless of the order of summation) of the
series
∞
ξk
(1)
k=1
are proved.
Throughout the paper I denote by J the set of all finite subsets and by S the set of
all subsets of the set of natural numbers. Let
ξk
(2)
ξ(A) =
k∈A
provided the series on the right converges with probability 1 regardless of the order of
summation. Let F (x, A) denote the distribution function and f (t, A) the characteristic
function of the random variable ξ(A).
Let 0 < λ < 1, and let ξ denote a random variable. We say that Q(λ) is a λ-quantile
of ξ if P (ξ ≤ Q(λ)) ≥ λ and P (ξ ≥ Q(λ)) ≥ 1 − λ. An arbitrary λ-quantile of the random
variable ξ(A) will be denoted by Q(λ, A).
In the literature many necessary and sufficient conditions are given concerning the
convergence of the series (1). These conditions are expressed generally in terms of mean
values, dispersions and characteristic functions. In what follows such conditions are given
in terms of the compactness of certain sets of distribution functions and of the quantiles.
1
§ 1. Preliminary lemmas
In this § three lemmas are proved. The first two contain some assertions relative to the
set of random variables {ξz , z ∈ Z}, where Z is an arbitrary given set. Let F (x, z) denote
the distribution function and f (t, z) the characteristic function of the random variable ξz .
P. Lévi has introduced the notion of distance between two distribution functions. The
distance L(F1 , F2 ) between F1 (x) and F2 (x) is defined as the lower bound of the values h,
for which
(3)
F1 (x − h) − h ≤ F2 (x) ≤ F1 (x + h) + h.
It is known that the axioms of the metric space are fulfilled relative to the distance L:
a) L(F1 , F2 ) = 0 if and only if F1 (x) ≡ F2 (x);
b) L(F1 , F2 ) = L(F2 , F1 );
c) L(F1 , F3 ) ≤ L(F1 , F2 ) + L(F2 , F3 ).
Let us consider the set F of all one dimensional distribution functions. From what has
been said it follows that F is a metric space relative to the distance L. According to (3), F
is bounded and by Theorem 2 of [2], p. 42. it is also complete. In Theorem 1 of [2], p. 38.
it is proved, that the relation L(Fn , F ) → 0 holds if and only if Fn (x) → F (x) at every
continuity point of F (x). From this fact it is easy to see that the space F is not compact.
For instance sequences of distribution functions can be given, which converge in every
point to 0. The first two lemmas give answers to the question: under what conditions is
a subset F = {F (x, z), z ∈ Z} of the space F compact?
To fixed values of λ, and z there corresponds generally not only one Q(λ, z), therefore
the set {Q(λ, z), z ∈ Z} is generally not uniquely determined.
Lemma 1 If to every λ, for which 0 < λ < 1, the quantiles Q(λ, z) can be chosen in
such a way that |Q(λ, z)| < K(λ) for z ∈ Z, where K(λ) is independent of z, then the set
F is compact.
Conversely, if the set F is compact, then to every λ for which 0 < λ < 1, there corresponds a K(λ) such that |Q(λ, z)| < K(λ) for every λ-quantile Q(λ, z) of the variable ξz .
Proof of the first part. Let ε > 0, 0 < λ < ε and x < −K(λ). From our
hypothesis it follows that
F (x, z) ≤ F (−K(λ), z) = P (ξz < −K(λ)) ≤ P (ξz < Q(λ, z)) ≤ λ < ε.
On the other hand if λ > 1 − ε and x > K(λ), then we obtain
F (x, z) ≥ F (K(λ), z) = P (ξz < K(λ)) ≥ P (ξz ≤ Q(λ, z)) ≥ λ > 1 − ε.
It follows that
F (x, z) → 0 if x → −∞,
F (x, z) → 1 if x → +∞
2
uniformly in z. According to Helly’s theorem we can choose a sequence F (x, zk ) which
converges to a non-decreasing, left continuous function F (x) at every point of continuity
of the latter. By the preceding relation F (x) must be a distribution function.
Proof of the second part. Obviously the relations
F (x, z) → 0 if x → −∞,
F (x, z) → 1 if x → +∞
hold uniformly in z. If 0 < ε < 1, then there is a positive number K(ε) such that
K(ε)
,
2
K(ε)
.
x≥
2
F (x, z) < ε if
x≤−
1 − F (x, z) < ε if
In this case
−K(ε) < Q(ε, z) < K(ε) for z ∈ Z,
as was to be proved.
Lemma 2 The set F is compact if and only if the characteristic functions f (t, z),
z ∈ Z are equicontinuous at t = 0, i.e. if to every ε > 0 there corresponds a δ such that
|1 − f (t, z)| < ε
if
|t| < δ.
(4)
Proof of the first part. Suppose that for a fixed pair ε, δ of positive numbers the
relation (4) holds. In view of
1−
1
sin x
>
x
10
if
|x| ≥ 1,
we obtain
δ
∞ δ
1
1
|1 − f (t, z)| dt ≥
(1 − eitx ) dt dF (x, z)
ε>
2δ −δ
2δ −∞ −δ
∞
1
1
sin δx
sin δx
dF (x, z) ≥
dF (x, z) ≥ P |ξz | >
,
1−
1−
=
δx
δx
10
δ
−∞
|x|> 1
δ
whence
1
.
δ
To every ε there corresponds such a δ, therefore if F (x, zk ) is a sequence of F , lim F (x, zk )
F (−x, z) + 1 − F (x, z) < 10ε if x >
k→∞
= F (x) at every point of continuity of the latter, where F (x) is a left continuous function,
then F (x) must be a distribution function.
Proof of the second part. If ε > 0, then there is a number K such that
ε
P (|ξz | > K) < .
4
3
Let δ =
ε
. Then for |t| < δ we have
2K
∞
itx
|1 − f (t, z)| = (1 − e ) dF (x, z)
−∞
∞
|1 − eitx | dF (x, z)
≤
−∞
ε
|x| dF (x, z) + 2P (|ξz | > K) < |t|K + < ε.
≤ |t|
2
|x|≤K
Lemma 3 Let η1 , η2 , . . . denote a sequence of random variables for which
P lim ηn = ∞ = 1.
n→∞
Then if 0 < λ < 1 and Q(λ, n) is a λ-quantile of ηn , we have
lim Q(λ, n) = ∞.
n→∞
Proof. Let Ωmn denote the event that
ηn > m.
Since
lim P (Ωmn ) = 1,
n→∞
there exists a number N such that if n > N ,
P (Ωmn ) > 1 − λ.
But in this case
Q(λ, n) > m.
As m can be chosen arbitrarily, this implies the assertion of Lemma 3.
Corollary If the random variables ξ1 , ξ2 , . . . are non negative and there exists a number λ (0 < λ < 1) for which
Q(λ, A) ≤ K
if
A ∈ J,
where K is a constant, then the series (1) converges with probability 1.
Proof. By the zero-one law (see [3], p. 60.) either
P
∞
ξk < ∞
= 0,
k=1
or
P
∞
ξk < ∞
= 1.
k=1
But now the first case is impossible because it would imply that for every λ
lim Q(λ, {1, 2, . . . , n}) = ∞.
n→∞
4
§ 2. Necessary and sufficient conditions of the convergence
of the series (1)
In this § two theorems are proved.
Theorem 1 If the set {F (x, A), A ∈ J } is compact, the series (1) converges with
probability 1 regardless of the order of summation.
Conversely, if the series (1) converges with probability 1 regardless of the order of
summation, then the set {F (x, A), A ∈ S} is compact.
Proof of the first part. Let us consider the sequence
F (x, {1}), F (x, {1, 2}), . . . , F (x, {1, 2, . . . , n}), . . . .
Since the set {F (x, A), A ∈ J } is compact, there is an increasing subsequence nk of the
natural numbers with the property that the sequence
F (x, {1, 2, . . . , nk })
converges to a distribution function F (x) at every point of continuity of the latter. It
follows that if
∞
eitx dF (x),
f (t) =
−∞
then
lim
nk
k→∞
Since the sequence
f (t, {l}) = f (t).
l=1
n
|f (t, {l})|
l=1
is non-increasing, we obtain that
∞
|f (t, {l})| = |f (t)|.
l=1
Since f (t) is a characteristic function, there is a number T > 0 such that |f (t)| > 0 if
|t| ≤ T . According to Theorems 2.7 and 2.6 of [1], Chapter III, there is a sequence c1 , c2 , . . .
of real numbers such that the series
∞
(ξk − ck )
k=1
converges with probability 1 regardless of the order of summation. Therefore it is sufficient
to prove that
∞
|ck | < ∞.
k=1
5
Let us suppose that this is not true. In this
a rearrangement ci1 , ci2 , . . . of
∞case there exists
the sequence c1 , c2 , . . ., for which either k=1 cik = +∞ or ∞
k=1 cik = −∞. It suffices to
consider the first case as the second can be reduced to the first by considering the variables
ζk = −ξk . It follows that
n
k=1
ξik =
n
(ξik − cik ) +
k=1
n
cik → ∞ if n → ∞.
(5)
k=1
Let An = {i1 , i2 , . . . , in }. By (5) and Lemma 3 we obtain that for every fixed λ (0 < λ < 1)
Q(λ, An ) → ∞ if n → ∞.
But this contradicts our supposition that the set {F (x, A), A ∈ J } is compact because
by Lemma 1 to every λ (0 < λ < 1) there corresponds a number K(λ) for which
|Q(λ, A)| ≤ K(λ),
A ∈ J.
Proof of the second part. If the series (1) converges with probability 1 regardless
of the order of summation then the series
∞
|1 − f (t, {k})|
k=1
converges uniformly in every finite t-interval (See [1], Chapter III, Theorem 2.7). Using
the inequality
|1 − z1 z2 . . . zr | ≤ |1 − z1 | + |1 − z2 | + · · · + |1 − zr |
valid for every sequence z1 , z2 , . . . , zr of complex numbers with the property |z1 | ≤ 1,
|z2 | ≤ 1, . . . , |zr | ≤ 1, we obtain that
r
r
f (t, {jl }) ≤
|1 − f (t, jl )|.
1 −
l=1
l=1
Let us put A = {j1 , j2 , . . .}. If r → ∞, we obtain the following inequality:
|1 − f (t, A)| ≤
∞
|1 − f (t, {jl })| ≤
k=1
∞
|1 − f (t, {k})|.
(6)
k=1
The series on the right of (6) is uniformly convergent, hence the continuity of the terms
implies the continuity of the sum. Since the relation (6) holds for every A ∈ S, the
characteristic functions f (t, A), A ∈ S, are equicontinuous at t = 0, and thus by Lemma 2
the set {F (x, A), A ∈ S} is compact, as it was to be proved.
Corollary If for every λ (0 < λ < 1) the quantiles Q(λ, A) (A ∈ J ) can be chosen in
such a way that they are bounded, then the series (1) converges with probability 1 regardless
of the order of summation.
Conversely, if the series (1) converges with probability 1 regardless of the order of
summation, then to every λ (0 < λ < 1) there is a K(λ) such that
|Q(λ, A)| ≤ K(λ),
6
A ∈ S.
Proof. The assertion is an immediate consequence for Lemma 1 and Theorem 1.
In the above corollary to Theorem 1 the condition implying the convergence of the
series (1) can be essentially reduced. This fact is contained in the following theorem.
Theorem 2 Let us suppose that there can be found two numbers λ1 , λ2 for which
0 < λ1 < λ2 < 1, such that the quantiles Q(λ1 , A) and Q(λ2 , A) can be chosen in such a
manner, that the sets Q1 = {Q(λ1 , A), A ∈ J } and Q2 = {Q(λ2 , A), A ∈ J } are bounded.
Then the series (1) converges with probability 1 regardless of the order of summation.
Proof. Let K be such a number for which by choosing appropriately the values
Q(λ1 , A) and Q(λ2 , A), we have
|Q(λ1 , A)| ≤ K,
Q(λ1 , A) ∈ Q1
|Q(λ2 , A)| ≤ K,
Q(λ2 , A) ∈ Q2
and
for A ∈ J . It follows that
sup P (|ξ(A)| > K) ≤ sup P (ξ(A) < −K) + sup P (ξ(A) > K) ≤ λ1 + 1 − λ2 < 1.
A∈J
A∈J
A∈J
It follows simply that
inf P (|ξ(A)| ≤ K) = ρ > 0.
A∈J
Thus
n
lim P ξk ≤ K ≥ ρ > 0.
n→∞
k=1
The last inequality, together with Theorems 2.9 and 2.6 of [1], Chapter III, implies the
existence of constants c1 , c2 , . . . which have the property that the series
∞
(ξk − ck )
k=1
converges with probability 1 regardless of the order of summation. Applying the same
reasoning as in the proof of Theorem 1 it can be shown that
∞
|ck | < ∞,
k=1
because in the contrary case all the sets {Q(λ, A), A ∈ J }, 0 < λ < 1 were unbounded.
Taking into account the equality
n
k=1
ξik =
n
n
(ξik − cik ) +
cik ,
k=1
k=1
we obtain that the series (1) converges with probability 1 regardless of the order of summation. Hence Theorem 2 is proved.
7
Corollary Let us suppose that the random variables ξk are symmetrically distributed.
1
Then if there exists λ = such that with a convenient choice of the quantiles Q(λ, A), the
2
set {Q, (λ, A), A ∈ J } is bounded, the series (1) converges with probability 1 regardless of
the order of summation.
1
Proof. The assertion follows from the fact that in this case 0 is a -quantile of ξ(A)
2
for A ∈ J , hence the conditions of Theorem 2 are fulfilled.
Remark It is easy to see that the boundedness of only one set of the type {Q, (λ, A),
A ∈ J } does not imply the convergence of the series (1). For instance if the random
variables ξk are equally and symmetrically distributed, moreover P (ξk = 0) < 1, then the
1
series (1) diverges for every ordering of the terms, but 0 is a -quantile of ξ(A) for A ∈ J .
2
Finally I mention a problem. Let us suppose that the series (1) converges with probability 1 regardless of the order of summation.
Let A1 , A2 , . . . be a sequence of sets of S
such that if Bn = An + An+1 + · · · , then ∞
n=1 Bn = 0. By the inequality (6) it follows
that
if k → ∞.
ξ(Ak ) ⇒ 0,
Does the ordinary limit of the sequence ξ(Ak ) exist with probability 1? If the series (1)
converges absolutely, then this is true, but the series (1) may be convergent by every
ordering of the terms and not absolutely convergent. Such series is the following:
∞
sgn sin 2n πx
,
n
0 ≤ x ≤ 1.
n=1
References
[1] Doob, J. L. (1953). Stochastic Processes, New York, London.
[2] Gnedenko, B. V. and Kolmogorov, A. N. (1949). Predel’nüe raszpredelenija dlja
szumm nezaviszimüh szlucsajnüh velicsin, Moszkva.
[3] Kolmogoroff, A. N. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung, Berlin.
8