Download The strong law of large numbers - University of California, Berkeley

THE STRONG LAW OF LARGE
NUMBERS
KAI LAI CHUNG
CORNELL UNIVERSITY
1. Introduction
A well known unsolved problem in the theory of probability is to find a set of
necessary and sufficient conditions (nasc's) for the validity of the strong law of
large numbers (SLLN) for a sequence of independent random variables. This problem will not be solved in the present paper. To avoid a possible misunderstanding
it must be stated at once that nasc's have been found, and several sets of them will
be given in section 3, but they are all unsatisfactory. Presumably all (or shall we
say most) mathematicians will agree on a satisfactory set of such conditions if and
when they are exhibited, but before they are it does not seem easy to lay down
criteria of satisfactoriness. On the other hand it is safe to rule out certain conditions as unsatisfactory, for example those in which sums of random variables enter;
the conditions to be given in section 3 all have this undesirable property.
The purpose of this paper is to give an account of the latest information on this
problem, at least in some directions. While undoubtedly much that follows is
known to experts in the field or, so to speak, lurks in the corners of their minds, it
is hoped that some of the results below are printed here for the first time and not
sufficiently known to a wider circle of probabilists. It is to acquaint this latter
group with the present status of knowledge of the problem that this paper is
written.
The paper is divided into three sections. Section 2 is quite independent of the
others and deals with the case of identically distributed, independent random variables (r.v.'s). In this case it is known, after Kolmogorov,1 that a nasc for the
validity of the SLLN is the finiteness of the first absolute moment of the common
distribution function (d.f.). For use in certain statistical applications Professor
Wald raised the question of the uniformity of the strong convergence with respect
to a family of d.f.'s (see section 2). A nasc for this is given in section 2, which includes Kolmogorov's theorem as a special case. The method of proof is classical.
In section 3 several sets of nas, but unsatisfactory, conditions for the validity
of the SLLN are given and their interrelations, mostly trivial, are explored. The
results of this section includes Kawata's partial result2 in this direction, and ProThis research was prepared at the University of California under the partial support of the
Office of Naval Research.
I See Kolmogorov [1, p. 671. As far as the author is aware the proof was never published by him.
The proof of the sufficiency part is given in Frechet [2]. The necessity part has been given without
centering at the medians; see Feller [3], for more general results.
2 Kawata [4]. He stated the theorem with zero expectations, an assumption which he never
used.
341
342
SECOND BERKELEY SYMPOSIUM: CHUNG
horov's result recently announced [5].* The proof given here of Prohorov's result is
different from and somewhat longer than his,3 but it is hoped that it brings out the
connections more clearly. As an application a simple proof of a sufficient condition
which includes Kolmogorov's [7] and Brunk's [8] is given, as also announced by
Prohorov.
In section 4 satisfactory nasc's for the SLLN are found for r.v.'s which are individually bounded and whose bounds satisfy certain restrictive order conditions.
Such a result was also announced by Prohorov. By using a deep estimate due to
Cramer and Feller [9], Prohorov's result is extended to slightly more general cases
In the following { X,, }, n = 1, 2, ... will always denote a sequence of independn
ent real valued r.v.'s, and Sn = E Xk. If X is a r.v., m(X) denotes a median4
k=1
of X; XO denotes the centered r.v. X - m(X); E(X) the expectation of X. If A is
an event, P(A) denotes its probability. The letters "i.o." are an abbreviation of
the phrase "infinitely often," namely, "for an infinite number of values of whatever
subscript is in question." The symbol e denotes an arbitrarily small positive number, thus a proposition involving e should read: "For every e > 0, etc."
If there exists a sequence of real numbers {Cn} such that
P(lim Sn
(1.1)
n
co
Cn
n
o)
1
we say that the sequence { Xnj obeys the SLLN. In this case it is trivial that we
can replace cn by m(Sn). Thus (1.1) is equivalent to
P(\nlim+co S0nn = 0
(1. 1 bis)
or to the following:
(1.2)
P
(lSI > nE i.o.)
=1
= 0
or to
P(ISn|
(1.3)
rne forall n > N) = 1.
Note that in (1.3) the N is allowed to depend not only on E, but also on the sample
sequence {X,,). Thus (1.3) is equivalent to the following: given any f > 0, there
exists a fixed No depending on E but no longer on the sample sequence such that
(1.4)
P
(IS°jI
ne for all n _ No) > 1-f .
2. The identically distributed case
Let all Xn have the same d.f. F(x). Kolmogorov (see footnote 2) proved that
*Added in proof: Prohorov's complete account has in the meanwhile appeared in Izvestia
Akad. Nauk. USSR, Vol. 14 (1950), pp. 523-536.
3 His proof depends on a new inequality of Kolmogorov, the idea of which is very close to one
of P. Levy [6, p. 138].
'Throughout this paper we could use instead of the median, any number u(X) such that
P[X _ W(X)] > X, P[X _ ,(X)] _ X for some fixed X: 0 < X < 1.
STRONG LAW OF LARGE NUMBERS
a nasc that
343
IX4I obeys the SLLN is that
f: IxdF(x) <in.
_co
The proof of the necessity part is trivial; as to the proof of the sufficiency part
there are three essentially different methods:
(i) Khintchine-Kolmogorov's method which depends on truncation and Kolmogorov's famous inequality with or without the intervention of infinite series;
(ii) A special case of G. D. Birkhoff's individual ergodic theorem, ever so many
proofs of which have been given;5
(iii) Doob's [11] very elegant proof using the theory of martingales.
The proof of the following more general theorem uses method (i) and is in essence
nothing but a precision of that method. It is not clear whether the other methods
will be applicable.
Let a family of d.f.'s F(x, 0) be given where 0 is the parameter of the family.
All the r.v.'s Xn have one and the same d.f. F(x, 0) from the family where 0 may
be any value of the parameter. The sequence { XI is said to obey the SLLN uniformly with respect to 0 if: given any e > 0, there exists a fixed No = No(e) not
depending on 6 such that (1.4) holds no matter what 0 is.
THEOREM. A sufficient conditionfor the sequence {X.} to obey the SLLN uniformly
with respect to 0 is the following: given any a there exists a number A (a) not depending
on 0 such that
f IxIdF(x, 0) < 6.
(2.1)
IjI>A(h)
If so we can replace S° in (1.4) by S,, - E(Sn). This condition is also necessary if
the median m(0) of F(x, 0) is a bounded function of 0.
PROOF. Sufficiency. From (.1) it follows that
f
I x I dF(x, 0) _A (1) + 1 = M.
Now choose N _ 2 and such that
f
(2.2)
Izi.N
x
IdF(x, 0) +!6 (N ,+6 f ixldF(x, 0)).j.
Iz
X
'
Having chosen N, choose No > N such that
(2.3)
1No
f
Noe
1xj>NX,/13
x I dF(
0"' NM
No
4~
We have
(2.4)
f
EP(IXkI_k)< 2xIN
keN
xldF(x, 0).
x
6 Extensions of methods (i) and (ii) to the case of dependent r.v.'s which includes the case of
independence have been announced by M. Loeve [10].
SECOND BERKELEY SYMPOSIUM: CHUNG
344
Next,
(2.5)
k-N
fIxi2dF(x,
)
I
=
kal<k
N-N 1
N +2
+ N
I
0<
II<NI/4
Z
f JxI2dF(x,6)0
*2
j-max(N,k) I
k=Ok.IzI<k+1
Ix2dF(x, 0)
+N $+1 zI.T|NI/4
Jf lxJdF(x,
'd O< 2
ff 1xldF(x,
= N1/2+ 6 J
x
0)
dF(x, 0).
IzIINll/'
Izlx2N+1
Define
lXk|
I Xk |
if
Xk
X'-=j
k- I 0
if
<
k,
k
Xk = Xk - E (Xk) .
Then
E(X 2). f
IxI2dF(x,
6).
lxl<k
By Kolmogorov's inequality, and (2.3)-(2.5),
Xkj-E (X;)
p(
>
for at least one nN)<4 -
It follows from Kronecker's lemma that
PK(I| E [Xk-E(Xk)| >4for at leastone n _ N)< .
Moreover, we have
( X1+ ..XNI >E <N
f
dF(x, 0) <-N2
lxl >ne/4N
Izl >ne/3N
IxJdF(x, 0) <E
and if n _ No, by (2.2) and (2.3)
E(X;) |n+ f
|
IxI
IkNE
xIdF(x, 0) <.
Altogether we conclude that
p(
S-E (Sn)
I> eforatleastone n2 NO)< 2
or
P[ISn-E(S")
5 ne foralln2No]
21-e.
If f < it follows by the definition of a median that for all n _ No,
-?_ E tS
< ne
MX
1Sn
(2.6
1
STRONG LAW OF LARGE NUMBERS
345
Thus
P [IS -m (Sn) < 2nf for all n _ No] _ 1-eThis implies (1.4), whatever 0 is.
Necessity. Suppose that (1.4) holds where No does not depend on 9. Then if
n > No,
P [IX,, - m (Sn) + m (Sn-,) I < 2nfl _ 1 - .
If e < 2 this entails
-m (X.) + m (S-,)- m (Sn) | < 2ne .
Since by hypothesis I m (X.) I = I m (0) < m where m does not depend on 0,
there exists a number N1, not depending on 0, such that if n _ N1
(2.7)
Im (S,,-)-m (S,) I < 3ne .
Now suppose that (2.1) did not hold and we wish to reach a contradiction. If (2.1)
did not hold there exists a 5 > 0 such that for any N there is a ON for which
f
lxldF(x, ON) -8
IZI>N
Hence
k=N+l
f|dF(x, ON)+(N+1) Ilx>N
f dF(x,O)_
.
It follows that one of the following two cases would occur:
Case (i). For a sequence Ni T , there corresponds a sequence Oi such that if
all X, have the d.f. F(x, Os), then
co
P(IX.1 >- n)_2.i
n=Ni+l
Hence for this sequence { X. I we have
P ( X. I < n) - e -8/2 .
n=Ni+l
P (XXn_ n for at least one n 2 Ni + 1) 1-e2.
We have X. = Sn -m(S) - [S,- m (S.-,)] + m(S^) - m(Sn.1); by (2.7),
since Ni _ N1, if f < 1/6, (2.8) entails
(2.8)
P [ISn -m (Sn)
_
n
for at least one n > N] _ 1-- e-812.
Since Ni T -, (1.4) becomes false for e < min (1/4, 1 -e-/2)
Case (ii). For a sequence N' T c, there corresponds a sequence OA such that if
all Xn have the d.f. F(x, OA), then
2Ni
IP
n=N'.
5XnI > N')
>
SECOND BERKELEY SYMPOSIUM: CHUNG
346
whence
P (IX.I > N' for at least one n: N: < n < 2N') _ 1 -e-
The same argument as in case (i) finishes the proof.
Remark 1. If the family of d.f.'s consists of a single d.f., then the theorem reduces to Kolmogorov's.
Remark 2. Without the assumption of the boundedness of the medians, the
condition stated in the theorem is not necessary.
Example. Let e run over the positive integers and define F(x, n) to be the d.f.
which has a single jump at the point x = n.
Remark 3. The following simpler version may be more useful for applications;
its proof is similar but simpler. Suppose that for every 0,
f xdF(x, o) =0,
J
IxIdF(xa,)< o
Then the condition stated in the theorem is a nasc that: give'ktany e > 0, there
exists a No depending on e but not on 0 nor on the sample sequence, such that
P (IS,, _ ne for all n _ No) > 1- e.
3. Necessary and sufficient conditions and a sufficient condition
We return now to the general case. We shall consider, besides the SLLN embodied in formula (1.1), also a modified form, namely,
P(lim S =o)= 1 .
(3.1)
PC+ o n
=
For any given sequence cn, (1.1) can be reduced to (3.1) by an obvious change of
variables: X.* = Xn - cn + c,1i. Thus while (1.1) answers the question: does
there exist some sequence {cn, such that (1.1) holds; (3.1) answers the question:
does (1.1) hold with a given sequence c,,. The second question will of course be
lim m (Sn)n - = 0 or not, but
answered via the first if we can decide whether n-co
there seems in general no control over m(SM).
To simplify writing, "convergence in probability" will be denoted by an arrow
; "convergence with probability one" or "almost sure convergence" by a double
arrow . If A and B are two propositions, A : B means "A implies B"; A B
means "A and B are equivalent."
Consider the following:
_^O
(1)
S._4
Sn ;
(2)
(3)
6
°
;2n
Instead of 2" we can take any sequence of positive integers such that 0 < Al <
At <
-.
q.+,/qn <
347
STRONG LAW OF LARGE NUMBERS
(4) 2
S2-n±i - S2---),0
2"
| 52 1 -S2n > 2nf ) <C>
P (2
(5)
n
EP (|S2n
(6)
> 2nC) <
.
The following relations obtain:
I. (1) v (2): Well known.
II. (1) D (3) _ (4) a (5): The equivalence of (3) and (4) is a simple analytical fact; that of (4) and (5) is a consequence of the Borel-Cantelli lemma.
III. (5) c (6) v (3): That (6) implies (3) is a consequence of one half of the
Borel-Cantelli lemma; that (6) implies (5) follows from Boole's inequality.
IV. (2) and (3) : (1).
PROOF. (3) is equivalent to
P (IS2,,i > 2-e i.o.) = 0.
For every positive integer k define n(k) by 2n(k)-1 < k < 2n(k). Since
P
(IS2f(k) - Sk|
_ 2n(k)E) _ 1
- P
(I S2 I()
> 2n(k)-1e) - P
(ISki > 2n(k)-1E)
by (2) we have if k > ko = 2n,
P
(IS2nCk
- Sk|
-< 2n(k)e)
> a
Now SkI > 2n(k)+le and S2(k) - SkI < 2n(k)e together imply I S2"() > 2n(k)e
and the first two events are independent, hence by a simple argument,
(ISkI > 2.(k)+le for some k > ko) _ 2P (iS2-(k)1 > 2n(k), for
we obtain
Letting ko P (ISkI > 4ke i.o.) < 2P (IS2n1 > 2"e i.o.) = 0.
P
some k > ko)
V. (1) _ (2) and (3) -(2) and (4) a (2) and (5) c (2) and (6): from I-IV.
The implication (2) and (6) O (1) was proved by Kawata [4]. Proposition (2) is
one form of the weak law of large numbers (WLLN). Thus the relations in V show
that the SLLN, in the form (2.1), is equivalent to the corresponding WLLN plus
the SLLN for the subsequence S2n- Now satisfactory nasc for the WLLN have
been given by Kolmogorov [12] and Feller [13]. Hence we can, if we prefer, replace (2) everywhere in V by these conditions. The significance of Prohorov's result below lies in the elimination of the WLLN as part of the sufficient conditions
for the SLLN, and this is done by centering (at the medians).
Now we consider the relations (1)-(6) with Sn S21 S2n+i - S2n replaced by
2 is not replaced by
S°, 5,,, (S2n+'- S,2)O respectively. (Note that S2fI+"We add a new propowe
call
The
(10)-(60).
resulting
propositions
+son2
S2,,!).
sition
(70)
O
2
348
SECOND BERKELEY SYMPOSIUM: CHUNG
We notice first that (70) entails
S2n+- - S2 - m (S2x+ l) - m (S2n) -* 0
2hence it follows that
(80)
m
(S2-+i)
-
m
(S2-)
-
m
(S2n+ -S2..)
0
All the relations I-III above carry over for the circled propositions and the
corresponding relations will be referred to as I°-III°. We have only to consider
Xn- m(S..) + m(Sn.-) as new r.v.'s and apply I-III; the fact that (70) O (80)
has to be used in several places.
IVO. (30) 3 (70) n (20).
PROOF. Only the second implication needs a proof and this is easiest done by
resorting to characteristic functions (c.f.'s). Let Xn have the c.f. f.(t). (70) implies
lim
2x
2-
K() I
nco
uniformly in any finite interval It _T. This immediately implies
/2n(k) t
ki
limH
fj
(
-. -_) |=1
uniformly in I|t < T/2, where n(k) is defined in IV, whence (20).
VO. (5°) D (10).
PROOF. (50) -(30) D (70) D (80). Since (80) holds (50) is equivalent
1:P ( I S2°x+i °2n I > 21e) <
(90)
-
-
to
.
Moreover by II0 and IVO, (50) * (20). But (20) and (90) imply (10), by the third
proposition in V.
VIO. (10) = (30) (40) =- (50) c (60): from II, III0, IVO, VO.
The equivalence (10) = (50) is Prohorov's theorem [5].
We shall now prove the following theorem which gives a sufficient condition
for the SLLN and includes Kolmogorov's (r = 1) and Brunk's (r integer > 1).
THEOREM. Let E(X.) = O for every n, and E(| Xn 2r) < oo for some real number
r
2 1. If
E
(3.2)
IX12r
nr+l
<
co
Then (i.1) holds.
PROOF. We need the following inequality
(3.*3 )
(
Xk
E |
12) <_ XAnr-1'YE (Il Xk 12r
where A depends only on r. This is easily proved if we use an inequality due to
Marcinkiewicz and Zygmund [14] (trivial if r is an integer) according to which
E
Xk 2r)
k-1i-
[(
2)]
349
STRONG LAW OF LARGE NUMBERS
Now by Holder's inequality
(EXk)
_
nr-1
Xkl1 2r;
k-I
k=l
hence (3.3) follows.
From (3.2) it follows firstly by Kronecker's lemma
lim E
S.
)
lm
r+lE (|Xk|2r)=O-
Hence by Tschebicheff's inequality, we have proposition (2) above. Next, again by
Tschebicheff's inequality, and (3.3)
P (1S|51
-5
>
2nE)
< A
...E(l+I2r)
21
-
(2r
E
(kIr
k=2E2r
k= 2-+1
Hence from (3.2) follows proposition (5). Since (2) and (5) (1) by VI we
have (1).
Remark. Using truncated variables the theorem can be stated without assuming any moments. We shall not insist on this, and also other more or less trivial
extensions of the theorem [15].
4. Necessary and sufficient conditions for some special cases
For easier reference we shall rewrite some of the previous formulas:
so
n0
(4.1)
Sn 0
(4.2)
If (4.1) holds, it is easy to see that we have
(4.3)
EP (IXn
_
ne:) <X-
Define X = X° if IXnI < ne, and Xn = 0 if IX|I > ne; then under (4.3) the
sequences { Xn} and { X^I are equivalent in the sense of Khintchine. If the SLLN
is valid for { Xn} it is by definition also valid for I Xn and so for I Xn} ; conversely
if the SLLN is valid for { Xn, and (4.3) is assumed, then it is also valid for {X"}.
Hence we may confine ourselves to r.v.'s {XnI satisfying the following condition:
(4.4)
sup I Xn = o (n) .
Under (4.4) we may, without loss of generality, assume that
(4-.5)
E(XR)=O,(X2) =c,2,
2=
S2 s2 (n)], s2 (2n+1)-S2 (2n) d,n
=
kc-i
LEMMA. Under (4.4) and (4.5), (4.1) and (4.2) are equivalent.
PROOF. We need only prove that (4.1) implies (4.2). We shall first prove that
(4.1) implies Sn = o(n). Otherwise let nk T o and Snk >_ 5k for all k with a > 0.
SECOND BERKELEY SYMPOSIUM: CHUNG
350
Then by (4.4) we have max sup IXj = o(snk). By a classical theorem of
If j:nk
P. Levy ([6], p. 102) the central limit theorem holds for the sequence Snk7 in particular for every I > 0
lim P (Snk _ lm
Sn_k) = iM Pe(Sk< 21 .l)
It follows that m(Sflk) = o(snk). Consequently the d.f. of s,-k (SOk) tends to that of
the normal and (4.1) cannot be true.
Therefore s,n= o(n) and it follows that n-lSn -O 0 (in probability !). This and
(4.1) imply that m(Sm) = o(n) and hence (4.2). q.e.d.
Of the results in section 3 we shall use the following which, combined with the
lemma above, will be referred to as (P).
(P) Under (4.4) and (4.5): if Sn = o(n) and also
IP(IS2n+l-S2nI
(4.6)
>2ne) < CO
then both (4.1) and (4.2) hold; if (4.1) or (4.2) holds, then (4.6) holds.
In the following (4.4) and (4.5) will be assumed.
THEOREM 1. If the further condition is satisfied:
(4.7)
Mn = max sup
k
o
(d
then a nasc for (4.1) or (4.2) is
exp ( -E22nd -2) <
(4.8)
co
PROOF. Sufficiency. If (4.8) holds, then Sn = o(n) because Sn is nondecreasing.
Furthermore, by a theorem of Feller and using his notation, see [9],
max supjXkj_Xndn, Xn=2-dn=o(1), x=e2ndd-i, O < XnX=
2fl<k.92f+ln
we have
(4.9)
P( S2n+l-S2- I
> 2n
) Cdn exp { ( -e222nd-2) (1 + 6)}
where 5 -÷0 as e -O0, and C is an absolute constant. Thus (4.8) implies (4.6).
(4.2) and (4.1) follow by (P).
Necessity. If (4.1) or (4.2) holds, (4.6) holds by (P). Also from the proof of the
lemma, we have Sn = o(n). Hence the estimate (4.9) is applicable and (4.6) reduces to (4.8).
Remark. Obviously (4.8) is a nasc for (4.1) or (4.2) if all the Xn have a normal
distribution with zero mean.
THEOREm 2. If instead of (4.7) we assume
(4.10)
supIXnI
=
°
(lg
n)
then (4.8) is again a nasc for (4.1) or (4.2).
PROOF. It follows from (4.10) that Mn = o(2nlgln). Let the values of n for
35I
STRONG LAW OF LARGE NUMBERS
which M. > 2-nd2 be nk, k = 1, 2. Let M* = e-'Mn( > Mn if f < 1). Then
2
dnk
e2 <M*k= 0(Igk).
By an inequality of Kolmogorov [16], we have if n = nk
(4.11)
P (S22n+l-S2n J > 2nE)
_
max [exp (
= exp
2E), exp (-22)]
-n).
It follows from (4.11) that
(4.12)
:P(JSnk+1-Sftk| >2nke) <
2
k
2
On the other hand it is trivial that
exp [-e22nkd I (nk) I <
(4.13)
co
k
Now, if (4.1) or (4.2) holds, then (4.6) holds by (P). If n F$ nk, (4.9) is applicable, hence
exp [e22nd-2 (n) I < c.
I
nonk
This and (4.13) give (4.8).
Conversely, if (4.8) holds, then by (4.9)
EP(|S2n+I-S2nt
>
2ne) <
o.
npEnk
This and (4.12) give (4.6). (4.1) and (4.2) follow by (P).
Theorem 2 was announced by Prohorov. If Sn is of a greater order of magnitude
than n(lg lg n)-12, theorem 1 provides an extension. Although these two theorems
are better than the crude results which can be obtained directly from the law of
the iterated logarithm, the domain of their applicability is essentially the same as
that of the latter, since we use the estimate (4.9) which leads to it.
The following examples, due to Dr. Erd6s, show that in general (4.8) is neither
necessary nor sufficient for (4.1) or (4.2) even under (4.4) and (4.5).
Example 1. Xk 0 if k 7$ 2n; X2" = ± 2 (Ig lg n)-1 each with probability 2.
Example 2. Xn = 0 with probability 1 - 2n(lg lg n)-2; Xn = n(lg lg lg n)
each with probability n-1(lglg n)-2.
5. Concluding remarks
An opinion may be ventured in conclusion. It is quite possible that the strong
law of large numbers will be solved by an approach entirely different from that
sketched here. It is even possible that it will be solved by a stroke of great cunning,
circumventing all the difficulties inherent in the present methods. Or it may be
solved as a result of obtaining sharp asymptotic estimates for probabilities of the
form P (IS > ne) in the general case.
SECOND BERKELEY SYMPOSIUM: CHUNG
352
It would appear from the necessary and sufficient conditions given in section 3
that such estimates may be indispensable, but this is not true in the special case of
identically distributed random variables (see section 2). However, one thing
should be said: the appraisal of such probabilities is one of the fundamental problems of the theory of probability, and any real progress in this direction will be
of more importance than the solution of a specific problem.
"One hates that power does not comefrom oneself,
but one does not care if the task is done by oneself."
-Confucius
REFERENCES
[11 A. N. KOLMOGOROV, "Grundbegriffe der Wahrscheinlichkeitsrechnung," Ergebnisse der
Mathematik, Vol. 2 (1937); English translation, Chelsea, 1950.
[2] M. FRECHET, Recherches Theoriques Modernes sur la Theorie des Probabilites, Book 1, Gauthier-Villars, Paris, 1937.
[3] W. FELLER, "A limit theorem for random variables with infinite moments," Amer. Jour.
Math., Vol. 58 (1946), pp. 257-262.
[4] T. KAWATA, "On the strong law of large numbers," Proc. Imperial Acad. Tokyo, Vol. 16
(1940), pp. 109-112.
[5] U. V. PROHOROV, "On the strong law of large numbers," (in Russian), Doklady Akad. Nauk.,
Vol. 69 (1949), pp. 607-610.
[6] P. LEvy, Theorie de l'Addition des Variables AlUtoires, Gauthier-Villars, Paris, 1937.
[7] A. N. KOLMOGOROV, "Sur la loi forte de grandes nombres," C. R. Acad. Sci., Paris, Vol. 191
(1930), pp. 910-911.
[8] H. D. BRUNK, "The strong law of large numbers," Duke Math. Jour., Vol. 15 (1948), pp. 181195.
[9] W. FELLER, "Generalization of a probability limit theorem of Cramer," Trans. Amer. Math.
Soc., Vol. 54 (1943), pp. 361-372.
[10] M. LokvE, "Remarques sur la convergence presque sure," C. R. Acad. Sci., Paris, Vol. 230
(1950), pp. 52-53; see also his Symposium paper in the present volume.
[11] J. L. DooB, "Application of the theory of martingales," Le Calcul des Probabilites etSes Applications, Colloques Internationaux du Centre National de la Recherche Scientifique, No. 13 (1949),
pp. 23-27.
[12] A. N. KOLMOGOROV, "Ueber die Summen durch den Zufall bestimmter unabhiangiger Grossen," Math. Annalen, Vol. 99 (1928), pp. 300-319; Vol. 102 (1930), pp. 484-488.
[13] W. FELLER, "Ueber das Gesetz der grossen Zahlen," Acta Litt. ac Sci., Sect. Sci. Math.,
Szeged, Vol. 8 (1936-37), pp. 191-201.
[14] J. MARcINKIEwIcz and A. ZYGMUND, "Sur les fonctions independantes," Fund. Math., Vol. 29
(1937), pp. 60-90.
[15] K. L. CHUNG, "Note on some strong laws of large numbers," Amer. Jour. Math., Vol. 69
(1947), pp. 189-192.
[16] A. N. KOLMOGOROV, "Ueber das Gesetz des iterierten Logarithms," Math. Annalen, Vol. 101
(1929), pp. 126-135.