Download 5 Sampling Distributions and Approximations

5
Sampling
Distributions
and Approximations
5.1
Distributions of Sums of Random Variables
Functions of random variables are usually of interest in statistical applications; and, of course, we have already considered some functions, called statistics, of the observations of a random sample. Two important statistics arc
the sample mean X and the sample variance S2. Of course, in a particular
sample, say xi, x ^ , . . . , x ^ , we observed definite values of these statistics, x and
s 2 ; however, we should recognize that each value is only one observation of
the respective random variables, X and S2. That is, each X or S2 (or, more
generally, any statistic) is also a random variable with its own distribution. IQ
this chapter we determine the distributions of some of the important statistics
and, more generally, distributions of functions of random variables. The derivations of distributions of these functions fall in that area of statistics usually
referred to as sampling distribution theory.
We begin with an example that illustrates how we can find the distributions
of two different functions of the random variables in a random sample.
Example 5.1-1 Let us cast an unbiased four-sided die two independent
times and observe the outcomes on these trials, say X, and X , . This if
equivalent to saying that X ^ and X^ are the observations of a random
sample of size n = 2 from a distribution with p.d.f.
284
5.1
Distributions of Sums of Random Variables
/M=^,
x=l,2,3,4.
We shall find the distribution of the sum Y = A"i + JC; and of the range
W = max (A'i, A',) - min (A-;, Ay.
We first consider the distribution of Y = X^ + X^ and immediately recognize that the space of Y is R = {2, 3, 4, 5, 6, 7, 8}. To determine the p.d.f.
of V, say g(y) = P(Y = y), y e R, we first look at some special cases. Let
y = 3. We note that the event {Y = 3} can occur in two mutually exclusive
ways: {A'i = 1, X^ = 2} and [X^ = 2, X^ = 1}. Thus recalling that X , and
X^ are independent, we obtain
g(3) = P(Y = 3) = P(Xt = 1, A-, = 2) + P(A-i = 2, X; = 1)
Similarly,
9(5) = F(r = 5) = P(JCi = 1, A-, = 4) + P(Xi = 2, X., = 3)
+ P(A-i = 3, X; = 2) + P(Jfi = 4, A'.; = 1)
In general,
g(y) = PfV = y) = 'S /(t)/(y - k),
where some of the summands could equal zero.
We can give the p.d.f. either by Table 5.1-1 or by the formula
8(>')=4—1——^1-,
10
y = 2, 3, 4, 5, 6, 7, 8.
We can compute the mean of Y from the p.d.f. of Y to obtain
E(Y)= ^ra(y)
-%HiiH^)<Mil)<)
=5.
285
Sampling Distributions and Approximations
Y
SI.Y)
2
3
4
5
6
7
8
1/16
2/16
3/16
4/16
3/16
2/16
1/16
To find the distribution of the range, W, note that the event {^=0}
occurs if [ X ^ = X ^ } , and thus
fi(0)=.P(W=0)=^.
The event {W = 3} occurs if {A"i = 1, X, = 4} or (A", = 4, A', = 1}. Thus
fc(3)=P(W=3)=^.
Similarly, A(l) = P(»V = 1) = 6/16, and A(2) = P(iy = 2) = 4/16. To find the
mean of W, we have
^
©
©
©
©
i
In Example 5.1-1, since Y = X i + X;, we anticipate that £(V) = 5, which is
computed using the p.d.f. of Y, would be equal to E(Xi + X^), which is computed using the joint p.d.f. ofA'i and X ^ , namely
Of course, this is true because
E(X, + X,) = E(X,) + E(X^ - J + | = 5 .
This special result with a linear function of two random variables extends to
more general functions of several random variables. We accept the following
without proof.
286
6.1
Distributions of Sums of Random Variables
THEOREM 5.1-1 Let X ^ , X ^ , ..., X.be n random variables with joint p.d.f.
f ( x ^ , x;,..., jc,,). Let the random variable Y = u(X^, X ^ , ..., XJ have the p.d.f.
g(y}. Then, in the discrete case,
E(Y) = £
provided that these summations exist. For random variables of the continuous
type, integrals replace the summations.
Recall (Section 4.2) that if X ^ and X; are independent random variables,
then
E^(X,)u,(X,f] = £[Ui(Xi)]£[:U2(A-2):l,
provided that these expectations exist. This can be extended to products of
functions of n mutually independent random variables.
THEOREM 5.1-2 If X ^ , X ^ , .... X , are mutually independent random variables having p.d.f:'s /i(xi), fi(x^, ...,/.(x,) and £[u,(X,)], i = 1, 2, ..., n, exist,
then
E^u,(XMX,) •
Proof; In the discrete case, we have that
Etu,(XMX,) •
=
£
£
•
•
•
£ "l(»l)"2^2) •
= £
= £[ui(A'i)]£[»2(^2)] •
REMARK Sometimes students recognize that X2 = X •
£(X2) is equal to [£(X)][£(A")] = [£(X)]2 because the above theorem states
that the expected value of the product is the product of the expected values.
However, note the hypothesis of independence in the theorem, and certainly X
is not independent of itself. Incidentally, if E(X2) did equal [£W]2, then the
variance of X
a2 = E(X2) - [£(X)]2
would always equal zero. This really happens only in the case of degenerate
(one point) distributions.
287
Sampling Distributions and Approximations
I
Example 5.1-2 It is interesting to note that these two theorems allow i
determine the mean, the variance, and the moment-generating function
function such as V = A\ 4- X ^ , where X i and X^ have been define
Example 5.1-1. We have already seen that /iy = 5.
The variance of Y is
a1, = £[(Y - ^)2] = EUX, + X , - /i, - ^)2],
where fi, = E(X,) = 5/2, i = 1, 2. Thus
a2 = £[(A-i - 11,} + (Xi - ^)]2
= £[(A-i - ^i)2 + 2(A-, - pi)(-y, - /^) + (A-; - ^)2].
But, from an earlier result about the expected value being a linear opei
we have
a2 = £[(Jf, - /^i)2] + 2£[(X, - ^)(A-, - ^.;)] + £[(^ - fi;)2].
However, since Xi and X^ are independent, then
£[(^i - fi)(A'2 - vS\ = Wt - ^WX, - fi,)]
= (/'I - f l ) ( f 2 - /'2) = 0.
Thus
"i = "i + "i.
where
(rf = £[(^, - ^,)2],
i = 1, 2.
In the case in which X^ and X^ are the outcomes on two independent
of a four-sided die, we have af = 5/4, i = 1,2, and, hence
- 5 5 5
"^^^lFinally, the moment-generating function is
M,(t} = E(e") = .E|>w.+.'•«>] = E(e•x•e"").
288
5.1
Distributions of Sums of Random Variables
The independence of X ^ and X, implies that
M^t) = E(e"W).
For our example in which X^ and X; have the same p.d.f.
f(x) = ^
x = 1, 2, 3, 4,
and thus the same moment-generating function
^O = 4
\ <•' +4^ e2' +4\e1' +4 \ '•"•
we have that My(i) equals
[M^ = ^^ ^ ^ ^ ^.+ ^ ^-
Note that the coefficient of e" is equal to the probability P(Y = b); for illustration, 4/16 = P(Y = 5). This agrees with the result found in Example 5.1-1,
and thus we see that we could find the distribution of Y by determining its
moment-generating function.
In this section we shall restrict attention to those functions that are linear
combinations of random variables. We shall first prove an important theorem
about the mean and the variance of such a linear combination.
THEOREM 5.1-3 I f X ^ , X ^ , . . . , X , are n independent random variables with
respective means f i ^ ^,..., ^ and variances f f \ , ffj, ..., (T^ , then the mean and
the variance of Y = ^^ i a, X ^ , where a^, a;,.. -, a,, are real constants, are
fiy=^a,ti,
and
o-y = ^ a f a f ,
respectively.
Proof: We have that
(^Em^S 0 ,^.^ ia.£W)= E a,H,
\.-i
/ .=1
i=i
289
Sampling Distributions and Approximations
Ch, B|
I
because the expected value of the sum is the sum of the expected values (i.e., £
is a linear operator). Also
i
o2 = £[( V - (^)2] = J( Y, a, X, - 1 a, ft.) ]
|
= E^i a,(X, - ^)Tl = J 1 f; a,fl/X. - AX^ - n,)}
;
Again using the fact that £
^ = 1=1
£
j-1
However, if i 96 y, then from the independence of X,and X j we have
£[(X, - /<,X-y; - ?<,)] = E(X, - /i,)£(A-, - /!,) = (p, - ^,X(<, - ^) = 0.
Thus the variance can be written as
IT? = ^ af£[(A-, - /i,)2] = S afff,2.
D
We give two illustrations of the theorem.
Example 5.1-3 Let the independent random variables X i and -Y; have
respectively means fi, = —4 and /i; = 3 and variances u2 = 4 and irj = 9.
The mean and the variance of Y = 3X^ - IX 3 are, respectively,
^ y = ( 3 X - 4 ) + ( - 2 X 3 ) = -18
and
ff 2 =(3) 2 (4)+(-2) 2 (9)=72.
Example 5.1-4 Let X ^ , X , , . . . . X. be a random sample of size n from a
distribution with mean y. and variance cr2. First let Y = X ^ — X^, then
/i,=/x-/i=0
and
ff 2 = (I)2;!2 + (-l) 2 ^ = 2ff 2 .
Now consider the sample mean
X , + X, + •
X='290
S.1
Distributions of Sums of Random Variables
which is a linear function with each a, = l/n. Then
"^.I.C;)'""
and
'^^O'^-
That is, the mean of X is that of the distribution from which the sample
arose, but the variance of X is that of the underlying distribution divided
by n.
In some applications it is sufficient to know the mean and variance of a
linear combination of random variables, say V. However, it is often helpful to
know exactly how Y is distributed. The next theorem can frequently be used
to find the distribution of a linear combination of independent random variables.
THEOREM 5.1-4 I f X ^ X ^ , . . . , X^ are independent random variables with
respective moment-generating functions My;((), i = 1, 2, 3, ..., n, then the
moment-generating function ofY = ^?=i Oi-^i is
M,(t)=f\M^t).
Proof: The moment-generating function of Y is given by
My{t) = £[e'1'] = E^e1^1^^ ••-+a-('•']
=E\ieal'xlea2txl•••ea'•tx^
= £[e•'l'yl]JE[e''2tx2] ••- EEe0"'^]
using Theorem 5.1-2. However, since
E^e'xi) = M^(t),
then
E{ea•txi}=M^a,t).
Thus we have that
My(t} = M^t)M^ t) •
=f\M^a,t).
1=1
D
A corollary follows immediately, and it will be used in some important
examples.
291
Sampling Distributions and Approximations
Ch. 5
COROLLARY 5.1-1 //-^n -^2' •
•
•
' ^n are observations of a random sample
from a distribution with moment-generating function M(t), then
(i) the moment-generating function of Y = Y.1= i ^ils
l
My(t)= n^(o=[^(o]";
1=1
(ii) the moment-generating function ofX == ^^i (l/n)Xi is
^'-n^KC)]"
I
,
;
:
Proof: For (i), let a, = 1, i = 1, 2,..., n, in Theorem 5.1-4. For (ii), take a, =
l/n,i= l,2,...,n.
D
The following examples and the exercises give some important applications
of Theorem 5.1-4 and its corollary.
Example 5.1-5 Let A",, X ^ , . . . , X , denote the outcomes on n Bernoulli
trials. The moment-generating function of X , , i == 1, 2,..., n, is
M(t) = q + fie'.
Y = £^,,
then
A'r(t)= FT (9 +;"'')= te+Pe')".
1=1
Thus we again see that Y is 6{n, p).
Example 5.1-6 Let X ^ , X ^ , X y be the observations of a random sample of
size n = 3 from the exponential distribution having mean 8 and, of course,
moment-generating function M(r) = 1/(1 —
generating function of Y = X^ + X^ + -¥3 is
My(() = [(1 - fl()-1]3 = (1 - fitF3,
t < 1/9,
which is that of a gamma distribution with parameters a = 3 and ft Thus Y
292
5.1
Distributions of Sums of Random Variables
has this distribution. On the other hand, the moment-generating function of
xis
^o-rn-r
t<3/0;
and hence the distribution of X is gamma with parameters a = 3 and 0/3,
respectively.
Exercises
5.1-1 Let X ] and X ^ be observations of a random sample of size n = 2 from a distribution with p.d.f./(x) = x/6, x = 1, 2, 3. Find the p.d.f of Y = X , + X ^ . Determine the
mean and the variance of the sum in two ways.
5.1-2 Let X , and X; be a random sample of size n = 1 from a distribution with p.d.f.
f { x ) = 6x(l - x), 0 < x < 1. Find the mean and the variance of Y = Xi + X;.
5.1-3 Let Xp X;, A" 3 be a random sample of size 3 from the distribution with p.d.f.
f { x ) = 1/4, x = 1, 2, 3, 4. For example, observe three independent rolls of a fair
four-sided die.
(a» Find the p.d.f. of Y = X^ + X, + X ^ .
(b) Sketch a bar graph of the p.d.f. of Y.
5.1-4 Let Xi and X^ be two independent random variables with respective means /^i
and ^ and variances o\ and a\. Show that the mean and the variance of Y = X ^ X ^
are /i^ and o^o-j + ^fffj + ^ a\, respectively.
HINT: Note that E(Y} = E(Xi)E[X^} and £(V2) = E(Xi)E(Xi).
5.1-5 Let X ^ and X ^ be two independent random variables with respective means 3
and 7 and variances 9 and 25. Compute the mean and the variance of Y = —2Xi
+X,.
5.1-6 Let X, and X ^ have independent distributions &(«i, p) and fc(n;, p), respectively.
Find the moment-generating function of Y = X i + X ^ . How is Y distributed?
5.1-7 Let A\, X;, X y be mutually independent random variables with Poisson distributions having means 2,1, 4, respectively.
(a) Find the moment-generating function of the sum Y = X ^ + X ^ + X ^ .
(b) How is V distributed?
(c) Compute P(3 < Y < 9).
5.1-8 Generalize Exercise 5.1-7 by showing that the sum of n independent Poisson
random variables with respective means fi,, ^,..., ^ is Poisson with mean
^l + ^ 2 + - - - + / V
5.1-9 Let A'i, X-i, X , , , X ^ , X., be a random sample of size 5 from a geometric distribution with p = 1/3.
(a) Find the moment-generating function of Y = X ^ + X ^ + X ^ + X^ + X ^ (b) How is r distributed?
293
Sampling Distributions and Approximations
Ch. 5
5.1-10 Let W = X\ + X ^ + •
distributed exponential random variables with mean 6. Show that W has a gamma
distribution with mean hQ.
5.1-11 Let X ^ X ^ , X y denote a random sample of size 3 from a gamma distribution
with a = 7 and 0 = 5 .
(a) Find the moment-generating function of Y = X ^ + X^ + X ^ (b) How is r distributed?
5.1-12 Let X and Y, with respective p-d.f.'s f [ x ) and g(y), be independent discrete
random variables, each of whose support is a subset of the nonnegative integers 0,1,
2 , . . . . Show that the p.d.f. of W = X + Y is given by the convolution formula
hiw) = S Mg(w - x), w = 0. 1, 2,.
HINT: Argue that /i(w) = P(W = w} is the probability of the w + 1 mutually exclusive
events (x, w —
5.1-13 Let X ^ , X ^ , X ^ , X^ be a random sample from a distribution having p.d.f.
/(x)=(x+l»/6,x=0, 1,2.
(a) Use Exercise 5.1-12 to find the p.d.f. of W, = X ^ + X ^ .
(b) What is the p-d.f. of W^ - X,, + X^
(c) Now find the p.d.f. of W = W^ + W^ = X ^ + X ^ + X ^ + X ^ .
5.1-14 Roll a fair 4-sided die eight times and denote the outcomes by A], X ^ , . . . , X y .
Use Exercise 5.1-12 to find the p.d.f.'s of
(a) X , + X , ,
(b) S^,,
(c) Y,X,.
5.2
Random Functions Associated with
Normal Distributions
In statistical applications, it is often assumed that the population from which a
sample is taken is normally distributed, N(^i, f f 2 ) . There is then interest in estimating the parameters p. and a2 or in testing conjectures about these parameters. The usual statistics that are used in these activities are the sample mean
X and the sample variance S 2 ; thus we need to know something about the
distribution of these statistics or functions of these statistics.
"THEOREM 5.2-1 If' X^, X - ^ , ..,, X^ are observations of a random sample of
size n from the normal distribution N{p., o"2), then the distribution of the sample
mean X = (1/n) ^?_i X, is N(fi, o-Vn).
294
5.2
Random Functions Associated with Normal Distributions
Proof Since the moment-generating function of each X is
/
ff't^
M^t) = exp ( ^ ( + -^-1,
the moment-generating function of
X=
1
YX,
n
i= 1
is, from the corollary of Theorem 5.1-4, equal to
^44(^];"
2 2
r
(ff /")' '!
= exp f t + ——j— .
However, the moment-generating function uniquely determines the distribution of the random variable. Since this one is that associated with the normal
distribution N(fi, o^fn), the sample mean X is N(/f, ^/n).
D
Theorem 5.2-1 shows that if X^ X ^ , ..., X^ is a random sample from the
normal distribution, N(^, o1}, then the probability distribution of X is also
normal with the same mean ^ but a variance a ^ f n . This means that X has a
greater probability of falling in an interval containing p, than does a single
observation, say X i . For example, if fi = 50, o2 = 16, n = 64, then P(49 <
X < 51) = 0.9544, whereas F(49 < Xi < 51) = 0.1974. This is illustrated again
in the next example.
Example 5.2-1 Let X ^ , X ^ , . . . , X , be a random sample from the JV(50, 16)
distribution. We know that the distribution o f X i s N(50, 16/n). To illustrate
the effect of n, the graph of the p.d.f. of X is given in Figure 5.2-1 for
n = 1, 4, 16, and 64. When n = 64, compare the areas that represent P(49 <
X < 51) and P(49 < X\ < 51).
Before we can find the distribution of S2, or (n —
liminary theorems that are important in their own right. The first is a direct
result of the material in Section 5,1 and gives an additive property for indept chi-square random variables.
(S^EOREM 5.2-2 Let the distributions o f X , , X ,,..., X^ be ^(i-i), ^(r;),....
^(i-i,), respectively. I f X ^ X ^ , . . . , X,, are independent, then Y = X, + A"; + •
+ X, is ^(r, + r; + •
295
Sampling Distributions and Approximations
Proof We give the proof for k = 2 but it is similar for a general k.
moment-generating function of Y is given by
Mi,(t) = E[_e'^ == E^1^2^ = E^^^.
Because Xi and X^ are independent, this last expectation can be factorei
that
M^) = E^W^ = (1 - It)-^! - 20-^
t < -,
since X^ and X^ have chi-square distributions. Thus
M^)^!^)^'^/2,
«
^
the moment-generating function for a chi-square distribution with r = r,
degrees of freedom. The uniqueness of the moment-generating function im]
that Y is ^(r^ + '•2). For the general k, Y is -^-(r^ + r^ + ••• + r^).
The next theorem combines and extends the results of Theorems 3.5-1
5.2-2 and gives one interpretation of degrees of freedom.
296
6.2
Random Functions Associated with Normal Oixtributions
THEOREM 5.2-3 Lei Z^, Z;, ..., Z, have standard normal distributions,
N(0, 1). If these random variables are mutually independent, then W = 7.\ + 7.\
+•
Proof: By Theorem 3.5-1, Z? is /'(I) for i = 1, 2,..., r. From Theorem 5.2-2,
with k = r, V = W, and r, = 1, we see that W is /•'(r).
0
COROLLARY 5.2-1 If X ^ , X , , .... X. have mutually independent normal
distributions N(u,, a1), i = 1, 2,..., r, respectively, then the distribution of
w
-^
is^(r).
Proof We simply note that Z, = ( X , - u,)/a, is N(0, 1), i = 1, 2,..., r.
Q
Note that the number of terms in the summation and the number of degrees
of freedom are equal in Theorem 5.2-3 and the corollary.
The following theorem gives an important result that will be used in statistical applications. In connection with those, we will use the sample variance
sl=-li(x,-x)l
n
~
l
1=1
2
to estimatethe variance, f f , when sampling from the normal distribution,
N{^, a2). More will be said about S2 at the time of its use.
THES&REM 5.2-4 ^/-^i> ^2, •••, ^n are observations of a random sample of
size nfrom the normal distribution N(^i, a2),
^4,t^
and
s'-^IW-W
,„
ns2
ZW-^) 2
1
5
(
i)
*
"
—
—
'
—
—
^
—
—
—
—
—
—
i
s
^
n
l
)
.
297
]
Sampling Distributions and Approximations
Ch,^
Proof: We are not prepared to prove (i) in this book; so we accept it without
proof here. To prove (ii), note that
y (x' - t1}2 „
1-1 \ " )
.=i L
"
J
-IK-^}^
'-'
because the cross-product term is equal to
^-^.-^2(^^_^
,=i
"
"
i-i
But Y, = (X, —
normal variables. Hence W = Y.^i y? i5 /("I by Corollary 5.2-1. Moreover,
since X is A^/i, a2/"), then
^ _ / . g - ^ Y- _ ^ - ^
"L/^;^
^
is y2;!) by Theorem 3.5-1. In this notation, equation (5.2-1) becomes
^(»_^z'.
2
a
2
However, from (i), X and S are independent; thus Z2 and S2 are also independent. In the moment-generating function of W, this independence permits
us to write
^^e•l<"-l>sa;•2+^'l] = ^[(..(•-Ds'/.'e.z']
= E[.e•(•-w2"'~}E[,e'z"l.
Since W and Z 2 have chi-square distributions, we can substitute their
moment-generating functions to obtain
(1 - 2t)-"'2 = El:^"-11""2^! - 2t)-"2.
Equivalently, we have
£[6""-""'°'] = (1 - 2t)-'"-'"2,
( < •1.
This, of course, is the moment-generating function of a ^(n - 1) variable, and
accordingly (n - IIS2^2 has this distribution.
Q
298
6.2
Random Functions Associated with Normal Distributions
Combining the results of Corollary 5.2-1 and Theorem 5.2-4, we see that
when sampling from a normal distribution,
,^M
is ^(n), and
^^_D!
is ^(n - 1). That is, when the population mean, fi, in ^ (X, - ^)2 is replaced
by the sample mean, X , one degree of freedom is lost. There are more general
situations in which a degree of freedom is lost for each parameter estimated in
certain chi-square random variables (e.g., see Section 8.5).
Example 5.2-2 Let X^, X ^ , X ^ , X^ be a random sample of size 4 from the
normal distribution, JV(76.4, 383). Then
u=T.
(X, - 76.4)2
383
is rt4),
-(X^O.
-i
383
is it^), and, for examples,
P(0.711 < U < 7.779) = 0.90 - 0.05 = 0.85,
P(0.352 :£ W & 6.251) = 0.90 - 0.05 = 0.85.
In later sections we shall illustrate the importance of the chi-square distribution in applications.
It is often true in statistical applications that there is interest in estimating
the mean y. or testing hypotheses about the mean when the variance a1 is
unknown. Theorem 4.7-1, in which a T random variable is defined, along with
Theorems 5.2-1 and 5.2-4, provide a random variable that can be used in such
situations. That is, if X^, X ^ , ..., X^ is a random sample from the normal
distribution N(^, a1), then (X - tt)l(a!^n) is N(0, 1), (n - W/a^ is ^(n - 1),
and the two are independent. Thus we have that
^._
(X - it)l(al^n)
^(n - W/a^n - 1)
_X-ii
S/^n
299
Sampling Distributions and Approximations
Ch. 5
has a r distribution with r = n —
able like this one that motivated Gosset's search for the distribution of T. This
t statistic will play an important role in statistical applications.
We now prove another theorem which deals with linear functions of independent normally distributed random variables.
THEOREM 5.2-5 //^i» X ^ , .-., X^ are n mutually independent normal variables with means ^i, /;;, ..., u^ and variances a}, o-j, ..., o-^, respectively, then
the linear function
Y = ^c,X,
has the normal distribution
MS'-,/,,, ^c.^2 .
\l-l
r=l
/
Proof: By Theorem 5.1-4, we have
M,(t) = flM^(c,t) = flexp (fi,c,t + aWl-t)
because M , f t ) = exp (;;, t + aft 2 /!), i = 1, 2,..., n. Thus
MM = exp [( E'-.f,)' + [l.^i)^')]
This is the moment-generating function of a distribution which is
N(ic,^,,
icw}/
\i=i
1=1
and thus Y has this normal distribution.
Q
From Theorem 5.2-5 we can make the observation that the difference of two
independent normally distributed random variables, say Y = X i —
the normal distribution N(u^ —
ahead to statistical applications in which we compare the means for two
normal distributions. This, in turn, is used to construct the following t random
variable, which is also important in statistical applications.
Let -X\, X y _ , ..., Xn and Y^ Y^, ..., Y^ be random samples from the independent normal distributions, N{ux, ffjt) and N{uy, a^}, respectively. The distribution of X is N(ujc, a^/n) and the distribution of Y is N(^y, a^/m). Since X
and V are independent, the distribution X —
300
5.2
Random Functions Associated with Normal Distributions
and
_
x-y-fe-^,)
Z=-
^/ffi/n + s\lm
is N(0, 1). The statistics
,„
1,C2
SW--?)2
I" - I f i x
i-l_________
and
(.-1^
^-y)2
have distributions that are '^(n —
normal distributions are independent, these chi-square statistics are independent, and thus the distribution of
^ _ (n - l)Sj
(m - 1)S;
is ^(n + m —
r =n +m —
T=^U/(" + m - 2)
[ X - Y - (^ - )ir)]/v/'d/" + "i/m
^/[(n - 1)S^ + (m - l)S?/<i?]/[n + m - 2]
In the statistical applications we sometimes assume that the two variances are
the same, say a\ = a\ =- a2, in which case
X - Y - (fi, - 11,)
7-=^{[(n - 1)S^ + (m - l)Sa/(n + m - 2)}[(l/n) + (1/m)]
and neither T nor its distribution depend on a2.
Example 5.2-3 Let X i, X ^ , . . . , Xg and Yi, Y;, . - -, Vie be random samples
of sizes «
301
Sampling Distributions and Approximations
2
2
Ch. 5
2
A/(^y, (7 ) and /V(/iy, o" ), respectively, where ff is unknown. Then
p( -2.306 ^ x——^ < 2.306) = 0.95
V
s,/^
)
(5.2.2)
because (X —
freedom. Also because
^_
X-Y~{^-^)
^[(SS2 + 15S2)/23][(1/9) + (1/16)]
has a t distribution with r = 9 + 16—2= 23 degrees of freedom,
P(- 1.714 < T < 1.714) = 0.90.
(5.2-3)
Note that in both equation (5.2-2) and equation (5.2-3), after the data have
been observed, X, Y, S 2 -, and S2 can be calculated so that only p.^ and ^
will be unknown. In the next chapter we shall use this information to construct interval estimates of unknown means and differences of means.
The F distribution also has many important applications in normal sampling theory, one of which uses the following random variable. Let X ^ , X ^ ,
..., X^ and YI, y;, ..., y,n be random samples of sizes n and m from two
independent normal distributions /V(/^y, o^) and .N(;Uy, o-2), respectively. We
know that (n - l)S^/o-^ is ^(n - 1) and (m - l^y/o 2 is ^(m - 1). The independence of the distributions implies that S^ and Sy are independent so that
(m - l^/^m - 1) _ SW
(n - l)S^{n - 1) Si/o2
has an F distribution with r^ = m —
This result is used in the next chapter.
Exercises
5.2-1 Let Xi, X;, ..., X^ be a random sample from a normal distribution N{77, 25).
Compute
(a) P(77 < X < 79.5),
(b) P(74.2 < X < 78.4).
5.2-2 Let X be N(50, 36). Using the same set of axes, sketch the graphs of the probability density functions of
(a) X ;
(b) X , the mean of a random sample of size 9 from this distribution;
(e) X , the mean of a random sample of size 36 from this distribution.
5.2-3 Let X equal the widest diameter (in millimeters) of the fetal head measured
between the 16th and 25th weeks of pregnancy. Assume that the distribution of X is
302
5.2
Random Functions Associated with Normal Distributions
N(46.58, 40.96), Let X be the sample mean of a random sample of n = 16 observations of X .
(a) Give the value of £(-?).
(b) Give the value of Var(^).
(c) Find P(44.42 ^ X ^ 48.98).
5.2-4 Let X equal the weight of the soap in a "6-pound" box. Assume that the distribution o f X is N(6.05, 0.0004).
(a) Find P(X < 6.0171).
(b) If nine boxes of soap are selected at random from the production line, find the
probability that at most two boxes weigh less than 6.0171 each.
(c) Let X be the sample mean of the nine boxes. Find P[X ^ 6.035).
5.2-5 Let Zp Z;, . . . , Z-, be a random sample of size n = 1 from the standard normal
distribution N(0, 1). Let W = Z\ + 2| + •
5.2-6 If A], X ^ , ..., X,6 is a random sample of size n = 16 from the normal distribution N(50, 100), determine
/
16
\
(a) P) 796.2 ^ S (X; - 50)2 ^ 2630 I,
(b) P 726.1 < ^ (X, - X}2 ^ 2500 1,
\
1=1
/
5.2-7 Let X and S2 be the sample mean and sample variance associated with a random
sample of size n = 16 from a normal distribution Nf/i, 225).
(a) Find constants a and b so that
P(a ^ S2 ^ b) - 0.95.
(b) Find a constant c so that
^_^^^.0.95.
5.2-8 Let X and Y denote the wing lengths (in millimeters) of a male and a female
gallinule, respectively. Assume that the respective distributions of X and Y are
N(184.09, 39.37) and N(171.93, 50.88) and that X and Y are independent. Let X and
Y equal the sample means of random samples of sizes n = m = 16 birds of each sex.
(a) FindP(X- V^9.12).
(b) Let S2. and S^ be the respective sample variances. Find a constant c so that
(
X - Y- 12.16
\
P( -c ^ .
^ c - 0.95.
\
V[(15S^-H5^)/30](2/16» /
5.2-9 Let X i , X ^ , ..., X q be a random sample of size 9 from a normal distribution
N(54, 10) and let Vp Y ^ , Y ^ , Y^ be a random sample of size 4 from an independent
303
Sampling Distributions and Approximations
Ch. 5
normal distribution N(54, 12). Compute
s^-?) 2
/
\
1
4
l
—
—
l
^ ^
5.2-10 Compute P{X < Y) == P(0 < Y —
variables with normal distributions /V(4, 9) and N(7, 16), respectively.
5.2-11 Let X denote the wing length in millimeters of a male gallinule and Y the wing
length in millimeters of a female gallinule. Assume that X is N(184.09, 39.37) and V
is N(171.93, 50.88), and that X and V are independent. If a male and a female gallinule are captured, what is the probability that X is greater than V?
5.2-12 Suppose that for a particular population of students SAT mathematics scores
are N{529, 5732) and SAT verbal scores are N(474, 6368). Select two students at
random, and let X equal the first student's math score and Y the second student's
verbal score. Find P(X > Y).
5.2-13 Suppose that the length of life in hours, say X , of a light bulb manufactured by
company A is N(800, 14,400) and the length of life in hours, say V, of a light bulb
manufactured by company B is N(850, 2500). One bulb is selected from each
company and is burned until "death."
(a) Find the probability that the length of life of the bulb from company A exceeds
the length of life of the bulb from company B by at least 15 hours.
(b) Find the probability that at least one of the bulbs " lives " for at least 920 hours.
5.3
The Central Limit Theorem
In Section 5.1 we found that the mean -Y of a random sample of size n from a
distribution with mean f i and variance a1 > 0 is a random variable with the
properties that
E(X)= f i
and
Var(J?)=°-.
n
Thus, as n increases, the variance of X decreases. Consequently, the distribution of X clearly depends on n, and we see that we are dealing with sequences
of distributions.
In Theorem 5.2-1 we considered the p.d.f. of X when sampling from the
normal distribution JV(/i, a1}. We showed that the distribution of X is N(fi,
f f 2 / ! ! } , and in Figure 5.2-1, by graphing the p.d.f.'s for several values of n, we
illustrated that as n increases, the probability becomes concentrated in a small
interval centered at ^. That is, as n increases, X tends to converge to (i, or
(X —
304
5.3
The Central Limit Theorem
In general, if we let
W=^(J?-,)=^
(7
ff/^/n
where X is the mean of a random sample of size n from some distribution with
mean p. and variance a2, then for each positive integer n,
^^1=^^=0
"f^P
Lff/VnJ
-ff/yij
a/,/))
(7/,/n
and
VarW = E(^) = fe^l =E^-^
|_ (T-/n J
a~/n
^ = 1.
(r-/n
Thus, while X —
"spreads out" the probability enough to prevent this degeneration. What then
is the distribution of W as n increases? One observation that might shed some
light on the answer to this question can be made immediately. If the sample
arises from a normal distribution then, from Theorem 5.2-1, we know that X is
N{^i, a^/n) and hence W is N(0, 1) for each positive n. Thus, in the limit, the
distribution of W must be N(0, 1). So if the solution of the question does not
depend on the underlying distribution (i.e., it is unique), the answer must be
/V(0, 1). As we will see, this is exactly the case, and this result is so important it
is called the Central Limit Theorem, the proof of which is given in Section 5.5.
THEOREM 5.3-1 (Central Limit Theorem) // X is the mean of a random
sample Xp X^ , ..., X,, of size n from a distribution with a finite mean ^ and a
finite positive variance o"2, then the distribution of
.^^
^/v"
v" °
is N(0, 1) i« the limit as n -* oo.
A practical use of the Central Limit Theorem is approximating, when n i
"sufficiently large," the distribution function of W, namely
,r—^
P(W < w) »
J-^J2n
'
J^
305
Sampling Distributions and Approximations
Ch.
We present some illustrations of this application, discuss " sufficiently large
and try to give an intuitive feeling for the Central Limit Theorem.
Example 5.3-1 Let X denote the mean of a random sample of size n = 1
from the distribution whose p.d.f. is/(x) = (3/2)x2, — l < x < l . Here ji =
and a2 = 3/5. Thus
P(0.03 < X < 0.15) = P(4^ < -£-*- < 4^)
\7375/^/T5
^3/5/^15
^/WyW
=P(0.15^ W<0.75)
»<1)(0.75) - <I'(0.15)
= 0.7734 - 0.5596 = 0.2138.
Example 5.3-2 Let Xi, X ^ , . . . , X^o denote a random sample of size ^
from the uniform distribution U(0, 1). Here E(X,) = 1/2 and Var(^",) = 1/1
j = 1,2, . . . , 2 0 . I f r = ^ i + X ^ +••• +X2o,then
-p
p(y < 0,) = pf-^2) , 9___^ ^ ^ ^ _^
a<Ii(-0.697)
= 0.2423.
Also,
0 1
^^y^ii.^p^
^ -^^11-7-^
8
-(
^5/3
^Sft
^5/3
=P(-1.162< W < 1.317)
»<t(1.317)-<!)(-1.162)
=0.9061 -0.1226=0.7835.
Example 5.3-3 Let X denote the mean of a random sample of size 25 froi
the distribution whose p.d.f. is/(x) = x3/^, 0 < x < 1. It is easy to show thi
^1=8/5= 1.6 and cr2 = 8/75. Thus
/ 1.5 - 1.6
X - 1.6
1.65 - 1.6 \
P(1.5 $ X < 1.65 = P( ,—— ,— & .—— ^- < ,——
\V8/75/y25 ^S/TS/^IS V8/75/v'25/
=P(-1.531 < W ^ 0.765)
»1>(0.765)-<1)(-1.531)
=0.7779-0.0629=0.7150.
306
,
6.3
The Central Limit Theorem
These examples have shown how the Central Limit Theorem can be used
for approximating certain probabilities concerning the mean X or the sum
Y = ^=1 X : of a random sample. That is, X is approximately N(^, o^/n), and
V is approximately N(n^i, na2) when n is "sufficiently large," where /t and a2
are the mean and the variance of the underlying distribution from which the
sample arose. Generally, if n is greater than 25 or 30, these approximations
will be good. However, if the underlying distribution is symmetric, unimodal,
and of the continuous type, a value of n as small as 4 or 5 can yield a very
adequate approximation. Moreover, if the original distribution is approximately normal, X would have a distribution very close to normal when n
equals 2 or 3. In fact, we know that if the sample is taken from N(^i, a2), X is
exactly N(^, a2/^ for every n = 1, 2, 3 , . . . .
The following examples will help to illustrate the previous remarks and will
give the reader a better intuitive feeling about the Central Limit Theorem. In
particular, we shall see how the size of n affects the distribution of X and
V = ^ X , for samples from several underlying distributions.
Example 5.3-4 Let X ^ , X ^ , X ^ , X^ be a random sample of size 4 from the
uniform distribution (7(0, 1). Then ^ = 1/2 and a2 = 1/12. We shall compare
the graph of the p.d.f. of
Y= "LX,
with the graph of the N[n(l/2), n(l/12)] p.d.f. for n = 2 and 4, respectively.
To find the p.d.f. of Y = X ^ + X ^ , we first find the distribution function
of V. The joint p.d.f. of-Yi and X^ is
f { x ^ , x;) = 1,
0 < x, < 1, 0 <. X2 < 1.
The distribution function of Y is
G(y)=P(Y<y)=P(X,+X^<y).
I f 0 < > - < 1,
^
G(y)=
and if 1 < y < 2,
v-^
-I:J:.,'
1
G(y) = 1 -
^
Idx^dx^^-,
•
r r'
J,-i J>-»i
(2 - v}2
1 dx, dx, = 1 - *
—
—
"
2
307
Sampling Distributions and Approximations
Ch. 5
Thus the p.d.f. of Y is
(v
,(,)=GW=^
0 < v < 1,
,^^
Adding the point g(l) = 1, we obtain the triangular p.d.f. that is graphed in
Figure 5.3-l(a). In this figure the N[2(l/2), 2(1/12)] p.d.f. is also graphed.
308
I
The Central Limit Theorem
The p.d.f. of y = Xi + X^ + X , + X^ could be found in a similar, but
somewhat more difficult, manner. It is
y_
0<y<1,
6'
3
2
-3y + 12y - lly + 4
9W = i
1 < V < 2,
2
3j,a - 24y + 60y - 44
3
2 < y < 3,
2
-y + 12y - 48y + 64
3 ^ y < 4.
This p.d.f. is graphed'in Figure 5.3-l(b) along with the N[4(1/2), 4(1/12)]
p.d.f. If we are interested in finding F(1.7 <; Y < 3.2), this could be done by
evaluating
1 g(y) dy,
r-
Jl-7
which is tedious (see Exercise 5.3-10). It is much easier to use a normal
approximation.
In Example 5.3-4 we showed that even for a small value of n, like n = 4, the
sum of the sample items has an approximate normal distribution. The following example illustrates that for some underlying distributions n must be quite
large. In order to keep the scale on the horizontal axis the same for each value
of n, we will use the following result.
Let f { x ) and F(x) be the p.d.f. and distribution function of a random variable, X , of the continuous type having mean y. and variance <r2. Let
W = (X —
G(w) == P(W <w)= p(x—ajj- < w}
\
/
= P(X <vw+fi)= F(ffw + /i).
Thus the p.d.f. of W is given by
g(w) = F'(cw + f^)= of (aw + /i).
309
Sampling Distributions and Approximations
Ch. 5
Example 53-5 Let X ^ , X ^ , ..., A\oo be a random sample of size 100 from
a chi-square distribution with one degree of freedom. If
Y= f^X,,
then y is ^(n), and £(r) = n, Var(V) = In. Let
^y-'.
V2n
The p.d.f. of W is given by
^^^^^""'e-^^,
-./^2,;<>V<«.
r@2Note that w > - n/^/2n corresponds to y > 0. In Figure 5.3-2(a) and (b), the
graph of W is given along with the N(0, 1) p.d.f. for n = 20 and 100, respectively.
The next example will also help to give the reader a better intuitive feeling
about the Central Limit Theorem. We shall again see how the size of n affects
the distribution of W = (X - ^/(a/^/n).
Example 5.3-6 It is often difficult to find the exact distribution of W =
(X —
distribution of W by simulating random samples on the computer. Let A";,
A" 2, ..., X^ denote a random sample of size n from the distribution with
p.d.f./(x), mean ^i, and variance (T2. We shall generate 1000 random samples
of size n from this distribution and compute a value of W for each sample,
thus obtaining 1000 observed values of W. A histogram of these 1000 values
is constructed using 21 intervals of equal length. We depict the results of this
experiment as follows:
(1) In Figure 5.3-3, f ( x ) = (x + 1)/2, - 1 < x < 1, fi = 1/3, a2 = 2/9, for
n = 2 and 12 (see Exercise 8.5-10).
(2) In Figure 5.3-4,/(x) = (3/2)x2, - 1 < x < 1, ii = 0, <r2 = 3/5, for n = 2
and 12 (see Exercise 8.5-9).
;
The N(0, 1) p.d.f. has been superimposed on each histogram.
,
Note very clearly that these examples have not proved anything. They are
presented to give evidence of the truth of the Central Limit Theorem. So far all
the illustrations have concerned distributions of the continuous type.
However, the hypotheses for the Central Limit Theorem do not require the
310
I
The Central Limit Theorem
distribution to be continuous. We shall consider applications of the Central
Limit Theorem for discrete-type distributions in the next section.
Exercises
5.3-1 Let X be the mean of a random sample of size 12 from the uniform distribution
on the interval (0, 1). Approximate P(l/2 < X < 2/3).
311
5.3-2 Let Y = A\ + X^ + - - •
the distribution whose p.d.f. is/fx) = (3/2)x2, —
P ( - 0 . 3 ^ Y ^ 1.5).
5.3-3 Let X be the mean of a random sample of size 36 from an exponential distribution with mean 3. Approximate P(2.5 < X < 4).
312
The Central Limit Theorem
0.4
f
7
\/-~
-3.0
-2.4
-1,8
1
\]
0.2 -
/
y
^<
'\WO, l)
^
/
^
^
-1,2
-0.6
n= 12
( b)
0.6
1.2
>^
2.4
1.8
3,0
F igur e 5 3 -4
53-4 Approximate F(39.75 $ X $ 41.25), where X is the mean of a random sample of
size 32 from a distribution with mean f i = 40 and variance o-2 = 8.
5.3-5 Let Xj, X;, ..., X ^ y be a random sample of size 18 from a chi-square distribution with r = 1. Recall that f t = 1, a2 = 2.
(a) How is V = ^ X , distributed?
313
Sampling Distributions and Approximations
Ch. 6
(b) Using the result of part (a), we see from Table V in the Appendix that
P{Y ^ 9.390) = 0.05
and
P(Y < 34.80) = 0.99.
Compare these two probabilities with the approximations found using the
Central Limit Theorem.
53-6 A random sample of size n = 18 is taken from the distribution with p.d.f.
f ( x ) = 1 - x/2,0 < x < 2.
(a) Find ^ and ff 2 .
(b» Find, approximately, P(2/3 £
5.3-7 Let X equal the maximal oxygen intake of a human on a treadmill, where the
measurements are in milliliters of oxygen per minute per kilogram of weight. Assume
that for a particular population the mean of X is f i = 54.030 and the standard deviation is a = 5.8. Let X be the sample mean of a random sample of size n = 47. Find
P(52.761 < X ^ 54.453), approximately.
53-8 Let Xt, X ^ , . . . , X^ be a random sample of size n = 25 from a population that
has a mean of f i = 71.43 and a variance of a2 = 56.25. Let X be the sample mean.
Find
(a) E{X\_
(b) Var(X).
(c) P(68.91 < X ^ 71.97), approximately.
53-9 Let X equal the birth weight in grams of a baby born in the Sudan. Assume that
E(X) = 3320 and Var(Z) = 6602. Let X be the sample mean of a random sample of
size n = 225. Find P(3233.76 s£ X ^ 3406.24), approximately.
5.3-10 In Example 5.3-4, compute P(1.7 ^ Y <. 3.2) and compare this answer with the
normal approximation of this probability.
5.4
Approximations for Discrete Distributions
In this section we illustrate how the normal distribution can be used to ;
approximate probabilities for certain discrete-type distributions. One of the i
most important discrete distributions is the binomial distribution. To see how |
the Central Limit Theorem can be applied, recall that a binomial random |
variable can be described as the sum of Bernoulli random variables. That is, |
let -YI, X ^ , . . . , X^ be a random sample from a Bernoulli distribution with a |
mean ju = p and a variance a2 = p(l —
314
Approximations for Discrete Distributions
Figure 5.4-1
is b(n, p). The Central Limit Theorem states that the distribution of
,/np(l - p)
,/p(l - p)/n
is N(0, 1) in the limit as n -»• co. Thus, if n is "sufficiently large," probabilities
for the binomial distribution b{n, p) can be approximated using the normal
distribution N[np, np(l —
large " if np > 5 and n(l —
Note that we shall be approximating probabilities for a discrete distribution
with probabilities for a continuous distribution. Let us discuss a reasonable
procedure in this situation. For example, if V is N{^1, a2), P(a < V < b) is
equivalent to the area bounded by the p.d.f. of V, the v axis, v = a, and y = b.
If V is b(n, p), recall that the probability histogram for Y was denned as
follows. For each y such that k —
^-fe)-^-^'
fe
=o'l^•••'"•
Then P(Y == k) can be represented by the area of the rectangle with a height of
P(Y = I;) and a base of length 1 centered at k. Figure 5.4-1 shows the graph of
the probability histogram for the binomial distribution b(3, 1/3). When using
the normal distribution to approximate probabilities for the binomial distribution, areas under the p.d.f. for the normal distribution will be used to approximate areas of rectangles in the probability histogram for the binomial
distribution.
Example 5.4-1 Let Y be t>(10, 1/2). Then, by the Central Limit Theorem,
P(a < Y < b) can be approximated using the normal distribution with mean
10(1/2) = 5 and variance 10(1/2)(1/2) = 5/2. Figure 5.4-2 shows the graph of
the probability histogram for h(10, 1/2) and the graph of the p.d.f. of
315
Sampling Distributions and Approximations
0.15-
/
/
0.10
1
\
\
^
0.05-
-
W,})
/ ^
0.20-
0
]
2
3
4
5
6
7
\
^
8
^ ,
9
10
11
Figure 5.4-2
N(5, 5/2). Note that the area of the rectangle whose base is
H'*l)
and the area under the normal curve between k - 1/2 and k + 1/2 are
approximately equal for each integer k.
Example 5.4-2 Let Y be fc(18, 1/6). Because up = 18(1/6) = 3 < 5, the
normal approximation is not as good here. Figure 5.4-3 illustrates this by
depicting the probability histogram for i)(18, 1/6) and the p.d.f. of N(3, 5/2).
0.05
^
4
5
6
Figure 5.4-3
316
7
8
9
10
5.4
Approximations for Discrete Distributions
Example 5.4-3 Let Y have the binomial distribution of Example 5.4-1 and
Figure 5.4-2, namely 6(10, 1/2). Then
P(3 < Y < 6) = P(2.5 < Y s; 5.5)
because P(Y = 6) is not in the desired answer. But the latter equals
Pi2'5——5 < Y—^ <5'5—5) »
\^10/4 yio/4 yi0/4/
= 0.6240 - 0.0570 = 0.5670.
Using the binomial formula, we find that P(3 < Y < 6) = 0.5683.
Example 5.4-4 Let Y be 6(36, 1/2). Then
P(12 < V <; 18) = P(12.5 < V < 18.5)
/12.5-18
/12.5 - 18 'r - 18 18.5 - 1S\
\ ^9
- ^9 ^ )
"i-^^
»1>(0.167)- W, -1.833)
= 0.5329.
Note that 12 was increased to 12.5 because P(Y = 12) is not included in the
desired probability. Using the binomial formula, we find that
P(12 < Y < 18) = P(13 < V < 18) = 0.5334.
Also,
P(Y = 20) = P(19.5 < V < 20.5)
_/19.5 - 18 V - 18 20.5 - 18\
=P —
—
j
=
— < ——-/=- < ——T=—
\ ^9
^/9
^/9 1
w 1)(0.833) - <D(0.5)
= 0.1060.
Using the binomial formula, we have P(Y = 20) = 0.1063.
317
Sampling Distributions and Approximations
Ch. 5
Note that, in general, if Y is f)(n, p),
(k + l / 2 - n p \
P(Y <. k) »
^-,
and
P(Y < k)
- np\
^k - 1/2
fnpq
I
We now show how the Poisson distribution with large enough mean can be
approximated using a normal distribution.
Example 5.4-5 A random variable having a Poisson distribution with
mean 20 can be thought of as the sum Y of the items of a random sample of
size 20 from a Poisson distribution with mean 1. Thus
w^^
,/20
has a distribution that is approximately N(0, 1), and the distribution of Y is
approximately N(20, 20) (see Figure 5.4-4). So, for illustration,
P(16 < V S 21) = P(16.5 <. Y < 21.5)
_
A6.5 - 20
\ ^20
ss
y-20
^/20
-
21.5 - 20\
V'20 /
»(1(0.335)-<I'(-0.783)
= 0.4142.
Note that 16 is increased to 16.5 because Y = 16 is not included in the event
16 < r < 21. The answer using the Poisson formula is 0.4226.
In general, if Y has a Poisson distribution with mean ^, then the distribution
of
" ^
is approximately N(0, 1) when ). is sufficiently large.
Exercises
5.4-1 Let the distribution of Y be b(25, 1/2). Find the following probabilities in two
ways: exactly using Table II and approximately using the Central Limit Theorem.
Compare the two results in each of the three cases.
318
Approximations for Discrete Distributions
I7 \
\1
/ \t
/
12
14
^
6
8
10
/
7
\
\
\ Poisson, \ = 20
\A' (20,20)
\
\
\
16
18
20
22
24
26
,'b-^-
28
30
32
34
Figure 5.4-4
(a) P(10 < V ^ 12),
(b) P(12< r<15),
(c) P(V = 12).
5.4-2 Let Y be &(18,1/3). Use the normal distribution to approximate
(a) P(4 < V < 6),
(b) P(5 < r < 8),
(c) P(Y = 7).
5.4-3 Suppose that 20% of the American public believes that we could survive a
nuclear war. Let Y equal the number of people in a random sample of size n = 25
who believe this way. Use the Central Limit Theorem to find approximately
P(6 < V < 9) and compare this probability with that found using Table II. Since
b(25, 0.2) is a skewed distribution, the approximation is not as good as for the symmetric binomial distribution (i.e., when p = 0.5),
5.4-4 A die is rolled independently 240 times. Approximate the probability that
(a) more than 40 rolls are fives,
(b) the number of twos and threes is from 75 to 83, inclusive.
5.4-5 Let X ^ , X ^ , .,., X^ be a random sample of size 48 from the distribution with
p.d.f./(x) = 1/x2, 1 < x < oo. Approximate the probability that at most 10 of these
random variables have values greater than 4.
HINT : Let the dh trial be a success if X , > 4, i = 1,2,..., 48.
319
Sampling Distributions and Approximations
Ch. 5
5.4-6 A candy maker produces mints that have a label weight of 20.4 grams. Assume
that the distribution of the weights of these mints is N{21.37, 0.16).
(a) Let X denote the weight of a single mint selected at random from the production
line. Find P(X < 20.857).
(b) During a particular shift 100 mints are selected at random and weighed. Let Y
equal the number of these mints that weigh less than 20.857 grams. Find approximately P(Y < 5).
(c) Let X equal the sample mean of the 100 mints selected and weighed on a particular shift. Find P(21.31 ^ X ^ 21.39).
5.4-7 Let X equal the number of alpha particles emitted by barium-133 per second and
counted by a Geiger counter. Assume that X has a Poisson distribution with X = 49.
Find approximately P(45 < X < 60).
5.4-8 Let X equal the number of alpha particles counted by a Geiger counter during 30
seconds. Assume that the distribution of X is Poisson with a mean of 4829. Find
approximately P(4776 -£ X <. 4857).
5.4-9 Let A"], X-i, ..., X^o be a random sample of size 30 from a Poisson distribution
with a mean of 2/3. Approximate
(a) P ( l 5 < £-y,£22),
(b)
(c)
<
<
30
\
21^ S^-<27|,
30
\
15 £
i==1
/
5.4-10 In the casino game of roulette, the probability of winning with a bet on red is
p = 18/38. Let V equal the number of winning bets out of 1000 independent bets
that are placed. Find approximately P(Y > 500).
5.4-11 Let X denote the payoff for $1 bets in the game of chuck-a-luck. Then X can
equal -1, 1, 2, or 3, such that E(X) = -0.0787 and Var(X) = 1.2392. Given 300
independent observations of X , say X , , X^ ..., X ^ , let Y = ^?^ X,. Then Y
represents the amount won after 300 bets. Find approximately
(a) P ( V ^ -21),
(b) P(r>21).
5.4-12 Let Y equal the number of $499 prizes won by a gambler after placing n straight
independent $1 bets in the Michigan daily lottery, in which a prize of $499 is won
with probability 0.001. Thus V is b{n, 0.001). If n = 4200, a gambler is behind if
V <. 8; use the Poisson distribution to approximate P(Y < 8).
5.4-13 If X is b{l00,0.1), find the approximate value of P(12 < X ^ 14) using
(a) the normal approximation,
(b) the Poisson approximation,
(c) the binomial p.d.f.
320
5,5
Limiting Moment-Generating Functions
5.4-14 Let X ^ , X ^ , . . . , X^ be a random sample of size 36 from the geometric distribution with p.d.f./M = (1/4)*' '(3/4), x = 1, 2, 3, . . . . Approximate
(a)P('46£ ^X,£49\
(b) P(1.25 £
HINT: Observe that the distribution of the sum is of the discrete type.
S.4-15 A die is rolled 24 independent times. Let V be the sum of the 24 resulting values.
Recalling that V is a random variable of the discrete type, approximate
(a) P(Y 2 86),
(b) P(V < 86),
(c) P(70 < y < 86).
*5.5
Limiting Moment-Generating Functions
We would like to begin this section by showing that the binomial distribution
can be approximated by the Poisson distribution when n is sufficiently large
and p fairly small by taking the limit of a moment-generating function. Consider the moment-generating function of Y, which is b{n, p). We shall take the
limit of this as n-> oo such that up =^isaconstant;thusp-»O.The momentgenerating function of Y is
M ( f ) = (1 - p + pe')'.
Because p = ^/n, we have that
M^fl-^c'T
L
" " J
=[i^]"
Since
limfl+''Y=^
we have
lim M(() = e"
321
Sampling Distributions and Approximations
Ch. 5
which exists for all real (. But this is the moment-generating function of a
Poisson random variable with mean L Hence this Poisson distribution seems
like a reasonable approximation to the binomial one when n is large and p is
small. This approximation is usually found to be fairly successful if n > 20 and
p < 0.05 and very successful if n > 100 and np < 10. Obviously, it could be
used in other situations too; we only want to stress that the approximation
becomes better with larger n and smaller p.
Example 5.5-1 Let Y be fc(50, 1/25). Then
P(^D=V ^oyy =0.400.
/24\50
/ 1 \/24\49
Since 2 = np = 2, the Poisson approximation is
P(Y < 1)% 0.406,
from Table III in the Appendix.
The preceding result illustrates the theorem we now state: If a sequence of
moment-generating functions approaches a certain one, say M(t), then the limit of
the corresponding distributions must be the distribution corresponding to M(i).
This statement certainly appeals to one's intuition! In a more advanced
course, the proof of this theorem is given and there the existence of the
moment-generating function is not even needed, for we would use the characteristic function ^i(t) = £(e'") instead.
The preceding theorem is used to prove the Central Limit Theorem. To help
in the understanding of this proof, let us first consider a different problem, that
of the limiting distribution of the mean X of a random sample X^, X ^ , . . . , Xy
from a distribution with mean a. If the distribution has moment-generating
function M(r), the moment-generating function of X is [M(t/n)]". But, by
Taylor's expansion, there exists a number ti between 0 and t/n such that
M ( " ) = M(0) + M'(ti) L
W
"
_ ^ at ^ [M'(ti) - M'(0)]t
because M(0) = 1 and M'(0) = in. Since M'(t) is continuous at t = 0 and since
ti -»0 as n -* oo, we know that
lim [M'(ti) - M'(0)] = 0.
322
5.5
Limiting Moment-Generating Functions
Thus, using a result from advanced calculus, we obtain
,-hm r./^M"
r \
J, 1 +^—
[M'(f,) - M'{0)-]t\"
M = hm
.-a, L WJ
n^ [
n
n
J
=e"',
for all real t. But this limit is the moment-generating function of a degenerate
distribution with all of the probability on ^.. Accordingly, X has this limiting
distribution, indicating that X converges to ^ in a certain sense. This is one
form of the law of large numbers.
We have seen that, in some probability sense, X converges to ^ in the limit,
or, equivalently, X —
by some function of n so that the result will not converge to zero. In our
search for such a function, it is natural to consider
^ _ X - fi _ ^/n{X -^ ^Y-n^i
a/^/n
o
^nrr '
where Y is the sum of the items of the random sample. The reason for this is
that W is a standardized random variable and has mean 0 and variance 1 for
each positive integer n.
Proof of the Central Limit Theorem.
We first consider
W'1-'}}} /
...„-^...„l
•
[(W\-'[(7^1
-«.„
- ^ - . ...../• v^
[(W}}'W^^
£[exp (tW)-] == E^exp (
—
—
—
)
( ^ X, - nfi
I L\^/na/\i=
which follows from the mutual independence of X i , X ^ , . . . , X ^ . Then
a
.[expW] =[.(-)]",
where
m(t) = f{exp r/^^——^ll
•[<'
-„<-<„,
-h < t < h,
323
Sampling Distributions and Approximations
Ch.
is the common moment-generating function of each
^^,
,=l,2,...,n.
(T
Since £(^) = 0 and £(Y?) = 1, it must be that
m(0) = 1,
m'(0) = E \ ' ~ 1 } = 0,
\ " /
m"(0) = E\ (x!—p-\ = 1.
L\ " ! J
Hence, using Taylor's formula with a remainder, we can find a number t
between 0 and t such that
.(0=.(0)+»w+'"^=l+''f(il)t2.
By adding and subtracting t'A we have that
^^^m^y.
Using this expression of m(t) in £[exp (tW)], we can represent the momeni
generating function of W by
£[exp (tW)-] = {l + ' (——) + \ [m"(ti) - l]f——Y}"
I
'• \^/n/
-'
\Vn/ J
f , t 2 [""(ti) - lit 2 ')"
=
^
1
+
—
+
—
—
—
—
—
—
>
,
^
In
2n
J
r,
^,
-^nh<t<^nh,
where now t^ is between 0 and i/^/n. Since ffi"(t) is continuous at t = 0 an
r; -* 0 as n -> oo, we have that
lim [m"((i) - 1] = 1 - 1 = 0.
Thus, using a result from advanced calculus, we have that
limECexp^fl^im^l^'"""-'-1^
n--co
n-oi {,
in
In
J
^Ji^l"^,
.-„ l " J
324
5.5
Limiting Moment-Generating Functions
for all real (. This means that the limiting distribution of
^ ^
Y n i^.-^
x
- ^ _i=l_______
a/^/n
^/no-
is N(0, 1). This completes the proof of the Central Limit Theorem.
Q
Examples of the use of the Central Limit Theorem as an approximating
distribution have been given in Sections 4.5 and 4.6.
Exercises
5.5-1 Let V be the number of defectives in a box of 50 articles taken from the output of
a machine. Each article is defective with probability 0.01. What is the probability
that V = 0,1, 2, or 3
(a) using the binomial distribution?
(b) using the Poisson approximation?
5.5-2 The probability that a certain type of inoculation takes effect is 0.995. Use the
Poisson distribution to approximate the probability that at most 2 out of 400 people
given the inoculation find that it has not taken effect.
HINT: Let p = 1 - 0.995 = 0.005.
5.5-3 Let S2 be the sample variance of a random sample of size n from N(^i, a2). Show
that the limit, as n -* GO, of the moment-generating function of S2 is e"2'; thus, in the
limit, the distribution of S1 is degenerate with probability 1 at o-2.
5.5-4 Let V be ^(n). Use the Central Limit Theorem to demonstrate that W =
{Y - n)A/2n has a limiting distribution that is JV(0, •
HINT: Think of Y as being the sum of a random sample from a certain distribution,
5.5-5 Let Y have a Poisson distribution with mean 3n. Use the Central Limit Theorem
to show that W = (Y —
325