Download Independent random variables

8 Independent random variables.
Independent Random Variables. Recall that two random variables S and T are independent if knowledge
of the values of one of them doesn't influence the probability that the other assumes various values, i.e.
Pr{S  A, T  B} = Pr{S  A} Pr{T  B}
(1)
for any two sets A and B. In particular, one has
(2)
Pr{ a < S < b, c < T < d} = Pr{a < S < b} Pr{c < T < d}
or
(3)
Pr{ a < S < b | c < T < d} = Pr{a < S < b}
for any numbers a, b, c and d. Thus the conditional probability that the value of S is in one interval given
that the value of T is in another is no different from the probability that S is in the first interval with no
knowledge of the values of T.
If S and T are independent and have density functions fS(s) and fT(t), then (3) can be written as
b
Pr{ a < S < b, c < T < d} =
d
 fS(s) ds 
 fT(t) dt

a
c
or
Pr{ (S, T)  A } =
(4)


 fS(s) fT(t) dsdt
A
where A is the rectangle consisting of all points (s,t) such that a < s < b, c < t < d. If we observe both S and
T then we can put the two observations into an ordered pair (S, T). Equation (4) says that the probability
that the ordered pair lies in some rectangle is just the double integral of fS(s) fT(t) over the rectangle. In fact
(4) holds for any set A in the plane. We can use this to answer questions involving two independent random
variables.
Example 1. Suppose male customers arrive at an bank at a rate of 20 per hour and female customers arrive
at the bank at a rate of 30 per hour. Suppose the two arrival rates are exponentially distributed and
independent. What is the probability that the next customer is a male?
Let
S = time (in min) until the next male customer arrives at a bank
T = time until the next female customer arrives at a bank.
f(s) = probability density function of S = e-s/3/3for s > 0
8-1
g(t) = probability density function of T = e-t/2/2for t > 0
We want to know Pr{(S,T)  A} where A = {(s,t): s < t}. Using (4) we have

Pr{(S,T)  A} =

 f(s,t) dsdt = 


e-s/3 e-t/2
[ 
 3 2 dt ] ds =
0
A


t=s
-s/3
 e e-s/2 ds
 3
0

=
1
2
2

 e-5s/6 ds = - 5 e-5s/6 0 = 5
3
|
0
The following proposition is a generalization of Example 1.
Proposition 1. If S and T are independent exponential random variables with means 1/ and 1/
respectively then
(5)
Pr{S < T} =

+
More generally, if T1, ..., Tn are independent exponential random variables with means 1/1, …, 1/n
respectively, then
(6)
Pr{T1 is less than all of T2, ..., Tn} =
1
1 +  + n
Proof. S and T have density functions fS(t) = e-t and fT(t) = e-t for s  0 and t  0 and fS(t) = fT(t) = 0 for
s < 0 or t < 0. Since S and T are independent, it follows form (5) that
-t -t
 e e
fS,T(s,t) = 
0
if s  0 and t  0
if s < 0 or t < 0
If A = {(s,t): s < t}, then
Pr{S < T} =
 

0 s
0
-t -t
-t -t

 e e dsdt

 f(s,t) dsdt = 

 e e dsdt = 
A

=

-(+)t
dsdt = e-(+)t

 e
+
0
|

0
=

+
This proves (5). We prove (6) by induction. The case n = 2 is (5). Suppose it is true for n – 1. Note that
Pr{T1 is less than all of T2, ..., Tn} = Pr{T1 < R} where R = min{T2, ..., Tn}. By Proposition 2 in the next
section R is an exponential random variable with mean 1/(2 +  + n). So (6) follows from (5). //
Here is an example similar to Example 1.
8-2
Example 2. The time it takes Bob to do his Math homework is uniformly distributed between 0 and 1 hour.
The time it takes him to do his English homework is uniformly distributed between 0 and 3 hours and is
independent of the time it takes to do his Math homework. On a given evening what is the probability that
Bob will finish both his Math and English homework in 2 hours?
Let
S = time for Bob to do his Math homework
T = time for Bob to do his English homework
R = S + T = time to do both Math and English homework.
fS(s) = probability density function of S =
fT(t) = probability density function of T =
1

0
 1/3

0
if 0  s  1
otherwise
if 0  t  3
otherwise
We want to find Pr{R < 2} = Pr{S+T < 2} = Pr{(S,T)  A} = 

 fS(s) fT(t) dsdt where
A
A = {(s,t}: s + t  2}. One has
fS(s) fT(t) =
if 0  s  1 and 0  t  3
 1/3

0
otherwise
Let A' = A  {(s,t): 0 s 1 and 0 t 3} = {(s,t): 0 s 1 and 0  t  2-s}. Then
Pr{R < 2} =
1
1
1
2+1


 3 dsdt = 3 × Area(A') = 3 × 1 × 2
= 1/2
A'
More Than Two Random Variables. T1, ..., Tn are independent if knowledge of the values of some of the
variables doesn't change the probability that the others assume various values, i.e.
Pr{a1 < T1 < b1, ..., an < Tn < bn} = Pr{a1 < T1 < b1}  Pr{an < Tn < bn}
Properties of expected values. The following proposition includes some of the properties of expected
values of discrete random variables that also hold for continuous random variables.
8-3
Proposition 2. Let X and Y be continuous random variables. Let c be a real number and z = g(x) and
z = h(x, y) be real valued functions. Then

(7)

 g(x)fX(x) dx
E(g(X)) =
-
(8)
E(X + Y) = E(X) + E(Y)
(9)
E(cX) =
cE(X)
(10)
E(XY) =
E(X)E(Y)
if X and Y are independent
The proofs are somewhat involved and will be omitted.
Sums of independent random variables. We often form new random variables by performing the usual
mathematical operations to existing random variables. If they are continuous random variables we often are
interested in the density function of the new random variable. We shall use sums to illustrate the technique
by which these are found. Here is an example to illustrate this operation.
Example 3. On the average it takes Bob 30 minutes to do his mathematics homework and 3 hours to do his
English homework. Suppose the times to do the homework in either class are exponential random variables
and they are independent. Tonight he both mathematics and English homework. Let
R = the time to do both assignments.
Find
h(r) = the probability density function of R
the probability that he will finish both assignments in 2.5 hours.
Solution. Let
S = time (in hours) to do his mathematics homework
T = time to do his English homework
f(s) = probability density function of S = 2e-2s for s > 0.
g(t) = probability density function of T =
e-t/3
for t > 0
3
Note that
R = S+T
8-4
The method by which we find h(r) is to first find H(r) = Pr{R  r} the cumulative distribution function of R.
Then h(r) = H'(r). One has
H(r) = Pr{S + T  r} = Pr{(S,T)  A}
where A = {(s,t): s + t  r}. Using (4) one has
 r-s
H(r) =



 f(s)g(t) dsdt = 

 f(s)g(t) dtds = 
 f(s)G(r-s) ds
- - 
A
-
where G(t) is the cumulative of T. So



 

 f(s)G(r-s) ds =   r [f(s)G(r-s)] ds = 
 f(s)g(r-s) ds
d
h(r) = H'(r) =
dr
-
-
-
where we have used Leibniz rule for differentiating under the integral sign. We can summarize this in the
following Proposition.
Proposition 3. Let S and T be independent random variables with density functions f(t) and g(t). Let
R = S + T and let h(r) be the density function of R. Then

(11)
h(r) =

 f(s)g(r-s) ds
-
If in addition f(s) = 0 for s < 0 and g(t) = 0 for t < 0 then h(r) = 0 for r < 0 and for r > 0 one has
r
(12)
h(r) =

 f(s)g(r-s) ds
0
The integrals (11) and (12) are called the convolutions of the functions f(s) and g(t). Convolutions come up
a lot in the solution of differential equations as well as in probability.
Now let's apply this to Example 3. We can use the formula (12) since the density functions of S and T are
zero for negative values. For r > 0 the integral (12) becomes
r
r
h(r) =
 f(s)g(r-s) dsdt =

0
-(r-s)/3
2e  - 3e
 3  5
-5s/3
|
s = r
s = 0
-r/3
0
0
-r/3
=
r

2e  e-5s/3 ds
 2e-2s e

 3  ds =  3  

=
-r/3
-5r/3
-r/3
-2r
2e  3 - 3e  = 2e - 2e
3
5
5
5
5



with h(r) = 0 for r < 0. So
8-5
2.5
 2e - 2e  dr = - 6e + e 
5 
5
 5
 5
-r/3
Pr{R < 2.5} =
-2r
-r/3
0
8-6
-2r
|
2.5
0
= -
6e-5/6 e-5 6 1
+ +  0.47983
5
5 5 5