Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Further Topics on Random Variables Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Probability Derived Distribution Often, a random variable is defined through its relation to another random variable. Let X be a continuous random variable and Y = g(X) Distribution of Y can be derived from distribution of X. Two steps FY (y) = P(Y ≤ y) = P(g(X) ≤ y) fY (y) = Prof. C. Chen d FY (y) dy Further Topics on Random Variables Example 4.1 Square Root of Uniform Random Variable Let X be uniform on [0, 1], and √ Y = X What is the PDF of Y ? FY (y) = P(Y ≤ y) √ = P( X ≤ y) = P(X ≤ y 2 ) = FX (y 2 ) = y2 ⇒ fY (y) = d FY (y) = 2y, dy Prof. C. Chen 0≤y≤1 Further Topics on Random Variables Example 4.2 Trip Duration John Slow is driving from Boston to New York, a distance of 180 miles at a constant speed X, whose value is uniformly distributed between 30 and 60 miles per hour. What is the PDF of the duration of the trip Y ? Y = ⇒ FY (y) = P(Y ≤ y) = P =1−P X ≤ =2− 180 y 180 X 180 180 ≤y =P X≥ X y = 1 − FX 180 y =1− 180 y 6 y ⇒ fY (y) = Prof. C. Chen 6 , y2 3≤y≤6 Further Topics on Random Variables − 30 30 Example 4.3 Square Suppose X is a random variable with PDF fX (x), and Y = X2 What is the PDF of Y ? FY (y) = P (Y ≤ y) = P X2 ≤ y √ = P (|X| ≤ y) √ √ = P (− y ≤ X ≤ y) √ √ = FX ( y) − FX (− y) ⇒ fY (y) = d 1 1 √ √ FY (y) = √ fX ( y) + √ fX (− y) , dy 2 y 2 y Prof. C. Chen y≥0 Further Topics on Random Variables Linear Functions Suppose X is a random variable with PDF fX (x), and Y = aX + b where a 6= 0 and b are scalars. Then y−b 1 fX fY (y) = |a| a For a > 0 FY (y) = P(Y ≤ y) = P(aX + b ≤ y) y−b y−b = FX =P X≤ a a 1 y−b ⇒ fY (y) = fX a a Similarly, for a < 0 1 y−b fY (y) = − fX a a Prof. C. Chen Further Topics on Random Variables Linear Function of an Exponential Random Variable Suppose X is an exponential random variable with parameter λ, and Y = aX + b What is the PDF of Y ? y−b 1 fX |a| a y−b λ −λ( y−b ) a = ≥0 ? e : 0 a |a| fY (y) = For a > 0 and b = 0 ⇒ fY (y) = λ −( λ )y e a , a i.e. y>0 λ a Y ∼ exponential Prof. C. Chen Further Topics on Random Variables Example 4.5 Linear Function of a Gaussian Suppose X is a normal random variable with mean µ and variance σ 2 , and Y = aX + b What is the PDF of Y ? fY (y) = 1 fX |a| y−b a 2 ( y−b −µ) 1 1 − a 2 2σ √ = e |a| 2πσ (y−(aµ+b))2 1 1 √ e− 2a2 σ2 = |a| 2πσ Indeed Y ∼ N (aµ + b, a2 σ 2 ) Prof. C. Chen Further Topics on Random Variables Monotonic Function Suppose X is a random variable and Y = g(X) where g(·) is strictly monotonic on the image of X. Let the inverse function of g(·) be h(·), meaning Y = g(X) ⇒ X = h(Y ) Then, then PDF of Y is related to the PDF of X by dh(y) dy fY (y) = fX (h(y)) Prof. C. Chen Further Topics on Random Variables Proof The event {X ∈ [h(y), h(y) + dx]} corresponds to the event {Y ∈ [y, y + dy]} ⇒ fY (y)|dy| = fX (h(y))|dx| dx dy dh(y) = fX (h(y)) dy ⇒ fY (y) = fX (h(y)) Prof. C. Chen Further Topics on Random Variables Example 4.2 (continued) Let us apply this result to Example 4.2. X ∼ uniform(30, 60), Y = ⇒ X= 180 = g(X) X 180 = h(Y ) Y dh(y) ⇒ fY (y) = fX (h(y)) dy 1 180 · 30 y 2 6 = 2, 3 ≤ y ≤ 6 y = Prof. C. Chen Further Topics on Random Variables Example 4.6 Square of Uniform Random Variable Let X be a continuous uniform random variable on the interval [0, 1], and Y = X2 What is the PDF of Y ? Y = X2 ⇒ X= √ Y = h(Y ) dh(y) dy ⇒ fY (y) = fX (h(y)) 1 =1· √ , 2 y Prof. C. Chen 0≤y≤1 Further Topics on Random Variables Functions of Multiple Random Variables A function of multiple random variables is, again, a random variable. The PDF of such a random variable may be derived, often through consideration of CDF. Prof. C. Chen Further Topics on Random Variables Example 4.7 Archers Two archers shoot at a target. The distance of each shot from the center of the target is uniformly distributed from 0 to 1, independent of the other shot. What is the PDF of the distance of the losing shot from the center? Let the distances of the shots from center be X and Y . The distance of the losing shot is Z = max(X, Y ) ⇒ {Z ≤ z} = {X ≤ z} ∩ {Y ≤ z} ⇒ FZ (z) = FX (z)FY (z) = 0, z2, 1, ⇒ fZ (z) = 2z, Prof. C. Chen z<0 0≤z≤1 z>1 0≤z≤1 Further Topics on Random Variables Example 4.8 Ratio of Uniform Random Variables Let X and Y be independent continuous uniform random variables over the interval [0, 1]. What is the PDF of Z= Y X 0, Y FZ (z) = P(Z ≤ z) = P ≤ z = z2 , X 1 − ⇒ fZ (z) = 0, 1 2, 1 , 2z 2 Prof. C. Chen z<0 0≤z≤1 1 2z , z > 1 z<0 0≤z≤1 z>1 Further Topics on Random Variables Example 4.9 Difference of Exponential Variables Romeo and Juliet have a date at a given time. Each will arrive at the meeting place with a random delay (X for Juliet, Y for Romeo) that is exponentially distributed with parameter λ. What is the PDF of Z =X −Y For z > 0 FZ (z) = P(X − Y ≤ z) Z ∞ Z z+y fX,Y (x, y)dx dy = 0 0 Z ∞ = λe−λy Z0∞ λe−λx dx dy 0 λe = Z z+y −λy 1 − e−λ(z+y) dy 0 = 1 − e−λz Z ∞ 0 Prof. C. Chen 1 λe−λ(2y) dy = 1 − e−λz 2 Further Topics on Random Variables For z < 0 FZ (z) = P(X − Y ≤ z) Z ∞ Z ∞ fX,Y (x, y)dy dx = x−z 0 Z ∞ = Z ∞ λe −λx λe −λx −λ(x−z) x−z 0 Z ∞ = 0 = eλz λe−λy dy dx Z ∞ e dx λe−λ(2x) dx 0 1 = eλz 2 ⇒ fZ (z) = dFZ (z) λ = e−λ|z| dz 2 Prof. C. Chen Further Topics on Random Variables Sum of Independent Random Variables The PDF (PMF) of the sum of two independent random variables is the convolution of PDFs (PMFs). Prof. C. Chen Further Topics on Random Variables Discrete Convolution Let X and Y be independent discrete random variables with PMF pX (x) and pY (y). The PMF of Z = X + Y is pZ (z) = X pX (x)pY (z − x) x pZ (z) = P(X + Y = z) X = P(X = x, Y = y) {(x,y)|x+y=z} = X P(X = x, Y = z − x) x = X P(X = x)P(Y = z − x) x = X pX (x)pY (z − x) x Prof. C. Chen Further Topics on Random Variables Continuous Convolution Let X and Y be independent continuous random variables with PDF fX (x) and fY (y). The PDF of Z = X + Y is Z fZ (z) = fX (x)fY (z − x)dx The conditional CDF of Z given X = x is FZ|X (z|x) = P(Z ≤ z|X = x) = P(Y ≤ z − x) = FY (z − x) ⇒ fZ|X (z|x) = fY (z − x) ⇒ fZ (z) = Z fXZ (x, z)dx Z fX (x)fZ|X (z|x)dx = Z = Prof. C. Chen fX (x)fY (z − x)dx Further Topics on Random Variables Example 4.10 Sum of Uniform Random Variables The random variables X and Y are independent, and uniformly distributed in the interval [0, 1]. What is the PDF of Z =X +Y Z ∞ fZ (z) = −∞ ( 1, 0, if 0 ≤ x ≤ 1 otherwise ( 1, 0, if 0 ≤ (z − x) ≤ 1 otherwise fX (x) = fY (z − x) = ( ⇒ fZ (z) = fX (x)fY (z − x)dx min(1, z) − max(0, z − 1), 0, Prof. C. Chen 0≤z≤2 otherwise Further Topics on Random Variables Example 4.11 Sum of Independent Gaussians Let X and Y be independent Gaussian random variables with means µx , µy and variances σx2 , σy2 respectively, and Z =X +Y What is the PDF of Z? Prof. C. Chen Further Topics on Random Variables Z fZ (z) = Z = fX (x)fY (z − x)dx − 1 e 2πσX σY (x−µX )2 2σ 2 X − e (z−x−µY )2 2σ 2 Y dx The exponent is a second-order polynomial of z, so Z is a Gaussian. Furthermore, since X and Y are independent E[Z] = E[X] + E[Y ] = µX + µY 2 + σY2 var(Z) = var(X) + var(Y ) = σX 2 ⇒ Z ∼ N (µX + µY , σX + σY2 ) Prof. C. Chen Further Topics on Random Variables Covariance The covariance of random variables X and Y is defined by cov(X, Y ) = E[(X − E[X])(Y − E[Y ])] cov(X, Y ) = E[XY ] − E[X]E[Y ] cov(X, aY + b) = a cov(X, Y ) cov(X, Y + Z) = cov(X, Y ) + cov(X, Z) X⊥ ⊥ Y ⇒ cov(X, Y ) = 0 cov(X, X) = var(X) Prof. C. Chen Further Topics on Random Variables Example 4.13 The pair of random variables (X, Y ) takes the values of (1, 0), (0, 1), (−1, 0), (0, −1) each with probability 1/4. What is the covariance of X and Y ? Prof. C. Chen Further Topics on Random Variables Correlation Coefficient The correlation coefficient of random variables X and Y is defined by cov(X, Y ) p ρ(X, Y ) = p var(X) var(Y ) It can be shown that −1 ≤ ρ(X, Y ) ≤ 1 Prof. C. Chen Further Topics on Random Variables Example 4.14 Consider n independent tosses of a coin with probability p of a head. Let X and Y be the numbers of heads and of tails, respectively. What is the correlation coefficient of X and Y ? Prof. C. Chen Further Topics on Random Variables X +Y =n ⇒ E[X + Y ] = n ⇒ E[X] + E[Y ] = n ⇒ X − E[X] = −(Y − E[Y ]) ⇒ cov(X, Y ) = E[(X − E[X])(Y − E[Y ])] = −var(X) = −var(Y ) cov(X, Y ) var(X)var(Y ) −var(X) = var(X) = −1 ⇒ ρ(X, Y ) = p Prof. C. Chen Further Topics on Random Variables Variance of Sum of Random Variables var(X + Y ) = var(X) + var(Y ) + 2cov(X, Y ) var(X + Y ) = E[((X + Y ) − E[(X + Y )])2 ] = E[((X + Y ) − (E[X] + E[Y ]))2 ] = E[((X − E[X]) + (Y − E[Y ]))2 ] = E[(X − E[X])2 ] + E[(Y − E[Y ])2 ] + 2E[(X − E[X])(Y − E[Y ])] = var(X) + var(Y ) + 2cov(X, Y ) var n X i=1 ! Xi = n X var(Xi ) + i=1 Prof. C. Chen n X X cov(Xi , Xj ) i=1 j6=i Further Topics on Random Variables Example 4.15 Hats Consider the hat problem discussed earlier, where n people throw their hats in a box and then pick a hat at random. What is the variance of the number of people who pick their own hat? Prof. C. Chen Further Topics on Random Variables Let {Xi = 1} be the event that the ith person picks his own hat, and {Xi = 0} be the event otherwise. We have 1 n Xi ∼ Bernoulli 1 1 ⇒ var(Xi ) = 1− n n ⇒ cov(Xi , Xj ) = E[Xi Xj ] − E[Xi ]E[Xj ] 2 = P(Xi = 1, Xj = 1) − 1 n 2 = P(Xi = 1)P(Xj = 1|Xi = 1) − 1 1 = · − n n−1 1 = 2 n (n − 1) Prof. C. Chen 1 n 2 1 n Further Topics on Random Variables The number of persons picking their own hats is H = X1 + · · · + Xn ⇒ var(H) = var n X ! Xi i=1 = n X var(Xi ) + i=1 n X X cov(Xi , Xj ) i=1 j6=i 1 1 1− =n· n n 1 1 =1− + n n =1 Prof. C. Chen + n(n − 1) · 1 − 1) n2 (n Further Topics on Random Variables Conditional Expectation The conditional expectation of X given Y is E[X|Y ] E[X|Y ] takes the value of E[X|Y = y] when Y = y. It is a function of Y It is a random variable Prof. C. Chen Further Topics on Random Variables Example 4.16 Random Probability We are given a biased coin and we are told that because of manufacturing defects, the probability of heads, denoted by Y , is itself random, with a known distribution over the interval [0, 1]. We toss the coin a fixed number n of times and we let X be the number of heads obtained. Then, for any y ∈ [0, 1], we have E[X|Y = y] = ny It follows that E[X|Y ] = nY Prof. C. Chen Further Topics on Random Variables The Law of Iterated Expectation The expectation of a random variable is the expectation of its conditional expectation given another random variable. That is E[X] = E [E[X|Y ]] for any random variables X and Y . Prof. C. Chen Further Topics on Random Variables Example 4.16 (continued) Suppose that Y is uniformly distributed over the interval [0, 1]. We toss the coin a fixed number n of times. What is the expectation of the number of heads X? E[X] = E [E[X|Y ]] = E[nY ] = nE[Y ] n = 2 Prof. C. Chen Further Topics on Random Variables Example 4.17 Break a Stick We start with a stick with length l. We break it at a point which is chosen randomly and uniformly over its length, and keep the piece that contains the left end of the stick. We then repeat the same process on the piece that we are left with. What is the expectation of the length of the piece that we are left with after breaking TWICE? Let Y be the length of the stick after the first time, and X be the length of the stick after breaking twice. E[X] = E[E[X|Y ]] = E Prof. C. Chen Y 2 1 l = E [Y ] = 2 4 Further Topics on Random Variables Conditional Expectation as an Estimator The conditional expectation of X given Y X̂(Y ) = E[X|Y ] is an estimator of X based on Y . An estimation error is the difference between a random variable and its estimator Z = X̂(Y ) − X Note E[Z] = E[E[X|Y ] − X] = E[E[X|Y ]] − E[X] = 0 Prof. C. Chen Further Topics on Random Variables The Law of Total Variance The variance of a random variable is the sum of the variance of conditional expectation the expectation of conditional variance var(X) = var(E[X|Y ]) + E[var(X|Y )] Prof. C. Chen Further Topics on Random Variables Proof X̂ = E[X|Y ] ⇒ X = X̂ − (X̂ − X) = X̂ − Z ⇒ var(X) = var(X̂ − Z) = var(X̂) + var(Z) − 2cov(X̂, Z) Prof. C. Chen Further Topics on Random Variables var(X̂) = var(E[X|Y ]) var(Z) = E[Z 2 ] − E 2 [Z] = E[Z 2 ] = E[E[Z 2 |Y ]] =E E h X − X̂ 2 |Y h = E E (X − E[X|Y ])2 |Y ii = E[var(X|Y )] cov(X̂, Z) = E[X̂Z] − E[X̂]E[Z] = E[X̂Z] = E[E[X̂(Y )Z|Y ]] = E[X̂(Y )E[Z|Y ]] = E[X̂(Y ) · 0] = 0 Prof. C. Chen Further Topics on Random Variables Example 4.16 (continued) We consider n independent tosses of a biased coin whose probability of heads Y is uniformly distributed over the interval [0, 1]. What is the variance of X, the number of heads obtained? var(X) = var(E[X|Y ]) + E[var(X|Y )] = var(nY ) + E[nY (1 − Y )] = n2 var(Y ) + nE[Y (1 − Y )] 1 1 − 2 3 = n2 E[Y 2 ] − E 2 [Y ] + n E[Y ] − E[Y 2 ] 1 1 − 3 4 n2 n = + 12 6 = n2 +n Prof. C. Chen Further Topics on Random Variables Example 4.17 (continued) We break twice a stick of length l at randomly chosen points. Here Y is the length of the piece left after the first break, and X is the length after the second break. What is the variance of X? var(X) = var(E[X|Y ]) + E[var(X|Y )] Y = var 2 " Y2 +E 12 # 1 l2 1 l2 + 4 12 12 3 7 2 = l 144 = Prof. C. Chen Further Topics on Random Variables Example 4.21 Consider a continuous random variable X with the PDF given in Fig. 4.13. Define an auxiliary random variable as follows: ( Y = x<1 x≥1 1, 2, Compute the variance of X by conditioning on Y . var(X) = var(E[X|Y ]) + E[var(X|Y )] = E[(E[X|Y ] − E[E[X|Y ]])2 ] + E[var(X|Y )] = 2 X pY (i)(E[X|Y = i] − E[X])2 + i=1 1 1 5 − 2 2 4 37 = 48 = 2 X pY (i)var(E[X|Y = i]) i=1 2 1 5 2− 2 4 + Prof. C. Chen 2 + 1 1 1 4 + 2 12 2 12 Further Topics on Random Variables Moment Generating Function Prof. C. Chen Further Topics on Random Variables Transform The transform associated with a random variable X is defined by h i MX (s) = E esX If X is discrete MX (s) = X pX (x)esx x If X is continuous Z MX (s) = fX (x)esx dx A probability function of x is transformed to a function of s. Prof. C. Chen Further Topics on Random Variables Example 4.22 A Discrete Random Variable Suppose pX (x) = 1 2, 1 , 6 1, 3 x=2 x=3 x=5 What is the transform associated with X? Prof. C. Chen Further Topics on Random Variables Example 4.23 Poisson Random Variable Let X be a Poisson random variable with parameter β. What is the transform associated with X? pX (k) = e−β βk , k! h sX ⇒ MX (s) = E e k = 0, 1, 2, . . . i = ∞ X k=0 esk e−β βk k! ∞ X (βes )k = e−β =e k=0 −β βes e = eβ(e Prof. C. Chen k! s −1) Further Topics on Random Variables Example 4.24 Exponential Random Variable Let X be an exponential random variable with parameter λ. What is the transform associated with X? fX (x) = λe−λx , h ⇒ MX (s) = E esX Z ∞ = x≥0 i esx λe−λx dx 0 Z ∞ = λe−(λ−s)x dx 0 = Prof. C. Chen λ λ−s Further Topics on Random Variables Example 4.25 Linear Function Let MX (s) be the transform associated with a random variable X. What is the transform associated with the random variable Y = aX + b h MY (s) = E esY i h = E es(aX+b) h = ebs E esaX i i = ebs MX (as) Prof. C. Chen Further Topics on Random Variables Example 4.26 Gaussian Random Variable Let X be a normal random variable with mean µ and variance σ 2 . What is the transform associated with X? X = σZ + µ where Z ∼ N (0, 1). h sZ ⇒ MZ (s) = E e 1 =√ 2π i 1 =√ 2π Z ∞ Z ∞ esz e−z 2 /2 dz −∞ 2 /2 e−(z−s) es 2 /2 dz −∞ 2 /2 = es ⇒ MX (s) = eµs MZ (σs) = e Prof. C. Chen σ2 2 s +µs 2 Further Topics on Random Variables Moment Generating Function The transform associated with a random variable is also known as the moment generating function (MGF) of the random variable. Prof. C. Chen Further Topics on Random Variables Derivative and the First Moment The derivative of the MGF of a random variable X is d d MX (s) = ds Zds = At s = 0 Z fX (x)esx dx xesx fX (x)dx d MX (s) = ds s=0 Z xfX (x)dx = E[X] Prof. C. Chen Further Topics on Random Variables Higher Moments More generally, the nth order derivative of MX (s) evaluated at s = 0 is dn dn M (s) = X dsn dsn s=0 Z = Z fX (x)e dx sx s=0 n x fX (x)dx = E[X n ] Prof. C. Chen Further Topics on Random Variables Example 4.27 Consider the PMF given in Example 4.22. Find the expectation and the second moment of X. Find the expectation and the second moment of an exponential random variable with parameter λ. Prof. C. Chen Further Topics on Random Variables MX (s) = λ λ−s d MX (s) ds s=0 λ = (λ − s)2 s=0 1 = λ d2 2 ⇒ E[X ] = 2 MX (s) ds s=0 ⇒ E[X] = 2λ = (λ − s)3 s=0 2 = 2 λ Prof. C. Chen Further Topics on Random Variables Inverse Transform The transform MX (s) associated with a random variable X uniquely determines the PDF of X, assuming MX (s) is finite for all s in some interval [−a, a], where a > 0. Prof. C. Chen Further Topics on Random Variables Example 4.28 Discrete Random Variable We are told that the transform associated with a random variable X is 1 1 1 1 MX (s) = e−s + + e4s + e5s 4 2 8 8 What is the probability distribution of X? Prof. C. Chen Further Topics on Random Variables Example 4.29 Geometric Random Variable We are told that the transform associated with a random variable X is pes MX (s) = 1 − (1 − p)es What is the probability distribution of X? Prof. C. Chen Further Topics on Random Variables Example 4.30 A Mixture of Two Distributions A bank has three tellers, two of them fast, one slow. The time to assist a customer is exponentially distributed with parameter λ = 6 at the fast tellers, and λ = 4 at the slow teller. Jane enters the bank and chooses a teller at random, each one with probability 1/3. Find the PDF of the time it takes to assist Jane and the associated transform. Prof. C. Chen Further Topics on Random Variables Sum of Independent Random Variables The transform associated with the sum of independent random variables is the product of the transforms associated with those random variables. Prof. C. Chen Further Topics on Random Variables Proof Suppose X ⊥ ⊥Y. Z =X +Y h ⇒ MZ (s) = E esZ i h = E es(X+Y ) h = E esX esY h = E esX i i i h E esY i = MX (s)MY (s) More generally, if X1 , . . . , Xn are independent MX1 +···+Xn (s) = MX1 (s) . . . MXn (s) Prof. C. Chen Further Topics on Random Variables Example 4.31 Binomial Random Variable Let X1 , . . . , Xn be independent Bernoulli random variables with a common parameter p. What is the transform associated with the binomial random variable Z = X1 + · · · + Xn MXi (s) = (1 − p)e0s + pes = (1 − p) + pes ⇒ MZ (s) = MX1 (s) . . . MXn (s) = (1 − p + pes )n Prof. C. Chen Further Topics on Random Variables Example 4.32 Independent Poisson Variables Let X and Y be independent Poisson random variables with parameters λ and µ, respectively, and Z =X +Y What is the transform associated with Z? MZ (s) = MX (s)MY (s) = eλ(e s −1) s −1) eµ(e s −1) = e(λ+µ)(e Poisson(λ) + Poisson(µ) ∼ Poisson(λ + µ) Prof. C. Chen Further Topics on Random Variables Example 4.33 Independent Gaussians Let X and Y be independent Gaussian random variables with means µx , µy , and variances σx2 , σy2 , respectively, and Z =X +Y What is the transform associated with Z? MZ (s) = MX (s)MY (s) 2 2 /2)+µ s x = e(σx s 2 2 = e((σx +σy )s 2 2 /2)+µ s y e(σy s 2 /2)+(µ x +µy )s N (µx , σx2 ) + N (µy , σy2 ) ∼ N (µx + µy , σx2 + σy2 ) Prof. C. Chen Further Topics on Random Variables Random Sum Prof. C. Chen Further Topics on Random Variables Definition A random sum is the sum of a random number of iid random variables. Specifically Y = X1 + · · · + XN where N is a random variable taking nonnegative integer values X1 , . . . , XN are iid random variables We use X to denote the random variable with the common distribution of Xi . Prof. C. Chen Further Topics on Random Variables Example Consider one day at a convenience store. a random number N of customers make purchases customer i makes a purchase of a random amount Xi Thus, the total amount of sale is S = X1 + · · · + XN Prof. C. Chen Further Topics on Random Variables Properties Y = X1 + · · · + XN ⇒ E[Y ] = E[E[Y |N ]] = E[N E[X]] = E[N ]E[X] ⇒ var(Y ) = E[var(Y |N )] + var(E[Y |N ]) = E[N var(X)] + var(N E[X]) = var(X) E[N ] + E 2 [X] var(N ) ⇒ MY (s) = MN (log MX (s)) Prof. C. Chen Further Topics on Random Variables Proof MY (s) = E[esY ] = E[E[esY |N ]] = E[(MX (s))N ] = X pN (n)(MX (s))n n MN (s) = E[esN ] = X pN (n)(esn ) n = X pN (n)(es )n n Comparison of MY (s) and MN (s) shows MY (s) = MN (log MX (s)) Prof. C. Chen Further Topics on Random Variables Example 4.34 Random Sum A remote village has 3 gas stations. Each gas station is open on any given day with probability 1/2, independent of the others. The amount of gas available in each station is uniformly distributed between 0 and 1000 gallons. What is the transform associated with the total amount of gas available at the gas stations that are open? Prof. C. Chen Further Topics on Random Variables The total amount of gas available is a random sum Y = X1 + · · · + XN where Xi is the amount of gas available at the ith open gas station and N is the number open gas stations. MN (s) = MB (s)3 = (1 − p + pes )3 = h sX MX (s) = E e i Z = esx f (x)dx = 1 (1 + es )3 8 1 1000s e −1 1000s e1000s − 1 1 1+ ⇒ MY (s) = MN (log MX (s)) = 8 1000s Prof. C. Chen !3 Further Topics on Random Variables Example 4.35 Great Expectations Jane visits a number of bookstores, looking for Great Expectations. Any given bookstore carries the book with probability p, independent of the others. In a bookstore visited, Jane spends a random amount of time, exponentially distributed with parameter λ, until she either finds the book or she determines that the bookstore does not carry it. We wish to find the mean, the variance, and the PDF of the total time spent in bookstores. Prof. C. Chen Further Topics on Random Variables The total time until Jane finds a copy is a random sum Y = X1 + X2 + · · · + XN where Xi is the time spent at the ith bookstore and N is the number of bookstores she visits. MN (s) = pes 1 − (1 − p)es MX (s) = ⇒ MY (s) = λ λ−s λ p λ−s pMX (s) = 1 − (1 − p)MX (s) 1 − (1 − p) ⇒ Y ∼ exponential(pλ), E[Y ] = Prof. C. Chen λ λ−s = pλ pλ − s 1 1 , var(Y ) = 2 2 λp λ p Further Topics on Random Variables Example 4.36 Let N be geometrically distributed with parameter p. Each random variable Xi is geometrically distributed with parameter q. All of these random variables are independent. What is the distribution of Y = X1 + · · · + XN MN (s) = pes , 1 − (1 − p)es MX (s) = qes 1 − (1 − q)es s qe p 1−(1−q)e s pMX (s) ⇒ MY (s) = = s qe 1 − (1 − p)MX (s) 1 − (1 − p) 1−(1−q)es = pqes 1 − (1 − pq)es ⇒ Y ∼ geometric(pq) Prof. C. Chen Further Topics on Random Variables