Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Functions of Random Variables Methods for determining the distribution of functions of Random Variables 1. Distribution function method 2. Moment generating function method 3. Transformation method Distribution function method Let X, Y, Z …. have joint density f(x,y,z, …) Let W = h( X, Y, Z, …) First step Find the distribution function of W G(w) = P[W ≤ w] = P[h( X, Y, Z, …) ≤ w] Second step Find the density function of W g(w) = G'(w). Example: Student’s t distribution Let Z and U be two independent random variables with: 1. Z having a Standard Normal distribution and 2. U having a c2 distribution with n degrees of freedom Find the distribution of Z t U n The density of Z is: 1 f z e 2 The density of U is: n z2 2 1 2 n 1 u 2 2 h u u e 2 n 2 Therefore the joint density of Z and U is: n 1 2 n z 2 u 1 2 2 f z, u f z h u u e 2 n 2 2 The distribution function of T is: Z t G t P T t P t P Z U n n U Then n 1 n 1 n 1 t2 2 2 2 t 2 g (t ) G t 1 K 1 n n n n 2 where n 1 2 K n n 2 Student’s t distribution t2 g (t ) K 1 n where n 1 n 1 2 K n n 2 2 Student – W.W. Gosset Worked for a distillery Not allowed to publish Published under the pseudonym “Student t distribution standard normal distribution Distribution of the Max and Min Statistics Let x1, x2, … , xn denote a sample of size n from the density f(x). Let M = max(xi) then determine the distribution of M. Repeat this computation for m = min(xi) Assume that the density is the uniform density from 0 to q. Hence 1 f ( x ) q 0 x q elsewhere and the distribution function 0 x F ( x) P X x q 1 x0 0 x q x q Finding the distribution function of M. G (t ) P M t P max xi t P x1 t , , xn t P x1 t P xn t 0 n t q 1 t0 0 t q t q Differentiating we find the density function of M. nt n 1 n g t G t q 0 0.12 0 t q otherwise 0.6 f(x) 0.1 0.5 0.08 0.4 0.06 0.3 0.04 0.2 0.02 0.1 0 0 0 2 4 6 8 10 g(t) 0 2 4 6 8 10 Finding the distribution function of m. G (t ) P m t P min xi t 1 P x1 t , , xn t 1 P x1 t P xn t 0 n t 1 1 q 1 t0 0 t q t q Differentiating we find the density function of m. n t 1 g t G t q q 0 n 1 0 t q otherwise f(x) g(t) 0.12 0.6 0.1 0.5 0.08 0.4 0.06 0.3 0.04 0.2 0.02 0.1 0 0 0 2 4 6 8 10 0 2 4 6 8 10 The probability integral transformation This transformation allows one to convert observations that come from a uniform distribution from 0 to 1 to observations that come from an arbitrary distribution. Let U denote an observation having a uniform distribution from 0 to 1. 1 0 u 1 g (u ) elsewhere Let f(x) denote an arbitrary density function and F(x) its corresponding cumulative distribution function. 1 X F (U ) Let Find the distribution of X. 1 G x P X x P F (U ) x P U F x F x Hence. g x G x F x f x Thus if U has a uniform distribution from 0 to 1. Then 1 X F (U ) has density f(x). X F 1 (U ) U Use of moment generating functions Definition Let X denote a random variable with probability density function f(x) if continuous (probability mass function p(x) if discrete) Then mX(t) = the moment generating function of X E etX tx e f x dx if X is continuous etx p x if X is discrete x The distribution of a random variable X is described by either 1. The density function f(x) if X continuous (probability mass function p(x) if X discrete), or 2. The cumulative distribution function F(x), or 3. The moment generating function mX(t) Properties 1. mX(0) = 1 2. mXk 0 k th derivative of mX t at t 0. k E X k E X 3. k k k x f x dx k x p x mX t 1 1t 2 2! t 2 X continuous X discrete 3 3! t 3 k k! t k . 4. Let X be a random variable with moment generating function mX(t). Let Y = bX + a Then mY(t) = mbX + a(t) = E(e [bX + a]t) = eatmX (bt) 5. Let X and Y be two independent random variables with moment generating function mX(t) and mY(t) . Then mX+Y(t) = mX (t) mY (t) 6. Let X and Y be two random variables with moment generating function mX(t) and mY(t) and two distribution functions FX(x) and FY(y) respectively. Let mX (t) = mY (t) then FX(x) = FY(x). This ensures that the distribution of a random variable can be identified by its moment generating function M. G. F.’s - Continuous distributions Name Continuous Uniform Exponential Gamma c2 nd.f. Normal Moment generating function MX(t) ebt-eat [b-a]t for t < t for t < t 1 1-2t n/2 for t < 1/2 t+(1/2)t22 e M. G. F.’s - Discrete distributions Name Discrete Uniform Bernoulli Binomial Geometric Negative Binomial Poisson Moment generating function MX(t) et etN-1 N et-1 q + pet (q + pet)N pet 1-qet pet k t 1-qe (et-1) e Moment generating function of the gamma distribution mX t E etX tx e f x dx where 1 x x e f x 0 x0 x0 e f x dx mX t E e tX tx 1 x e x e dx 0 1 t x x e dx 0 tx using or ba a 1 bx 0 a x e dx 1 a a 1 bx 0 x e dx ba then 1 t x mX t x e dx 0 t t t Moment generating function of the Standard Normal distribution mX t E etX tx e f x dx where 1 f x e 2 x2 2 thus mX t e tx 1 e 2 x2 2 dx 1 e 2 x2 tx 2 dx We will use 0 mX t e 1 e 2 x2 tx 2 x2 2tx 2 1 e 2 b t2 2 2 x a 2b2 dx 1 dx 1 e dx 2 2 x t x 2 2 tx t 2 t 2 t2 1 2 1 2 2 2 e e dx e e dx 2 2 Note: 2 3 4 x x x ex 1 x 2! 3! 4! 2 3 t t t2 2 2 2 t 2 mX t e 1 2 2! 3! 2 2 4 6 t t t 1 2 3 2 2 2! 2 3! Also mX t 1 1t 2 2! t 2 2 2m t m 2 m! 3 3! t 3 Note: 2 3 4 x x x ex 1 x 2! 3! 4! 2 3 t t t2 2 2 2 t 2 mX t e 1 2 2! 3! 2 2 Also 4 2 6 2m t t t t 1 2 3 m 2 2 2! 2 3! 2 m! 2 2 3 3 mX t 1 1t t t 2! 3! k k th moment x k f x dx Equating coefficients of tk, we get k 0 if k is odd and 2 m 1 for k 2m then m 2 m ! 2m ! hence 1 0, 2 1, 3 0, 4 3 Using of moment generating functions to find the distribution of functions of Random Variables Example Suppose that X has a normal distribution with mean and standard deviation . Find the distribution of Y = aX + b Solution: 2t 2 t mX t e 2 2 2 at at bt maX b t e mX at e e bt a b t e 2 2 a 2t 2 2 = the moment generating function of the normal distribution with mean a + b and variance a22. Thus Y = aX + b has a normal distribution with mean a + b and variance a22. Special Case: the z transformation Z X 1 X aX b 1 Z a b 0 2 1 2 2 2 2 Z a 1 Thus Z has a standard normal distribution . Example Suppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y Find the distribution of S = X + Y Solution: mX t e mY t e X t Y t X2 t 2 2 Y2 t 2 2 Now mX Y t mX t mY t e X t X2 t 2 2 e Y t Y2 t 2 2 or mX Y t e X Y t 2 X Y2 t 2 2 = the moment generating function of the normal distribution with mean X + Y and 2 2 variance X Y Thus Y = X + Y has a normal distribution 2 2 with mean X + Y and variance X Y Example Suppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y Find the distribution of L = aX + bY Solution: mX t e X t X2 t 2 mY t e 2 Y t Y2 t 2 2 Now maX bY t maX t mbY t mX at mY bt e X at X2 at 2 2 e Y bt Y2 bt 2 2 or a t 2 maX bY t e a X bY 2 X b2 Y2 t 2 2 = the moment generating function of the normal distribution with mean aX + bY 2 2 2 2 and variance a X b Y Thus Y = aX + bY has a normal distribution with mean aX + bY and 2 2 2 2 variance a X b Y Special Case: a = +1 and b = -1. Thus Y = X - Y has a normal distribution with mean X - Y and variance 1 2 X 1 Y X Y 2 2 2 2 2 Example (Extension to n independent RV’s) Suppose that X1, X2, …, Xn are independent each having a normal distribution with means i, standard deviations i (for i = 1, 2, … , n) Find the distribution of L = a1X1 + a1X2 + …+ anXn Solution: mX i t e Now ma1 X1 i t i2t 2 (for i = 1, 2, … , n) 2 an X n t ma X t 1 1 mX1 a1t e 1 a1t man X n t mX n ant 12 a1t 2 2 e n an t n2 an t 2 2 or ma1 X1 an X n t e a11 ... an n a t 2 2 2 2 ... a 1 1 n n t 2 2 = the moment generating function of the normal distribution with mean a11 ... an n 2 2 2 2 and variance a1 1 ... an n Thus Y = a1X1 + … + anXn has a normal distribution with mean a11 + …+ ann 2 2 2 2 and variance a1 1 ... an n Special case: a1 a2 1 2 12 12 1 an n n 12 2 In this case X1, X2, …, Xn is a sample from a normal distribution with mean , and standard deviations ,and 1 L X1 X 2 X n n X the sample mean Thus Y x a1 x1 ... an xn n 1 n x x1 ... 1 n has a normal distribution with mean x a11 ... an n 1 ... 1 n n and variance x2 a12 12 ... an2 n2 2 1 1 1 2 2 2 ... n n n n n 2 2 2 Summary If x1, x2, …, xn is a sample from a normal distribution with mean , and standard deviations ,then x the sample mean has a normal distribution with mean x and variance 2 x 2 n standard deviation x n Sampling distribution of x 0.4 0.3 Population 0.2 0.1 0 20 30 40 50 60 The Central Limit theorem If x1, x2, …, xn is a sample from a distribution with mean , and standard deviations ,then if n is large x the sample mean has a normal distribution with mean x and variance 2 x 2 standard deviation x n n Proof: (use moment generating functions) We will use the following fact: Let m1(t), m2(t), … denote a sequence of moment generating functions corresponding to the sequence of distribution functions: F1(x) , F2(x), … Let m(t) be a moment generating function corresponding to the distribution function F(x) then if lim mi t m t for all t in an interval about 0. i then lim Fi x F x for all x. i Let x1, x2, … denote a sequence of independent random variables coming from a distribution with moment generating function m(t) and distribution function F(x). Let Sn = x1 + x2 + … + xn then mSn t mx1 x2 = m t xn t =mx t mx t 1 2 mxn t n x1 x2 now x n xn Sn n n t t or mx t m 1 t mSn m Sn n n n Let z x n then mz t e n n t x n nt mx e n t nt m n t and ln mz t t n ln m n n n t t2 Let u or n and n 2 2 u u n t t Then ln mz t t n ln m n t 2 t2 2 2 2 ln m u u u n t 2 ln m u u 2 u2 Now lim ln mz t lim ln mz t n u 0 t2 t2 lim ln m u u 2 u 0 lim 2 u 0 u2 m u m u using L'Hopital's rule 2u m u m u m u t 2 lim 2 u 0 m u 2 2 2 using L'Hopital's rule again m u m u m u t m u 2 2 lim 2 u 0 t m 0 m 0 2 2 2 2 2 using L'Hopital's rule again 2 t E x E xi t2 2 2 2 2 2 i 2 2 t thus lim ln mz t and lim mz t e n n 2 t2 2 Now m t e t2 2 Is the moment generating function of the standard normal distribution Thus the limiting distribution of z is the standard normal distribution i.e. lim Fz x n x 1 e 2 u2 2 du Q.E.D.