Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Random Vectors and Matrices • A random vector is a vector whose elements are random variables. • The collective behavior of a p x 1 random vector is described by a joint probability density function f(x1,x2,…,xp) = f(x). • If the joint density of a p x 1 random vector can be factored as f(x1,x2,…,xp) = f1(x1) f2(x2)∙∙∙ fp(xp) then the p continuous random variables X1,X2,…Xp are mutually independent. STA347 - week 9 1 Mean and Variance of Random Vector • The expected value of a random vector is a vector of the expected values of each of its elements. That is, the population mean vector is • The population variance-covariance matrix of a px1 random vector x is a p x p symmetric matrix CovX E x x ' ij where σij = Cov(Xi, Xj) = E(Xi – μi)(Xj – μj). • The population correlation matrix of a px1 random vector x is a p x p symmetric matrix ρ = (ρij) where ij ij ii jj . STA347 - week 9 2 Properties of Mean Vector and Covariance Matrix STA347 - week 9 3 Functions of Random variables • In some case we would like to find the distribution of Y = h(X) when the distribution of X is known. • Discrete case PX x pY y PY y Ph X y P X h 1 y xh 1 y • Examples 1. Let Y = aX + b , a ≠ 0 1 PY y PaX b y P X y b a 2. Let Y X2 P X y P X y PY y P X 2 y P X 0 0 STA347 - week 9 if y 0 if y 0 if y 0 4 Continuous case – Examples 1. Suppose X ~ Uniform(0, 1). Let Y X 2 , then the cdf of Y can be found as follows FY y PY y P X 2 y P X y FX y The density of Y is then given by 2. Let X have the exponential distribution with parameter λ. Find the 1 density for Y X 1 3. Suppose X is a random variable with density x 1 f X x 2 0 , 1 x 1 , elsewhere Check if this is a valid density and find the density of Y X 2 . STA347 - week 9 5 Theorem • If X is a continuous random variable with density fX(x) and h is strictly increasing and differentiable function form R R then Y = h(X) has density fY y f X d 1 h y h y dy 1 for y R . • Proof: STA347 - week 9 6 Theorem • If X is a continuous random variable with density fX(x) and h is strictly decreasing and differentiable function form R R then Y = h(X) has density dyd h y f Y y f X h 1 y 1 for y R . • Proof: STA347 - week 9 7 Summary • If Y = h(X) and h is monotone then d 1 f Y y f X h y h y dy 1 • Example X has a density x3 f X x 4 0 for 0 x 2 otherwise Let Y X 6. Compute the density of Y. STA347 - week 9 8 Change-of-Variable for Joint Distributions • Theorem Let X and Y be jointly continuous random variables with joint density function fX,Y(x,y) and let DXY = {(x,y): fX,Y(x,y) >0}. If the mapping T given by T(x,y) = (u(x,y),v(x,y)) maps DXY onto DUV. Then U, V are jointly continuous random variable with joint density function given by f xu , v , y u , v J u , v fU ,V u , v X ,Y 0 if u, v DU ,V otherwise where J(u,v) is the Jacobian of T-1 given by x J u, v u y u x v y v assuming derivatives exists and are continuous at all points in DUV . STA347 - week 9 9 Example • Let X, Y have joint density function given by e x y f X ,Y x, y 0 Find the density function of U if x, y 0 otherwise X . X Y STA347 - week 9 10 Example • Show that the integral over the Standard Normal distribution is 1. STA347 - week 9 11 Example • A device containing two key components fails when and only when both components fail. The lifetime, T1 and T2, of these components are independent with a common density function given by e t fT t 0 t 0 otherwise • The cost, X, of operating the device until failure is 2T1 + T2. Find the density function of X. STA347 - week 9 12 Convolution • Suppose X, Y jointly distributed random variables. We want to find the probability / density function of Z=X+Y. • Discrete case X, Y have joint probability function pX,Y(x,y). Z = z whenever X = x and Y = z – x. So the probability that Z = z is the sum over all x of these joint probabilities. That is pZ z p X ,Y x, z x . x • If X, Y independent then pZ z p X x pY z x . x This is known as the convolution of pX(x) and pY(y). STA347 - week 9 13 Example • Suppose X~ Poisson(λ1) independent of Y~ Poisson(λ2). Find the distribution of X+Y. STA347 - week 9 14 Convolution - Continuous case • Suppose X, Y random variables with joint density function fX,Y(x,y). We want to find the density function of Z=X+Y. Can find distribution function of Z and differentiate. How? The Cdf of Z can be found as follows: FZ z P X Y z zx f x, y dydx X ,Y x y z f x, v x dvdx X ,Y x v z f x, v x dxdv. X ,Y v x If f x, v x dx XY is continuous at z then the density function of Z is given by x • If X, Y independent then f Z z f Z z f x, z xdx XY x f x f z xdx X Y x This is known as the convolution of fX(x) and fY(y). 15 Example • X, Y independent each having Exponential distribution with mean 1/λ. Find the density for W=X+Y. STA347 - week 9 16 Order Statistics • The order statistics of a set of random variables X1, X2,…, Xn are the same random variables arranged in increasing order. • Denote by X(1) = smallest of X1, X2,…, Xn X(2) = 2nd smallest of X1, X2,…, Xn X(n) = largest of X1, X2,…, Xn • Note, even if Xi’s are independent, X(i)’s can not be independent since X(1) ≤ X(2) ≤ … ≤ X(n) • Distribution of Xi’s and X(i)’s are NOT the same. STA347 - week 9 17 Distribution of the Largest order statistic X(n) • Suppose X1, X2,…, Xn are i.i.d random variables with common distribution function FX(x) and common density function fX(x). • The CDF of the largest order statistic, X(n), is given by FX n x PX n x • The density function of X(n) is then f X n x d FX n x dx STA347 - week 9 18 Example • Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the density function of X(n). STA347 - week 9 19 Distribution of the Smallest order statistic X(1) • Suppose X1, X2,…, Xn are i.i.d random variables with common distribution function FX(x) and common density function fX(x). • The CDF of the smallest order statistic X(1) is given by FX 1 x PX 1 x 1 PX 1 x • The density function of X(1) is then f X 1 x d FX x dx 1 STA347 - week 9 20 Example • Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the density function of X(1). STA347 - week 9 21 Distribution of the kth order statistic X(k) • Suppose X1, X2,…, Xn are i.i.d random variables with common distribution function FX(x) and common density function fX(x). • The density function of X(k) is f X k x n! FX x k 1 1 FX x nk f X x k 1!n k ! STA347 - week 9 22 Example • Suppose X1, X2,…, Xn are i.i.d Uniform(0,1) random variables. Find the density function of X(k). STA347 - week 9 23 Computer Simulations - Introduction • Modern high-speed computers can be used to perform simulation studies. • Computer simulation methods are commonly used in statistical applications; sometimes they replace theory, e.g., bootstrap methods. • Computer simulations are becoming more and more common in many applications such as quality control, marketing, scientific research etc. STA347 - week 9 24 Applications of Computer Simulations • Our main focus is on probabilistic simulations. Examples of applications of such simulations include: Simulate probabilities and random variables numerically. Approximate quantities that are too difficult to compute mathematically. Random selection of a sample from a very large data sets. Encrypt data or generate passwords. Generate potential solutions for difficult problems. STA347 - week 9 25 Steps in Probabilistic Simulations • In most applications, the first step is to specify a certain probability distribution. • Once such distribution is specified, it will be desired to generate one or more random variables having that distribution. • The build-in computer device that generates random numbers is called pseudorandom number generator. • It is a device for generating a sequence U1, U2, … of random values that are approximately independent and have approximately uniform distribution of the unit interval [0,1]. STA347 - week 9 26 Simulating Discrete Distributions - Example • Suppose we wish to generate X ~ Bernoulli(p), where 0 < p < 1. • We start by generating U ~ Uniform[0, 1] and then set: 1 X 0 Up Up • Then clearly X takes two values, 0 and 1. Further, P X 1 PU p p • Therefore, we have that X ~ Bernoulli(p). • This can be generalized to generate Y ~ Binomial(n, p) by generating U1, U2, … Un. Setting Xi as above and let Y = X1 + ∙∙∙ + Xn. STA347 - week 9 27 Simulating Discrete Distributions • In general, suppose we wish to generate a random variable with probability mass function p. • Let, x1 < x2 < x3 < ∙∙∙ be all the values for which p(xi) > 0. • Let U ~ Uniform[0, 1]. • Define Y by: j Y min x j : pxk U k 1 • Theorem 1: Y is a discrete random variable, having probability mass function p. • Proof: STA347 - week 9 28 Simulating Continuous Distributions - Example • Suppose we wish to generate X ~ Uniform[a, b]. • We start by generating U ~ Uniform[0, 1] and then set: X b aU a • Using one-dimensional change of variable theorem we can easily show that X ~ Uniform[a, b]. STA347 - week 9 29 Simulating Continuous Distributions • In general, simulating continuous distribution is not an easy task. • However, for certain continuous distributions it is not difficult. • The general method for simulating continuous distribution makes use of the inverse cumulative distribution function. • The inverse cdf of a random variable X with cumulative distribution function F is defined by: F 1 t min x : F x t for 0 < t < 1. STA347 - week 9 30 Inversion Method for Generating RV • Let F be any cumulative distribution function, and let U ~ Uniform[0, 1]. • Define a random variable Y by: Y F 1 U • Theorem 2: Y has cumulative distribution function given by F. That is, PY y F y • Proof: STA347 - week 9 31 Important Notes • The theorem above is valid for any cumulative distribution function whether it corresponds to a continuous distribution, a discrete distribution or a mixture of the two. • The inversion method for generating random variables described above can be used whenever the distribution function is not too complicated and has a close form. • For distributions that are too complicated to sample using the inversion method and for which there is no simple trick , it may still be possible to generate samples using Markov chain methods. STA347 - week 9 32 Example – Exponential Distribution • Suppose X ~ Exponential(λ). The probability density function of X is: e x f X x 0 • The cdf of X is: x0 otherwise x F x e t dt 1 e x 0 x • Setting U F x 1 e and solving for x we get… • Therefore, by theorem 2 above, X 1 ln 1 U where U ~ Uniform[0, 1], has an Exponential(λ) distribution. STA347 - week 9 33 Example – Standard Normal Distribution • Suppose X ~ Normal(0,1). The cdf of X is denoted by Ф(x). It is given by: t2 x 1 2 x e dt 2 • Then, if U ~ Uniform[0, 1], by theorem 2 above Y 1 U min x : x U has a N(0,1) distribution. • However, since both Ф and Ф-1 don’t have a close form, i.e., it is difficult to compute them, the inversion method for generating RV is not practical. STA347 - week 9 34