Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Statistics (I) Joint Probability Distributions Learning Objectives 1. Define two discrete random variables 2. Describe two continuous random variables 3. Discuss covariance and correlation 4. Define bivariate normal Distribution 5. Describe linear combination of random variables 6. Discuss Chebyshev’s inequality Why Two Discrete Random Variables ? To find out the simultaneous behavior of two random variables. Example: X and Y are random variables defined for the two dimensions of a part of a production line. What is the probability that X is within the range of 2.95 ~ 3.05 and Y is within the range of 7.60~7.80? Two Discrete Random Variables If X and Y are discrete random variables, the joint distribution of X and Y is a description of the set of points (x, y) in the range of (X, Y) along with the probability of each point. It is sometimes referred to as the bivariate probability distribution or bivariate distribution. Two Discrete Random Variables Definition The joint probability mass function of the discrete random variable X and Y, denoted as fXY(x,y) , satisfies 1. f XY ( x, y ) 0 2. f XY (x,y) 1 x y 3. f XY (x,y) P( X x, Y y ) Two Discrete Random Variables --Example A financial company uses X=1, 2,3 to represent low, medium, and high income customers respectively. Also they use Y=1,2,3,4 to represent mutual funds, bonds, stocks, and options respectively. Then the joint probability of X and Y could be represented by following table. Two Discrete Random Variables --Example Y=1 Y=2 Y=3 Y=4 X=1 0.1 0 0 0 X=2 0.2 0.1 0.2 0 X=3 0.1 0 0.1 0.2 (1) f XY ( x , y ) 0 (2) x y f XY ( x , y ) 1 Two Discrete Random Variables --Example 0.2 0.15 0.1 0.05 Y 0 1 2 3 4 1 2 3 Marginal Probability Distribution (1/2) f X ( x ) P( X x ) f XY ( x, y ) Rx fY ( y ) P(Y y ) f XY ( x, y ) Ry Marginal Probability Distribution (2/2) where Rx denotes the set of all points in the range of (X, Y) for which X=x and Ry denotes the set of all points in the range of (X, Y) for which Y=y Mean of Marginal Probability Distribution If the marginal probability distribution of X has the probability mass function fX(x), then Example(1/3) A financial company use X=1,2,3 to represent low, medium, and high income customers respectively. Also they use Y=1,2,3,4 to represent mutual funds, bonds, stocks, and options respectively. Then the joint probability of X and Y could be represented by following table. Example(2/3) Y=1 Y=2 Y=3 Y=4 X=1 0.1 0 0 0 X=2 0.2 0.1 0.2 0 X=3 0.1 0 0.1 0.2 Example(3/3) Please find marginal probability of fX(x,y), fY(x,y), and their mean and variance. Solution to Example(1/3) Solution to Example(2/3) Solution to Example(3/3) Y=1 Y=2 Y=3 Y=4 fX(x) X=1 0.1 0 0 0 0.1 X=2 0.2 0.1 0.2 0 0.5 X=3 0.1 0 0.1 0.2 0.4 0.1 0.3 0.2 1 fY(Y) 0.4 Conditional Probability Distribution Given discrete random variable X and Y with joint probability mass function fXY(x, y) the conditional probability mass function of Y given X = x is Properties of Conditional Probability Distribution Example(1/3) Same example mentioned above Y=1 Y=2 Y=3 Y=4 X=1 0.1 0 0 0 X=2 0.2 0.1 0.2 0 X=3 0.1 0 0.1 0.2 Example(2/3) Computed results Y=1 Y=2 Y=3 Y=4 fX(x) X=1 0.1 0 0 0 0.1 X=2 0.2 0.1 0.2 0 0.5 X=3 0.1 0 0.1 0.2 0.4 fY(Y) 0.4 0.1 0.3 0.2 1 Example(3/3) The conditional probability of Y given X = 2 is Properties of Conditional Probability Distribution Conditional mean of Y given X = x Conditional variance of Y given X = x Example Please find E(X|y=1) Example Please find V(X|y=1). Independence of Two Discrete Random Variables(1/2) For discrete random variables X and Y, if any one of the following properties is true, then the others are also true, and X and Y are independent . Independence of Two Discrete Random Variables(2/2) (1) f XY ( x , y ) f X ( x ) f Y ( y ) for all x and y ( 2 ) f Y | x ( y ) f Y ( y ) for all x and y with f X ( x ) > 0 ( 3) f X | y ( x ) f X ( x ) for all x and y with f Y ( y ) > 0 ( 4) P ( X x, Y y ) f X ( x ) f Y ( y ) for all ( x , y ) in the range of X and Y Example In above example, are X and Y independent ? Solution to Example P(X=1, Y=1)=0.1 fX(Y=1)=0.4 fY(X=1)=0.1 P(X=1, Y=1) ≠ fX(Y=1)*fY(X=1) Thus X and Y are dependent! Two Continuous Random Variables A joint probability density function for the continuous random variables X and Y, denoted as fXY(x, y), satisfied the following properties. Example Suppose X is the time to failure of a component, and Y is also the time to failure of its spare part. If the joint probability density function of X and Y is: Please find P(X≦1000, Y≦2000)=? Solution to Example 0.915 Why? Marginal Probability Distributions If the joint probability density function of continuous random variables X and Y is fXY (x,y), then the probability density functions of X and Y are Rx denotes the set of all points in the range of (X,Y) for which X=x and Ry denotes the set of all points in the range of (X,Y) for which Y=y Example From above example, please find P(Y>2000)=? Solution to Example 0.05 Why? Conditional Probability Distribution Given continuous random variable X and Y with joint probability density function fXY(x, y) the conditional probability density function of Y given X = x is f Y |x (x, y) for all f XY ( x , y ) fX (x) fX (x) > 0 Properties of Conditional Probability Distribution Example From above example, please find the conditional density function Y given that X=x. Solution to Example p.225 Properties of Conditional Probability Distribution Conditional mean of Y given X = x Conditional variance of Y given X = x Independence of Two Continuous Random Variables (1/2) For continuous random variables X and Y, if any one of the following properties is true, then the others are also true, and X and Y are independent . Independence of Two continuous Random Variables(2/2) (1) f XY ( x , y ) f X ( x ) f Y ( y ) for all x and y ( 2 ) f Y | x ( y ) f Y ( y ) for all x and y with f X ( x ) > 0 ( 3) f X | y ( x ) f X ( x ) for all x and y with f Y ( y ) > 0 ( 4 ) P ( X A, Y B) P ( X A) P(Y B) for any sets A and B in the range of X and Y, respectively. Covariance and Correlation Covariance Expected value of a function of two random variables h(x, y) R h( x, y ) f XY ( x, y ) X,Y discre te E[h( X , Y )] h( x, y ) f ( x, y)dxdy X,Y continuous XY R Covariance The covariance between the random variables X and Y, denoted as cov(X,Y) or σXY , XY E[( X X )(Y Y )] E( XY ) X Y Example(1/3) A financial company use X=1,2,3 to represent low, medium, and high income customers respectively. Also they use Y=1,2,3,4 to represent mutual funds, bonds, stocks, and options respectively. Then the joint probability of X and Y could be represented by following table. Example(2/3) Y=1 Y=2 Y=3 Y=4 X=1 0.1 0 0 0 X=2 0.2 0.1 0.2 0 X=3 0.1 0 0.1 0.2 Example(3/3) Please find the covariance of X and Y. Solution to Example XY E ( XY ) X Y 11 0.1 2 1 0.2 2 2 0.1 2 3 0.2 3 1 0.1 3 3 0.1 3 4 0.2 2.3 2.3 5.7 (2.3) 0.41 2 Correlation The correlation between the random variables X and Y, denoted as XY , ρ XY cov( X , Y ) σ XY V ( X )V (Y ) σ Xσ Y whe re - 1 ρ XY 1 Correlation If X and Y are independent random variables, then σ XY ρ XY 0 Example From the above example, please find the correlation of X and Y. Solution to Example ρ XY cov( X , Y ) σ XY V ( X )V (Y ) σ Xσ Y 0.41 (0.41)(1.41) Bivariate Normal Distribution Bivariate Normal Distribution The probability density function of a bivariate normal distribution f XY ( x, y;σ X ,σ Y ,μ X ,μ Y ,ρ ) 1 2σ Xσ Y - 1 ( x μ X ) 2 2ρ (x- X )( y μ Y ) ( y μ Y ) 2 exp 2 2 2 2 σ Xσ Y σ Y 1 ρ 2(1 -ρ ) σ X for -∞<x<∞ and -∞<y<∞, with parameters σX >0, σy >0 , -∞<μX<∞ and -∞< μY <∞, and -1<ρ<1 Bivariate Normal Distribution If X and Y have a bivariate normal distribution with joint probability density fXY(x,y; σX, σY, μX, μY, ρ), then the marginal probability distributions of X and Y are normal with means μX and μY, and standard deviations σX and σY, respectively. Bivariate Normal Distribution If X and Y have a bivariate normal distribution with joint probability density fXY(x,y; σX, σY, μX, μY, ρ), then the correlation between X and Y is ρ Bivariate Normal Distribution If X and Y have a bivariate normal distribution with ρ=0, then X and Y are independent. Linear Combinations of Random Variables Linear Combinations of Random Variables Given random variables X1, X 2, …., Xp and constants c1, c 2, …., cp , then Y= c1 X1+ c 2 X 2 …., cp Xp is a linear combination of X1, X 2, …., Xp . Linear Combinations of Random Variables If Y= c1 X1+ c 2 X 2 …., cp Xp , then E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp) Linear Combinations of Random Variables If Y= c1 X1+ c 2 X 2 …., cp Xp , then V (Y ) c V ( X 1 ) c V ( X 2 ) ... c V ( X p ) 2 1 2 2 2 p 2 ci c j cov( X i , X j ) i j 2 Furthermore, if X1, X 2, …., Xp , are independent, then V (Y ) c12V ( X1 ) c22V ( X 2 ) ... c 2pV ( X p ) Linear Combinations of Random Variables If X ( X1 X2 ... X p ) / p with E(Xi)=μ for i=1,2,…,p, then E( X ) μ Furthermore, if X1, X 2, …., Xp , are also independent, with V(Xi ) σ 2 for i=1,2,…,p, then V (X ) σ 2 / p Reproductive Property of the Normal Distribution If X1, X 2, …., Xp , are independent, normal 2 random variables with E(Xi)=μi and V(Xi)=σi , for i=1,2,..,p, then Y= c1 X1+ c 2 X 2 …., cp Xp is a normal random variable with E(Y)= c1 E(X1)+ c 2 E(X 2)…., cp E(Xp) and V (Y ) c12 21 c22 2 2 ... c 2p 2 p Chebyshev’s Inequality Chebyshev’s Inequality For any random variable X with mean μ and variance σ2 , P(|X- μ| ≧ c σ) ≦ 1/c 2 for c >0 Example The process of drilling holes in PCB produces diameters with a standard deviation of 0.01 millimeter. How many diameters must be measured so that the probability is at least 8/9 that the average of the measured diameters is within 0.005 of the process mean. Solution to Example(1/2) Let X1, X2, …, Xn be the independent random variables that denote the diameters of n holes. So the average measured diameter is : X ( X 1 X 2 ... X n ) / n E ( X ) and V ( X ) 0.01 / n 2 σ X 0.01 / n 2 Solution to Example(2/2) P(|X- μ| ≧ c σ) ≦ 1/c 2 So P (| X | c(0.012 / n)1/ 2 ) 1 / c 2 let c 3 P (| X | 3(0.012 / n)1/ 2 ) 1 / 9 i.e. P (| X | 3(0.01 / n) 2 So 3(0.012 / n)1/ 2 0.005 and n 36 1/ 2 ) 8/9