Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
STAT 3610: Review of Probability Distributions Mark Carpenter Professor of Statistics Department of Mathematics and Statistics August 25, 2015 Support of a Random Variable Definition The support of a random variable, say X , denoted as X , is defined to be the set of all points on the real line for which the pdf/pmf is non-zero. That is, X = {x ∈ R : fX (x) > 0} , where the braces indicates a set. Support of a Random Variable I The support of a random variable is usually denoted by the script form of the letter corresponding to random variable I I I I the random variable X has support X the random variable Y has support Y the random variable Z has support Z The support of a random variable is one of the first characteristics we can use to help identify the distribution of a random variable. Continuous versus Discrete Random Variables Definition Continuous Random Variable: A random variable is said to be continuous if its support is a continuous set (made up of unions and intersections of real intervals). The CDF* of a continuous random variable must be a continuous function on the real line Definition Discrete Random Variable: A random variable is said to be discrete if its support is a discrete set. The CDF* of a discrete random variable is not continuous, but it is right continuous. *CDF stands for Cumulative Distribution Function Difference between Discrete and Continuous Random Variables So, whether the random variable, X , is a continuous or discrete random variable depends on whether its support is continuous or discrete. I In the discrete case, the pmf* is a discontinuous function with a positive mass (probability) at each point in the support. I In the continuous case, the pdf** itself does not have to be a continuous function everywhere, but it usually a continuous function on intervals in the support. *pmf stands for probability mass function **pdf stands for probability density function Exponential Random Variable and Exponential Distribution Example 1: (continuous support) Suppose X ∼ exponential (θ). From Section 3.2 (pp. 95-113) of the textbook, the exponential distribution, indexed by the scale parameter θ (θ > 0) is 1 −x/θ 1 −x/θ x ≥0 θe I[0,∞) (x) = f (x; θ) = e 0 otherwise. θ which means {x ∈ R : f (x) > 0} = [0, ∞) and the support of X is X = [0, ∞), a continuous set. We see that the pdf for the exponential is zero for all points below zero, then jumps to λe 0 = λ at x = 0 and is continuous on [0, ∞). pdf and cdf for the Standard Exponential 1 f (x) 0.5 F (a) = 1 − e −a 1 a x 1 F (x) 0.5 F (a) = 1 − e −a 1 a x Exponential is special case of Gamma and Weibull You can verify that the non-truncated gamma and Weibull distributions, from which the exponential is a special case, share this same support. If X is a normal random variable, then the support is X = (−∞, ∞) = R. Mean or Expected Value of a Random Variable Recall, for any random variable, X , with pdf/pmf f (x), a measure of central tendency of the population is the population mean µ, or the expected value/long run average for X . More formally, Population Mean: For any random variable, X , with pdf/pmf f (x), the population mean µ = E (X ) where Z ∞ xf (x)dx if X is continuous −∞ µ = E (X ) = X xf (x) if X is discrete x∈X Population Variance or Variance of a Random Variable Population Variance: For any random variable, X , with pdf/pmf f (x), the population variance is σ 2 = E (X − µ)2 , where Z ∞ (x − µ)2 f (x)dx if X is continuous −∞ σ 2 = E (X − µ)2 = X (x − µ)2 f (x) if X is discrete x∈X Sometimes Easier Way to Compute Population Variance Note that it is often easier to compute the variance by noting that , σ 2 = E (X −µ)2 = E (X 2 −2X µ+µ2 ) = EX 2 −2µEX +µ2 = EX 2 −(EX )2 . So, rather than going through the original express, one need only compute E (X 2 ) and µ = E (X ) and plug the results in to the following expression σ 2 = E (X 2 ) − µ2 . Expectations of Functions of a Random Variable You might notice that each of E (X ), E (X 2 ), and E (X − µ)2 are the expected value of different functions, g1 (x) = x, g2 (x) = X 2 and g3 (x) = (x − µ)2 . The expected value for any function is defined below. Moment Generating Functions Whenever it exists, the moment-generating function for a random variable X , denoted MX (t), is the continuous function of t ∈ (−∞, ∞) given as h i MX (t) = E e tX , t ∈ (−h, h), h > 0. The interval (−h, h) is referred to as the radius of convergence. Properties of a Moment Generating Function (mgf) This function is called the moment-generating function because you can find the nth moment for the random variable, X, by computed its nth derivative with respect to t then setting t = 0, as follows d (n) n . E (X ) = MX (0) = MX (t) dt t=0 Notice that the moment generating function is a continuous and differentiable function of |t| < h, whether or not X is continuous. In fact, the moment generating function is mathematically independent of the original variable (since it was integrated or summed over the support) and only relates to the variable X through the moments of the distribution and any related parameters. Properties of Exponential We will show on chalkboard that if X ∼ Exp(θ) then Z ∞ Z ∞ 1 −x/θ I f (x)dx = e dx = 1. θ −∞ 0 Z ∞ Z ∞ x −x/θ I µ = E (X ) = e dx = λ. x · f (x)dx = θ −∞ 0 Z ∞ I σ 2 = E (X − µ)2 = (x − µ)2 f (x)dx = θ2 0 I Cumulative Distribution Function (cdf) for any w ≥ 0 is F (w ) = P(X ≤ w ) = 1 − e −w /θ I The moment generating function (mgf), denoted M(t) exists and 1 1 M(t) = , t< (1 − θt) θ Discrete Example (Binomial) Example: (discrete support) Suppose Y is a Binomial random variable with parameters n and p (see page 117 of textbook) ,then the probability mass function (pmf) is n p y (1 − p)n−y y = 0, 1, . . . , n y fY (y ) = 0 otherwise which means {y : f (y ) > 0} = {0, 1, . . . , n} and the support of Y is Y = {0, 1, . . . , n}, a discrete (and finite) set of points. Recall that, n y = n! . y !(n − y )! cdf for Binomial Random Variable Example 2 (binomial): Suppose X is a Binomial (4, 0.5), then the pmf is 1 4 f (x; n = 4, p = 1/2) = , x ∈ X = {0, 1, 2, 3, 4} x 2n Table : CDF for Binomial (n=4, p=1/2) (−∞, 0) P(X < 0) = 0 0 = 0 [0, 1) P(X ≤ 0) = P(0) 1 16 = 1 16 [1, 2) P(X ≤ 1) = P(0) + P(1) 1 16 4 + 16 = 5 16 + 4 16 = 11 16 [2, 3) P(X ≤ 2) = P(0) + P(1) + P(2) 1 16 [3, 4) P(X ≤ 3) = P(0) + P(1) + P(2) + P(3) 1 16 4 + 6 + 4 + 16 16 16 = 15 16 [4, ∞) P(X ≤ 4) = P(0) + P(1) + P(2) + P(3) + P(4) 1 16 4 + 6 + 4 + 1 + 16 16 16 16 = 1 + 6 16 cdf for Binomial Random Variable 1 15/16 11/16 5/16 1/16 0 1 2 3 4 5 Binomial, hypergeometric, geometric One can verify that the supports for the negative binomial (r , p), the Geometric(p), and the Poisson random variables (see page 126) are the same countably infinite set {0, 1, 2, . . .}. The support of the hypergeomtric(n, M, N) is discrete/finite set {max(0, n − N + M), . . . , min(n, M).} Poisson Random Variable Suppose X is a discrete random variable with a Poisson (λ) distribution, then the probability mass function (pmf) is λx e −λ x = 0, 1, 2, ..., x! f (x) = 0 otherwise where λ > 0. Recall that the MacLaurin series of an Exponential function, e y , is ∞ X yi ey = , where i! represents the factorial function for an i! i=0 integer i. Some Properties of a Poisson Random Variable I X f (x; λ) = x! x=0 x∈X I ∞ X λx e −λ µ = E (X ) = X =1 x · f (x) = λ x∈X I σ 2 = E (X − µ)2 = X (x − µ)2 · f (x) = λ x∈X I M(t) = E (e tX )= X x∈X e tx · f (x) = e λ(e t −1) , −∞ < t < ∞.