Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Random variables M. Veeraraghavan A random variable is a rule that assigns a numerical value to each possible outcome of an experiment. Outcomes of an experiment form the sample space S . Definition: A random variable X on a sample space S is a function X : S → ℜ that assigns a real number X ( s ) to each sample point s ∈ S . We define the event space A x to be the subset of S to which the random variable X assigns the value x . Ax = { s ∈ S X ( s ) = x } (1) Image of a rv X is the set of all values taken by X . Inverse image is A x . Example: Take 3 Bernoulli trials. Sample space has 8 possible outcomes: 000, 001, 010, 100, 011, 110, 101, 111. But if we only interested in the number of successes, then 001, 100, 010 sample points map to the value 1 (for number of 1s within the three trials). Similarly, 011, 110, 101 map to 2. The event space has only 4 outcomes 0, 1, 2, 3 instead of the original sample space which has 8 points. Typically, we are only interested in the event space A x rather than in the whole sample space S if our interest lies only in the experimental values of the rv X . Discrete random variables: A random variable is discrete if it can take on values from a discrete set of numbers. Probability mass function (pmf): P ( Ax ) = ∑ P ( s ) = P [ X = x ] = pX ( x ) X(s) = x Probabiltiy distribution function or Cumulative distribution function (cdf): 1 (2) FX ( t ) = P ( X ≤ t ) = ∑ pX ( x ) (3) x≤t Often we will see the statement, “X is a discrete rv with pmf P X ( x ) “ with no hint or mention of the sample space. Know the pmf, cdf, mean (expected value) and variance for the following types of discrete random variables: 1. Bernoulli - one parameter p (of success) 2. Binomial - count number of successes in n independent trials with p as probability of success of each trial - two parameters n and p . 3. Geometric - count number of trials upto 1st success (while binomial counts number of successes) - one parameter p pX ( i ) = p ( 1 – p ) i–1 FX ( t ) = 1 – ( 1 – p ) for i = 1, 2, … and t for ( t ≥ 0 ) (4) (5) 4. Negative binomial - count number of trials until r th success - two parameters p and r . 5. Poisson - approximation of binomial if n is large and p is small. One parameter α = np . –α k e α p X ( k ) = -------------k! for k = 0, 1, 2, … (6) 6. Hypergeometric - sampling without replacement while binomial is sampling with replacement; N components of which d are defective. In a sample of n components what is the probability that k are defective. 7. Uniform - one parameter - range N . 8. Constant - one parameter c 9. Indicator - one parameter p . Continuous random variables: Probability density function (pdf): dF X ( x ) f X ( x ) = ----------------dx 2 (7) Probabiltiy distribution function or Cumulative distribution function (cdf): t ∫ fX ( r ) dr FX ( t ) = P ( X ≤ t ) = (8) –∞ Know the pdf, cdf, mean (expected value) and variance for the following types of random variables: 1. Exponential - 1 parameter λ 1 – e – λx F(x) = 0 if ( 0 ≤ x < ∞ ) otherwise λe –λx f( x) = 0 if ( x > 0 ) otherwise (9) (10) 1 E [ X ] = --λ (11) 1 Var [ X ] = ----2λ (12) 2. Gamma, chi-square, Student-t distributions 3. Erlang distribution 4. Hypoexponential distribution 5. Hyperexponential distribution 6. Normal (Gaussian) distribution 7. Pareto distribution 8. Weibull distribution Relationships between random variables 1. Mutually exclusive events P(A ∪ B) = P(A) + P(B ) ; P(A ∩ B) = 0 . (13) In general P ( A ∪ B ) = P ( A ) + P ( B ) – P ( A ∩ B ) (14) 2. Independent events P ( A ∩ B ) = P ( A )P ( B ) 3. Bayes rule 3 (15) P ( A B j )P ( B j ) P ( B j A ) = ----------------------------------------∑ P ( A Bi )P ( Bi ) (16) i 4. Theorem of total probability n P(A) = ∑ P ( A Bi )P ( Bi ) (17) i=1 5. Correlation of two random variables r XY = E [ XY ] E [ XY ] = ∑ ∑ (18) xyP X, Y ( x, y ) for a discrete r.v. (19) x ∈ SX y ∈ SY If X and Y are independent random variables then because P X, Y ( x, y ) = P X ( x )P Y ( y ) , 2 E [ XY ] = E [ X ]E [ Y ] (see [2, page 107]). Second moment of a single random variable is E [ X ] . 6. Covariance of two random variables cov ( X, Y ) = E [ X – µ X ] [ Y – µ Y ] = E [ XY ] – µ X µ Y , where µ X = E [ X ] and µ Y = E [ Y ] . Relate this to variance of a single random variable 2 2 Var [ X ] = E [ ( X – µ X ) ] = E [ X 2 ] – µ X (20) cov ( X, Y ) ρ XY = --------------------------------------var ( X )var ( Y ) (21) 7. Correlation coefficient: Memoryless property: Two distributions, the exponential continuous r.v. distribution and geometric discrete r.v. distribution enjoy this property. Let X be the lifetime of a component. Suppose we have observed that the system has been operational for time t . We would like to know the probability that it will be operational for y more hours. Say Y = X – t . We will show that 4 P ( Y ≤ y X > t ) = P ( Y ≤ y ) , in other words, how long it lasts beyond some time t is independent of how long the component has been operational so far, i.e. t . P(Y ≤ y X > t) = P(X – t ≤ y X > t) = P(X ≤ y + t X > t) (22) P ( X ≤ y + t and X > t ) P(t < X ≤ y + t) = ----------------------------------------------------- = ------------------------------------P(X > t) P(X > t) Exponential distribution: (y + t) ∫ P( Y ≤ y X > t) = λe – λx dx t ------------------------------∞ – λx ∫ λe dx – λt – λy e ( 1 – e -) – λy = -------------------------------= (1 – e ) = P(Y ≤ y) – λt e (23) t Geometric distribution: Let X be a rv with a geometric distribution and Y = X – t . We will show that P( Y ≤ y X > t) = P( Y ≤ y ) . y+t t t y 1 – (1 – p) – ( 1 – ( 1 – p ) )( 1 – p ) ( 1 – ( 1 – p ) )P ( Y ≤ y X > t ) = --------------------------------------------------------------------------------= -----------------------------------------------------------= P(Y ≤ y) t t (1 – p) (1 – p) Note: Y = X – t has the same distribution as X because t is a constant. See the stat lecture to see distributions of functions of random variables. Relation between exponential distribution (for continuous rv) and geometric distribution (for discrete rv): [2] If X is an exponential random variable with parameter a , then K = X is a geometric random –a variable with parameter p = 1 – e . References [1] [2] K. S. Trivedi, “Probability, Statistics with Reliability, Queueing and Computer Science Applications,” Second Edition, Wiley, 2002, ISBN 0-471-33341-7. R. Yates and D. Goodman, “Probability and Stochastic Processes,” Wiley, ISBN 0-471-17837-3. 5