Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Lecture Notes 2 Random Variables EE278 Prof. B. Prabhakar Statistical Signal Processing Autumn 02-03 • Definition • Discrete Random Variables • Continuous Random Variables • Functions of a Random Variable c Copyright °2000–2002 Abbas El Gamal 2-1 EE 278: Random Variable Random Variable • A random variable (r.v.) X is a real-valued function X(ω) over the sample space Ω, i.e., X : Ω → R Ω ω X(ω) • Notations: – Always use upper case letters for random variables (X, Y , . . .) – Always use lower case letters for values of random variables: X = x means that the random variable X takes on the value x EE 278: Random Variable 2-2 • Examples: 1. n coin flips: Here Ω = {H, T }n, define the random variable X ∈ {0, 1, 2, . . . , n} to be the number of heads 2. Let Ω = R, define the two random variables a. X = ω, ( b. Y = 1 for ω ≥ 0 −1 otherwise 3. Packet arrival times in the interval (0, T ]: Here Ω is the set of all finite length strings (t1, t2, . . . , tn) ∈ (0, T ]∗, define the random variable X to be the length of the string n = 0, 1, . . . 2-3 EE 278: Random Variable Specifying a Random Variable • Specifying a random variable means being able to determine the probability that X ∈ A for any set A ⊂ R, e.g., any interval • To do so, we consider the inverse image of the set A under X(ω), {w : X(ω) ∈ A} R set A inverse image of A under X(ω) • So, X ∈ A iff ω ∈ {w : X(ω) ∈ A}, thus P({X ∈ A}) = P({w : X(ω) ∈ A}), or in short P{X ∈ A} = P{w : X(ω) ∈ A} EE 278: Random Variable 2-4 Discrete Random Variables • A random variable is said to be discrete if for some countable set X ⊂ R, i.e., X = {x1, x2, . . .}, P{X ∈ X } = 1 • Examples 1, 2-b, and 3, are discrete random variables x1x2 x3 . . . xn . . . R Ω • Here X(ω) partitions Ω into the sets {ω : X(ω) = xi}, for i = 1, 2, . . ., so to specify X, it suffices to know P{X = xi} for all i • A discrete random variable is thus completely specified by its probability mass function (pmf) pX (x) = P{X = x}, for all x ∈ X 2-5 EE 278: Random Variable • Clearly pX (x) ≥ 0 and P x∈X pX (x) = 1 • So pX (x) can be simply viewed as a probability measure over a discrete sample space (even though the original sample space may be continuous as in examples 2-b and 3) • The probability of any set A ⊂ R is given by X pX (x) P{X ∈ A} = x∈A∩X • Notation: We use X ∼ pX (x) or simply X ∼ p(x) to mean that the discrete random variable X has pmf pX (x) or p(x) EE 278: Random Variable 2-6 Famous Discrete Random Variables • Bernoulli r.v.: X ∼ Br(p), for 0 ≤ p ≤ 1 has pmf pX (1) = p, and pX (0) = 1 − p • Geometric r.v.: X ∼ Geom(p), for 0 ≤ p ≤ 1 has pmf pX (k) = p(1 − p)k−1, for k = 1, 2, . . . This r.v. represents, for example, the number of coin flips until the first heads shows up (assuming independent coin flips) • Binomial r.v.: X ∼ B(n, p), for integer n > 0, and 0 ≤ p ≤ 1 has pmf µ ¶ n k pX (k) = p (1 − p)(n−k), for k = 0, 1, 2, . . . , n k This r.v. represents, for example, the number of heads in n independent coin flips • Poisson r.v.: X ∼ Poisson(λ), for λ > 0 has pmf pX (k) = λk −λ e , for k = 0, 1, 2, . . . k! 2-7 EE 278: Random Variable This represents the number of random events in interval (0, 1], e.g., arrivals of packets, photons, customers, etc f l Continuous Random Variables • To specify a random variable, we need to be able to determine P{X ∈ A} for any set A ⊂ R, i.e., any set generated by countable unions, intersections, and complements of intervals • Thus to specify X, it suffices to specify P{X ∈ (a, b]} for all intervals, the probability of any other set can then be determined using the axioms of probability • Equivalently, to specify a random variable it suffices to specify its cumulative distribution function (cdf) FX (x) = P{X ≤ x}, for all x ∈ R FX (x) 1 x 2-9 EE 278: Random Variable • Properties of FX (x): 1. FX (x) ≥ 0 is monotonically nondecreasing, i.e., if a > b then FX (a) ≥ FX (b) 2. limx→∞ FX (x) = 1 and limx→−∞ FX (x) = 0 3. FX (x) is right continuous, i.e., limx→a+ FX (x) = FX (a) 4. P{X = a} = FX (a) − FX (a−) 5. P{X ∈ A} for any Borel set A can be determined from FX (x) • For a discrete random variable FX (x) consists of a countable set of steps • A random variable is said to be continuous if its cdf FX (x) is a continuous function • Examples: F1 1 x EE 278: Random Variable 2-10 F2 1 x F3 1 x F4 1 x • If FX (x) is continuous and differentiable (except over a countable set), then X has a probability density function (pdf) fX (x) such that Z x fX (α)dα FX (x) = −∞ 2-11 EE 278: Random Variable • If FX (x) is differentiable everywhere, then fX (x) = P{x < X ≤ x + ∆x} dFX (x) = lim ∆x→0 dx ∆x • Properties of fX (x): 1. fX (x) ≥ 0 R∞ 2. −∞ fX (x)dx = 1 3. For any event A ∈ R Z P{X ∈ A} = for example P{x1 < X ≤ x2} = fX (x)dx, x∈A R x2 x1 fX (x)dx • Important note: fX (x) should not be interpreted as the probabilty that X = x, in fact it is not a probability measure, e.g., it can be > 1 • Notation: X ∼ fX (x) (or f (x)) means that X has pdf fX (x) • Remark: Using δ(.) functions, we can define pdf for a r.v. with discontinuous cdf, e.g., a discrete r.v. EE 278: Random Variable 2-12 Famous Continuous Random Variables • Uniform r.v.: X ∼ U[a, b], for b > a has pdf ( 1 for a ≤ x ≤ b b−a f (x) = 0 otherwise • Exponential r.v.: X ∼ Exp(λ), for λ > 0 has pdf ( λe−λx for x ≥ 0 f (x) = 0 otherwise This r.v. represents interarrival time in a queue, e.g., time between two consecutive packet or customer arrivals, also service time in a queue, and lifetime of a particle, etc • Example: Let X ∼ Exp(0.1) be the service time of customers at a bank (in minutes). The person ahead of you has been served for 10 minutes. What is the probability that you will wait another 10 minutes or more before getting served? 2-13 EE 278: Random Variable We want to find P{X > 20|X > 10} By definition P{X > 20, X > 10} P{X > 10} P{X > 20} = P{X > 10} e−2 = −1 e = e−1, P{X > 20|X > 10} = but P{X > 10} = e−1, i.e., the conditional probability of waiting more than 10 miutes is the same as the unconditional probability of waiting more than 10 minutes ! • This is because the exponential r.v. is memoryless, which in general means that, for any 0 ≤ x1 < x2 P{X > x2|X > x1} = P{X > x2 − x1} EE 278: Random Variable 2-14 Gaussian Random Variable • Gaussian r.v.: X ∼ N (µ, σ 2) has pdf fX (x) = √ 1 − (x−µ)2 2σ 2 e , 2πσ 2 where µ is the mean and σ is the standard deviation N (µ, σ 2) x µ • Gaussian r.v.s are frequently encountered in nature, e.g., thermal and shot noise in electronic devices are gaussian, and very frequently used in modelling various social, biological, and other phenomena (a lot more on gaussian r.v.s later) 2-15 EE 278: Random Variable • The Φ, Q, and erfc functions: Let X ∼ N (0, 1), and define its cdf as Z x 1 − ξ2 √ e 2 dξ Φ(x) = 2π −∞ Also define the function Q(x) = 1 − Φ(x) N (0, 1) Q(x) x As we shall soon see, the Q(·) function can be used to quickly compute P{X > a} for any gaussian r.v. X √ The function erfc(x) = 2Q( 2x), for x > 0 EE 278: Random Variable 2-16 Functions of a Random Variable • We are often given a r.v. with known distribution (pmf, cdf, or pdf), a function y = g(x), and would like to specify the random variable Y = g(X) • If X ∼ pX (x), i.e., discrete r.v., then Y is also discrete with pmf X pY (y) = pX (x) {x:g(x)=y} g(xi) = y x1 x2 x3 ... y1 y2 . . . y . . . • If X ∼ fX (x), i.e., a continuous r.v., and the function g is differentiable, then we can find the pdf of Y as follows 2-17 EE 278: Random Variable Note that P{y < Y ≤ y + ∆y} ∆y→0 ∆y So we first find the inverse image of the set (y, y + ∆y] fY (y) = lim x y y + ∆y y {x : y < g(x) ≤ y + ∆y} Thus P{y < Y ≤ y + ∆y} = P{x : y < g(x) ≤ y + ∆y}, or fY (y)∆y ≈ P{x : y < g(x) ≤ y + ∆y} from which we can find fY (y) ( as we shall demonstrate in the following examples) EE 278: Random Variable 2-18 • Example: Linear Function of a r.v. Let Y = aX + b y y + ∆y y x y−b y+∆y−b a a fY (y)∆y ≈ P{x ½ : y < g(x) ≤ y + ∆y} ¾ y − b ∆y y−b <X≤ + = P a a a ¶ µ y − b ∆y ≈ fX a |a| Thus as we let ∆y → 0 we get ¶ µ 1 y−b fY (y) = fX |a| a 2-19 EE 278: Random Variable • Example: Quadratic Function of a r.v. Let Y = X 2 y y + ∆y y √ √ y x √ y + 2∆y y fY (y)∆y ≈ P{x ¾ ½ : y < g(x) ≤ y + ∆y} ∆y ∆y √ √ √ √ y < X ≤ y + √ or − y − √ < X ≤ − y = P 2 y 2 y µ ¶ 1 1 √ √ ≈ √ fX ( y) + √ fX (− y) ∆y 2 y 2 y Thus EE 278: Random Variable 1 √ √ fY (y) = √ (fX ( y) + fX (− y)) 2 y 2-20 • In general, let X ∼ fX (x) and Y = g(X) be differentiable, then fY (y) = k X fX (xi) i=1 |g 0(xi)| , where x1, x2, . . . are the solutions of the equation y = g(x) (in terms of y) and g 0(xi) is the derivative of g evaluated at xi • If the cdf of X is given and we wish to find the cdf of Y y x y {x : g(x) ≤ y} then we use FY (y) = P{Y ≤ y} = P{x : g(x) ≤ y} 2-21 EE 278: Random Variable • Example: Let X ∼ F (x) (cdf), where F (x) is continuous, and define Y = F (X). Find the cdf of Y . To do so, consider FY (y) = = = = P{Y ≤ y} P{F (X) ≤ y} © ª P X ≤ F −1(y) ¢ ¡ F F −1(y) = y, for 0 ≤ y ≤ 1 Thus Y ∼ U [0, 1] !! • Note: Even though we assumed here that F (x) is invertible, it does not need to be — if F (x) = a, a constant over some interval, i.e., the probability that X lies in this interval is zero, then without loss of generality we can take F −1(a) to be the leftmost point of the interval EE 278: Random Variable 2-22