Download Introduction to Random Variables (RVs)

ECE270: Handout 5
Introduction to Random Variables (RVs)
Outline:
1. informal definition of a RV,
2. three types of a RV: a discrete RV, a continuous RV, and a mixed RV,
3. a general rule to find probability of events concerning a RV,
4. cumulative distribution function (CDF) of a RV,
5. formal definition of a RV using CDF,
6. discrete RV: probability mass function (pmf ) and CDF,
7. continuous RV: probability density function (pdf ) and CDF,
8. basic properties of the CDF.
• The outcome of a random experiment need not be a numbers. Examples are: coding the
incoming patients in a hospital according to their insurance and health status, randomly
selecting a committee from a group of people, randomly selecting balls from an urn, randomly
selecting cards from a deck.
• Usually we are interested in some measurement or numerical attribute of the outcome. Examples are: counting the number of heads when tossing a coin 10 times, the number of
re-transmission needed until the receiver receives the data packet correctly, the number of errors in erroneous received data packets, the lifetime of a memory chip, the number of packets
arriving in t sec at a server, the number of queries arriving in t sec at a call center, the number
of particles emitted by a radioactive mass during a fixed time period, the random thermal
noise being added to the signal at the receiver of a communication system at a specific time.
• In these examples, we assign a real number to the outcome of the random experiment through
measurement. Since the outcomes are random, the results of the measurements will be random
too. Hence it makes sense to talk about the probabilities of the resulting numerical values.
F Informal Definition of a RV
• A RV X is a function that assigns a real valued number x = X(ξ) to each outcome ξ ∈ S
(Recall: a function is a rule for assigning a numerical value to each element of a set).
• Sample space S is the domain of the RV X and the set of all real numbers taken on by X,
Sx , Sx ⊂ R, is the range of the RV.
F Three Types of a RV
• Three types of RVs: i) discrete, ii) continuous, iii) mixed.
i) The range of a discrete RV X is a countable set (either finite or infinite) Sx =
{x1 , x2 , ...} or Sx = {x1 , x2 , ..., xn }.
ii) The range of a continuous RV X is an uncountable set, e.g., Sx = [0, ∞), Sx =
(−∞, ∞), Sx = [0, 1], Sx = [a, b] for −∞ < a < b < ∞.
iii) The range of a mixed RV X is the union of an uncountable and a countable sets.
• Notation: capital letters X, Y , Z, V , U , ... denote RVs, lowercase letters x, y, z, v, u, ... denote possible values of RVs.
• Consider the RV X, the function or rule that assigns a real number x = X(ξ) to the outcome
ξ ∈ S is fixed and deterministic, e.g., the rule of “count the number of heads when we toss a
coin 3 times”. The randomness in the experiment is complete as soon as we toss the coin 3
times. The process of counting is deterministic.
• In some random experiments the outcome ξ is already the numerical value we are interested in,
e.g., measure the lifetime of a chip under certain conditions, ⇒ X(ξ) = ξ (identity function).
• The distribution of the values of a RV X is determined by the probabilities of the basic events
of the underlying random experiment, i.e., we should be able to compute the probability of
the observed value of X in terms of the probability of the underlying event.
• The specification of the measurements on the outcomes of a random experiment defines a
function on S and hence a RV.
• A function of a RV is another RV.
EXAMPLE 1
A fair coin is tossed 3 times and the sequence of heads and tails is noted. The sample space
is S = {HHH, HHT, HT H, HT T, T HH, T HT, T T H, T T T } and the outcomes are equally
probable.
a) Let X be the number of heads in the 3 tosses. Sx = {0, 1, 2, 3}.
b) A player pays 1.5$ to play the following game: the player receives 1$ if X = 2 and
8$ if X = 3, but nothing otherwise. Let Y be the reward to the player. Sy = {0, 1, 8}.
c) Let Z be a function of X such that Z = 0 if X ∈ {0, 1, 2} and Z = 1 if X = 3.
Sz = {0, 1}.
ξ
X(ξ)
Y (ξ)
Z(ξ)
HHH
3
8
1
HHT
2
1
0
HTH
2
1
0
THH
2
1
0
HTT
1
0
0
P (X = 0) = P ({T T T }) = 1/8
P (X = 1) = P ({T T H, T HT, HT T }) = 3/8
P (X = 2) = P ({T HH, HHT, HT H}) = 3/8
Page 2
THT
1
0
0
TTH
1
0
0
TTT
0
0
0
P (X = 3) = P ({HHH}) = 1/8
P (Y = 0) = P ({T T T, T T H, T HT, HT T }) = 4/8 = 1/2
P (Y = 1) = P ({T HH, HT H, HHT }) = 3/8
P (Y = 8) = P ({HHH}) = 1/8
½
P (Z = 0) = P (X ∈ {0, 1, 2}) = P (X = 0) + P (X = 1) + P (X = 2) = 1/8 + 3/8 + 3/8 = 7/8
P (Z = 1) = P (X = 3) = 1/8
or alternatively:
½
P (Z = 0) = P ({T T T, T T H, T HT, HT T, HHT, HT H, T HH}) = 7/8
P (Z = 1) = P ({HHH}) = 1/8
F A General Rule to Find Probabilities of Events Concerning a RV X
• The example shows a general technique to find the probability of events involving RVs. To
find the probability of X ∈ B (where B ⊂ R) we need to find the set of outcomes A, A ⊂ S
that are mapped to B, i.e., the set A = {ξ : X(ξ) ∈ B}.
• If the experiment outcome ξ ∈ A then event A occurs. Hence X(ξ) ∈ B ⇒ event B occurs.
If event B occurs then X(ξ) ∈ B implies ξ ∈ A ⇒ event A occurs. So: P (X ∈ B) = P (A) =
P ({ξ : X(ξ) ∈ B}). We refer to A and B as equivalent events.
• In some random experiments the outcome ξ is already the numerical value we are interested
in. In such cases we simply let X(ξ) = ξ, i.e., the identity function is used to obtain a random
variable. Example: measure the received signal at a receive antenna.
F CDF of a RV X
• Regardless of X being discrete, continuous, or mixed, the cumulative distribution function
(CDF) of a RV X is defined as the probability of the event B = {X ≤ x}:
FX (x) = P (B) = P ({X ≤ x}) , P (X ≤ x)
for
−∞<x<∞
The event B = {X ≤ x} and its probability vary as x is varied. Hence, FX (x) is a function
of the variable x.
• In terms of the underlying random experiment FX (x) = P ({ξ : X(ξ) ≤ x}).
|
{z
}
an event
F Formal Definition of a RV Using CDF
• We are ready for a formal definition of a RV: Consider a random experiment with sample
space S. The RV X is a function from S to real line R with the property that the set
A = {ξ : X(ξ) ≤ b} is an event for every b ∈ R. This definition simply requires that every
set A has a well defined probability in the underlying random experiment.
Page 3
F Discrete RV: pmf and CDF
• A discrete RV X has the CDF FX (x) that is right continuous, non-decreasing staircase
function of x, with jumps at points x1 , x2 , ... where xi ∈ Sx , and it grows from 0 to 1:
X
FX (x) =
P (X = xi ) u(x − xi )
| {z }
i=1
pmf
• pX (x) = P (X = x) = P ({ξ : X(ξ) = x}), for x a real number, is called the probability mass
function (pmf) of X, and is the magnitude of the jump in CDF at the point x.
• pX (x) is a function of x over the real line and is nonzero only at the points x1 , x2 , ... ∈ Sx .
• pmf pX (x) for all x ∈ Sx provides all information required to calculate probabilities of any
event involving a discrete RV X. We can obtain CDF from pmf and vice versa.
• We can forget about the underlying random experiment, its sample space S and its associated
probability law and just work with Sx and the pmf of X, or equivalently the CDF of X.
• The pmf of X is the probability of allP
elementary events from Sx . The pmf
P pX (x) satisfies
three properties: 1) 0 ≤ pX (x) ≤ 1, 2) x∈Sx pX (x) = 1, 3) P (X ∈ B) = x∈B pX (x) where
B ⊂ R.
EXAMPLE 2 (CDF of a discrete RV, back to example 1)
Find and plot FX (x) in example 1.
F Continuous RV: pdf and CDF
• A continuous RV X has the CDF FX (x) that is continuous everywhere and is a non-decreasing
function of x that grows from 0 to 1 as x ranges from its minimum to its maximum values.
Also, FX (x) is smooth enough that it can be written as an integral of some nonnegative
function fX (x):
Z
x
FX (x) =
fX (x) dx
{z }
−∞ |
pdf
• Considering FX (x) we calculate probabilities as integral of probability density function (pdf)
over an interval of the real line (−∞, x]. We can obtain CDF from pdf and vice versa.
• For a continuous RV X we have P (X = x) = 0.
EXAMPLE 3 (CDF of a continuous RV)
We spin an arrow attached to the center of a circular board. Let θ be the angle of the arrow
where 0 < θ ≤ 2π. The probability that θ falls in a subinterval of (0, 2π] is equal to the length
of the subinterval divided by 2π. We define the RV X(θ) = θ/2π. What is FX (x)?
Page 4
EXAMPLE 4 (CDF of a mixed RV)
The waiting time X of a customer at a taxi stand is zero if the customer finds a taxi parked
at the stand, and a uniformly distributed random length of time in the interval [0, 1] (in hour)
if no taxi is found upon arrival. The probability that a taxi is at the stand when the customer
arrives is p. What is the CDF of X?
SOLUTION: using the total probability theorem we have
FX (x) = P (X ≤ x) = P (X ≤ x|find taxi)p + P (X ≤ x|no taxi)(1 − p)

½
 1 x>1
1 x≥0
x 0≤x≤1
P (X ≤ x|find taxi) =
P (X ≤ x|no taxi) =
0 x<0

0 x<0
⇒

x>1
 1 × p + 1 × (1 − p) = 1
1 × p + x(1 − p) = p + x(1 − p) 0 ≤ x ≤ 1
FX (x) = P (X ≤ x) =

0
x<0
F Basic Properties of the CDF
The three axioms of probability and the corresponding properties imply that the CDF has
the following properties:
(i) 0 ≤ FX (x) ≤ 1
(ii) limx→∞ FX (x) = 1
(iii) limx→−∞ FX (x) = 0
(iv) FX (x) is a nondecreasing function of x, that is, if a < b then FX (a) ≤ FX (b)
(v) FX (x) is continuous from the right, i.e., for h > 0, FX (b) = limh→0 FX (b + h) =
FX (b+ ). This property implies that at points of discontinuity, the CDF is equal to the
limit from the right.
These five properties confirm that, in general, CDF is a nondecreasing function of x that
grows from 0 to 1 as x increases from −∞ to ∞.
More properties follow:
(vi) P (a < X ≤ b) = FX (b) − FX (a)
(vii) P (X = b) = FX (b) − FX (b− ) = FX (b+ ) − FX (b− ). This property says that the
probability that X = b is equal to the magnitude of the jump of the CDF at the point
b. So if the CDF is continuous at a point b, then P (X = b) = 0.
(viii) P (X > x) = 1 − FX (x)
Page 5
Based on above properties we can compute P (a ≤ X ≤ b), P (a ≤ X < b), P (a < X < b).
Since {a ≤ X ≤ b} = {a} ∪ {a < X ≤ b}, we have:
P (a ≤ X ≤ b) = P (X = a) + P (a < X ≤ b)
= FX (a) − FX (a− ) + FX (b) − FX (a)
= FX (b) − FX (a− )
Since {a ≤ X ≤ b} = {b} ∪ {a ≤ X < b}, we have:
P (a ≤ X < b) = P (a ≤ X ≤ b) − P (X = b) = FX (b− ) − FX (a− )
Since {a ≤ X < b} = {a} ∪ {a < X < b}, we have:
P (a < X < b) = P (a ≤ X < b) − P (X = a) = FX (b− ) − FX (a)
• NOTE: if the CDF is continuous at the endpoints of an interval, then the endpoints have zero
probabilities, and therefore, they can be included in, or excluded from, the interval without
affecting the probabilities.
EXAMPLE 5 (back to example 1)
Consider example 1 and let A = {1 < X ≤ 2}, B = {0.5 ≤ X < 2.5}, C = {1 ≤ X < 2}. Find
P (A), P (B), P (C).
EXAMPLE 6 (back to example 3)
Find P (−0.5 < X < 0.25), P (0.3 < X < 0.65), P (|X − 0.4| > 0.2).
Page 6