Download 3. Discrete Random Variables

3.
Discrete Random Variables
A random variable is defined as a function that assigns a numerical value to the outcome
of the experiment. In this chapter we introduce the concept of a random variable and
methods for calculating probabilities of events involving a random variable.
The following basic concepts will be presented.



probability mass function
define the expected value of a random variable and relate it to our intuitive notion
of an average
introduce the conditional probability mass function for the case where we are
given partial information about the random variable
Throughout the book it is shown that complex random experiments can be analyzed by
decomposing them into simple subexperiments.
3.1
The Notion of a Random Variable
The outcome of a random experiment need not be a number. However, we are usually
interested not in the outcome itself, but rather in some measurement or numerical
attribute of the outcome.
A measurement assigns a numerical value to the outcome of the random experiment.
Since the outcomes are random, the results of the measurements will also be random.
Hence it makes sense to talk about the probabilities of the resulting numerical values
A random variable X is a function that assigns a real number, X(ζ),to each outcome in the
sample space of a random experiment.
Recall that a function is simply a rule for assigning a numerical value to each element of
a set, as shown pictorially in Fig. 3.1. The specification of a measurement on the outcome
of a random experiment defines a function on the sample space, and hence a random
variable. The sample space S is the domain of the random variable, and the set SX of all
values taken on by X is the range of the random variable. Thus SX is a subset of the set of
all real numbers.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
1 of 37
We will use the following notation: capital letters denote random variables, e.g., X or Y,
and lower case letters denote possible values of the random variables, e.g., x or y.
The above example shows that a function of a random variable produces another random
variable.
Aside: Is this a good game for the player to play if the coin is fair?
Result
Probability
Payout
Prob*Payout
HHH
0.125
8
1.0
HHT, HTH, THH
3*0.125
1
0.375
HTT,THT,TTH
3*0.125
0
0
TTT
0.125
0
0
Total
1.375
On average, you lose 0.125 cents every time you play. If the HHH payout were $9, the
payout would equal the bet … on average.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
2 of 37
For random variables, the function or rule that assigns values to each outcome is fixed
and deterministic, as, for example, in the rule “count the total number of dots facing up in
the toss of two dice.” The randomness in the experiment is complete as soon as the toss is
done. The process of counting the dots facing up is deterministic. Therefore the
distribution of the values of a random variable X is determined by the probabilities of the
outcomes ζ in the random experiment. In other words, the randomness in the observed
values of X is induced by the underlying random experiment, and we should therefore be
able to compute the probabilities of the observed values of X in terms of the probabilities
of the underlying outcomes.
Example 3.3 illustrates a general technique for finding the probabilities of events
involving the random variable X. Let the underlying random experiment have sample
space S and event class F. To find the probability of a subset B of R, e.g., B  xk  we
need to find the outcomes in S that are mapped to B, that is,
A   : X    B
as shown in Fig. 3.2. If event A occurs then X    B so event B occurs. Conversely, if
event B occurs, then the value X   implies that ζ is in A, so event A occurs. Thus the
probability that X is in B is given by:
PX    B  PA  P : X    B
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
3 of 37
An alternate definition of Random Variable
Random variable: A real function whose domain is that of the outcomes of an
experiment (sample space, S) and whose actual value is unknown
in advance of the experiment.
From: http://en.wikipedia.org/wiki/Random_variable
A random variable can be thought of as the numeric result of operating a nondeterministic mechanism or performing a non-deterministic experiment to generate a
random result.
Unlike the common practice with other mathematical variables, a random variable cannot
be assigned a value; a random variable does not describe the actual outcome of a
particular experiment, but rather describes the possible, as-yet-undetermined outcomes in
terms of real numbers.
3.1.1 *Fine Point: Formal Definition of a Random Variable
In going from Eq. (3.1) to Eq. (3.2) we actually need to check that the event A is in F,
because only events in F have probabilities assigned to them. The formal definition of a
random variable in Chapter 4 will explicitly state this requirement.
If the event class F consists of all subsets of S, then the set A will always be in F, and any
function from S to R will be a random variable. However, if the event class F does not
consist of all subsets of S, then some functions from S to R may not be random variables.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
4 of 37
3.2
Discrete Random Variables And Probability Mass Function
A discrete random variable X is defined as a random variable that assumes values from a
countable set, that is, S X  x1 , x2 , x3 , . A discrete random variable is said to be finite
if its range is finite, that is, S X  x1 , x2 , x3 ,, xn . We are interested in finding the
probabilities of events involving a discrete random variable X. Since the sample space
S X is discrete, we only need to obtain the probabilities for the events A   : X    xk 
in the underlying random experiment. The probabilities of all events involving X can be
found from the probabilities of the Ak ’s.
The probability mass function (pmf) of a discrete random variable X is defined as:
p X x  PX  x  P : X    x
Note that p X  x  is a function of x over the real line, and that p X  x  can be nonzero only
at the values x1 , x2 , x3 , … For xk in SX, we have p X  xk   PAk  .
Note:
•
•
for discrete random variables: probability mass functions (pmf)
for continuous random variables: probability density functions (pdf)
Cumulative Distribution Function (CDF)
FX x  PX  x ,
for    x  
The sum or integral of the pmf or pdf.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
5 of 37
Properties of the probability mass functions (pmf)
(1)
(2)
p X x  0 for all x
 p  x    p  x    P A   1
xS X
(3)
X
X
all k
k
k
all k
PX in B   p X x  where B  S X
xB
The pmf of X gives us the probabilities for all the elementary events from SX . The
probability of any subset of SX is obtained from the sum of the corresponding elementary
events.
Once the pmf of the elementary events have been defined, the probability for all other
sets containing the elementary events can be readily defined.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
6 of 37
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
7 of 37
Example 3.9 Message Transmissions
Let X be the number of times a message needs to be transmitted until it arrives correctly
at its destination. Find the pmf of X. Find the probability that X is an even number.
X is a discrete random variable taking on values from S X  1,2,3, . The event X  k
occurs if the underlying experiment finds k - 1 consecutive erroneous transmissions
(“failures”) followed by an error-free one transmission (“success”):
p X x   PX  x  P00001  1  p 
k 1
 p  q k 1  p
We call X the geometric random variable, and we say that X is geometrically distributed.
In Eq. (2.42b), we saw that the sum of the geometric probabilities is 1.



k 1
k 1
k 0
PXis even   p X 2  k   p   q 2k 1  p  q   q 2k 
pq
q

2
1 q
1 q
Note: textbook solution is wrong … as a simple example, what happens if p=1 and q=0?
Finally, let’s consider the relationship between relative frequencies and the pmf.
For a large number of trials, the relative frequencies should approach the pmf!
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
8 of 37
3.3
Expected Value and Moments of Discrete Random Variable
In this section we introduce parameters that quantify the properties of random variables.
The expected value or mean of a discrete random variable X is defined by
m X  E X  
 x  p x   x
X
xS X
k
 p X xk 
all k
The expected value E[X] is defined if the above sum converges absolutely, that is,
E X  
x
k
 p X xk   
all k
There are random variables for which the summation does not converge. In such cases,
we say that the expected value does not exist.
If we view p X  x  as the distribution of mass on the points in the real line, then E[X]
represents the center of mass of this distribution.
The use of the term “expected value” does not mean that we expect to observe E[X] when
we perform the experiment that generates X.
E[X] corresponds to the “average of X” in a large number of observations of X.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
9 of 37
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
10 of 37
General types of problems and the 3expected value
Note:
n
k 
k 1
n  n  1
and
2
n
k
2
k 1

n  n  1  2n  1
6
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
11 of 37
3.3.1 Expected Value of Functions of a Random Variable
Let X be a discrete random variable, and let Z  g  X  . Since X is discrete, will
Z  g  X  assume a countable set of values of the form g  xk  where xk  S X . Denote
the set of values assumed by g(X) by z1 , z 2 , . One way to find the expected value of Z
is to use Eq. (3.8), which requires that we first find the pmf of Z.
m X  E X    x k  p X  x k 
k
EZ   g m X 
Another way is to use the following result:
EZ   Eg  X    g xk   p X  xk 
k
To show this equation, group the terms x k that are mapped to each value z j








g
x

p
x

z

p
x
k k X k j j   X k   j z j  pZ z j   EZ 
xk :g  xk  z j

The sum inside the braces is the probability of all terms x k for which g  xk   z j . which
 
is the probability that Z  z j , that is, p Z z j .
Example 3.17 Square-Law Device
Let X be a noise voltage that is uniformly distributed in S X   3,1,1,3 with
p X k   1 4 for k in S X . Find E[Z] where Z  X 2 .
Using the first approach we find the pmf of Z
p Z 9  PX   3,3  p X  3  p X 3  1 2
p Z 2  PX   1,1  p X  1  p X 1  1 2
Therefore
1
1 10
EZ    z j  p Z z j   1   9    5
2
2 2
j
The second approach produces
20
2 1
2 1
2 1
2 1
EZ    g xk   p X xk    3    1  1  3  
5
4 4
4
4
4
k
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
12 of 37
Useful Properties of the expected value operator:
Eg  X   h X   Eg  X   Eh X 
Ea  g  X   a  Eg  X 
Ec  c
Example 3.18 Square-Law Device
The noise voltage X in the previous example is amplified and shifted to obtain
Y  2 X  10 and then squared to produce Z  Y 2 Find E[Z].
.
2
2
EZ   E Y  E 2 X  10
  

EZ   E4 X  40 X  100
2
 
EZ   4  E X 2  40  EX   100
but
 
20
2 1
2 1
2 1
2 1
E X 2   3    1  1  3  
5
4
4
4
4 4
1
1
1
1 0
EX    3    1  1  3    0
4
4
4
4 4
EZ   4  5  40  0  100  120
Example 3.19 Voice Packet Multiplexer
Let X be the number of voice packets containing active speech produced by n=48
independent speakers in a 10-millisecond period as discussed in Section 1.4. X is a
binomial random variable with parameter n and probability p = 1/3. Suppose a packet
multiplexer transmits up to M=2- active packets every 10 ms, and any excess active
packets are discarded. Let Z be the number of packets discarded. Find E[Z].
The number of packets discarded every 10 ms is the following function of X:
if X  M
0

Z  X  M   
 X  M if X  M
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
13 of 37
Then
 48  1   2 
p X k          
 k  3  3
k
48 k
for k  0,1,,48
,
 48  1   2 
EZ    k  20         
k  21
 k   3  3
k
48
48 k
 0.182
Every 10 ms EX   n  p  16 active packets are produced on average, so the fraction of
active packets discarded is EZ  / EX   0.182 / 16  1.14% which users will tolerate.
This example shows that engineered systems also play “betting” games where favorable
statistics are exploited to use resources efficiently. In this example, the multiplexer
transmits 20 packets per period instead of 48 for a reduction of 28/48=58% of the
capacity required for 100% packets for all 48 possible users.
See MATLAB example for pX(x) and computations.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
14 of 37
3.3.2 Variance of a Random Variable
The expected value E[X], by itself, provides us with limited information about X.For
example, if we know that EX   0 then it could be that X is zero all the time. However,
it is also possible that X can take on extremely large positive and negative values. We are
therefore interested not only in the mean of a random variable, but also in the extent of
the random variable’s variation about its mean.
Let the deviation of the random variable X about its mean be X  E X  which can take
on positive and negative values. Since we are interested in the magnitude of the
variations only, it is convenient to work with the square of the deviation, which is always
2
positive, D X    X  EX  . The variance of the random variable X is defined as the
expected value of D:

 
 X2  VARX   E  X  EX 2  E  X  m X 2
 X2 
 x  m 
 p X  x     xk  m X   p X  xk 
2
2
X
xS X

all k
The standard deviation of the random variable X is defined by:
 X  STDX   VARX 1 2
By taking the square root of the variance we obtain a quantity with the same units as X.
Aside: In making measurements, one often uses the variances as the “+/- error” as in
value  m X   X
An alternate equality for the variance can be derived as
2
2
VARX   E  X  m X   E X 2  2  m X  X  m X

 
 
VARX   E X 2  2  m X  EX   m X
 
VARX   E X 2  2  m X  m X  m X
 
 

2
2
VARX   E X 2  m X  E X 2  E X 
2
2
 
The value E X 2 is also referred to as the second moment of X. The Nth moment is
defined as:
  x
E XN 
xS X
N
 p X  x    xk  p X  xk 
N
all k
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
15 of 37
Some nice propertied of the variance: (1) adding a constant offset to the random variable
does not change the variance and (2) multiplying the random variable by a constant
results in the constant squared times the variance.
2
2
VARX  c  E  X  c  EX  c  E  X  c  EX   c 

 
VARX  c  E X  EX    VARX 

2
and

 
VARc  X   E c   X  EX    c
VARc  X   E c  X  Ec  X   E c  X  c  EX 
2
2
2
2
2

 VARX 
Now let X  c a random variable that is equal to a constant with probability 1, then
E X   m X  c

 

VARX   E c  m X   E c  c   E0  0
2
2
A constant random variable has zero variance.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
16 of 37
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
17 of 37
3.4
Conditional Probability Mass Function
In many situations we have partial information about a random variable X or about the
outcome of its underlying random experiment. We are interested in how this information
changes the probability of events involving the random variable. The conditional
probability mass function addresses this question for discrete random variables.
3.4.1
Conditional Probability Mass Function
Let X be a discrete random variable with pmf p X x  and let C be an event that has
nonzero probability, PC  . The conditional probability mass function of X is defined by
the conditional probability:
p X  x | C  
PX  x  C 
, for PrC   0
PC 
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
18 of 37
Most of the time the event C is defined in terms of X, for example C  X  10 or
C  a  X  b . For x k in S X we have the following general result:
 Px k 
,

p X  x k | C    PC 
0,

if x k  C
if x k  C
The above expression is determined entirely by the pmf of X.
Many random experiments have natural ways of partitioning the sample space S into the
union of disjoint events B1, B2, … , Bn. Let p X x | Bi  be the conditional pmf of X given
event Bi. The theorem on total probability allows us to find the pmf of X in terms of the
conditional pmf’s:
n
p X x    p X  x | Bi   PBi 
i 1
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
19 of 37
3.4.2 Conditional Expected Value
Let X be a discrete random variable, and suppose that we know that event B has
occurred. The conditional expected value of X given B is defined as:
m X | B  x   E X | B  
 x  p x | B    x
X
xS X
k
 p X  xk | B 
k
where we apply the absolute convergence requirement on the summation.
The conditional variance of X given B is defined as:


VARX | B  E X  m X |B  | B   xk  m X |B   p X xk | B 
2
2
k


VARX | B  E X 2 | B  m X |B
2
Note that the variation is measured with respect to m X |B and not m X .
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
20 of 37
Let B1, B2, … , Bn be the partition of S, and let p X x | Bi  be the conditional pmf of X
given event Bi. Then, E[X] can be calculated from the conditional expected values
E X | B i 
n
EX    EX | Bi   PBi 
i 1
Based on total probability
n

EX    k  p X xk    k   p X  xk | Bi   PBi 
k
k
 i 1

n
n
n

EX     k  p X xk | Bi   PBi    EX | Bi   PBi 
i 1  k
i 1

It can also be shown that
n
Eg  X    Eg  X  | Bi   PBi 
i 1
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
21 of 37
3.5
Important Discrete Random Variables
Certain random variables arise in many diverse, unrelated applications. The
pervasiveness of these random variables is due to the fact that they model fundamental
mechanisms that underlie random behavior.
3.5.1 The Bernoulli Random Variable
S X  0,1
p0  1  p  q and p1  p , for 0  p  1
m X  E X   p
 X 2  VARX   p  1  p   p  q
Remarks: The Bernoulli random variable is the value of the indicator function IA for
some event A; X=1 if A occurs and X=0 otherwise.
Every Bernoulli trial, regardless of the event A, is equivalent to the tossing of a biased
coin with probability of heads p. In this sense, coin tossing can be viewed as
representative of a fundamental mechanism for generating randomness, and the Bernoulli
random variable is the model associated with it.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
22 of 37
3.5.2 The Binomial Random Variable
S X  0,1,2,, n
 n
nk
pk     p k  1  p  ,
k 
for k  0,1,2,, n
m X  E X   n  p
 X 2  VARX   n  p  1  p 
Remarks: X is the number of successes in n Bernoulli trials and hence the sum of n
independent, identically distributed Bernoulli random variables
The binomial random variable arises in applications where there are two types of objects
(i.e., heads/tails, correct/erroneous bits, good/defective items, active/silent speakers), and
we are interested in the number of type 1 objects in a randomly selected batch of size n,
where the type of each object is independent of the types of the other objects in the batch.
Note that:
pk 1 n  k
p


pk
k 1 1 p
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
23 of 37
3.5.3 The Geometric Random Variable
First Version
S X  0,1,2,
pk  p  1  p  ,
for k  0,1,2,
k
m X  E X  
1 p
p
 X 2  VARX  
1 p
p2
Remarks: X is the number of failures before the first success in a sequence of
independent Bernoulli trials.
Note that:
pk 1
 1 p  q
pk
The geometric random variable is the only discrete random variable that satisfies the
memoryless property:
PM  k  j | M  j   PM  k  ,
for all j,k>1
The above expression states that if a success has not occurred in the first j trials, then the
probability of having to perform at least k more trials is the same as the probability of
initially having to perform at least k trials. Thus, each time a failure occurs, the system
“forgets” and begins anew as if it were performing the first trial.
Second Version
S X  1,2,
pk  p  1  p 
k 1
,
for k  1,2,
m X  EX ' 
 X 2  VARX ' 
1
p
1 p
p2
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
24 of 37
Remarks: X’=X+1 is the number of trials until the first success in a sequence of
independent Bernoulli trials.
k
PM  k    p  q
i 1
i 1
1 qk
 p  q  p 
 1 qk
1 q
i 0
k 1
i
The geometric random variable arises in applications where one is interested in the time
(i.e., number of trials) that elapses between the occurrence of events in a sequence of
independent experiments. Examples where the modified geometric random variable
arises are: number of customers awaiting service in a queueing system; number of white
dots between successive black dots in a scan of a black-and-white document.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
25 of 37
3.5.4 The Poisson Random Variable
In many applications, we are interested in counting the number of occurrences of an
event in a certain time period or in a certain region in space. The Poisson random variable
arises in situations where the events occur “completely at random” in time or space. For
example, the Poisson random variable arises in counts of emissions from radioactive
substances, in counts of demands for telephone connections, and in counts of defects in a
semiconductor chip.
S X  0,1,2,
pk 
k
k!
 e  ,
for k  0,1,2,
m X  E X   
 X 2  VARX   
where  is the average number of event occurrences in a specified time interval or region
in space.
Thus the Poisson pmf can be used to approximate the binomial pmf for large n and small
p, using  = np.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
26 of 37
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
27 of 37
3.5.5 The Uniform Random Variable
S X   j  1, j  2, j  L
pk 
1
,
L
for k   j  1, j  2, j  L
m X  E X   j 
L 1
2
 X 2  VARX  
L2  1
12
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
28 of 37
3.5.6 The Zipf Random Variable
The Zipf random variable is named for George Zipf who observed that the frequency of
words in a large body of text is proportional to their rank. Suppose that words are ranked
from most frequent, to next most frequent, and so on. Let X be the rank of a word, then
S X  1,2,, L where L is the number of distinct words. The pmf of X is:
S X  1,2,, L
pk 
1 1
 , for k  1,2,, L
cL k
where cL is a normalization constant. The second word has 1/2 the frequency of
occurrence as the first, the third word has 1/3 the frequency of the first, and so on. The
normalization constant, cL, is given by the sum:
1
j 1 j
L
cL  
L
L
k 1
k 1
m X  E X    k  p k   k 
 X 2  VARX  
1 1 L
 
cL k cL
L2  L L2

2  cL cL 2
The Zipf and related random variables have gained prominence with the growth of the
Internet where they have been found in a variety of measurement studies involving Web
page sizes, Web access behavior, and Web page interconnectivity.
These random variables had previously been found extensively in studies on the
distribution of wealth and, not surprisingly, are now found in Internet video rentals and
book sales.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
29 of 37
3.6
Generation of Discrete Random Variables
If you can generate the pmf, you can generate an approximation to the random variable.
1) generate a uniform random number between 0 and 1.
2) Using the pmf, assign values from 0 to pmf(1) to the 1st discrete value, pmf(1) to
pmf(2) to the second and so on.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
30 of 37
SUMMARY
• A random variable is a function that assigns a real number to each outcome of a random
experiment. A random variable is defined if the outcome of a random experiment is a
number, or if a numerical attribute of an outcome is of interest.
• The notion of an equivalent event enables us to derive the probabilities of events
involving a random variable in terms of the probabilities of events involving the
underlying outcomes.
• A random variable is discrete if it assumes values from some countable set. The
probability mass function is sufficient to calculate the probability of all events involving
a discrete random variable.
• The probability of events involving discrete random variable X can be expressed as the
sum of the probability mass function p X  x .
• If X is a random variable, then Y=g(X) is also a random variable.
• The mean, variance, and moments of a discrete random variable summarize some of the
information about the random variable X. These parameters are useful in practice because
they are easier to measure and estimate than the pmf.
• The conditional pmf allows us to calculate the probability of events given partial
information about the random variable X.
• There are a number of methods for generating discrete random variables with prescribed
pmf’s in terms of a random variable that is uniformly distributed in the unit interval.
CHECKLIST OF IMPORTANT TERMS
Discrete random variable
Equivalent event
Expected value of X
Function of a random variable
nth moment of X
Probability mass function
Random variable
Standard deviation of X
Variance of X
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
31 of 37
Mean Values and Moments
Mean Value: the expected mean value of measurements of a process involving a random
variable.
This is commonly called the expectation operator or expected value of …
and is mathematically described as:
X  EX  

x f
X
x   dx

X  EX  

 x  Pr X  x
x  
For laboratory experiments, the expected value of a voltage measurement can be thought
of as the DC voltage.
In general, the expected value of a function is:
E g  X  

 gX  f
X
x   dx

E g  X  

 g  X   Pr X  x
x  
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
32 of 37
Moments
The moments of a random variable are defined as the expected value of the powers of the
measured output or …

   x
X EX
n
n
 f X  x   dx
n


   x
X EX
n
n
n
 Pr  X  x 
x  
Therefore, the mean or average is sometimes called the first moment.
Expected Mean Squared Value or Second Moment
The mean square value or second moment is

   x
X EX
2
2
2
 f X  x   dx

 
X2 E X2 

x
2
 Pr  X  x 
x  
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
33 of 37
Central Moments
The central moments are the moments of the difference between a random variable and
its mean.
X  X 
n

    x  X 

E XX
n
 f X  x   dx
n


X  X 
n
E XX
    x  X 

n
n
 Pr  X  x 
x  
Notice that the first central moment is 0 …
The second central moment is referred to as the variance of the random variable …


2
2  X X

E XX
    x  X 
2

 f X  x   dx
2


  XX
2

2

E XX
    x  X 
2

2
 Pr  X  x 
x  
Note that:

2 E X X

   EX  X  X  X 
2
 2  E X 2  2 X  X  X
2

 2  E X 2   2  X  E X   X
 2  E X 2   2  X  X  X
2
2
 2  E X 2   X  E X 2   E X 2
2
2  X2 X
2
The variance is equal to the 2nd moment minus the square of the first moment …
The variance is also referred to as the standard deviation.
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
34 of 37
Hypergeometric Distribution
From: http://en.wikipedia.org/wiki/Hypergeometric_distribution
In probability theory and statistics, the hypergeometric distribution is a discrete
probability distribution (probability mass function) that describes the number of
successes in a sequence of n draws from a finite population without replacement.
A typical example is the following: There is a shipment of N objects in which D are
defective. The hypergeometric distribution describes the probability that in a sample of n
distinctive objects drawn from the shipment exactly x objects are defective.
 D  N  D
   

x   n  x 

Pr  x  X , N , D, n  
N
 
n
for
max 0, D  n  N   x  min n, D 
The equation is derived based on a non-replacement Bernoulli Trials …
Where the denominator term defines the number of trial possibilities, the 1st numerator
term defines the number of ways to achieve the desired x, and the 2nd numerator term
defines the filling of the remainder of the set.
Example Use: State Lottery
There are N objects in which D are of interest. The hypergeometric distribution describes
the probability that in a sample of n distinctive objects drawn from the total set exactly x
objects are of interest.
Lotteries …
N= number of balls to be selected at random
D = the balls that you want selected
n = the number of balls drawn
x = the number of desired balls in the set that is drawn
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
35 of 37
Example: Michigan’s Classic Lotto 47
Prize Structure For Classic Lotto 47 (web site data)
Match
Prize
Odds of Winning
6 of 6
Jackpot
1 in 10,737,573
5 of 6
$2,500 (guaranteed)
1 in 43,649
4 of 6
$100 (guaranteed)
1 in 873
$5 (guaranteed)
1 in 50
3 of 6
Overall Odds: 1 in 47
http://www.michiganlottery.com/lotto_47_info?#
Matlab Odds
Match
Odds of Winning 1 in
Percent Probability
6 of 6
10737573
0.0000%
5 of 6
43649
0.0023%
4 of 6
873
0.1146%
3 of 6
50
1.9856%
2 of 6
7.1
14.1471%
1 of 6
2.4
41.8753%
0 of 6
2.4
Chance of winning money is 2.1025%
Chance of 0 or 1 number 83.75%
41.8753%
One winner, even money Jackpot $7,826,573.
Matlab Note: binomial coefficient = nchoosek(n,k)
Matlab Example:
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
36 of 37
Black-Jack Card Point Count
The “value” in black-jack of cards is the value of the card for two through 9, 10 for 10’s
and face cards, and 11 for aces. What is the mean value of the cards in a deck?
And the probability mass function, f X  x   Pr  X  x  , is then
 4 , for x  2,3,4,5,6,7,8,9
 52
16 , for x  10
f X  x    52
 4 , for x  11
 52
0,
else
Mean Value
X  EX  

 x  Pr  X  x 
x  
X  EX  
4  44  16  10  4  11  380  7.31
4 9
16
4
  k   10   11 
52 k  2
52
52
52
52
Second Moment (mean squares)
  x
X2 E X2 

2
 Pr  X  x 
x  
 
X2 E X2 
4  284  16  100  4  121  3220  61.92
4 9 2 16
4
  k   10 2   112 
52 k 2
52
52
52
52
Variance
2  X2 X
2
 2  61.92  7.312  8.49
  2.91
So, if you want another card :
EX     7.31  2.91
What if we originally assumed that an Ace was worth 1 instead of 11 ?
So, if you want another card :
E X     6.53  3.15
Notes and figures are based on or taken from materials in the textbook: Alberto Leon-Garcia, “Probability, Statistics,
and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
37 of 37