Download Chapter 3: Discrete Random Variable and Probability Distribution

Chapter 3: Discrete Random Variable and
Probability Distribution
Walid Sharabati
Purdue University
February 3, 2014
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
1 / 31
Discrete Random Variables
Chapter Overview
Random Variable (r.v.)
Definition
Discrete and continuous r.v.
Probability distribution for discrete r.v.
Mass function
Cumulative distribution function (CDF)
Some discrete probability distribution
Binomial
Geometric and Hypergeometric
Poisson
Uniform
Expectation and variance
Expectation
Variance
Poisson Process
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
2 / 31
Discrete Random Variables
Random Variables
Random Variable
Definition
A random variable (r.v.) is a real valued function of the sample space S.
It is any rule that associates a number with each outcome in S.
i.e., a r.v. is some number associated with a random experiment.
Notation: we use upper case letters X, Y, Z, · · · , to denote random
variables. Lower case x will be used to denote different values of a
random variable X.
A random variable is a real valued function, so the expression:
X(s) = x means that x is the value associated with the outcome s
(in a specific sample space S) by the r.v. X.
Example
Toss a coin, the sample space is S = {H, T }. Let X be the r.v. associated
with this random experiment, and let X(H) = 1, X(T ) = 0. Or we may
simply write X = x, x = 0, 1.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
3 / 31
Discrete Random Variables
Random Variables
Random Variable Examples: Bernoulli rv
Example 3.1.1 Check if a manufactured computer component is defect. If
it is defect X = 1, if not X = 0.
Example 3.1.2 Let X = 1 if the life of a light bulb is over 1000 hours,
X = 0 if not.
Definition
A Bernoulli rv has two possible values 0 and 1. A Bernoulli rv is like an
“indicator” variable I:
1, If event A occurs;
I(A) =
0, If event A doesn’t occur.
Example 3.1.3 Toss a coin 3 times. Let I1 be the Bernoulli variable for
the first toss, I2 be the Bernoulli variable for the second toss, I3 be the
Bernoulli variable for the third toss. Ii = 1, if head, Ii = 0 if tail; for
i = 1, 2, 3. Let X be the totally number of heads tossed, we have:
X = I1 + I2 + I3
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
4 / 31
Discrete Random Variables
Random Variables
Types of Random Variables: Discrete & Continuous
A discrete rv is an rv whose possible values constitute a finite set or
a countably infinite set.
A continuous rv is an rv whose possible values consists of an entire
interval on the real line.
Example
Example
Example
Example
3.1.4
3.1.5
3.1.6
3.1.7
X
X
X
X
=
=
=
=
Walid Sharabati (Purdue University)
number of tosses needed before getting a head.
number of calls a receptionist gets in an hour.
life span of a light bulb.
weight of a Purdue female student.
Discrete Random Variable & Probability Dist.
Spring 2014
5 / 31
Discrete Random Variables
Probability Distributions
Probability Distribution for Discrete RVs
Distribution of an rv, vaguely speaking, is how an rv distributes its
probabilities on real numbers. For a discrete rv, we may list the values and
the probability for each value of the rv, this gives the probability
distribution of the discrete rv. Such rv is said to be an rv with discrete
distribution.
Definition
The probability distribution or probability mass function (pmf) of a
discrete rv is defined for every number x by:
p(x) = P (X = x) = P (all s ∈ S : X(s) = x).
Definition
Suppose that p(x) depends on a quantity that can be assigned any one of
a number of possible values, each with different value determining a
different probability distribution. Such a quantity is called a parameter of
the distribution.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
6 / 31
Discrete Random Variables
Probability Distributions
PMF Examples
Example 3.1.8 Tossing a die, let X = outcome of the die. X = 1, 2, ...6. Find pmf.
p(1) = P (X = 1) = P (outcome of the die is 1)
=
p(2) = P (X = 2) = P (outcome of the die is 2)
=
1
6
1
6
.........
=
..
.
p(6) = P (X = 6) = P (outcome of the die is 6)
=
1
6
Example 3.1.9 Toss a fair coin three times, let X = number of heads, find the pmf.
1
8
p(0)
=
P (X = 0) = P (TTT) =
p(1)
=
P (X = 1) = P (TTH) + P (THT) + P (HTT) =
p(2)
=
P (X = 2) = P (HHT) + P (THH) + P (HTH) =
p(3)
=
P (X = 3) = P (HHH) =
Walid Sharabati (Purdue University)
3
8
3
8
1
8
Discrete Random Variable & Probability Dist.
Spring 2014
7 / 31
Discrete Random Variables
Probability Distributions
PMF Examples
Example 3.1.8 Tossing a die, let X = outcome of the die. X = 1, 2, ...6. Find pmf.
p(1) = P (X = 1) = P (outcome of the die is 1)
=
p(2) = P (X = 2) = P (outcome of the die is 2)
=
1
6
1
6
.........
=
..
.
p(6) = P (X = 6) = P (outcome of the die is 6)
=
1
6
Example 3.1.9 Toss a fair coin three times, let X = number of heads, find the pmf.
1
8
p(0)
=
P (X = 0) = P (TTT) =
p(1)
=
P (X = 1) = P (TTH) + P (THT) + P (HTT) =
p(2)
=
P (X = 2) = P (HHT) + P (THH) + P (HTH) =
p(3)
=
P (X = 3) = P (HHH) =
3
8
3
8
1
8
Example 3.1.10 Let X = 1 if a specific product is defect, X = 0 otherwise. And suppose
p(1) = α, then p(0) = 1 − α. We write:

if x=1;
 α,
1 − α, if x=0;
p(x, α) =

0,
otherwise.
Here α is called the parameter of the distribution.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
7 / 31
Discrete Random Variables
Probability Distributions
PMF Examples
Example 3.1.8 Tossing a die, let X = outcome of the die. X = 1, 2, ...6. Find pmf.
p(1) = P (X = 1) = P (outcome of the die is 1)
=
p(2) = P (X = 2) = P (outcome of the die is 2)
=
1
6
1
6
.........
=
..
.
p(6) = P (X = 6) = P (outcome of the die is 6)
=
1
6
Example 3.1.9 Toss a fair coin three times, let X = number of heads, find the pmf.
1
8
p(0)
=
P (X = 0) = P (TTT) =
p(1)
=
P (X = 1) = P (TTH) + P (THT) + P (HTT) =
p(2)
=
P (X = 2) = P (HHT) + P (THH) + P (HTH) =
p(3)
=
P (X = 3) = P (HHH) =
3
8
3
8
1
8
Example 3.1.10 Let X = 1 if a specific product is defect, X = 0 otherwise. And suppose
p(1) = α, then p(0) = 1 − α. We write:

if x=1;
 α,
1 − α, if x=0;
p(x, α) =

0,
otherwise.
Here α is called the parameter of the distribution.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
7 / 31
Discrete Random Variables
Probability Distributions
Cumulative Distribution Function (CDF) for Discrete RVs
Definition
The cumulative distribution function (cdf) F (x) of a discrete rv X with
pmf p(x) is defined for every number x by
X
F (x) = P (X ≤ x) =
p(y).
y:y≤x
For any number x, F (x) is the probability that the observed value of X
will be at most x.
Proposition
For any two numbers a and b with a ≤ b,
P (a ≤ X ≤ b) = F (b) − F (a− ).
“a” represents the largest possible X value that is strictly less than a.
Notice that F (x) is a non-decreasing function.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
8 / 31
Discrete Random Variables
Probability Distributions
CDF Examples
Example 3.1.11 Roll a die. Let X = outcome of the die.
X = 1, 2, ...6. Find the cdf. What is the probability that 2 ≤ X ≤ 4?
x
p(x)
F (x)
1
2
3
4
5
6
1
6
1
6
1
6
1
3
1
6
1
2
1
6
2
3
1
6
5
6
1
6
−
P (2 ≤ X ≤ 4) = F (4) − F (2 ) =
1
2
3
−
1
6
= 12 .
Example 3.1.12 Toss a fair coin three times, let X = number of
heads, find the cdf, what is the probability to get at least 2 heads?
x
p(x)
F (x)
0
1
2
3
1
8
1
8
3
8
1
2
3
8
7
8
1
8
1
P (at least two H) = P (X ≥ 2) = 1 − P (X ≤ 1) = 1 − F (1) = 12 .
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
9 / 31
Discrete Random Variables
Probability Distributions
CDF Examples
Example 3.1.11 Roll a die. Let X = outcome of the die.
X = 1, 2, ...6. Find the cdf. What is the probability that 2 ≤ X ≤ 4?
x
p(x)
F (x)
1
2
3
4
5
6
1
6
1
6
1
6
1
3
1
6
1
2
1
6
2
3
1
6
5
6
1
6
−
P (2 ≤ X ≤ 4) = F (4) − F (2 ) =
1
2
3
−
1
6
= 12 .
Example 3.1.12 Toss a fair coin three times, let X = number of
heads, find the cdf, what is the probability to get at least 2 heads?
x
p(x)
F (x)
0
1
2
3
1
8
1
8
3
8
1
2
3
8
7
8
1
8
1
P (at least two H) = P (X ≥ 2) = 1 − P (X ≤ 1) = 1 − F (1) = 12 .
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
9 / 31
Discrete Random Variables
Probability Distributions
CDF Examples
Example
Here is a probability distribution for a random variable X:
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
10 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial Distribution
Definition (Binomial experiment)
1
A sequence of n trials.
2
Each trial is a dichotomous trial, i.e., has two results: success (S) or fail (F).
3
All trials are independent.
4
Probability of success is constant for all trials, and is denoted by p.
Definition (Binomial RV and Distribution)
Given a binomial experiment consisting of n trials, the binomial rv X associated
with this experiment is defined as:
X = the number of S’s among n trials
The probability dist associated with binomial rv X is the binomial distribution.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
11 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial Examples
Example 3.2.1 Toss a fair coin 4 times, let X = number of heads in
n tosses. Find the pmf.
4 1 4
p(0) = P (X = 0) = P (0H) =
0 2
4 1 4
p(1) = P (X = 1) = P (1H) =
1 2
... ...
4 1 4
p(4) = P (X = 4) = P (4H) =
4 2
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
12 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial Examples (continued.)
Example 3.2.2 Toss a fair coin n times, let X = number of heads in
n tosses. Find the pmf.
n 1 n
p(0) = P (X = 0) = P (0H) =
( )
0 2
n 1 n
p(1) = P (X = 1) = P (1H) =
( )
1 2
... ...
n 1 n
p(x) = P (X = x) = P (xH) =
( )
x 2
... ...
n 1 n
p(n) = P (X = n) = P (nH) =
( )
n 2
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
13 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial Examples (continued..)
Example 3.2.3 Toss an uneven coin, the probability that the coin is
head is p. Find the pmf.
n
p(0) = P (X = 0) = P (0H) =
(1 − p)n
0
n
p(1) = P (X = 1) = P (1H) =
p(1 − p)(n−1)
1
... ...
n x
p(x) = P (X = x) = P (xH) =
p (1 − p)(n−x)
x
... ...
n n
p(n) = P (X = n) = P (nH) =
p
n
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
14 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial PMF
Exercise
A card is drawn from a standard 52-card deck. If drawing a club is
considered a success, find the probability of
1
exactly one success in 4 draws (with replacement).
2
no successes in 5 draws (with replacement).
For any binomial experiment with n trials and each trial has a success
probability p, the binomial pmf is denoted by b(x; n, p), where n and p are
two parameters associated with the binomial dist. We have:
Walid Sharabati (Purdue University)
Theorem
Discrete Random Variable & Probability Dist.
Spring 2014
15 / 31
Discrete Random Variables
The Binomial Probability Distribution
Binomial Example (again)
Example 3.2.4 (exercise 3.51) 20% of all telephones of a certain type
are submitted for service while under warranty. Of these 60% can be
repaired, whereas the other 40% must be replaced with new units. If
a company purchases ten of these telephones, what is the probability
that exactly two will end up being replaced under warranty?
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
16 / 31
Discrete Random Variables
Geometric and Hypergeometric Distributions
Geometric and Hypergeometric Distribution
Definition (Geometric Distribution)
1
A sequence of trials
2
Each trial is dichotomous, with outcomes S or F and P (S) = p.
3
All trials are independent
4
Random variable X = the number of S before F appears is a Geometric rv,
or follows a Geometric dist.
5
p is the parameter.
Definition (Hyper Geometric Distribution)
1
N Dichotomous elements (S and F), M are S.
2
Draw n out of N elements without replacement.
3
Random variable X = number of S in n draws is a Hypergeometric rv, or
follows a Hypergeometric dist.
N, M, n are parameters.
Discrete Random Variable & Probability Dist.
4 Sharabati (Purdue University)
Walid
Spring 2014
17 / 31
Discrete Random Variables
Geometric and Hypergeometric Distributions
Geometric PMF
Example 3.3.1 Toss an uneven coin, the probability of getting a head is p, so the
probability of getting tail is 1 − p. Let X = the number of tails before getting a
head.
X has a geometric dist.
p(0) = P (X = 0) = P (H) = p
p(1) = P (X = 1) = P (T H) = (1 − p)p
......
p(x) = P (X = x) = P (x tails and 1H) = (1 − p)x p
Proposition
The pmf of a geometric rv X is given by:
p(x) = P (X = x) = P (xT and 1H) = (1 − p)x p
Where p is the probability of success.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
18 / 31
Discrete Random Variables
Geometric and Hypergeometric Distributions
Hypergeometric PMF
Example 3.3.2 A bag with 10 balls, 3 of them are black, now take out 6 balls from
the bag, X = number of black balls follows a hypergeometric dist. With N = 10,
M = 3, n = 6.
3 10−3
3 10−3
p(0) = P (X = 0) =
p(2) = P (X = 2) =
0
3
2
6 ;
10
6
10−3
4
;
10
6
p(1) = P (X = 1) =
p(3) = P (X = 3) =
1
3
3
5
10
6
10−3
3
10
6
If X is the number of S’s in a completely random sample of size n drawn from a
population consisting of M S’s and (N M ) F ’s, then the probability distribution of X is
called hypergeometric.
Proposition
The pmf of a hypergeometric rv X is given by:
p(x) = P (X = x) =
M
x
N −M
n−x
N
n
Where x is an integer satisfying max(0, n − N + M ) ≤ x ≤ min(n, M ), p(x) is denoted
by h(x; n, M, N ) in textbook.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
19 / 31
Discrete Random Variables
Poisson Distribution
Poisson Distribution
Definition
A random variable X is said to have a Poisson distribution with
parameter λ(λ > 0) if the pmf of X is:
p(x; λ) =
e−λ λx
x!
Where x = 0, 1, 2, ....
Proposition
Let λ > 0, limn→∞ b(x; n, pn ) = p(x; λ), if pn → 0 as n → ∞ and
npn → λ
i.e. Binomial approaches Poisson when n is large (→ ∞) and p is small
(→ 0). Thus we can use poisson to approximate binomial when n is large
and p is small.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
20 / 31
Discrete Random Variables
Poisson Distribution
Poisson Examples
Example 3.3.3 Let X = the number of calls a receptionist receives in an
hour, X follows a poisson dist with λ = 5. What is probability that the
receptionist receives at least one call in an hour?
P (at least one call) = 1 − P (no calls) = 1 − p(0) = 1 −
e−λ λ0
0!
λ = 5, so P (at least one call) = 1 − e−5
Example 3.3.4 0.2% feral cats are infected with the feline aids (FIV) in a
region. What is the probability that there are exactly 10 cats infected with
FIV among 1000 cats?
Let X = the number of cats with FIV among 1000 cats. X Binomial, with
n = 1000 and p = 0.002. So,
1000
P (10 FIV cats) =
0.00210 (1 − 0.002)(1000−10)
10
Complicated... Use poisson approximation: n = 1000, p = 0.002,
λ = np = 1000 × 0.002 = 2, so,
Walid Sharabati (Purdue University)
e−2 210
P
(10 Random
FIV cats)
=& p(10)
=Dist.
Discrete
Variable
Probability
Spring 2014
21 / 31
Discrete Random Variables
Uniform Distribution
Uniform Distribution
Definition
If an rv has any of n possible values k1 , ..., kn that are equally probable,
then X has a discrete uniform distribution with pmf:
p(ki ) =
1
, where i = 1, 2, ...n.
n
Example
Example 3.3.5 A very simple example is: Roll a fair die, and let X =
outcome of the die. pmf:
p(1) = p(2) = p(3) = p(4) = p(5) = p(6) =
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
1
6
Spring 2014
22 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Expectation of a Discrete Distribution
Definition
Let X be a discrete rv and let the set of all possible values of X be D and
pmf of X be p(x). The expectation or mean of X, denoted by E(X) or
µx is:
X
E(X) = µx =
x · p(x)
x∈D
Notice here that the expectation E(X) is the population mean µ when a
dist is given.
Example
Let X be outcome of a die, what is the expectation of X?
E(X) = 1 ×
1
6
Walid Sharabati (Purdue University)
+2×
1
6
+ ... + 6 ×
1
6
=
1 + 2 + ... + 6
= 3.5
6
Discrete Random Variable & Probability Dist.
Spring 2014
23 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Expectation of a Discrete Distribution
Exercise
Use the data below to find out the expected number of the number of
credit cards that a student will possess. x = # of credit cards.
x
P (X = x)
0
0.08
Walid Sharabati (Purdue University)
1
0.28
2
0.38
3
0.16
4
0.06
Discrete Random Variable & Probability Dist.
5
0.03
6
0.01
Spring 2014
24 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Properties of Expectation
1
For any given constant a and b, E(aX + b) = aE(X) + b
E(aX) = aE(X)
E(X + b) = E(X) + b
2
3
If the rv X has a set of possible values D and pmf p(x), then the
expectation P
of any function h(X), denoted by
E[h(X)] = D h(x) · p(x).
For random variables X1 , X2 , ..., Xn ,
E(a1 X1 + a2 X2 + ... + an Xn ) = a1 E(X1 ) + a2 E(X2 ) + ... + an E(Xn )
Example
Let X be the outcome of a die. What is the expectation of X 2 ?
X
X2
p(x)
E(X 2 ) = 1 ×
Walid Sharabati (Purdue University)
1
1
2
4
3
9
4
16
5
25
6
36
1
6
1
6
1
6
1
6
1
6
1
6
+4×
1
6
1
6
+ ... + 36 ×
Discrete Random Variable & Probability Dist.
1
6
≈ 15.17
Spring 2014
25 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Expectations of a List of Discrete Distributions
Binomial b(x; n, p):
E(X) = np
Geometric p(x; p):
E(X) =
1−p
p
Hyper Geometric h(x; n, M, N ):
E(X) = n ·
M
N
Poisson p(x; λ):
E(X) = λ
Uniform:
Pn
E(X) =
Walid Sharabati (Purdue University)
i=1 ki
n
Discrete Random Variable & Probability Dist.
Spring 2014
26 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Variance of a Discrete Distribution
Definition
Let X be an rv with pmf p(x) and expected value E(X) Then the
2 is:
variance of X, denoted by V ar(X) or σX
X
(x − E(X))2 · p(x) = E(X − E(X))2
V ar(X) =
D
The standard deviation σX is σX =
p
V ar(X)
Notice here that V ar(X) is the population variance σ 2 when a dist is
given.
A shortcut formula:
V ar(X) = E(X 2 ) − [E(X)]2
Example Still the die example. What is the variance of X = outcome
of a fair die?
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
27 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Variance of a Discrete Distribution
Exercises
1 5 cards are drawn, with replacement, from a standard 52-card deck. If
drawing a club is considered a success, find the mean, variance, and
standard deviation of X (where X is the number of successes).
2
If the probability of a student successfully passing this course (C or
better) is 0.82, find the probability that given 8 students
1
all 8 pass.
2
none pass.
3
at least 6 pass.
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
28 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Properties of Variance
1
Given two numbers a and b, V ar(aX + b) = a2 V ar(X)
V ar(aX) = a2 V ar(X)
V ar(X + b) = V ar(X)
2
V ar[h(X)] =
P
D {h(x)
− E[h(X)]}2 · p(x)
Example: Bernoulli rv I, p(0) = p, p(1) = 1 − p. What is
V ar(3I + 5)?
Example: Still the die example. What is the variance of 2X 2 ?
V ar(2X 2 ) = 4V ar(X 2 ) = 4 · E(X 4 ) − (E(X 2 ))2
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
29 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Variances of a List of Discrete Distributions
Binomial b(x; n, p):
V ar(X) = n · p · (1 − p)
Geometric p(x; p):
V ar(X) =
1−p
p2
Hyper Geometric h(x; n, M, N ):
N −n
M
M
V ar(X) =
·n·
· 1−
N −1
N
N
Poisson p(x; λ):
V ar(X) = λ
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
30 / 31
Discrete Random Variables
Expectation and Variance of a Discrete Distribution
Poisson Process
Poisson process is a very important application of Poisson distribution.
Definition (Poisson Process)
1
Given a rate α, i.e., expected number of ”occurrences” per unit time.
2
Then the X = number of occurrences during a time interval t follows a
Poisson distribution with λ = αt. The expected number of occurrences is
λ = αt.
Exercises
1
Every second there are 2 cosmic rays hit a specific spot on earth. What is
the probability that there are more than 20 cosmic rays hitting the spot
within 5 seconds?
2
2-D Poisson process. (problem 3.87) Trees are distributed in a forest
according to a 2-dimensional Poisson process with parameter α = the
Walid Sharabati (Purdue University)
Discrete Random Variable & Probability Dist.
Spring 2014
31 / 31