Download EC381/MN308 Probability and Some Statistics Lecture 5

EC381/MN308
Probability and Some Statistics
Lecture 5 - Outline
1. Random Variables
Yannis Paschalidis
2. Discrete Random Variables.
[email protected], http://ionia.bu.edu/
Dept. of Manufacturing Engineering
Dept. of Electrical and Computer Engineering
Center for Information and Systems Engineering
1
EC381/MN308 – 2007/2008
Chapter 2 Discrete Random Variables
Random Variable (RV)
An assignment of a value (real number) to each & every outcome in a
probability space
Mathematically: A random variable X is a function from the sample space SX to
the space of real numbers R
2
EC381/MN308 – 2007/2008
Types of Random Variables
• Discrete random variable
Possible values (range) of X is a countable set {x1, x2, …}
• Finite random variable
Range of X is a finite set {x1, x2, …, xk}
Sample
space
SX
• Continuous random variable
s2
sk
Range of X is neither countable nor finite.
s1
k outcomes
x1 x2
xk
R
Discrete Random Variable X
Notations:
The random variable is denoted by capital letter: X
& the values it takes are denoted by lower-case italic: x
Odd concept in Book. Improper Experiment: Random variable not defined
in all of outcome space; some outcomes lead to undefined random variables
Common definition: random variable defined for all outcomes.
3
EC381/MN308 – 2007/2008
2.1 Probability Mass Function (PMF)
Examples of Discrete Random Variables
A random
- Number of Geiger-counter counts n in 1 sec (n = 0,1,2,..)
- Number of photons
(n = 0,1,2,..)
4
EC381/MN308 – 2007/2008
variable X that takes values {x1, x2, …, xk}
associated with outcomes {s1, s2, …, sk}
is characterized by the probabilities
{P [X=x1] = P [s1], P [X=x2] = P [s2], …, P [X=xk] = P [sk] }
n collected by a photodetector in 1 μsec
These probabilities are also written as PX(x) or simply P [x]
- Fabricated semiconductor chip is tested for faults
faults. Random
variable X assigns 0 if chip is faulty, 1 if OK. X is discrete and
finite.
1
Discrete
Random Variable
X
Probability Mass Function PX(x)
P [x2]
P [x1]
P [xk]
0
EC381/MN308 – 2007/2008
5
x1
x2
Note: We can use PMFs directly to compute probabilities without going back to original
outcomes in probability space…
EC381/MN308 – 2007/2008
xk
6
1
Properties of PMF
Example 1
ƒ PX(xi) ≥ 0, i = 1, 2, … k
X = number of coin tosses until first head
– assume independent tosses, P [H] = p > 0
– PX (k) = P [X = k] = P [TT ··· TH]
– PX (k) = (1 − p)k−1p, k = 1, 2,...
ƒ ∑i PX(xi) = 1
Sample
space
sk
B
S
B
s2
s1
x1 x2
xk
Event: {s: X(s) ∈ B}
SX
E
Example
l 2
x∈B
B
Mathematical Statement of Properties
– For any x in SX, PX(x) ≥ 0
– ∑x in Sx PX(x) = 1
– For any set B ⊂ SX, the probability that X is in B is
P [{s ∈ S: X(s) ∈ B}] ≡ P [B] = ∑x in B PX(x)
Random variable N has PMF PN(n) = c/n, n = 1, 2, 3, 4,
or 0 otherwise
–Find value of c:
–Find P [N ≥ 2]:
Probability in original experiment is mapped into probability on subsets of the real line!
–Find P [N < 3]:
These properties follows from the underlying definition of probability space
7
EC381/MN308 – 2007/2008
Properties
The CDF of a random variable X is the total probability mass
from –∞ up to (and including) the point x.
FX(x) = P [X ≤ x] = P [X ∈ (- ∞, x)], –∞ < x < ∞
PX(x)
P [x1]
x1
•
The CDF is a non-negative real-valued function FX(x) ∈ [0,1] defined for all
real values of its argument x
•
The CDF of any discrete random variable is a staircase function. If X takes
on values x1, x2, … xk, with probabilities P [x1], P [x2], … , P [xk], then the
CDF has jumps at x1, x2, … xk, with heights P [x1], P [x2], … , P [xk] and is
flat in between the jumps
P [x2]
P [xk]
x2
xk
x
P [xk]
1
CDF
8
EC381/MN308 – 2007/2008
Cumulative Distribution Function (CDF)
PMF
c(1 + 1/2 + 1/3 + 1/4) = 1 Æ c = 12/25
c(1/2 + 1/3 + 1/4)
c(1 + 1/2)
FX(x)
CDF is not a very useful for Discrete RVs
0
P [x2]
P [x1]
EC381/MN308 – 2007/2008
x1
x2
PMF is easier to use, has all information needed. So, why bother?
xk
x
Because concept extends to ALL random variables, continuous, discrete, etc.
9
2.1 Statistics of Random Variables
10
EC381/MN308 – 2007/2008
Random Variable Statistics
Sample Statistics
A statistic based on a collection of sample values of a random variable is
a single real number such as the average value, the median, or the
standard deviation of the sample values.
Example
A statistic of a RV maps the RV into a real-valued quantity (computed
from the PMF).
Expected Value
The expected value (“average” or “mean”) of a random variable X is
Scores in midterm
X = score of a randomly picked student (uniformly)
PX(x) = fraction of students with grade x
also denoted
mX = E [X]
Interpretations:
– It is the weighted average of all possible values, using the PMF weights
– Center of gravity of PMF
– Average in large number of repetitions of the experiment (to be
substantiated later in this course)
EC381/MN308 – 2007/2008
11
EC381/MN308 – 2007/2008
12
2
Variance and Standard Deviation
Other Simple Statistics of Random Variable X
Median = any number xmed such that
Standard deviation
P [X < xmed] = P [X > xmed]
Moments
May not exist
May not be unique
Central Moments
Variance = Second central moment = Second moment - Mean2
Mode = any number xmod such that
Proof
P [X = xmod] ≥ P [X = x] for all x ∈ SX
May not be unique, but must exist
13
EC381/MN308 – 2007/2008
2.2 Families of Discrete Random Variables
14
EC381/MN308 – 2007/2008
Sequences and Series (p. 508)
1) Uniform
2) Bernoulli
3) Geometric
4) Binomial
5) Poisson
These families of discrete RVs describe common experiments.
Members of each family differ only in the value of some experimental
parameters.
15
EC381/MN308 – 2007/2008
1) Uniform Random Variables
Range:
{0, 1, 2, …, n}
PMF:
PX(x) = 1 / (n + 1)
Mean:
n
Variance:
n(n + 2) / 12
/2
16
EC381/MN308 – 2007/2008
More General Uniform Random Variables
PMF
parameter n
0 1
n
x
Range:
{k, k+1, …, n}
PMF:
PX(x) = 1/(n – k + 1)
Mean:
( n + k) / 2
PMF
parameters k, n
k k+1
n
x
Variance: (n-k)(n-k+2)/12 (same for Uniform {0,…,n-k})
Application: Used to model random experiments where all outcomes are
equally likely, e.g., fair coin tosses, die rolls, roulette wheels, etc
Proof
Proof
EC381/MN308 – 2007/2008
17
EC381/MN308 – 2007/2008
18
3
3) Geometric Random Variables
2) Bernoulli Random Variables
Range:
{0, 1} (={failure, success})
PMF:
PX(1) = p, PX(0) = 1 − p, parameter p ∈ (0,1)
Binary RV
mean:
p
Variance:
p (1 − p)
1−p
Application: Failure models, Bit errors
0
PMF
p
1
{1, 2, … }
PMF:
PX(x) = p(1 − p)x-1
Mean=
1/p
parameter p ∈ (0,1)
Variance= (1 − p) / p2
x
p
PMF
x
1 2 3
Application:
– Waiting time to first “success” for repeated independent Bernoulli
experiments
– Each bit in a communications channel has probability of error p; PMF
of first bit that is in error is geometric
– Similar examples using sequential testing of potentially faulty
components
Proof
EC381/MN308 – 2007/2008
Range:
19
20
EC381/MN308 – 2007/2008
4) Binomial Random Variable
Proof
Range:
{0, 1, .., n}
PMF:
Mean:
parameters
p ∈ (0,1) and n
np
Variance: np(1 − p)
Application:
pp
Number of failures in n independent
p
binary
y experiments
p
Proof
EC381/MN308 – 2007/2008
21
5) Poisson Random Variables
Example
Range:
{0, 1, 2, … }
n: number of customers
p: prob. customer requires service
s: no. of service persons
PMF:
PX ( x ) =
X: no. of service requests (RV)
Variance: α
• Service facility design (e.g. tellers at a bank)
–
–
–
–
22
EC381/MN308 – 2007/2008
Mean:
αx
x!
e −α ,
x = 0,1,L
parameter α
α
0 1 2
PMF
x
• What is p
probability
y that customer has to wait?
Applications:
Models the random number of events X (“arrivals”) arriving within a
fixed time window (e.g., photons or packets arriving at a
communications switch or computer node).
Arrival rate: λ per time unit
Time interval over which arrivals occur: T
EC381/MN308 – 2007/2008
23
EC381/MN308 – 2007/2008
)
α = λT
24
4
Proof
Relation between Bernoulli, Binomial and Poisson RVs
A Poisson RV can be viewed as the limit of a sum of
Bernoulli trials, i.e., as the limit of a binomial RV
– n Bernoulli trials, each with probability of success α/n
– Kn is the number of successes in n trials
– As n Æ ∞, PKn(k) converges to the PMF of a Poisson random
variable with parameter α
– Poisson RV = sum of an infinite number of infinitesimal Bernoulli
RVs
25
EC381/MN308 – 2007/2008
26
EC381/MN308 – 2007/2008
Median and Mode of Special RVs
Proof
• Bernoulli RV, parameter 0.4
– No median, mode = 0
• Geometric RV, parameter 0.5
– Median is any number between 1 and 2
2, mode = 1
• Binomial distribution, parameters 2, 0.5:
– Median = mode = 1
27
EC381/MN308 – 2007/2008
28
EC381/MN308 – 2007/2008
Summary of Special Discrete Random Variables
PMF
Bernoulli
PX(1) = p
PX(0) = 1 - p
1-p
0
p
Binomial
p → 0, n → ∞, and np →α
1
n
2
0
1
α
. . . ∞x
2
p
EC381/MN308 – 2007/2008
p (1 − p)
PX ( x ) = p (1 − p )
2
3
... ∞
α
Discrete RVs in MATLAB Statistics Toolbox
Distribution
PMF
CDF
Sample
Discrete
uniform 1:N
unidpdf(X,N)
unidcdf(X,N)
unidrnd(N)
Bernoulli p
binopdf(X,1,p)
binocdf(X,1,p)
binornd(1,p)
Binomial n, p
binopdf(X n,p)
binopdf(X,
binocdf(X n,p)
binocdf(X,
binornd(n,p)
Geometric p
geopdf(X,p)
geocfd(X,p)
geornd(p)
Poisson α
poisspdf(X,α)
poisscdf(X,α)
poissrnd(α)
x −1
1/p
1
np (1 − p)
x
Geometric
Number Bernoulli trials needed to achieve
one success
p
np
0
Poisson
Number of successes in n Bernoulli trials as
Variance
x
1
Number of successes in binary experiment
p = probability of success
Number of successes in n Bernoulli trials
Mean
(1− p) / p2
x
29
EC381/MN308 – 2007/2008
30
5
Example
MATLAB m-files on Web Site
CDF of a Binomial RV with parameters n = 5, p = 0.53
X = 5*linspace(-0.1,1.1);
Y = binocdf(X,5, 0.53);
stairs(X,Y)
These and some others …
Table 2.1 (p. 88)
MATLAB Functions
EC381/MN308 – 2007/2008
31
EC381/MN308 – 2007/2008
32
6