Download TMS-062: Lecture 2. Part 2. Discrete Random Variables

Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Random variables
TMS-062: Lecture 2. Part 2.
Discrete Random Variables
A Random variable is a numerical X value associated with
every outcome ω of a random experiment, so it is just a function
X : Ω 7→ R on the sample space Ω.
Sergei Zuyev
Sergei Zuyev
The Distribution PX ( · ) describes how probable different ranges
are: PX (A) = P{X ∈ A}, A ⊂ R.
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete random variables
X is a discrete R.V. if it takes values on a discrete (finite or
countable) set, i. e. its values can (in principal) be listed:
{x1 , . . . , xn , . . . }.
The distribution of X can be easily expressed vis p.m.f.:
Write pi = P(X = xi ) = P{ω : X (ω) = xi }, i = 1, 2, . . . . The
function fX (xi ) which associates the probabilities pi with the
values xi is called the probability mass function (or p.m.f.). It is
often represented as a table:
P(X = xi ) = xi } = pi
P(xm ≤ X ≤ xn ) =
n
X
pi .
i=m
In general, for any set B ⊆ R, P(X ∈ B) = sum of all pi ’s such
that xi ∈ B.
X
fX
x1
p1
x2
p2
Sergei Zuyev
...
...
(total)
1.00
(1)
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Example
Distribution Function
Let X be the number of Tails in a toss of two distinquishable
symmetrical coins. The Probability space
Ω
X (ω)
P
HH
0
1/4
HT
1
1/4
TH
1
1/4
TT
2
1/4
A general way to describe the distribution of a numerical R.V.’s
is via the Cumulative Distribution Function (c.d.f) which is
FX (t) = P(X ≤ t).
In the ’Tails’ example above:
1.00
1/4
X
PX
0
1/4
1
1/2
2
1/4
1/2
1/4
1.00
0
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
1
FX (t )
p2
Sergei Zuyev
2
t
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Indeed, if t < x1 , then P(X ≤ t) = 0, as x1 is the smallest
possible value. If x1 ≤ t < x2 then P(X ≤ t) = P(X = x1 ) = p1
for all such t. When t = x2 we have
P(X ≤ x2 ) = P(X = x1 ) + P(X = x2 ) = p1 + p2 , so FX (t) has a
jump of size p2 there, etc.
X
FX (t) = P(X ≤ t) =
pi and
xi ≤t
p1
x1
1
Discrete Random Variables
Expectation
Important Distributions
Generally, the c.d.f. of a discrete r.v. X with p.m.f. (1) is a step
function with jumps pi at xi :
0
FX (t)
3/4
Hence we have the following p.m.f.:
t
x2
Sergei Zuyev
P(X = xk ) = FX (xk ) − FX (xk −)
x3
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Properties of the c.d.f.
Expected value of a R.V.
0 ≤ FX (t) ≤ 1;
FX (t) is non-decreasing and FX (−∞) = 0, FX (+∞) = 1;
FX (t) is continuous from the right: FX (t+) = FX (t);
The expected value or expectation or the average or the
mean value of a discrete R.V. X is
X
EX =
xk P{X = xk }
k
P(a < X ≤ b) = FX (b) − FX (a)
P(a ≤ X ≤ b) = FX (b) − FX (a−)
P(a < X < b) = FX (b−) − FX (a)
P(a ≤ X < b) = FX (b−) − FX (a−) .
Sergei Zuyev
One needs to distinguish a theoretical mean µ = E X and
its sample counterpart X – the sample mean. In general, X
for population X = {X1 , X2 , . . . , XN } is an estimate of the
random variable X ’s mean µ which may be unknown.
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Function of a random variable
Take a function g : R 7→ R. Then g(X ) is also a R.V. taking
value g(X (ω)) when an elementary outcome ω is
observed. Since X is discrete, the range of Y = g(X ) is
also discrete: (y1 , y2 , . . . ) and the p.m.f. of Y is then
X
P{Y = yj } =
pi .
i: g(xi )=yj
One then has
E g(X ) =
TMS-062: Lecture 2. Part 2. Discrete Random Variables
X
g(xk )P(X = xk )
Example:
Consider g(x) = x 2 and the following R.V. X :
X
PX
-1
1/3
0
1/3
1
1/3
Since P{X 2 = 1} = P{X = −1} + P{X = 1},
Y = X2
PY
0
1/3
E X = −1·1/3+0·1/3+1·1/3 = 0;
1
2/3
E X 2 = 0·1/3+1·2/3 = 2/3.
k
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Variance
Variance of a R.V. X is a theoretical analogue of the sample
variance – the average square deviation from the mean:
Example: Exam results: 2 students got 5, 2 – 4, 5 – 3 and 1 – 2
(failed). A student is picked at random and his class X is
written. Find distribution of X and its first 2 moments (E X and
var X ).
Probabilistic model:
X
P
σ 2 = var X = E(X − E X )2 = E X 2 − (E X )2
The standard deviation is σ =
Sergei Zuyev
√
var X .
4
0.2
3
0.5
2
0.1
E X = 5 · 0.2 + 4 · 0.2 + 3 · 0.5 + 2 · 0.1 = 3.5 (compare with
X̄ = 1 · 2/10 + 2 · 2/10 + . . . = E X here – the way we have
chosen the model!)
E X 2 = 52 · 0.2 + 42 · 0.2 + 32 · 0.5 + 22√· 0.1 = 13.1, thus
var X = 13.1 − (3.5)2 = 0.85 and σ = 0.85 ≈ 0.92.
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
5
0.2
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Properties of E and var
Independence
Suppose X , Y are R.V.’s and a, b, c are constants.
Two R.V.’s X , Y are independent if for any A, B ⊆ R one has
P(X ∈ A, Y ∈ B) = P(X ∈ A) · P(Y ∈ B)
E(aX ) = a E X
E(b) = b
E(X + Y ) = E X + E Y
Example: Tossing two symmetric distinguishable coins. Xi –
what shows i-th coin. Then X1 , X2 are independent. Indeed
E(XY ) = E X · E Y
if X , Y are independent
P(X1 = H, X2 = H) = P(HH) = 1/4
= 1/2 · 1/2 = P(X1 = H) · P(X2 = H)
var(aX ) = a2 var X
var(b) = 0
Similarly, for other combinations of H and T .
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
var(X + Y ) = var X + var Y
Sergei Zuyev
if X , Y are independent
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Discrete uniform distribution
Let S = {x1 , . . . , xn } be a finite set. A R.V. X is uniformly
distributed over S (notation: X ∼ Unif(S)) if
As consequence
E(aX + bY + c) = a E X + b E Y + c
and if X , Y are, in addition, independent, then
var(aX + bY + c) = a2 var X + b2 var Y .
X
P
x1
1/n
x2
1/n
...
...
xn
1/n
For example, if S = {1, . . . , n} then
E X = 1/n
n
X
i=
i=1
n+1
,
2
var X = left as a Matlab exercise
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Binomial distribution
A trial has two possible outcomes: a ‘Success’ (S) or a ‘Failure’
(F ) with probs. p = P(S) = 1 − P(F ).
The R.V. X – the No. of successes in n independent such trials
has Binomial Bin(n, p) distribution:
n k
P(X = k) =
p (1 − p)n−k , k = 0, . . . , n
k
Indeed, on {X = k} an outcome has form (FFSFS . . . FS) with
exactly k S’s and n − k F ’s. Each such outcome
has prob.
n
k
n−k
p (1 − p)
(independence!) and there are k such outcomes
since there are that many ways to distribute k S’s over n places.
E X = np
and
var X = np(1 − p)
(2)
Let ξi be R.V. equal 1 if there is a success in the i-th trial and 0
otherwise. We have E ξi = 1 · p + 0 · (1 − p) = p = EP
ξi2 and
2
2
2
var ξi = E ξi − (E ξi ) = p − p = p(1 − p). But X = ni=1 ξi
and ξi are independent, thus we have (2) by properties 3 and 7
above.
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Discrete Random Variables
Expectation
Important Distributions
Example: Suppose that 20% of a large batch of resistors are
defective and 5 are chosen at random. What are probs. that
there are (i) exactly 4 are good; (ii) No more than 4 are good?
(iii) What are the mean and the st.dev. of the number of good
resistors in the sample?
5
(i) P(X = 4) =
0.84 0.21 = 0.4096
4
(ii) P(X ≤ 4) = 1 − P(X > 4) = 1 − P(X = 5)
Note the Binomial distr. is appropriate here because the
sample is small and the batch is large: the proportion of
defective transistors still may be assumed to stay at 20%.
In general, when sampling without replacement from a
not-so large finite population, we use the Hypergeometric
distribution (not covered in this course).
= 1 − 0.85 = 0.67232
(iii) E X = 0.8 · 5 = 4, var X = 5 · 0.2 · 0.8 = 0.8
hence σ =
√
0.8 ≈ 0.894
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Discrete Random Variables
Expectation
Important Distributions
Poisson Distribution (optional)
Consider a fixed finite length (time) interval divided into a very
large number n of very short slots. During each such slot an
event may either occur with small prob. pn or not. Slots are so
short that no more than 1 occurrence is virtually possible. Then
the No. X of occurrences of the event follows Binomial
Bin(n, pn ) distribution.
Let now n → ∞ and pn → 0, but so that npn → λ > 0 (e. g. time
slots have length 1/n and pn = λ/n). I can be shown that in the
limit
λk −λ
P(X = k ) =
e , k = 0, 1, . . .
k!
and we say that X follows Poisson distribution Po(λ).
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables
Very useful property is that E X = λ = var X .
Sergei Zuyev
TMS-062: Lecture 2. Part 2. Discrete Random Variables