Download Document

Random variables
Some random experiments may yield a sample space whose elements
(events) are numbers, but some do not. For mathematical purposes, it is
desirable to have numbers associated with the outcomes.
A random variable X is a function that assigns a real number, X(ζ), to
each outcome ζ in the sample space of a random experiment.
The sample space S is the domain of the random variable and the set
SX of all values taken on by X is the range of the random variable.
Note that SX ⊂ R, R is set of all real numbers.
S
ζ
X(ζ
)
=x
real number line
SX
Example A random experiment of tossing 3 fair coins.
Sample space S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}.
Let X be the number of heads; then SX = {0, 1, 2, 3}.
eg.
1
3
3
1
X (THH ) = 2; P[ X = 0] = , P[ X = 1] = , P[ X = 2] = , P[ X = 3] = .
8
8
8
8
Equivalent events
Let A be the set of outcomes ζ in S that leads to the set of values X(ζ)
in B.
A
B
A = X −1 ( B ) = {ζ ∈ S : X (ζ ) ∈ B}
eg. in the above coins tossing example,
X−1({2, 3}) = {HHT, HTH, THH, HHH}
= set of all preimages of elements in B = {2, 3}.
Since event B in SX occurs whenever event A in S occurs, and vice
versa. Hence P[B] = P[A] = P[{ζ: X(ζ) in B}]. A and B are called
equivalent events with respect to X.
If we assign probabilities in this manner, then the probabilities assigned
to subsets of the real line will satisfy the three axioms of probability.
1. P[B] ≥ 0 for all B ⊂ SX.
2. P[SX] =1.
3. If B1 and B2 are mutually exclusive, then
P[B1 ∪ B2] = P [B1] + P[B2].
In the tossing coins experiment, we observe
1
1
7
P[ X ≤ 0] = , P[ X ≤ 1] = , P[ X ≤ 2] = , P[ X ≤ 3] = 1.
8
2
8
Hence, P[X ≤ x] is a number whose value depends on x, and so it is a
function of x.
Example
Consider the random experiment of tossing 3 coins
S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
X = no of heads in the 3 coins, SX = {0, 1, 2, 3}
A1 = {HTT, TTT}
A2 = {HHT, HTH, THH, TTT}
A3 = {HTT, THT, TTH, TTT}
X(A1) = {0, 1} = set of all values taken by X(ζ), ζ ∈ A1
X(A2) = {0, 2}
X−1({0, 1})
= set of all preimages of elements in {0, 1}
= {HTT, THT, TTH, TTT} = A3.
Note that A3 and {0, 1} are equivalent events since
1
= P[ A3 ] = P[ X = 0 or X = 1].
2
Note that A3 ⊂ S and {0, 1} ⊂ SX.
Consider another random variable:
Y = number of heads – number of tails
then Y can assume the values –3, −1, 1 and 3.
Now, Y−1({−3, −1}) = {TTT, HTT, THT, TTH}, so
{TTT, HTT, THT, TTH}, and {−3, −1} are equivalent events.
Example
A point is selected at random from inside the unit circle centered at the
origin. Let Y be the random variable representing the distance of the
point from the origin.
(a)
SY = {y: 0 ≤ y ≤ 1} = range of Y.
center ×
y
×
selected
point
(b) The equivalent event in the sample space S for the event {Y ≤ y} in
SY is that the selected point falls inside the region centered at the
origin and with radius y.
(c)
P[Y ≤ y]
(0 ≤ y ≤ 1)
= probability of selecting a point inside the unit circle, and whose
2
π
y
distance is less than or equal to y =
= y2.
π
Let Z be the random variable representing the distance of the selected
⎛1 ⎞
point from ⎜ ,0 ⎟.
⎝2 ⎠
3⎫
⎧
S
=
z
:
0
≤
z
≤
(a) Z ⎨
⎬
2⎭
⎩
(b) The equivalent event in S for the event
{Z ≤ z} is the region formed by the
intersection of the circles:
⎧x2 + y 2 ≤ 1
⎪
2
1⎞
⎨⎛
2
2
x
y
z
−
+
≤
⎜
⎟
⎪
2⎠
⎩⎝
y
z
×
⎛1 ⎞
⎜ ,0 ⎟
⎝2 ⎠
x
Cumulative distribution function (cdf)
The cdf of a random variable X is defined as
−∞ < x < ∞.
FX(x) = P[X ≤ x],
Axioms of probability ⇒ following properties of cdf
1. 0 ≤ FX ( x) ≤ 1
2.
FX ( x) = 1
lim
x →∞
(sure event)
3.
FX ( x) = 0
lim
x → −∞
(impossible event)
4.
FX (x) is a non-decreasing function of x
This is obvious since for x2 > x1, we have
P[ X ≤ x1 ] ≤ P[ X ≤ x2 ].
5. FX (x) is continuous from the right i.e. for h > 0
FX (b) = lim FX (b + h) = FX (b + )
h→0 +
Example The tossing coins experiment again, where X = number of
heads appearing in tossing 3 coins.
Take h > 0 and h → 0+,
FX (1 − h) = P[ X ≤ 1 − h] = P{0 head} =
1
FX (1) = P[ X ≤ 1] = P{0 or 1 head} =
2
1
FX (1 + h) = P[ X ≤ 1 + h] = .
2
1
8
FX(x)
Hence, the cdf of X is continuous
from the right.
1
7
8
Define the unit step function:
⎧0 x < 0
u ( x) = ⎨
⎩1 x ≥ 0,
1
2
1
8
x
1
2
3
1
3
3
1
FX ( x) = u ( x) + u ( x − 1) + u ( x − 2) + u ( x − 3).
8
8
8
8
The jump at x = 0 is given by P[X = 0], and similarly, for the jump at
x = 1, 2 and 3.
6. P[a < X ≤ b] = FX(b) – FX(a)
since {X ≤ a} ∪ {a < X ≤ b} = {X ≤ b},
and {X ≤ a} and {a < X ≤ b} are mutually exclusive
so FX(a) + P[a < X ≤ b] = FX(b).
Suppose we take a = b − h , h > 0,
P[b − h < X ≤ b] = FX(b) – FX(b − h).
As h → 0+, P[X = b] = FX(b) – FX(b−).
The probability that X takes on the special value b is given by the
magnitude of the jump of the cdf FX(x) at b.
h If the cdf is continuous at b, then the event {X = b}
has probability zero (essentially).
h If the cdf is continuous at x = a and x = b, then P[a < X < b],
P[a ≤ X < b], P[a < X ≤ b], P[a ≤ X ≤ b] have the same value.
7. P[X > x] = 1 – FX(x).
Example Let T be the random variable which equals the life of a diode.
Suppose the cdf of T takes the form
FT (t ) = P[T ≤ t ]
⎧ 0
=⎨
-µt
⎩1 - e
t<0
t≥0
= u (t )(1 − e − µt ),
then the probability that the diode fails between times a and b is
P[a < T ≤ b] = P[T ≤ b] − P[T ≤ a] = e−µa − e−µb.
FT(t)
1
t
Three types of random variables
1. Discrete random variable
The cdf is a right-continuous, staircase function of x with jumps at
a countable set of points x1, x2, …
FX ( x) = ∑ PX ( xk )u ( x − xk )
k
where PX(xk) = P[X = xk] gives the magnitude of the jump at X = xk
in the cdf.
2. Continuous random variable
The cdf FX(x) is continuous everywhere, so
P[X = x] = 0 for all x.
3. Random variable of mixed type
The cdf has jumps on a countable set of points and also increases
continuously over at least one interval of values of x
FX(x) = p F1(x) + (1 – p) F2(x),
cdf of a
discrete random
variable
cdf of a
continuous random
variable
0 < p < 1.
Example Let X be the time instant that a customer in a queue is being
served. We have: X is zero if the system is idle and exponentially
distributed if the system is busy.
P[ X ≤ x] = P[ X ≤ x | idle]P[idle] + P[ X ≤ x | busy]P[busy ]
p = probability that the system is idle.F (x)
x
X = pX idle + (1 − p ) X busy .
1
x<0
⎧0
FX ( x) = ⎨
− λx
p
(
1
p
)(
1
e
) x≥0
+
−
−
⎩
= pu ( x) + (1 − p )u ( x)(1 − e −λx )
p
0
x
Xidle is a discrete random variable with P[Xidle = 0] = 1 so that
FXidle (x) = u(x); Xbusy is continuous with FXbusy (x) = u(x) (1- e−λx).
fX(x)
x
x + ∆x
P[x < X ≤ x + ∆x] ≈ fX(x) ∆x
Probability density function
dF ( x)
pdf, if exists, is defined as f X ( x) = X
dx
since
P[ x < X ≤ x + ∆x]
= FX ( x + ∆x) − FX ( x) =
FX ( x + ∆x) − FX ( x)
∆x.
∆x
x
Properties of pdf
1. fX(x) ≥ 0 since cdf is a non-decreasing function of x
x
2. Fx ( x) = ∫−∞ f X (t ) dt
d
FX ( x), we obtain
Proof: From f X ( x) =
dx
x
FX ( x) = ∫ f X (t ) dt.
c
The constant c is determined by FX(−∞) = 0, given c = −∞.
b
3. P[a < X ≤ b] = ∫ f X (t ) dt
a
Proof:
∞
P[a < X ≤ b] = FX (b) − FX (a )
4. 1 = ∫−∞ f X (t ) dt
b
a
b
−∞
−∞
a
= ∫ f X (t ) dt − ∫ f X (t ) dt = ∫ f X (t ) dt
Example Let radius of bull-eye = b and radius of target = a.
Probability of the dart striking between r and r + dr is
⎡ ⎛ r ⎞2 ⎤
P[r ≤ R ≤ r + dr ] = C ⎢1 − ⎜ ⎟ ⎥ dr.
⎣⎢ ⎝ a ⎠ ⎦⎥
R = distance of hit from the center of the target.
a
The density function takes the form
⎡ ⎛ r ⎞2 ⎤
f R (r ) = C ⎢1 − ⎜ ⎟ ⎥.
⎢⎣ ⎝ a ⎠ ⎥⎦
b
How to determine C?
Assume that the target is always hit:
2
3
⎛r⎞
,
C ∫ 1 − ⎜ ⎟ dr = 1 ⇒ C =
0
2a
⎝a⎠
a
probability of hitting bull-eye = P[0 ≤ R ≤ b] = ∫
b
0
b(3a 2 − b 2 )
.
f R (r ) dr =
3
2a
pdf for a discrete random variable
The delta function δ(x) is related to u(x) via
x
∞
d
δ ( x) = u ( x) or u ( x) = δ (t ) dt. Note that ∫−∞ δ (t )dt = 1.
−∞
dx
Recall that
∫
FX ( x) = ∑ PX ( xk )u ( x − xk )
k
probability mass function
x
FX ( x) = ∫ f x (t ) dt , we then have
−∞
According to
∞ when x = xk
⎧
f X ( x) = ∑ PX ( xk )δ ( x − xk ), where δ ( x − xk ) = ⎨
.
k
⎩0 otherwise
δ(x – xk)
u(x – xk)
∞
1
xk
x
xk
x
Example The coins tossing experiment
cdf:
1
3
3
1
FX ( x) = u ( x) + u ( x − 1) + u ( x − 2) + u ( x − 3).
8
8
8
8
pdf:
1
3
3
1
f X ( x) = δ ( x) + δ ( x − 1) + δ ( x − 2) + δ ( x − 3).
8
8
8
8
P[1 < X ≤ 2] =
∫
2
1+
3
f X ( x) dx = P[ X = 2] = .
8
Note that the delta function located at 1 is excluded but the delta
function located at 2 is included.
Similarly,
P[2 ≤ X < 3] =
∫
3−
3
f X ( x) dx = P[ X = 2] = .
2
8
Conditional cdf of X given A
FX ( x | A) =
P[{ X ≤ x} ∩ A]
if P[ A] > 0.
P[ A]
h cdf of X with reference to the reduced sample space A
Conditional pdf of X given A
d
f X ( x | A) =
FX ( x | A).
dx
S
{X ≤ x}
A
Example
The lifetime X of a machine has a continuous cdf, FX(x). Find the
conditional cdf and pdf given the event A = {X > t}, that is, the machine
is still working at time t.
Conditional cdf
FX ( x | X > t ) = P[ X ≤ x | X > t ]
=
(i) x ≤ t
{X > t}
P[{ X ≤ x} ∩ { X > t}]
.
P[ X > t ]
(ii) x > t
{X ≤ x}
{X > t}
{X ≤ x}
Note that
x≤t
⎧φ
{ X ≤ x} ∩ { X > t} = ⎨
,
⎩{t < X ≤ x} x > t
so
0
⎧
⎪
FX ( x | X > t ) = ⎨ FX ( x) − FX (t )
⎪⎩ 1 − FX (t )
x≤t
x > t.
Conditional pdf is found by differentiating FX ( x | X > t ) with respect to x
0
⎧
⎪ f ( x)
f X (x | X > t) = ⎨ X
⎪⎩1 − FX (t )
x≤t
x>t.
Note that FX(x | X>t) is continuous at x = t, but fX(x | X>t) has a jump
at x = t.
Example Tossing of 3 coins; X = number of heads; A = {X > 2}.
FX(x|A)
FX ( x | A) = FX ( x | X > 2)
=
P[{ X ≤ x} ∩ { X > 2}]
P[ X > 2]
if x ≤ 2
⎧⎪0
= ⎨ P[2 < X ≤ x] if x > 2
⎪⎩ 1 − P[ X ≤ 2]
⎧0 if x < 3
.
=⎨
⎩1 if x ≥ 3
1
1
Note that P[X > 2] = P[X = 3] = and
8
⎧⎪ 0 if x < 3
P[2 < X ≤ x] = ⎨ 1 if x ≥ 3.
⎪⎩ 8
x
2
3