Download Random Vectors

RANDOM VECTOR
Tutorial 6, STAT1301 Fall 2010, 02NOV2010, MB103@HKU
By Joseph Dong
RECALL: CARTESIAN PRODUCT OF SETS
Two discrete sets
Two Continuous sets
2
RECALL: SAMPLE SPACE OF A RANDOM VARIABLE
3
THE MAKING OF A RANDOM VECTOR AS JOINT
RANDOM VARIABLES: 𝑋1 , ⋯ , 𝑋𝑛
A CRASH COURSE OF LATIN NUMBER PREFIXES
•
Uni-variate : 1 random variable
•
Bi-variate : 2 random variables bind together to become a 2-tuple random vector like
𝒳 = 𝑋1 , 𝑋2
•
Tri-variate : 3 random variables bind together to become a 3-tuple random vector like
𝒳 = 𝑋1 , 𝑋2 , 𝑋3
•
……
•
n-variate : n random variables bind together to become a 3-tuple random vector like
𝒳 = 𝑋1 , ⋯ , 𝑋𝑛
•
You can even have infinite-dimensional random vectors! Unimaginable!
Prefix
Uni-
Bi-
Tri-
Quadri-
Quinti-
Num.
1
2
3
4
5
Sexa- Septi6
7
Octo-
Novem-
Deca-
8
9
10
4
RANDOM VECTOR AS A FUNCTION ITSELF:
𝑓: 𝛺1 × ⋯ × 𝛺𝑛 ∋ 𝜔1 , ⋯ , 𝜔𝑛 ↦ 𝑥1 , ⋯ , 𝑥𝑛 ∈ ℝ𝑛
•
How to distribute total probability mass
1 on the sample space of the random
vector?
•
Is this process completely fixed?
•
If not fixed, is this process completely
arbitrary?
•
If neither arbitrary, what are the rules
for distributing total probability mass 1
onto this state space?
•
“Marginal PDF/PMF” imposes an
additive restriction.
• There is a lot to discover here…
5
INDEPENDENCE AMONG RANDOM VARIABLES
•
Recall: What are independence among events?
• ℙ 𝐴𝐵 = ℙ 𝐴 ⋅ ℙ 𝐵
•
Q: What does a random variable do to its state space?
• It partitions the state space by the atoms in the sample space!
•
𝑥 is an atom in the sample space and 𝑋 −1 𝑥
•
𝑥1 , 𝑥2 is a union of atoms in the sample space and 𝑋 −1 𝑥1 , 𝑥2
in the state space.
•
We can talk about whether 𝑋 = 𝑥 and 𝑌 = 𝑦 are independent
• because they mean two events: 𝑋 −1 𝑥
•
is a block in the state space.
is a union of blocks
and 𝑌 −1 𝑦
We can talk about whether 𝑋 ∈ 𝐴 and 𝑌 ∈ 𝐵 are independent
• because they mean two events: 𝑋 −1 𝐴 and 𝑌 −1 𝐵
•
Goal: Generalize this connection to the most extent: Establish the meaning of
independence between whole random variables 𝑋 and 𝑌.
6
TWO RANDOM VARIABLES ARE INDEPENDENT
IF…
•
Each event in the state space of 𝑋 is
independent from each event in the state space
of 𝑌.
ℙ 𝑋 ∈ 𝐴, 𝑌 ∈ 𝐵 = ℙ 𝑋 ∈ 𝐴 ℙ 𝑌 ∈ 𝐵 ,
∀𝐴 ⊂ 𝑋 Ω1 , ∀𝐵 ⊂ 𝑌 Ω2
•
Further, this is true if each atom in the state
space of 𝑋 is independent from each atom in
the state space of 𝑌.
ℙ 𝑋 = 𝑎, 𝑌 = 𝑏 = ℙ 𝑋 = 𝑎 ℙ 𝑌 = 𝑏 ,
∀𝑎 ∈ 𝑋 Ω1 , ∀𝑏 ∈ 𝑌 Ω2
•
How many terms are there if you expand
𝑎2 + 𝑎3 + 𝑎3 𝑏3 + 𝑏4 + 𝑏5 ?
•
One more equivalent condition:
ℙ 𝑋 ≤ 𝑎, 𝑌 ≤ 𝑏 = ℙ 𝑋 ≤ 𝑎 ℙ 𝑌 ≤ 𝑏 ,
∀𝑎 ∈ 𝑋 Ω1 , ∀𝑏 ∈ 𝑌 Ω2
7
INDEPENDENCE OF CONTINUOUS RANDOM
VARIABLES
•
Previous picture deals with the discrete random variables case.
•
Two continuous random variables 𝑋 and 𝑌 are independent if
• ℙ 𝑋 ∈ 𝐴, 𝑌 ∈ 𝐵 = ℙ 𝑋 ∈ 𝐴 ℙ 𝑌 ∈ 𝐵 , ∀𝐴 ⊂ 𝑋 Ω1 , ∀𝐵 ⊂ 𝑌 Ω2 or/and
• 𝑓 𝑋,𝑌 𝑎, 𝑏 = 𝑓𝑋 𝑎 𝑓𝑌 𝑏 , ∀𝑎 ∈ 𝑋 Ω1 , ∀𝑏 ∈ 𝑌 Ω2 or/and
• 𝐹
𝑋,𝑌
𝑎, 𝑏 = 𝐹𝑋 𝑎 𝐹𝑌 𝑏 , ∀𝑎 ∈ 𝑋 Ω1 , ∀𝑏 ∈ 𝑌 Ω2
8
DETERMINE INDEPENDENCE SOLELY FROM THE
JOINT DISTRIBUTION
•
If you are only given the form of ℙ(𝑋 = 𝑎, 𝑌 = 𝑏) or 𝑓 𝑋,𝑌 𝑎, 𝑏 how do you know that
𝑋 and 𝑌 are independent?
• Check if ℙ(𝑋 = 𝑎, 𝑌 = 𝑏) or 𝑓 𝑋,𝑌 𝑎, 𝑏 can be factorized into a product of two
functions, one is solely a function of 𝑥, the other solely a function of 𝑦.
• ℙ 𝑋 = 𝑎, 𝑌 = 𝑏 or 𝑓 𝑋,𝑌 𝑥, 𝑦 = 𝑔 𝑥 ℎ 𝑦 ⟹ 𝑋, 𝑌 are independent
• Clearly vice versa
• Pf.
9
EXPECTATION VECTOR 𝔼 𝑋 , 𝔼 𝑌
•
Define the expecation of a random vector as
𝔼 𝑋, 𝑌, ⋯
•
≔ 𝔼 𝑋 ,𝔼 𝑌 ,⋯
It’s still the (multi-dimensional) coordinate of the center of mass of the joint sample space
(Cartesian product of each individual sample spaces).
• E.g. The center of mass of a massed region in a plane.
• E.g. The center of mass of a massed chunk in a 3D space.
•
For the expectation of a scalar-valued function of random vector can be computed using
Lotus as:
𝔼 𝜑 𝑋, 𝑌, ⋯
•
= ∫ ⋯ ∬ 𝜑(𝑥, 𝑦, ⋯ )𝑓(𝑋,𝑌,⋯ ) (𝑥, 𝑦, ⋯ )𝑑𝑥𝑑𝑦 ⋯
Expectation of independent product: If 𝑋 and 𝑌 are independent, then
𝔼 𝑋𝑌 = 𝔼 𝑋 ⋅ 𝔼 𝑌
• Pf.
•
MGF of independent sum: If 𝑋 and 𝑌 are independent, then
𝑀𝑋+𝑌 𝑡 = 𝑀𝑋 𝑡 ⋅ 𝑀𝑌 𝑡
• Pf.
10
A SHORT SUMMARY FOR INDEPENDENT
RANDOM VARIABLES
•
First of all, the bedrock (joint sample space) must be a rectangular region.
•
•
Refer to the problem on Slide 9 of Tutorial 2.
Then you must be careful to equip each point in that region with a probability mass (for discrete
case) or a probability density (for continuous case).
•
The rules are
• Total probability mass is 1
• The probability mass/density distributed on each column must sum/integrate to the
that column’s marginal probability mass/density.
• The probability mass/density distributed on each row must sum/integrate to the that
row’s marginal probability mass/density.
•
Your goal is to make either of the following true at every point 𝑥, 𝑦 in the joint space
• ℙ 𝑋 = 𝑥, 𝑌 = 𝑦 = ℙ 𝑋 = 𝑥 ℙ 𝑌 = 𝑦
• 𝑓 𝑋,𝑌 𝑎, 𝑏 = 𝑓𝑋 𝑎 𝑓𝑌 𝑏
11
CONTINUOUS RANDOM VECTOR (OR JOINTLY
CONTINUOUS RANDOM VARIABLES)
•
Intuition: there cannot be cave-like vertical openings of the density surface over the joint
sample space.
•
Rigorous definition:
• There exists density function 𝑓 everywhere on the joint sample space.
• ℙ 𝑋, 𝑌 ∈ 𝐶 = ∬ 𝑥,𝑦
∈𝐶
𝑓 𝑥, 𝑦 𝑑𝑥𝑑𝑦 , ∀𝐶 ∈ 𝑗𝑜𝑖𝑛𝑡 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑝𝑎𝑐𝑒
12
JOINT CDF 𝐹
𝑋,𝑌
𝑥, 𝑦 ≔ ℙ 𝑋 ≤ 𝑥, 𝑌 ≤ 𝑦
•
Check more properties of joint CDF
and the relationship between joint CDF
and joint PMF/PDF in the review part
of handout.
13
EXERCISE TIME
14