Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
RANDOM VECTOR Tutorial 6, STAT1301 Fall 2010, 02NOV2010, MB103@HKU By Joseph Dong RECALL: CARTESIAN PRODUCT OF SETS Two discrete sets Two Continuous sets 2 RECALL: SAMPLE SPACE OF A RANDOM VARIABLE 3 THE MAKING OF A RANDOM VECTOR AS JOINT RANDOM VARIABLES: π1 , β― , ππ A CRASH COURSE OF LATIN NUMBER PREFIXES β’ Uni-variate : 1 random variable β’ Bi-variate : 2 random variables bind together to become a 2-tuple random vector like π³ = π1 , π2 β’ Tri-variate : 3 random variables bind together to become a 3-tuple random vector like π³ = π1 , π2 , π3 β’ β¦β¦ β’ n-variate : n random variables bind together to become a 3-tuple random vector like π³ = π1 , β― , ππ β’ You can even have infinite-dimensional random vectors! Unimaginable! Prefix Uni- Bi- Tri- Quadri- Quinti- Num. 1 2 3 4 5 Sexa- Septi6 7 Octo- Novem- Deca- 8 9 10 4 RANDOM VECTOR AS A FUNCTION ITSELF: π: πΊ1 × β― × πΊπ β π1 , β― , ππ β¦ π₯1 , β― , π₯π β βπ β’ How to distribute total probability mass 1 on the sample space of the random vector? β’ Is this process completely fixed? β’ If not fixed, is this process completely arbitrary? β’ If neither arbitrary, what are the rules for distributing total probability mass 1 onto this state space? β’ βMarginal PDF/PMFβ imposes an additive restriction. β’ There is a lot to discover hereβ¦ 5 INDEPENDENCE AMONG RANDOM VARIABLES β’ Recall: What are independence among events? β’ β π΄π΅ = β π΄ β β π΅ β’ Q: What does a random variable do to its state space? β’ It partitions the state space by the atoms in the sample space! β’ π₯ is an atom in the sample space and π β1 π₯ β’ π₯1 , π₯2 is a union of atoms in the sample space and π β1 π₯1 , π₯2 in the state space. β’ We can talk about whether π = π₯ and π = π¦ are independent β’ because they mean two events: π β1 π₯ β’ is a block in the state space. is a union of blocks and π β1 π¦ We can talk about whether π β π΄ and π β π΅ are independent β’ because they mean two events: π β1 π΄ and π β1 π΅ β’ Goal: Generalize this connection to the most extent: Establish the meaning of independence between whole random variables π and π. 6 TWO RANDOM VARIABLES ARE INDEPENDENT IFβ¦ β’ Each event in the state space of π is independent from each event in the state space of π. β π β π΄, π β π΅ = β π β π΄ β π β π΅ , βπ΄ β π Ξ©1 , βπ΅ β π Ξ©2 β’ Further, this is true if each atom in the state space of π is independent from each atom in the state space of π. β π = π, π = π = β π = π β π = π , βπ β π Ξ©1 , βπ β π Ξ©2 β’ How many terms are there if you expand π2 + π3 + π3 π3 + π4 + π5 ? β’ One more equivalent condition: β π β€ π, π β€ π = β π β€ π β π β€ π , βπ β π Ξ©1 , βπ β π Ξ©2 7 INDEPENDENCE OF CONTINUOUS RANDOM VARIABLES β’ Previous picture deals with the discrete random variables case. β’ Two continuous random variables π and π are independent if β’ β π β π΄, π β π΅ = β π β π΄ β π β π΅ , βπ΄ β π Ξ©1 , βπ΅ β π Ξ©2 or/and β’ π π,π π, π = ππ π ππ π , βπ β π Ξ©1 , βπ β π Ξ©2 or/and β’ πΉ π,π π, π = πΉπ π πΉπ π , βπ β π Ξ©1 , βπ β π Ξ©2 8 DETERMINE INDEPENDENCE SOLELY FROM THE JOINT DISTRIBUTION β’ If you are only given the form of β(π = π, π = π) or π π,π π, π how do you know that π and π are independent? β’ Check if β(π = π, π = π) or π π,π π, π can be factorized into a product of two functions, one is solely a function of π₯, the other solely a function of π¦. β’ β π = π, π = π or π π,π π₯, π¦ = π π₯ β π¦ βΉ π, π are independent β’ Clearly vice versa β’ Pf. 9 EXPECTATION VECTOR πΌ π , πΌ π β’ Define the expecation of a random vector as πΌ π, π, β― β’ β πΌ π ,πΌ π ,β― Itβs still the (multi-dimensional) coordinate of the center of mass of the joint sample space (Cartesian product of each individual sample spaces). β’ E.g. The center of mass of a massed region in a plane. β’ E.g. The center of mass of a massed chunk in a 3D space. β’ For the expectation of a scalar-valued function of random vector can be computed using Lotus as: πΌ π π, π, β― β’ = β« β― β¬ π(π₯, π¦, β― )π(π,π,β― ) (π₯, π¦, β― )ππ₯ππ¦ β― Expectation of independent product: If π and π are independent, then πΌ ππ = πΌ π β πΌ π β’ Pf. β’ MGF of independent sum: If π and π are independent, then ππ+π π‘ = ππ π‘ β ππ π‘ β’ Pf. 10 A SHORT SUMMARY FOR INDEPENDENT RANDOM VARIABLES β’ First of all, the bedrock (joint sample space) must be a rectangular region. β’ β’ Refer to the problem on Slide 9 of Tutorial 2. Then you must be careful to equip each point in that region with a probability mass (for discrete case) or a probability density (for continuous case). β’ The rules are β’ Total probability mass is 1 β’ The probability mass/density distributed on each column must sum/integrate to the that columnβs marginal probability mass/density. β’ The probability mass/density distributed on each row must sum/integrate to the that rowβs marginal probability mass/density. β’ Your goal is to make either of the following true at every point π₯, π¦ in the joint space β’ β π = π₯, π = π¦ = β π = π₯ β π = π¦ β’ π π,π π, π = ππ π ππ π 11 CONTINUOUS RANDOM VECTOR (OR JOINTLY CONTINUOUS RANDOM VARIABLES) β’ Intuition: there cannot be cave-like vertical openings of the density surface over the joint sample space. β’ Rigorous definition: β’ There exists density function π everywhere on the joint sample space. β’ β π, π β πΆ = β¬ π₯,π¦ βπΆ π π₯, π¦ ππ₯ππ¦ , βπΆ β πππππ‘ π πππππ π ππππ 12 JOINT CDF πΉ π,π π₯, π¦ β β π β€ π₯, π β€ π¦ β’ Check more properties of joint CDF and the relationship between joint CDF and joint PMF/PDF in the review part of handout. 13 EXERCISE TIME 14