Download 05-Laws of division of casual sizes

Laws of division of casual sizes.
Binomial law of division.
1
Sensor Model
• The prior (|I)
• The sensor reliability
P(|I)
• Likelihood p(y|,,I)
2
Outline
•
•
•
•
•
•
•
•
Random experiments
Samples and Sample Space
Events
Probability Space
Axioms of Probability
Conditional Probability
Bayes’ Rule
Independence of Events
3
Random Experiments
• A random experiment is an experiment in which
the outcome varies in a unpredictable fashion
when the experiment is repeated under the
same condition.
• A random experiment is specified by stating an
experimental procedure and a set of one or
more measurements or observations.
• Examples:
– E1:Toss a coin three times and note the sides facing
up (heads or tail)
– E2:Pick a number at random between zero and one.
– E : Poses done by a rookie dancer
4
Samples and Sample Space
• A sample point (o) or an outcome of a
random experiment is defined as a result
that cannot be decomposed into other
results.
• Sample Space (S): defined as the set of all
possible outcomes from a random
experiment.
• Countable or discrete sample space, oneto-one correspondence between outcomes
5
and integers
Events
• A event is a subset of the sample space S,
a set of samples.
• Two special events:
– Certain event: S
– Impossible or null event: 
6
Probability Space {S,E,P}
• S: Sample space, the space of the
outcomes from a random experiment {o}
• E: Event space, collection of subsets of
the sample space, {A}
• P: Probability measure of a event P(A),
ranges [0,1], encoding how likely an event
will happen.
S
7
Axioms of Probability
1.
2.
3.
4.
0P(A)
P(S)=1
If A ∩ B =, then P(A U B)=P(A)+P(B)
Given a sequence of event, Ai, if  ij, Ai
 Bj =,
P(U i=1 Ai )= i=1 P(Ai)
referred to as countable additivity
8
Some Corollaries
•
•
•
•
•
P(Ac) = 1 – P(A)
P() = 0
P(A)  1
Given a sequence of event, Ai,..., An, if  ij, Ai ∩
Bj =,
A∩B
P(U ni=1 Ai )= ni=1 P(Ai)
P(AUB)=P(A)+P(B)-P(A ∩ B)
B
A
9
Conditional Probability
P( A  B )
P( A | B )  A ∩ B
P
(
B
)
S
Imagine that P(A) is proportional to the size of the area
B
A
10
Theorem of Total Probability
Let {Bi} be a partition of the sample space S
B7
B1
B6
A
B2
B3
B4
B5
P(A)=  7i=1 P(A ∩ Bi ) = 7i=1 P(A | Bi ) P(Bi)
11
Bayes’ Rule
Let {Bi} be a partition of the sample space S.
Suppose that event A occurs, what is the probability of
a priori
the event Bj?
By the definition of conditional
probability we have
P( A  B j )
P( A | B j ) P( B j )
P( B j | A) 
 n
P( A)
 P( A | Bk ) P( Bk )
k 1
a posteriori
12
Independence of Events
• If knowledge of the occurrence of an event B does
not alter the probability of some other event A, then
it would be natural to say that event A is
A  B)
independent
of
B.B)  P( A) PP((B
P
(
A

)
P )  P( A | B ) 
P( B )
• The most common application of the independence
concept is in making the assumption that the
events of separate experiments are independent,
which are referred as independent experiments.
13
An example of Independent
Events
•
•
•
•
Experiment: Toss a coin twice
E1: the first toss is a head
E2: the second toss is a tail
Consider the experiment
T constructed by
H
concatenating two separate random
H (H,H) (H,T)
experiment, toss a coin once.
T
(T,H)
(T,T)
14
Last time …
•
•
•
•
•
•
•
•
Random experiments
Samples and Sample Space
Events
Probability Space
Axioms of Probability
Conditional Probability
Bayes’ Rule
Independence of Events
15
Random Variables
• A random variable X is a function that
assigns a real number, X(), to each
outcome  in the sample space of a
S
X()
random experiment.
Real line

x
SX
16
Random Variables
• Let SX be the set of the sample space.
• X() can be considered as a new random
experiments with outcomes X() as a
function of , the outcome of the original
experiment.
S
X()
Real line

x
SX
17
Examples
• E1:
– Toss a coin three times
– S={HHH,HHT,HTH,HTT, THH,THT,TTH,TTT}
– X()=number of heads in three coins tosses. Note that
sometimes a few  share the same value of X().
– SX={0,1,2,3}
– X is then a random variable taking on the values in
the set SX
• If the outcome  of some experiment is already a
numerical value, we can immediately reinterpret
the outcome as a random variable defined by
18
X()= .
Probabilities and Random
Variables
• Let A be an event of the original
experiment. There is a corresponding
equivalent event B in the new experiment,
with X() as outcome, such that A={S:
X()  B} orS B={X()S
: A}
X() X
Real line
B
A
SX
19
Probabilities and Random
Variables
• P(B)=P(A)=P ({: X()B})
• Two typical events:
– B:{X=x}
– B: {X in I}
S
X()
Real line
B
A
SX
20
The Cumulative Distribution
Function
• The cumulative distribution function (cdf)
of a random variable X is defined as the
probability of the event {Xx}:
FX(x)=P(Xx) for -<x<+
• In term of the underling random
experiment,
FX(x)=P(Xx) =P ({: X()  x})
• The cdf is simply a convenient way of
specifying the probability of all semiinfinite intervals
21
Major Properties of cdf
1.
2.
3.
4.
0 FX(x) 1
limx=1
limx-=0
FX(x) is a non-decreasing function of x,
that is, if a < b, then FX(a)  FX(b).
22
Probability of an event
Let A = {a<Xb} and b>a
P(A)=P{a<Xb}=FX(b) - FX(a).
23
The Probability Mass Function
• Discrete random variables are those random
variables taking values at the countable set
of points.
• The probability mass function (pmf) is the set
of probability pX(xk)=P(X=xk) of the elements
S={HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}
in
SX.
SX={0,1,2,3}, pX(0)=1/8, pX(1)=3/8, pX(2)=3/8, pX(3)=1/8
FX(x)
pX(x)
1
7/8
3/8
1/8
1/2
0 1
2
3
X
1/8
0 1
2
3
X
24
The Probability Density
Function
• A continuous random variable is defined as a
random variable whose cdf is continuous
everywhere, and which,x in addition, is sufficiently
( x )be
 written
f (t )as
dt an integral of
smooth that F
it Xcan

some nonnegative function f(x):

dFX ( x )
f ( x) 
dx
• The probability density function of X (pdf) is
defined as the derivative of FX(x):
25
The Probability Density
Function
p
 ( x)dx  1


X
pX(x)
P( a  X  b)  FX (b)  FX ( a )
b
a


  p X (t )dt   p X (t )dt
b
  p X (t )dt
a
When a b, P(aX b) pX((a+b)/2) |b-a|
a b
Support of X
X
26
An Example
pX(x) = 1, when x [0,1], otherwise 0
pX(x)
1
0
1
X
FX(x)
1
0
1
X
27
Multiple R.V.
• Joint cdf : FX,Y(x,y)=P(Xx,Y y)
• Conditional cdf: if P(Yy) > 0,
FX|Yy(x)=P(Xx|Yy)=P(Xx,Y y)/P(Yy)
• Joint pdf: pX,Y(x,y)
the 2nd order derivatives of the joint cdf,
usually independent of the order when
pdfs are continuous.
• Marginal pdf: pY (y) = X pX,Y(x,y)dx
• Conditional pdf: pX|Y=y(x)=pX,Y(x,y)/ pY (y) 28
Expectation
• The expectation of a random variable is given
by the weighted average of the values in the
support of the random variable
N
E{ X }   xk p X ( xk )
k 1

E{ X }   xpX ( x )dx

29
Smoothing Property of Conditional
Expectation
• EY|X {Y|X=x}=g(x)
• E{Y}=EX{EY|X {Y|X=x}}
30
Fundamental Theorem of
Expectations
• Let Y=g(X)




E{Y }   ypY ( y )dy   g ( x ) p X ( x )dx
• Recall that E{Y}=EX{EY|X {Y|X=x}}
31
Variance of a R.V.
• Weighted difference from the expectation
Var(X)=X (x-E(X))2 pX(x) dx
32
Last Time …
• Random Variables
• cdf, pmf, pdf,
• Expectation, variances,
33
Correlation and Covariance of Two
R.V.
• Let X and Y be two random variables.
• The correlation between X and Y is given
by
E{XY}= X,Y xy pX,Y(x,y) dxdy
• Covariance of X and Y is given by
COV(X,Y)=E{(X-E{X})(Y-E{Y})}
=X,Y (X-E{X})(Y-E{Y}) pX,Y(x,y) dxdy
=E{XY}-E{X}E{Y}
• When COV(X,Y)=0 or E{XY}=E{X}E{Y}, X
and Y are (linearly) uncorrelated. When 34
Correlation coefficient
 X ,Y
COV ( X , Y )

VAR( X )VAR(Y )
 1   X ,Y  1
35
Independent R.V.s
• If pX|Y=y(x)=pX,Y(x,y)/ pY (y)= pX (x) for all x
and y, X and Y are independent random
variables, i.e.
Independence  pX,Y(x,y) = pX (x)pY (y)  x,
y
36
Independent v.s. Uncorrelated
• Independent  Uncorrelated
• Uncorrelated  Independent
• Example: Uncorrelated but dependent
random variables: Let  U[0,2] and
X=cos(), Y=sin()
E{X}= (1/2)02 cos() d = 0, E{Y}=0;
COV(X,Y)= (1/2)02 cos()sin() d
= (1/4)02 sin(2) d
=0
• If X and Y are jointly Gaussian,
Independent Uncorrelated
37
Covariance Matrix of Random
Vector
• X = (X1,X2,…,Xn)T
• The covariance matrix of random vector X is
given by
CX = E{(X-E{X})(X-E{X})T}
• CX(i,j) = COV(Xi,Xj)
• Properties of CX
– Symmetric, CX = CXT, i.e. CX(i,j) = CX(j,i)
– Semi-positive definite Rn
T CX  = 0
Since VAR(T X) = T CX  = 0
38
First Order Markov Process
• Let {X1,X2,…,Xn} be a sequence of random
variables (or vectors), e.g. the joint angle
vectors in a gait cycle over a period of time.
• We call this process is a first order Markov
process if the following Markov property is
true:
P(Xn=xn|Xn-1=xn-1,…,X1=x1) = P(Xn=xn|Xn1=xn-1)
i.e.
39
Chain Rule
P(Xn=xn,Xn-1=xn-1,…,X1=x1)
= P(Xn=xn|Xn-1=xn-1,…,X1=x1) P(Xn-1=xn1,…,X1=x1)
= P(Xn=xn|Xn-1=xn-1) P(Xn-1=xn-1,…,X1=x1)
= P(Xn=xn|Xn-1=xn-1) P(Xn-1=xn-1|Xn-2=xn-2)…
P(X1=x1)
= P(X1=x1) k=2n P(Xk=xk|Xk-1=xk-1)
40
Dynamic Systems
• System and Observation equations
 xt  Qt ( xt 1 )  v  xt ~ qt ( xt | xt 1 )


,
 yt  Ft ( xt )  n  yt ~ f t ( yt | xt ) ,
p( x0 )
• Chain rule
 t ( x0:t )  p( x0:t , y1:t )   ( x0 )  k 1 q( xk xk 1 ) f ( yk xk )
 t 1 ( x0:t 1 )   t ( x0:t )q( xt 1 xt ) f ( yt 1 xt 1 )
t
41
Discrete State Markov Chains
• Given a finite discrete set S of states, a Markov
chain process possesses one of these states at
each unit of time.
• The process either stays in the same state or moves
to some other state in S.
S1
S2
S1 S2 S3
1/ 1/2 0
2
1/ 2/3 0
42
3
Good luck
43