Download Modes of Convergence Outline Deterministic Convergence Example

Outline
• Stochastic vs. deterministic convergence.
• Modes of stochastic convergence.
• Relations between convergence modes.
Modes of Convergence
M. Sami Fadali
EE782 Random Signals& Estimation
1
2
Example: Deterministic
Deterministic Convergence
Convergence of a Sequence: A sequence
of points in C converges to a point in C if
, an integer such that
whenever
.
nth partial sum of a series:
Convergence of a Series: Convergence of the
to , = sum of the series.
sequence
3
• Divergent Series
• Convergent Series
4
Stochastic Convergence
Example: Sample Mean
•
•
•
•
• Governs a sequence of random variables.
• Must be defined in an averaged sense.
• Many standard definition of convergence
are available.
• Important in assessing estimators
(asymptotic theory).
Consider samples from the same population.
Treat as i.i.d. random variables.
Use sample mean as estimate of mean.
How does the estimate change when we add
more sample points (increase )?
5
6
Convergence in Law
Sequence of Random Vectors
converges in law to , if
• Vector
• Real random entries
• Sequence of Random Vectors:
Lim
→
x
x
• x where FX(x) is continuous.
• Also called convergence in distribution
or weak convergence.
• Denoted by
L
• Joint Distribution Function
7
8
Convergence in Probability
Convergence in the rth Mean
converges in probability to , if
converges in the
Lim
mean to , if
Lim
→
→
Denoted by
• In Quadratic Mean:
(most useful)
• Denoted by
9
10
Almost Sure Convergence
Basic Relationships
converges almost surely to , if
. .
Lim
→
L
• Also called convergence with probability 1
(w.p. 1) or strong convergence
Hence the terms strong and weak convergence.
• Denoted by
. .
11
12
Variance of Sample Mean Yn
Example: Sample Mean Unbiased
Unbiased Estimator
of
i.i.d. random variables
Sample Mean: Estimator of
Lim
13
Convergence of Sample Mean
→
Lim
→
→
14
References
• Sample mean converges in mean square to the
population mean.
• Mean square convergence  convergence in
probability.
• Convergence in probability  convergence in
distribution.
• Prove convergence in probability by proving
convergence in mean square.
15
1. T. M. Apostol, Mathematical Analysis, Addison
Wesley, Reading, MA, 1974.
2. Thomas S. Ferguson, A Course in Large Sample
Theory, Chapman & Hall, London, 1996.
3. R. G. Brown and P. Y. C. Hwang, Introduction
to Random Signals and Applied Kalman
Filtering, 3ed, J. Wiley, NY, 1997.
16