Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Outline • Stochastic vs. deterministic convergence. • Modes of stochastic convergence. • Relations between convergence modes. Modes of Convergence M. Sami Fadali EE782 Random Signals& Estimation 1 2 Example: Deterministic Deterministic Convergence Convergence of a Sequence: A sequence of points in C converges to a point in C if , an integer such that whenever . nth partial sum of a series: Convergence of a Series: Convergence of the to , = sum of the series. sequence 3 • Divergent Series • Convergent Series 4 Stochastic Convergence Example: Sample Mean • • • • • Governs a sequence of random variables. • Must be defined in an averaged sense. • Many standard definition of convergence are available. • Important in assessing estimators (asymptotic theory). Consider samples from the same population. Treat as i.i.d. random variables. Use sample mean as estimate of mean. How does the estimate change when we add more sample points (increase )? 5 6 Convergence in Law Sequence of Random Vectors converges in law to , if • Vector • Real random entries • Sequence of Random Vectors: Lim → x x • x where FX(x) is continuous. • Also called convergence in distribution or weak convergence. • Denoted by L • Joint Distribution Function 7 8 Convergence in Probability Convergence in the rth Mean converges in probability to , if converges in the Lim mean to , if Lim → → Denoted by • In Quadratic Mean: (most useful) • Denoted by 9 10 Almost Sure Convergence Basic Relationships converges almost surely to , if . . Lim → L • Also called convergence with probability 1 (w.p. 1) or strong convergence Hence the terms strong and weak convergence. • Denoted by . . 11 12 Variance of Sample Mean Yn Example: Sample Mean Unbiased Unbiased Estimator of i.i.d. random variables Sample Mean: Estimator of Lim 13 Convergence of Sample Mean → Lim → → 14 References • Sample mean converges in mean square to the population mean. • Mean square convergence convergence in probability. • Convergence in probability convergence in distribution. • Prove convergence in probability by proving convergence in mean square. 15 1. T. M. Apostol, Mathematical Analysis, Addison Wesley, Reading, MA, 1974. 2. Thomas S. Ferguson, A Course in Large Sample Theory, Chapman & Hall, London, 1996. 3. R. G. Brown and P. Y. C. Hwang, Introduction to Random Signals and Applied Kalman Filtering, 3ed, J. Wiley, NY, 1997. 16