Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
I An Introduction to Probability Theory and Its Applications WILLIAM FELLER (1906-1970) Eugene Higgins Professor of Mathematics Princeton University VOLUME I THIRD EDITION Revised Printing John Wiley & Sons, Inc. New York • London • Sydney Contents CHAPTER PAGE INTRODUCTION: 1. 2. 3. 4. 5. I THE SAMPLE SPACE 1. 2. 3. 4. 5. 6. 7. 8. II THE NATURE OF PROBABILITY THEORY . The Background Procedure "Statistical" Probability Summary Historical Note 1 1 3 4 5 6 7 The Empirical Background Examples The Sample Space. Events Relations among Events Discrete Sample Spaces Probabilities in Discrete Sample Spaces: Preparations The Basic Definitions and Rules Problems for Solution ELEMENTS OF COMBINATORIAL ANALYSIS 1. Preliminaries 2. Ordered Samples 3. Examples 4. Subpopulations and Partitions *5. Application to Occupancy Problems *5a. Bose-Einstein and Fermi-Dirac Statistics *5b. Application to Runs 6. The Hypergeometric Distribution 7. Examples for Waiting Times 8. Binomial Coefficients 9. Stirling's Formula Problems for Solution: 10. Exercises and Examples 7 9 13 14 17 19 22 24 26 26 28 31 34 38 40 42 43 47 50 52 54 54 * Starred sections are not required for the understanding of the sequel and should be omitted at first reading. xiii XIV CONTENTS CHAPTER PAGE 11. Problems and Complements of a Theoretical Character 12. Problems and Identities Involving Binomial Coefficients *IH FLUCTUATIONS IN COIN TOSSING AND RANDOM WALKS . 1. 2. 3. 4. *5. 6. 7. 8. 9. 10. *IV COMBINATION OF EVENTS 1. 2. 3. 4. 5. 6. V VI General Orientation. The Reflection Principle . . . Random Walks: Basic Notions and Notations . . . The Main Lemma Last Visits and Long Leads Changes of Sign An Experimental Illustration Maxima and First Passages Duality. Position of Maxima An Equidistribution Theorem Problems for Solution 63 67 68 73 76 78 84 86 88 91 94 95 98 Union of Events Application to the Classical Occupancy Problem The Realization of m among N events Application to Matching and Guessing Miscellany Problems for Solution CONDITIONAL PROBABILITY. 58 . . 98 101 106 107 109 Ill STOCHASTIC INDEPENDENCE . 114 1. Conditional Probability 2. Probabilities Defined by Conditional Probabilities. Urn Models 3. Stochastic Independence 4. Product Spaces. Independent Trials *5. Applications to Genetics *6. Sex-Linked Characters *7. Selection 8. Problems for Solution 114 THE BINOMIAL AND THE POISSON DISTRIBUTIONS . . . . 1. 2. 3. 4. Bernoulli Trials The Binomial Distribution The Central Term and the Tails The Law of Large Numbers 118 125 128 132 136 139 140 146 146 147 150 152 CONTENTS XV CHAPTER PAGE 5. 6. 7. 8. 9. 10. VII The Poisson Approximation The Poisson Distribution Observations Fitting the Poisson Distribution . . . . Waiting Times. The Negative Binomial Distribution The Multinomial Distribution Problems for Solution . ^ THE NORMAL APPROXIMATION TO THE BINOMIAL DISTRIBUTION 1. 2. 3. 4. 5. *6. 7. *VIII Infinite Sequences of Trials Systems of Gambling The Borel-Cantelli Lemmas The Strong Law of Large Numbers The Law of the Iterated Logarithm Interpretation in Number Theory Language Problems for Solution RANDOM VARIABLES; 1. 2. 3. 4. 5. 6. *7. *8. 9. X 174 The Normal Distribution Orientation: Symmetric Distributions The DeMoivre-Laplace Limit Theorem Examples Relation to the Poisson Approximation Large Deviations Problems for Solution 174 179 182 187 190 192 193 UNLIMITED SEQUENCES OF BERNOULLI TRIALS 1. 2. 3. 4. 5. 6. 7. IX 153 156 159 164 167 169 . . . . 196 196 198 200 202 204 208 210 EXPECTATION 212 Random Variables Expectations Examples and Applications The Variance Covariance; Variance of a Sum Chebyshev's Inequality Kolmogorov's Inequality The Correlation Coefficient Problems for Solution 212 220 223 227 229 233 234 236 237 LAWS OF LARGE NUMBERS 1. Identically Distributed Variables *2. Proof of the Law of Large Numbers 3. The Theory of "Fair" Games 243 243 246 248 XVI CONTENTS CHAPTER *4. 5. *6. *7. 8. XI *XII 264 264 266 270 275 279 280 283 BRANCHING PROCESSES . . . Sums of a Random Number of Variables The Compound Poisson Distribution Processes with Independent Increments Examples for Branching Processes Extinction Probabilities in Branching Processes . . . The Total Progeny in Branching Processes Problems for Solution 286 286 288 292 293 295 298 301 RECURRENT EVENTS. 1. 2. 3. 4. 5. 6. *7. *8. 9. 10. *11. 12. 251 253 256 258 261 Generalities Convolutions Equalizations and Waiting Times in Bernoulli Trials Partial Fraction Expansions Bivariate Generating Functions The Continuity Theorem Problems for Solution COMPOUND DISTRIBUTIONS. 1. 2. 2a. 3. 4. 5. 6. XIV The Petersburg Game Variable Distributions Applications to Combinatorial Analysis The Strong Law of Large Numbers Problems for Solution INTEGRAL VALUED VARIABLES. GENERATING FUNCTIONS . 1. 2. 3. 4. 5. *6. 7. XIII PAGE RENEWAL THEORY Informal Preparations and Examples Definitions The Basic Relations Examples Delayed Recurrent Events. A General Limit Theorem The Number of Occurrences of 8 Application to the Theory of Success Runs More General Patterns Lack of Memory of Geometric Waiting Times . . . Renewal Theory Proof of the Basic Limit Theorem Problems for Solution RANDOM WALK AND RUIN PROBLEMS 1. General Orientation 2. The Classical Ruin Problem I 303 303 307 311 313 316 320 322 326 328 329 335 338 342 342 344 CONTENTS XV11 CHAPTER PAGE 3. Expected Duration of the Game *4. Generating Functions for the Duration of the Game and for the First-Passage Times *5. Explicit Expressions 6. Connection with Diffusion Processes *7. Random Walks in the Plane and Space 8. The Generalized One-Dimensional Random Walk (Sequential Sampling) 9. Problems for Solution XV MARKOV CHAINS 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. *11. *12. 13. 14. *XVI . . 428 428 432 436 438 443 . 444 444 446 448 451 454 458 General Theory Examples Random Walk with Reflecting Barriers Transient States; Absorption Probabilities Application to Recurrence Times General Orientation. Markov Processes The Poisson Process The Pure Birth Process Divergent Birth Processes The Birth and Death Process Exponential Holding Times 363 367 372 375 382 384 387 390 392 399 404 406 407 414 419 424 . . . . THE SIMPLEST TIME-DEPENDENT STOCHASTIC PROCESSES 1. 2. 3. *4. 5. 6. 349 352 354 359 372 ALGEBRAIC TREATMENT OF FINITE MARKOV CHAINS 1. 2. 3. 4. 5. XVII Definition Illustrative Examples Higher Transition Probabilities Closures and Closed Sets Classification of States Irreducible Chains. Decompositions Invariant Distributions Transient Chains Periodic Chains Application to Card Shuffling Invariant Measures. Ratio Limit Theorems Reversed Chains. Boundaries The General Markov Process Problems for Solution 348 CONTENTS XVlll CHAPTER 7. 8. 9. 10. PAGE Waiting Line and Servicing Problems The Backward (Retrospective) Equations General Processes Problems for Solution 460 468 470 478 ANSWERS TO PROBLEMS 483 INDEX 499