Download Document

An Introduction to
Stochastic Processes
with Applications to Biology
Linda J. S. Allen
Department of Mathematics and Statistics
Texas Tech University
PEARSON EDUCATION, INC., Upper Saddle River, New Jersev 07458
Contents
Preface
xi
1 Review of Probability Theory and an Introduction to
Stochastic Processes
1.1 Introduction
1.2 Brief Review of Probability Theory
1.3 Generating Functions
1.4 Central Limit Theorem
1.5 Introduction to Stochastic Processes
1.6 An Introductory Example: A Simple Birth Process
1.7 Exercises for Chapter 1
1.8 References for Chapter 1
1.9 Appendix for Chapter 1
1.9.1 MATLAB and FORTRAN Programs
1.9.2 Interevent Time
1
1
3
18
22
24
27
32
35
37
37
38
2
Discrete Time Markov Chains
2.1 Introduction
2.2 Definitions and Notation
2.3 Classification of States
2.4 First Passage Time
2.5 Basic Theorems for Markov Chains
2.6 Stationary Probability Distribution
2.7 Finite Markov Chains
2.7.1 Mean Recurrence Time and Mean First Passage Time
2.8 The n-Step Transition Matrix
2.9 An Example: Genetics Inbreeding Problem
2.10 Unrestricted Random Walks in Two and Three Dimensions
2.10.1 Two Dimensions
2.10.2 Three Dimensions
2.11 Exercises for Chapter 2
2.12 References for Chapter 2
2.13 Appendix for Chapter 2
2.13.1 Power of a Matrix
2.13.2 Genetics Inbreeding Problem
41
41
42
45
51
56
62
65
69
71
75
77
77
78
80
86
88
88
89
viii
Contents
3
Biological Applications of Discrete Time Markov Chains
3.1 Introduction
3.2 Restricted Random Walk Models
3.3 Gambler's Ruin Problem
3.3.1 Probability of Absorption
3.3.2 Expected Time until Absorption
3.3.3 Probability Distribution for Absorption
3.4 Gambler's Ruin Problem on a Semi-Infinite Domain . . . .
3.5 General Birth and Death Process
3.5.1 Expected Time to Extinction
3.6 Logistic Growth Process
3.7 Quasistationary Probability Distribution
3.8 SIS Epidemic Model
3.8.1 Deterministic SIS Epidemic Model
3.8.2 Stochastic SIS Epidemic Model
3.9 Chain Binomial Epidemic Models
3.9.1 Greenwood Model
3.9.2 Reed-Frost Model
3.9.3 Duration and Size of the Epidemic
3.10 Exercises for Chapter 3
3.11 References for Chapter 3
3.12 Appendix for Chapter 3
3.12.1 MATLAB Programs
3.12.2 Maple Program
91
91
92
93
95
98
101
104
106
107
109
112
115
117
118
121
122
124
125
127
133
135
135
137
4
Discrete Time Branching Processes
4.1 Introduction
4.2 Definitions and Notation
4.3 Probability Generating Function of Xn
4.4 Probability of Population Extinction
4.5 Mean and Variance of Xn
4.6 Multitype Branching Processes
4.7 An Example: Age-Structured Model
4.8 Exercises for Chapter 4
4.9 References for Chapter 4
139
139
140
143
145
151
155
159
164
169
5
Continuous Time Markov Chains
5.1 Introduction
5.2 Definitions and Notation
5.3 The Poisson Process
5.4 Generator Matrix Q
5.5 Embedded Markov Chain and Classification of States
5.6 Kolmogorov Differential Equations
5.7 Finite Markov Chains
5.8 Generating Function Technique
171
171
172
174
178
. . . 181
186
189
194
Contents
5.9
5.10
5.11
5.12
5.13
6
Interevent Time and Stochastic Realizations
Review of Method of Characteristics
Exercises for Chapter 5
References for Chapter 5 . . . .
Appendix for Chapter 5
5.13.1 MATLAB Program
Continuous Time Birth and Death Chains
6.1 Introduction
6.2 General Birth and Death Process
6.3 Stationary Probability Distribution
6.4 Simple Birth and Death Processes
6.4.1 Simple Birth Process
6.4.2 Simple Death Process
6.4.3 Simple Birth and Death Process
6.4.4 Simple Birth and Death Process with Immigration .
6.5 Queueing Processes
6.6 Probability of Population Extinction
6.7 Expected Time to Extinction and First Passage Time . . .
6.8 Logistic Growth Process
6.9 Quasistationary Probability Distribution
6.10 An Explosive Birth Process
6.11 Nonhomogeneous Birth and Death Process
6.12 Exercises for Chapter 6
6.13 References for Chapter 6
6.14 Appendix for Chapter 6
6.14.1 Generating Functions for the Simple Birth and Death
Process
6.14.2 Proofs of Theorems 6.2 and 6.3
6.14.3 Comparison Theorem
7 Epidemic, Competition, Predation and Population
Genetics Processes
7.1 Introduction
7.2 Continuous Time Branching Processes
7.3 SI and SIS Epidemic Processes
7.3.1 Stochastic SI Epidemic Model
7.3.2 Stochastic SIS Epidemic Model
7.4 Multivariate Processes
7.5 SIR Epidemic Process
7.5.1 Stochastic SIR Epidemic Model
7.5.2 Final Size of the Epidemic
7.5.3 Expected Duration of an SIR Epidemic
7.6 Competition Processes
7.6.1 Stochastic Competition Model
ix
197
203
204
208
209
209
211
211
212
215
218
219
222
224
228
232
236
237
242
247
249
252
254
261
263
263
265
268
269
269
270
275
277
280
281
284
286
288
291
293
295
x
Contents
7.7 Predator-Prey Processes
7.7.1 Stochastic Predator-Prey Model
7.8 Other Population Processes
7.8.1 SEIR Epidemic Model
7.8.2 Spatial Predator-Prey Model
7.8.3 Population Genetics Model
7.9 Exercises for Chapter 7
7.10 References for Chapter 7
7.11 Appendix for Chapter 7
7.11.1 MATLAB Programs
297
298
300
300
302
304
308
313
316
316
8 Diffusion Processes and Stochastic Differential Equations
8.1 Introduction
8.2 Definitions and Notation
8.3 Random Walk and Brownian Motion
8.4 Diffusion Process
8.5 Kolmogorov Differential Equations
8.6 Wiener Process
8.7 Ito Stochastic Integral
8.8 Ito Stochastic Differential Equation
8.9 Numerical Methods for Solving SDEs
8.10 Ito SDEs for Interacting Populations
8.11 Epidemic, Competition, and Predation Processes
8.11.1 Competition Model
8.11.2 Predator-Prey Model
8.11.3 SIR Epidemic Model
8.12 Population Genetics Process
8.13 Expected Time to Extinction and First Passage Time . . .
8.14 Exercises for Chapter 8
8.15 References for Chapter 8
8.16 Appendix for Chapter 8
8.16.1 Derivation of Kolmogorov Equations
8.16.2 MATLAB Programs
321
321
322
324
327
328
333
335
341
348
351
357
357
358
360
362
365
367
373
376
376
377
Index
381