Download Stochastic Processes

Stochastic Processes
Math-432
Instructor: Sultan Sial
Office: 130 Academic Block
Office Extension: 2130
Office Hours: TBA
Term: Spring, 2004-2005
Email: [email protected]
Lectures: Monday and Wednesday 11:45-13:25
Pre-requisites: Math-131 (or an equivalent course) or permission from the Instructor
Lecture notes: Lecture notes covering approximately the first half of the course will be
available for copying before the start of classes. The rest of the lecture notes will be
available for copying a little later.
Textbook: Classical and Spatial Stochastic Processes, by Rinaldo B. Schinazi, 1999.
Course Description: This course is an introduction to stochastic processes. As such it
assumes only knowledge of calculus and elementary probability. This course quickly
reviews basic probability theory and then deals with stochastic processes. Topics include
discrete and continuous Markov chains, random walk, branching process, stationary
distributions, birth and death chains, Brownian motion, and martingales. We will cover a
variety of applications to different disciplines.
Goals: Goals of the course are to present general theory and applications of stochastic
processes and to develop probabilistic thinking and intuition.
Grading:
Quizzes
Midterm
Final
30%
30%
40%
Topics to be covered (not necessarily in this order)
Review of basic probability: Sample space, event, conditional probability, independent
events, Baye’s formula, random variable, distribution, cumulative distribution, Bernoulli
distribution, binomial distribution, Poisson distribution, geometric distribution, density
function, exponential distribution, gamma distribution, expectation, variance, standard
deviation, joint distribution, covariance, correlation, mean, moment generating function,
Chebyshev’s inequality, Law of large numbers, central limit theorem. It is expected that
the student has already seen these topics and these will not be covered in depth.
Finite Markov chains: basics, examples, 1-step and n-step transition probabilities,
stationary distributions, classifying states, periodicity of classes, absorption of transient
states, reversibility, ruin problem.
Branching processes: probability generating function, compound distribution,
generations of offspring, extinction, total progeny, generalizations.
Renewal theory: sequence generating function, pattern generation, consecutive
successes, mean number of trials, breaking even, mean number of occurrences,
comparison of patterns, probability of winning, expected duration.
Markov processes: Poisson processes, extensions, pure birth processes, Yule process,
pure death processes, birth and death processes, linear case, linear growth with
immigration, M/M/infinity queue, power supply problem, stationary distributions,
examples, absorption, continuous time Markov chains, n-state case.
Brownian motion: definition, no drift case, reaching a point, avoiding zero, returning to
zero, drift.