Download 0301334 - The University of Jordan :: Faculty of Science

University of Jordan
Faculty of Science
Mathematics Department
COURSE OUTLINE
Course Name and Number : Stochastic Processes 0331334
References:
1) Elements of Applied Stochastic Processes,3rd edition 2002
By: Bhat,U. and Miller ,G.
2) Lectures on Stochastic Processes, By: William Faires
3) Elementary Stochastic Processes, Lecture Notes, By: William Anderson
4) Notes on Stochastic processes ,By: Kiyoshi Igusa 2006
5) Stochastic Processes lecture Notes ,by Jiahua Chen.
Examinations : First Exam 20% Second Exam 30% and Final Exam 50%
Course Contents
1. Discrete Time Markov Chains: (10 lectures)
Conditional probability, Generating functions, random sum of random variables.
Examples of Markov Chains( Random Walk, Gambler's ruin, Ehrenfest Chain,
Branching chains and queueing chains). Chapman-Kolmogorov equations.
Transition function and initial distribution. Transient ,positive recurrent and null
recurrent states. Periodic and aperiodic Markov chains. Reducible and Irreducible
Markov chains. Erogodicity. First step conditional analysis.
2.Stationary and Limiting Matrix distribution for Markov chains:( 10 lectures)
Conditions on the Markov chain to converge to stationary distribution.
Stationary and Limiting matrix distribution for irreducible Markov chains.
Stationary and Limiting matrix distributions for reducible Markov chains.
First passage(hitting) time and mean first passage(hitting) time
First recurrence time and mean first recurrence time. Mean Time spent in a
transient state. Time Reversible Markov chains . Local and global
balance equations. Conditional expectations and martingales. Optimal stopping.
3.Markov Pure jump processes: ( 15 lectures)
Stationary and independent increment processes. Time homogeneity.
Jump times and holding times. Birth-Death processes. Forward and Backward
Chapman-Kolmogorov equations. Embedded Markov chains. Poisson process and
Compound Poisson Process.Exponential distribution and Poisson Process.Renewal
Processs.Time reversibility. Convergence of Markov Processes. Inter-arrival and
waiting time distributions. Stationary distributions for Markov Processes. Twostate birth-death process.Pure birth process. Infinite server queue and M/M/m
queueing systems.Branching processes, probability of extinction and distribution of
total number of progeny.
4. Second order processes: ( 7 lectures):
Weak and strong stationarity. Mean auto-covariance functions. Autocovariances
and Autocorrelations of different lags. Guassian Processes. Weiner process .
Brownian Motion and stochastic calculus.