Download stochastic processes

Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
LECTURE 1: STOCHASTIC PROCESSES
Salha Mamane
School of Statistics and Actuarial Science
First Floor, Central Block
East Annexe, E58B
[email protected]
February 21, 2011
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Course schedule
1
Lectures: 10:15 12:00 Mondays (CB15)
2
Tutorials: 14:15 - 15:00 Wednesdays (CB128)
3
Consultations: 14:00-17:00 Fridays (E58B)
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Course schedule
1
Lectures: 10:15 12:00 Mondays (CB15)
2
Tutorials: 14:15 - 15:00 Wednesdays (CB128)
3
Consultations: 14:00-17:00 Fridays (E58B)
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Course schedule
1
Lectures: 10:15 12:00 Mondays (CB15)
2
Tutorials: 14:15 - 15:00 Wednesdays (CB128)
3
Consultations: 14:00-17:00 Fridays (E58B)
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Assessment scheme
Test (18/05/2011): 30%.
Exam : 70%
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Assessment scheme
Test (18/05/2011): 30%.
Exam : 70%
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Assessment scheme
Test (18/05/2011): 30%.
Exam : 70%
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Course plan
1
Stochastic Processes
2
Markov chains
3
Markov jump processes
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Introduction
What is this course about?
Stochastic Processes
What are stochastic processes?
Models for systems that evolve randomly in time.
Why do we study stochastic processes?
Stochastic processes arise in many fields such as economics,
finance, biology, physics, telecommunication . . .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Introduction
What is this course about?
Stochastic Processes
What are stochastic processes?
Models for systems that evolve randomly in time.
Why do we study stochastic processes?
Stochastic processes arise in many fields such as economics,
finance, biology, physics, telecommunication . . .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Introduction
What is this course about?
Stochastic Processes
What are stochastic processes?
Models for systems that evolve randomly in time.
Why do we study stochastic processes?
Stochastic processes arise in many fields such as economics,
finance, biology, physics, telecommunication . . .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Introduction
What is this course about?
Stochastic Processes
What are stochastic processes?
Models for systems that evolve randomly in time.
Why do we study stochastic processes?
Stochastic processes arise in many fields such as economics,
finance, biology, physics, telecommunication . . .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Introduction
What is this course about?
Stochastic Processes
What are stochastic processes?
Models for systems that evolve randomly in time.
Why do we study stochastic processes?
Stochastic processes arise in many fields such as economics,
finance, biology, physics, telecommunication . . .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
1
How do future rates depend on past rates?
2
What is the proportion of time the South African Rand is
going up?
3
What is the yearly average rate?
4
What is the probability that the exchange rate will go down to
6.00 given that it is at 7.00?
5
How long would it take before the exchange rate exceeds 9.00?
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Mathematical definition
We need a rigorous definition of a stochastic process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Stochastic process
Definition
A stochastic process {Xt : t ∈ T } also denoted (Xt )t∈T is a
collection of chronologically ordered random variables defined on
the same probability space Ω.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Time space and state space
Definition
1 The set T ⊂ R is called the time space of the process
(Xt )t∈T .
2
The set S of all possible values for Xt , t ∈ T is called the
state space of the process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Time space and state space
Definition
1 The set T ⊂ R is called the time space of the process
(Xt )t∈T .
2
The set S of all possible values for Xt , t ∈ T is called the
state space of the process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Definition
1 The time space is said discrete if the set T is countable.
Otherwise, it is said continuous.
2
The state space is said discrete if for all fixed t, the random
variable Xt is discrete. It is said continuous if for all t, the
random variable Xt is continuous.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Definition
1 The time space is said discrete if the set T is countable.
Otherwise, it is said continuous.
2
The state space is said discrete if for all fixed t, the random
variable Xt is discrete. It is said continuous if for all t, the
random variable Xt is continuous.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Examples
1
(At )t∈T the stochastic process that models the availability of
a book at the time of inventory.
2
(Xt )t∈T the stochastic process that models the state of health
of policyholders of a life insurance company
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Examples
1
(At )t∈T the stochastic process that models the availability of
a book at the time of inventory.
2
(Xt )t∈T the stochastic process that models the state of health
of policyholders of a life insurance company
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Examples
1
Model for the daily maximal temperatures observed in
Johannesburg.
2
Model for the stock price of a company
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Examples
1
Model for the daily maximal temperatures observed in
Johannesburg.
2
Model for the stock price of a company
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Sample paths
Sample paths model possible evolutions of the dynamic system.
Definition
A sample path of the stochastic process (Xt )t∈T is any realization
{xt , t ∈ T } of the chronologically ordered random variables
{Xt , t ∈ T }.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Full characterization ⇔ Probability to paths.
Full characterization ⇔ Distributions of
(Xt1 , . . . , Xtn ) , t1 , . . . , tn ∈ T .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Finite-dimensional distributions
Definition
The distributions of (Xt1 , . . . , Xtn ) , t1 , . . . , tn ∈ T are called the
finite-dimensional distributions of the stochastic process.
In other terms, the full specification of the stochastic process
requires that for all n, we know P(Xt1 ≤ x1 , Xt2 ≤ x2 , . . . , Xtn ≤ xn )
for all t1 , . . . , tn and x1 , x2 , . . . , xn .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Mean and covariance functions
Definition
1 The mean function of a process (X )
t t∈T is the function m
defined by
m(t) = E (Xt ), ∀t ∈ T .
2
The covariance function of a process (Xt )t∈T is the function
K defined by
K (s, t) = cov(Xs , Xt ),
Salha Mamane
∀(s, t) ∈ T 2 .
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Mean and covariance functions
Definition
1 The mean function of a process (X )
t t∈T is the function m
defined by
m(t) = E (Xt ), ∀t ∈ T .
2
The covariance function of a process (Xt )t∈T is the function
K defined by
K (s, t) = cov(Xs , Xt ),
Salha Mamane
∀(s, t) ∈ T 2 .
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
K (t, t) = var(Xt ).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Classification of stochastic processes
Assigning probabilities to all sample paths: impossible!
Specifying the finite dimensional distributions: less
intimidating....... but still impossible!
So now what?
We study particular cases where the process has some
simplifying probability structure.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Classification of stochastic processes
Assigning probabilities to all sample paths: impossible!
Specifying the finite dimensional distributions: less
intimidating....... but still impossible!
So now what?
We study particular cases where the process has some
simplifying probability structure.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Classification of stochastic processes
Assigning probabilities to all sample paths: impossible!
Specifying the finite dimensional distributions: less
intimidating....... but still impossible!
So now what?
We study particular cases where the process has some
simplifying probability structure.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Classification of stochastic processes
Assigning probabilities to all sample paths: impossible!
Specifying the finite dimensional distributions: less
intimidating....... but still impossible!
So now what?
We study particular cases where the process has some
simplifying probability structure.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Strict stationarity
Definition
A stochastic process is said to be stationary or strictly stationary if
the finite-dimensional distributions are invariant under a time shift,
that is, for all s, t1 , t2 , . . . , tn ,
(Xt1 , Xt2 , . . . , Xtn ) and (Xt1 +s , Xt2 +s , . . . , Xtn +s ) have the same
distribution.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Example
Example
Let (Xt )t∈T a stochastic process such that the Xt , t ∈ T are i.i.d.
Then (Xt )t∈T is a stationary process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Weak stationarity
Definition
Consider a stochastic process (Xt )t∈T with finite second order
moments. (Xt )t∈T is said to be weakly stationary if its mean
function and covariance function are invariant under a time shift.
i.e
m(t) = constante = m and K (t, t + s) = γ(s).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
1 Weak stationarity is also called second order stationarity.
2
Strict stationarity implies weak stationarity.(Provided the
process has finite second order moments).
3
Weak stationarity implies constant variance.
4
The function γ is called the autocovariance function of the
weakly stationary process. It satisfies:
γ(−t) = γ(t).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
1 Weak stationarity is also called second order stationarity.
2
Strict stationarity implies weak stationarity.(Provided the
process has finite second order moments).
3
Weak stationarity implies constant variance.
4
The function γ is called the autocovariance function of the
weakly stationary process. It satisfies:
γ(−t) = γ(t).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
1 Weak stationarity is also called second order stationarity.
2
Strict stationarity implies weak stationarity.(Provided the
process has finite second order moments).
3
Weak stationarity implies constant variance.
4
The function γ is called the autocovariance function of the
weakly stationary process. It satisfies:
γ(−t) = γ(t).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
1 Weak stationarity is also called second order stationarity.
2
Strict stationarity implies weak stationarity.(Provided the
process has finite second order moments).
3
Weak stationarity implies constant variance.
4
The function γ is called the autocovariance function of the
weakly stationary process. It satisfies:
γ(−t) = γ(t).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
White noise
Example
A white noise is a weakly stationary process with a mean function
and an autocovariance function given respectively by
m(t) = 0
(
0
if t 6= 0
γ(t) =
2
σ
if t = 0
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Gaussian processes
Definition (Multivariate normal distribution)
A random vector X = (X1 , . . . , Xn ) is said to be Gaussian if the
joint distribution of (X1 , . . . , Xn ) is a multivariate normal
distribution. That is the joint density function is given by
1
1
0 −1
f (x) = f (x1 , . . . , xn ) =
exp − (x − m) Γ (x − m)
n√
2
(2π) 2 det Γ
where m = (m1 , . . . , m2 ) is a vector and Γ is a symmetric positive
definite matrix.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Theorem
E(X ) = E(X1 , . . . , Xn ) = m
cov(Xi , Xj ) = Γij
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Theorem
A random vector X = (X1 , . . . , Xn ) is Gaussian if and only if for all
(α1 , α2 , . . . , αn ) ∈ Rn ,
n
X
αi Xti = α1 X1 + α2 Xt2 + . . . + αn Xtn
i=1
has a normal distribution.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Gaussian processes
Definition
A process (Xt )t∈T is called a Gaussian process if all its
finite-dimensional distributions are multivariate normal
distributions. That is for all (t1 , t2 , . . . , tn ) ∈ T n ,
(Xt1 , Xt2 , . . . , Xtn ) has a multivariate normal distribution.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Theorem
A Gaussian process is strictly stationary if and only if it is weakly
stationary.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Proof
We only have to show that for a Gaussian process, second order
stationarity implies strict stationarity since the converse holds in
general for any stochastic process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Increments
Definition
The increment of the process (Xt )t∈T between time s and t, t > s
is the difference Xt − Xs .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Independent increments
Definition
A process (Xt )t∈T is said to be an independent increment process
if for all 0 ≤ t1 < t2 < . . . < tn , the increments
Xt2 − Xt1 , Xt3 − Xt2 , . . . , Xtn − Xtn−1
are independent.
In particular,
cov(Xti+1 − Xti , Xtj+1 − Xtj ) = 0,
Salha Mamane
Stochastic Processes
∀i 6= j
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Stationary increments
Definition
A process (Xt )t∈T is said to have stationary increments if for all
t, t + s ∈ T the distribution of the increment Xt+s − Xt only
depends on s.
i.e
∀s, t ∈ T , Xt+s − Xt has the same distribution as Xs − X0 .
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
It should be noted that in all the definitions above, the time
intervals do not overlap.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Random Walk
Definition
A stochastic process (Xn )n∈N is called a general random walk if
Xn = X0 +
n
X
Yi ,
i=1
where Y1 , Y2 , . . . are independent identically distributed and
independent of X0 .
A random walk has independent and stationary increments.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Brownian motion
A Brownian motion is the continuous version of random walk
processes. It plays a prominent role in the general theory of
continuous time stochastic processes with continuous state space.
Definition
A Brownian motion (Wt )t∈R also called a Wiener process is a
continuous time stochastic process that satisfies the following
properties:
1
W0 = 0;
2
(Wt )t∈R has independent increments;
3
Wt+s − Wt ∼ N(0, s).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Markov processes
Definition
A process (Xt )t∈T is said to be a Markov process if it satisfies the
following property called the Markov property
P(Xtn+1 ∈ B|Xt1 , . . . , Xtn ) = P(Xtn+1 ∈ B|Xtn )
for all t1 < t2 < . . . < tn ∈ T and B ⊂ S.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
A discrete time space Markov process with discrete
state space is called a Markov chain. Markov chains
will be studied in details in chapter 2.
A continuous time space Markov process with
discrete state space is called a Markov jump process.
Markov jump processes will be studied in details in
chapter 3.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Theorem
An independent increment stochastic process has the Markov
property.
Proof.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Counting process
Definition
A process (Nt )t∈T is called a counting process if
1
the state space of (Nt )t∈T is N
2
Nt is a non-decreasing function of t.
Counting processes model the number of occurrences of random
events.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Bernoulli processes
Definition
A Bernoulli process is a stochastic process (Xn )n∈N such that the
Xn are iid Bernoulli distributed random variables.
(
1 with probability p
Xn =
0 with probability 1 − p
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Binomial process
The Bernoulli process models the occurrence of random events.
The process (Nn )n∈N∗ defined by
Nn =
n
X
Xi
i=1
is thus a counting process that models the number of events that
occurred in the interval [1, n]. (Nn )n∈N∗ is called the Binomial
counting process.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
We have
Nn ∼ B(n, p).
That is
n k
P(Nn = k) =
p (1 − p)n−k .
k
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
We also derive
1
the process (Tn )n≥1 defined by
Tn = min {k, Nk = n}.
Tn models the time of occurrence of the nth event.
2
the process (Yn )n≥1 defined by
(
Y1 = T1
.
Yn = Tn − Tn−1 , for n ≥ 2.
It models the time elapsed between occurrences of two
consecutive events.
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Remark
Tn =
n
X
Yi ,
∀ n ≥ 1.
i=1
(
X1 = N1
Xn = Nn − Nn−1 , for n ≥ 2.
{Nn ≥ m} ⇐⇒ {Tm ≤ n}
Nn =
∞
X
I{Tk ≤n}
k=1
Yn ∼ Geom(p)
i.e P(Tn = m) = (1 − p)m p
Tn follows a negative binomial distribution N B(n, p).
Salha Mamane
Stochastic Processes
Introduction
Stochastic Processes
Characterization of stochastic processes
Classification of stochastic processes
Independent increments-Stationary increments
Two examples of independent and stationary increments processes
Markov processes
Counting processes
Some books
1
2
3
4
5
6
7
8
Stochastic processes in science, engineering, and finance /
Frank Beichelt.
Elements of applied stochastic processes / U. Narayan Bhat,
Stochastic processes / Sheldon M. Ross.
Introduction To Probability Models./ Ross, Sheldon M.
Modeling and analysis of stochastic systems / Vidyadhar G.
Kulkarni.
Stochastic processes and models / David Stirzaker.
Finite Markov chains/ Kemeny, John G.
Continuous-time Markov Chains: An Applications-oriented
Approach / Anderson William J.
Salha Mamane
Stochastic Processes