Download Stochastic Processes: An Overview

STOCHASTIC
PROCESSES:
AN OVERVIEW
M AT H 1 8 2 2 N D S E M AY 2 0 1 6 - 2 0 1 7
STOCHASTIC PROCESS
 Suppose we have an index set 𝑻 ⊆ [𝟎, ∞)
We usually call this “time”
 𝑿𝒕 where 𝒕 ∈ 𝑻 is a stochastic or random
process
The set of random variables indexed by
“time” 𝑻 is called a stochastic or random
process
STOCHASTIC PROCESS
When 𝑻 is countable (e.g., 𝑻 = ℕ ∪ {𝟎})
𝑿𝒕 is a discrete-time process (e.g., 𝑿𝟎 , 𝑿𝟏 , 𝑿𝟐 , 𝑿𝟑 , … )
When 𝑻 is an interval in the real line (e.g., 𝑻 = 𝟎, 𝟏𝟎 )
𝑿𝒕 is a continuous-time process
When 𝑻 is a singleton (e.g., 𝑻 = {𝟎})
The process can be studied using a single random
variable (MATH 181)
STOCHASTIC PROCESS
The value of the random variable 𝑿𝒕 is the state of
the system at time t.
Note: 𝑿𝒕 can be discrete or continuous (recall discrete and
continuous probability distributions in MATH 181)
When 𝑿𝒕 can only assume discrete values
𝑿𝒕 is a discrete-state process (chain)
When 𝑿𝒕 can assume continuous values
𝑿𝒕 is a continuous-state process
EXAMPLE
Let the random process 𝑿𝟎 , 𝑿𝟏 , 𝑿𝟐 , 𝑿𝟑 represent the number
of customers that have entered McDo Vega at time period 0
(8:00-8:59), 1 (9:00-9:59), 2 (10:00-10:59) and 3 (11:00-11:59).
We have a discrete-time discrete-state stochastic process (or
simply, a discrete-time chain).
Stochastic processes usually model the evolution of a random
system through time.
STOCHASTIC PROCESS AS A
FUNCTION OF TWO VARIABLES
Recall, random variable:
𝝎 ↦ 𝑿𝒕 (𝝎)
Random variable is a function of outcome
“a procedure assigning a number to an outcome”
STOCHASTIC PROCESS AS A
FUNCTION OF TWO VARIABLES
We know that the outcome is uncertain. For now,
let us consider a fixed outcome 𝝎𝟎 .The random
variable in a stochastic process with fixed outcome:
𝒕 ↦ 𝑿𝒕 (𝝎𝟎 )
Random variable as a function of time
Actually, 𝑿𝒕 (𝝎𝟎 ) is a function of time and outcome!
SAMPLE PATH
Again, the outcome is uncertain which means 𝑿𝒕 (𝝎)
can have different possible states at time t.
For example, at time 0, 𝑿𝟎 (𝝎𝟎 ) and 𝑿𝟎 (𝝎𝟏 ) are
possible states where 𝝎𝟎 and 𝝎𝟏 are the possible
outcomes.
This means a stochastic process 𝑿𝒕 (𝝎) can have
different “realizations”.
SAMPLE PATH
A realization of the stochastic process is called a
sample path (or sample trajectory).
1000
X_t
800
600
400
200
0
0
2
4
6
t
sample path 1
sample path 2
8
10
TIME-SLICES OF A STOCHASTIC PROCESS
1000
600
30 runs
400
200
0
0
2
4
6
t
5sample path 1
4
3
2
1
0
8
10
Histogram of X_0
sample path 2
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1000
X_t
800
TIME-SLICES OF A STOCHASTIC PROCESS
1000
600
30 runs
400
200
0
0
2
4
6
t
4sample path 1
8
10
Histogram of X_2
sample path 2
3
2
1
0
0
50
100
150
200
250
300
350
400
450
500
550
600
650
700
750
800
850
900
950
1000
X_t
800
UNCERTAINTIES
Uncertainties does NOT mean there is no pattern!
There are many cases where patterns exist, and
mostly they are described by
SPECIAL PROBABILITY DISTRIBUTIONS!
and
SPECIAL STOCHASTIC PROCESSES!
UNCERTAINTIES
SPECIAL PROBABILITY DISTRIBUTIONS
and
SPECIAL STOCHASTIC PROCESSES
have well-studied properties and we can use these
properties to describe systems with randomness.
EXAMPLE
Customers arrive RANDOMLY at a bakery at an
average of 18 per hour on weekday mornings.
The arrival distribution can be described by a
POISSON distribution with a mean of 18.
The saleslady can serve a customer in an average of
three minutes; this time can be described by an
exponential distribution with a mean of 3.0 minutes.
EXAMPLE
Using Queueing Theory, we can determine the
Average number of customers waiting in line
Average time customers wait in line
The maximum expected number waiting in line
The probability of zero customers in the system
The probability of n customers in the system
etc.
SOME FAMOUS STOCHASTIC PROCESSES
Bernoulli process
Binomial process
Branching process
Markov chain
Moran process
Random walk
Birth-death process
Wiener or Brownian motion process
Contact process
SOME FAMOUS STOCHASTIC PROCESSES
Diffusion process
Interacting particle system
Jump process
Lévy process
Ornstein-Uhlenbeck process
Poisson process
Gaussian process
Martingale
Renewal process