Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Nonlinear dimensionality reduction wikipedia, lookup

Transcript
```DISCRETE -TIME
MARKOV CHAIN
( C O N T I N U AT I O N )
PROBABILITY OF ABSORPTION
If state j is an absorbing state, what is the probability of going
from state i to state j?
Let us denote the probability as π΄ππ .
Finding the probabilities is not straightforward, especially
when there are two or more absorbing states in a Markov
chain.
PROBABILITY OF ABSORPTION
What we can do is to consider all the possibilities for the first
transition and then, given the first transition, we consider the
conditional probability of absorption into state j.
π
π΄ππ =
πππ π΄ππ
π=0
PROBABILITY OF ABSORPTION
We can obtain the probabilities by solving a system of linear equations
π
π΄ππ =
πππ π΄ππ for π = 0,1, β¦ , π
π=0
subject to
π΄ππ = 1
π΄ππ = 0 if state π is recurrent and π β  π
1
EXERCISE: FIND π¨ ππ
State
0
.3
1
State
3
p=0.7
0.3
State
2
p=0.7
State
1
ENDING SLIDES
CHAINS
TIME REVERSIBLE MARKOV CHAINS
Consider a stationary (i.e., has been in operation for a long
time) ergodic Markov Chain having transition probabilities πππ
and stationary probabilities ππ . Suppose that starting at some
time we trace the sequence of states going backward in time.
TIME REVERSIBLE MARKOV CHAINS
Starting at time n, the stochastic process
ππ , ππβ1 , ππβ2 , β¦ , π0 is also a Markov Chain! The transition
probabilities are
ππ πππ
πππ =
.
ππ
If πππ = πππ for all π, π then the Markov Chain is time
reversible. Or ππ πππ = ππ πππ which means the rate at which
the process goes from i to j is equal to the rate at which it
goes from j to i.
FOR REPORTING:
β’Hidden Markov Chains applied to data
analytics/mining
β’Markov Chain Monte Carlo in data fitting
```
Related documents