Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Survey

Transcript

DISCRETE -TIME MARKOV CHAIN ( C O N T I N U AT I O N ) PROBABILITY OF ABSORPTION If state j is an absorbing state, what is the probability of going from state i to state j? Let us denote the probability as π΄ππ . Finding the probabilities is not straightforward, especially when there are two or more absorbing states in a Markov chain. PROBABILITY OF ABSORPTION What we can do is to consider all the possibilities for the first transition and then, given the first transition, we consider the conditional probability of absorption into state j. π π΄ππ = πππ π΄ππ π=0 PROBABILITY OF ABSORPTION We can obtain the probabilities by solving a system of linear equations π π΄ππ = πππ π΄ππ for π = 0,1, β¦ , π π=0 subject to π΄ππ = 1 π΄ππ = 0 if state π is recurrent and π β π 1 EXERCISE: FIND π¨ ππ State 0 .3 1 State 3 p=0.7 0.3 State 2 p=0.7 State 1 ENDING SLIDES ABOUT MARKOV CHAINS TIME REVERSIBLE MARKOV CHAINS Consider a stationary (i.e., has been in operation for a long time) ergodic Markov Chain having transition probabilities πππ and stationary probabilities ππ . Suppose that starting at some time we trace the sequence of states going backward in time. TIME REVERSIBLE MARKOV CHAINS Starting at time n, the stochastic process ππ , ππβ1 , ππβ2 , β¦ , π0 is also a Markov Chain! The transition probabilities are ππ πππ πππ = . ππ If πππ = πππ for all π, π then the Markov Chain is time reversible. Or ππ πππ = ππ πππ which means the rate at which the process goes from i to j is equal to the rate at which it goes from j to i. FOR REPORTING: β’Hidden Markov Chains applied to data analytics/mining β’Markov Chain Monte Carlo in data fitting