Download Master equation

3 Stochastic processes
in some arbitrarity high dimensional state space that permits discrete countable (possibly) infinite states so that if the probability
p(n, t )
is the proability of finding the system in state n at time t then
X
n
p(n, t ) = 1
In this case the CK-Eq reads
p(n3 , t 3 |n1 , t 1 ) =
X
n2
p(n3 , t 3 |n2 , t 2 )p(n2 , t 2 |n1 , t 1 )
The state space can be something quite abstract, e.g. the conformation of a protein, the state
of a biological cell,whatever. As long as we can count the states in principle and the Markov
assumption is fullfilled, this equation must hold.
Master equation
The Chapman Kolmorov equation is an intuitve equation but it’s equally a complicated integral equation that a function of 4 parameters namely has to fullfill. In actual applications it is
usually more common to make assumtions about the short time properties of the conditional
probability for instance concerning
p(n, t + ¢t |m, t )
what do we know about this quantity. Well, first we know that the system must be in some state
so
X
p(n, t + ¢t |m, t ) = 1
n
What else? We must also have
p(n, t |m, t ) = ±n,m
where ±n,m is zero, except for when n = m in which case this function is 1. The above equation
means that a particle cannot be in two different states at the same time. Now let’s assume that
p(n, t + ¢t |m, t ) º ¢t £ w(n|m) n 6= m
This means two things. First, we assume that a transition form state m to state n is proportional
to time when time gets small. Note also that this is only specified for transitions into different
states (m 6= n). The quantity
p(n, t + ¢t |m, t )
¢t !0
¢t
w(n|m) = lim
68
n 6= m
3 Stochastic processes
is a probability rate, e.g. how much probability per unit time is moving form m to n. The above
definition is only for n 6= m we know that
p(n, t + ¢t |m, t ) º ¢t £ w(n|m) n 6= m
is the proability that a transition is made from m to n so that
X
Q(m) = 1 ° ¢t
w(n|m)
n6=m
is the probability that no state transition is made at all in the small time interval ¢t . So altogether we can write
p(n, t + ¢t |m, t ) º ¢t w(n|m) +Q(m)±n,m
where the first term is only defined for n 6= m we can also just say that w(n|n) = 0 of course. Now
let’s see what the CK-Eq. tells us. We can write
X
p(n, t + ¢t |n0 , t 0 ) =
p(n, t + ¢t |m, t )p(m, t |n0 , t 0 )
m
X
£
§
=
¢t w(n|m) +Q(m)±n,m p(m, t |n0 , t 0 )
m
X
X
= ¢t w(n|m)p(m, t |n0 , t 0 ) + Q(m)±n,m p(m, t |n0 , t 0 )
m
m
X
= ¢t w(n|m)p(m, t |n0 , t 0 ) +Q(n)p(n, t |n0 , t 0 )
m
"
#
X
X
= ¢t w(n|m)p(m, t |n0 , t 0 ) + 1 ° ¢t
w(m|n) p(n, t |n0 , t 0 )
m
=
p(n, t |n0 , t 0 ) + ¢t
So that
@t p(n, t |n 0 , t 0 ) =
X£
m
X£
m
m6=n
w(n|m)p(m, t |n0 , t 0 ) ° w(m|n)p(n, t |n0 , t 0 )
w(n|m)p(m, t |n0 , t 0 ) ° w(m|n)p(n, t |n0 , t 0 )
§
§
This is the differential form of the Chapman Kolmogorov equation and is known as the master
equation. It informs us how the conditional proability changes as a function of time. It’s best
understood if we integrate out the condition. Recall that
X
p(n, t |n0 , t 0 )p(n0 , t ) = p(n, t )
n0
this means that multiplying the master equation by p(n0 , t 0 ) and summing over n0 gives
X£
§
@t p(n, t ) =
w(n|m, t )p(m, t ) ° w(m|n, t )p(n, t )
m
This is also the master euqation but it’s easier to interpret. The quantity p(n, t ) is the probability
of the system being in state n. How does this change over time? Well, a system can be in any
state m 6= n and move into the state m. The sytstem is in state m with probability p(m, t ) and
69
3 Stochastic processes
from any of these states m it moves to the state n with probability w(n, m)p(m, t ). However, it
could also already be in state n and leave to go to one of the other states m that proability is
X
m
w(m|n)p(n, t ).
So the master equation is a probability flux balance equation, best understood if p(n, t ) is interpreted as a concentration of probability. The change in that proability is the difference of what
comes in and what leaves.
Equilibrium and detailed balance
Typically one cannot solve the master equation. However, sometimes it is possible to solve for
the stationary solution, which means that the probability p(n, t ) is time independent so that
p(n, t ) = p ? (n). Let’s see if we can find such a condition
0=
X£
m
w(n|m, t )p(m, t ) ° w(m|n, t )p(n, t )
§
Now let’s assume that w(n|m) > 0 for all n 6= m that means that a sufficient condition for this to
be true is
w(n|m)p ? (m) = w(m|n)p ? (n)
This is the condition of detailed balance. IT means that in equilibrium there’s as much probability flux going from n to m as the other way around. From this we can already see that if the
probabiliy rates are symmetric
w(n|m) = w(m|n)
then
p ? (n) = p ? (m)
which means that the stationary distribution is constant.
Birth and Death processes
Let’s consider the 1-d Masterequation
@t p(n, t ) =
X£
m
§
w(n|m)p(m, t ) ° w(m|n)p(n, t )
for a discrete dynamical process n(t ). In many situations n(t ) is a quantity that only changes by
a single step so we have either
n ! n +1
or
n ! n °1
70