* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Martingale problem approach to Markov processes
Lateral computing wikipedia , lookup
Routhian mechanics wikipedia , lookup
Scheduling (computing) wikipedia , lookup
Computational fluid dynamics wikipedia , lookup
Mathematical optimization wikipedia , lookup
Computational complexity theory wikipedia , lookup
Mathematical economics wikipedia , lookup
Computational electromagnetics wikipedia , lookup
Mathematical physics wikipedia , lookup
Perturbation theory wikipedia , lookup
Martingale problem approach to Markov processes Rajeeva L. Karandikar Director Chennai Mathematical Institute [email protected], [email protected] Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chains and Markov processes Markov chains and Markov processes have played a very important role in applications of probability theory to real world problems. The early development of probability theory focused on experiments that could be repeated large number of times leading to the law of large numbers and central limit theorems for partial sums of an i.i.d. sequence of random variables. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chains and Markov processes However, when applications to evolutionary phenomenon were considered, it was clear that one has to consider situations other than i.i.d- independent and identically distributed. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chains and Markov processes An early example of a situation where one has to go away from i.i.d. nature is a Queueing system: where even if we assume that arrivals in an hour are i.i.d. and number of services completed in an hour is also i.i.d. - the resulting queueing process is not partial sum of i.id. variables, as there can be no service when the queue is empty! Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chains and Markov processes Branching processes and birth and death processes, used to model spread of an infection or to model growth of population are also examples that do not fit into partial sums of i.i.d. random variables. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chains and Markov processes For a Markov chain, in many applications one is interested in knowing if the Markov chain admits a stationary distribution, and if so, is it unique. The discrete time Markov Chain on a finite state space are easiest to analyze: the transition probability matrix P and its eigenvalues and eigenvectors yield most of the answers. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov chain in continuous time For a Markov chain in continuous time, but finite state space S, the quantity of interest is the transition probability function pij (t) = P(Xt+s = j | Xs = i), i, j ∈ S. pij (t) satisfies pij (t + s) = ∑ pik (t)pkj (s). k∈S It can be shown that pij (t) is differentiable. Let qij = Rajeeva L. Karandikar Martingale problem approach to Markov processes d pij (t)t=0 . dt Director, Chennai Mathematical Institute Forward and Backward equations It can be shown that pij (t) satisfies the equations (shown on next slide) known as Kolmgorov backward equation and Kolmogorv forward equation Kolmgorov backward equation: d pij (t) = ∑ qik pkj (t) dt k∈S Kolmogorv forward equation: d pij (t) = ∑ pik (t)qkj dt k∈S Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute The Q matrix When the state space is countable, under mild regularity conditions, the forward equation still is valid and yields the transition probability function. Thus, the matrix Q = ((qij )) can be used to recover properties of pij (t) . Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov processes on a continuous state space Let the state space S = R or S = Rd or S be a general complete separable metric space. For a Markov process X , the function P: P(t, Xs , A) = P Xt+s ∈ A | {Xu : 0 ≤ u ≤ s, } is the transition probability function and determines the properties of the process. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Markov process and its semigroup Here too, one has Z P(t + s, x, A) = P(s, x, dy )P(t, y , A). (1) The Semi-group theory comes in useful for the study of Markov process. The operators Tt is defined for a bounded continuous f on S (f ∈ Cb (S)) by Z (Tt f )(x) = f (y )P(t, x, dy ). Using (1) one has: Tt ◦ Ts = Tt+s (the semi-group property). Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Generator of the semi-group The generator L of the semi-group is defined as follows: Let D0 (L) be the class of f ∈ Cb (S) such that t 7→ Tt f (x) is continuous for all x. Let D(L) (domain of L) be the class of f ∈ Cb (S) such that Tt f (x) − f (x) = g (x) t t↓0 lim (2) exists for all x ∈ S and g ∈ D0 (L). For f ∈ D(L), Lf is defined to be g where g is given by (2). Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Stationary distribution One of the questions of interest for a Markov process is : Does it admit a stationary distribution π? i.e. if X0 has distribution π, then for each t, distribution of Xt is also π. It can be shown that π is a stationary distribution if and only if Z (Lf )dπ = 0 ∀f ∈ D(L) Rajeeva L. Karandikar Martingale problem approach to Markov processes (3) Director, Chennai Mathematical Institute Evolution equation Notation: for a function f and measure µ on S, let hf , µi R denote the integral fd µ. For initial distribution µ0 (distribution of X0 ), let µt denote the distribution of Xt . Then (µt ) satisfies the equation hf , µt i = hf , µ0 i + Z t 0 hLf , µu idu ∀f ∈ D(L), ∀t ≥ 0. (4) Indeed, (under mild conditions on the process (Xt )) the equation (4) admits a unique solution and thus characterizes µt - the law of Xt Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Evolution equation Let (Xt ) be the Markov process with generator L and initial distribution µ0 . For g ∈ Cb (S) and 0 ≤ t < ∞, let the measure νt be defined by h i R νt (A) = E 1A (Xt ) exp( 0t g (Xs )ds) . Then it can be shown that νt satisfies hf , νt i = hf , µ0 i + Z t 0 hLf , νu idu + Z t 0 hgf , νu idu ∀f ∈ D(L) (5) and that the perturbed evolution equation (5) admits a unique solution. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Filtering theory Filtering theory: Suppose a signal (Xt ) (assumed to be Markov) is transmitted over a noisy channel and one observes Z t Yt = 0 h(Xs )ds + Wt where Wt is a wiener process. The interest is in computing the conditional distribution of Xt given {Yu : u ≤ t}: πt (A)(Y ) = E (Xt ∈ A | Yu : u ≤ t). Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute FKK and Zakai equations of filtering In early seventies, it was shown that the non linear filter satisfies an equation called the Fujisaki-Kallianpur-Kunita (FKK) equation- this is a non liner perturbation of the evolution equation (4). In a special case Zakai introduced a quantity σt (A)(Y ) such that σt satisfies a linear equation (a perturbation of (4)) and when normalized to have measure one, yields the conditional distribution πt . Hence σt has been called the unnormalized conditional distribution. Both FKK and Zakai equations are infinite dimensional stochastic differential equations. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute FKK and Zakai equations of filtering The question of uniqueness of solution to the FKK and Zakai equation is connected with uniqueness question for the perturbed evolution equation (5). The uniqueness of solution to Zakai equation and the FKK equation was shown in the case when the function h is bounded. Even in this special while the equations characterize the filter πt , a major difficulty is that like the perturbed evolution equation (5), the class of test functions is D(L). Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute FKK and Zakai equations of filtering In most cases, the class D(L) is difficult to characterize. Even in the simplest case when S = Rd and Xt is the Brownian motion, it can be shown that Cb2 (Rd ) - the class of twice continuously differentiable functions - is contained in D(L) and for f ∈ Cb2 (Rd ), 1 Lf = ∆f . 2 Thus L is an extension of the Laplacian, but L is not equal to the Laplacian. It is difficult to give a complete description of D(L) and hence a criterion which involves ∀f ∈ D(L) is not useful. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem Question: Can we restrict the operator L to a subdomain D(A) such that the restricted operator A (A = L on D(A)) characterizes the process: for example, Does Z (Lf )dπ = 0 ∀f ∈ D(A) (6) imply that π is a stationary distribution? Does the equation hf , µt i = hf , µ0 i + Z t 0 hLf , µu idu + Z t 0 hgf , µu idu ∀f ∈ D(A), (7) admit a unique solution? Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem When (Xt ) is a Markov process and L is its generator, it can be shown that Mtf = f (Xt ) − Z t 0 (Lf )(Xu )du (8) is a martingale for all f ∈ D(L). Indeed, If for a process (Xt ), Mtf is a martingale for all f ∈ D(L), then (Xt ) is the Markov process corresponding to L. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem Question: Can we choose D(A) such that the restriction A of L to D(A) satisfies the following: The requirement that Mtf = f (Xt ) − Z t 0 (Af )(Xu )du (9) is a martingale for all f ∈ D(A) implies that (Xt ) is the Markov process corresponding to L? For Brownian motion, we can take D(A) to be Cb2 (Rd ) or even Cb∞ (Rd ) and Af = 12 ∆f and the answer is Yes. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem This leads to the Martingale problem: Let A be an operator on Cb (S) with domain D(A) ⊆ Cb (S), A is a function from D(A) into Cb (S). Does there exist a process (Xt ) with r.c.l.l. paths such that Mtf = f (Xt ) − Z t 0 (Af )(Xu )du is a martingale ∀f ∈ D(A) (10) and does the requirement (10) determine the process (i.e. its law or its distribution) uniquely? If answer is YES, the martingale problem is said to be well posed. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem The martingale problem for A is said to be well posed if for all probability measures µ0 on S, there exists a process (Xt ) with r.c.l.l. paths with distribution of X0 being µ0 and Mtf = f (Xt ) − Z t 0 (Af )(Xu )du is a martingale ∀f ∈ D(A). Further, if (Yt ) is another process with r.c.l.l. paths with distribution of Y0 being µ0 and if Ntf = f (Yt ) − Z t 0 (Af )(Yu )du is a martingale ∀f ∈ D(A) then the distribution of the process (Xt ) equals distribution of the process (Yt ). Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale Problem Note that we have defined well posedness of martingale problem for an operator A without any reference to a Markov process or a semigroup. We have just started with a linear operator whose domain and range are subsets of Cb (S). Stroock-Varadhan introduced and studied Martingale problem when S = Rd and A is a second order differential operator with D(A)=Cb2 (Rd ) and constructed diffusion processes. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Diffusion processes For f ∈ D(A)=Cb2 (Rd ), let (Af )(x) = d 1 d ∂ ∂2 f ](x) + bi (x)[ f ](x) (11) a (x)[ ij ∑ ∑ 2 ij=1 ∂ xi ∂ xj ∂ xi i=1 where aij ∈ Cb (Rd ), bi ∈ Cb (Rd ) 1 ≤ i, j ≤ d (12) and for all x ∈ Rd ((aij (x))) is a (strictly) positive definite symmetric matrix. (13) Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Diffusion processes Under conditions (11), (12) and (13), Stroock-Varadhan proved that the martingale problem for A is well posed and that the resulting solution is a Markov process and the generator of the Markov process is an extension of the differential operator A. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale problem The same technique has been used to show that similar result is true in general. Under a mild regularity condition on the A, well posedness of the martingale problem in the class of processes with r.c.l.l. paths leads to its solution being a Markov process with the generator of the Markov process being an extension of A. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale problem Subsequently Eicheveria (a student of Varadhan) showed : Let S be locally compact and let martingale problem for A be well posed and suppose that D(A) is an algebra. Then Z (Lf )dπ = 0 ∀f ∈ D(A) (14) implies that π is a stationary distribution. Further hf , µt i = hf , µ0 i + Z t 0 hLf , µu idu + Z t 0 hgf , µu idu ∀f ∈ D(A), (15) admits a unique solution. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale problem The requirement that S be locally compact was removed by Karandikar and Bhatt : it was shown that (i) if D(A) is an algebra (ii) there exists a solution to the martingale problem in the class of r.c.l.l. processes (iii) if uniqueness holds in the class of measurable solutions then the results on stationary distribution and evolution equations are valid. They further extended the result to cover the case when Af may be unbounded and has discontinuities of certain kind. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Martingale problem B. V. Rao, Karandikar and Bhatt showed that if the martingale problem is well posed in the class of measurable processes, then the results on stationary distributions and evolution equations are true. These extensions were crucial for application to Filtering theory. Kallianpur, Karandikar and Bhatt used these results to prove that Fujisaki-Kallianpur-Kunita equation and Zakai equation (these are infinite dimensional stochastic differential equations) admit unique solution and thus characterize the filter. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute Filtering theory Indeed, using these techniques, Kallianpur, Karandikar and Bhatt have proven the uniqueness of solution to FKK and Zakai equation, about 25 years after these equations were derived. These techniques are also useful is proving robustness of the filter- an important property. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute References 1. A. G. Bhatt, G. Kallianpur, and R. L. Karandikar. Uniqueness and robustness of solution of measure-valued equations of nonlinear filtering. Ann. Probab., 23(4):1895 − 1938, 1995. 2. A. G. Bhatt and R. L. Karandikar. Invariant measures and evolution equations for markov processes charachterized via martingale problems. Ann. Probab., 21(4):2246 − 2268, 1993 3. A. G. Bhatt and R. L. Karandikar. Evolution equations for markov processes: application to the white-noise theory of filtering. Appl. Math. Optim., 31(3):327 − 348, 1995. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute References 4. A. G. Bhatt and R. L. Karandikar. Characterization of the optimal filter: the non-markov case. Stochastics Stochastics Rep., 66:177 − 204, 1999. 5. A. G. Bhatt, R. L. Karandikar, and B. V. Rao. On characterisation of markov processes via martingale problems. Proc. Indian Acad. Sci. Math. Sci., 116(1):83 − 96, 2006. 6. P. Echeverra. A criterion for invariant measures of markov processes. Z. Wahrsch. Verw. Gebiete, 61(1):1 − 16, 1982. 7. S. N. Ethier and T. G. Kurtz. Markov Processes: Characterization and Convergence. Wiley, New York, first edition, 1986. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute References 8. G. Kallianpur and R. L. Karandikar. Measure-valued equations for the optimum filter in finitely additive nonlinear filtering theory. Z. Wahrsch. Verw. Gebiete, 66(1):1 − 17, 1984. 9. G. Kallianpur and R. L. Karandikar. White noise calculus and nonlinear filtering theory. Ann. Probab., 13(4):1033 − 1107, 1985. 10. G. Kallianpur and R. L. Karandikar. White noise theory of Prediction, Filtering and Smoothing, volume 3 of Stochastics Monographs. Gordon and Breach, New York, first edition, 1988. Rajeeva L. Karandikar Martingale problem approach to Markov processes Director, Chennai Mathematical Institute