Download Duality of Markov processes with respect to a function

Duality of Markov processes
with respect to a function
Sabine Jansen
Ruhr-Universität Bochum, Germany
joint work with Noemi Kurt (TU Berlin)
Probab. Surveys 11, 59 –120 (2014)
Mini-workshop Duality of Markov processes and applications to
spatial population models,
Berlin, 6-7 November 2014
Background
Duality is a useful tool in a variety of areas – e.g., random walks with absorbing
/ reflecting barriers, queuing theory Feller, Asmussen, diffusions, interacting
particle systems Liggett, hydrodynamic limits, population genetics...
Basic idea: in order to understand the long-time behavior of some complicated
process, follow a simpler, dual process backwards in time.
Related: duality wrt a measure (potential theory Hunt ’57-58).
General theory available.
Question: general theory for duality wrt a function?
Outline
1. Duality with respect to a measure
2. Duality of linear operators. Existence and uniqueness
3. Spectrum
4. Convex geometry: cone duality.
5. Quantum mechanics
Definition
I
E and F Polish spaces (Borel σ-algebra).
I
(Xt ) and (Yt ) Markov processes with state spaces E and F
I
H : E × F → R bounded, measurable (duality function).
(Xt ) and (Yt ) are dual with respect to the function H if for all t > 0, x ∈ E
and y ∈ F ,
Ex H(Xt , y ) = Ey H(x, Yt )
Ex : (Xt ) started in x, Ey : (Yt ) started in y .
In terms of the semi-groups (Pt ) and (Qt ):
Z
0
0
Z
Pt (x, dx )H(x , y ) =
E
Qt (y , dy 0 )H(x, y 0 )
F
Finite state spaces: matrix equation (AT : transposed)
Pt H = HQtT .
Special case: H = diag(1/µ(x)):
µ(x)Pt (x, y ) = µ(y )Qt (y , x).
Duality wrt measure ≈ duality wrt diagonal function.
Pathwise duality
Fix a time horizon T > 0, starting points x ∈ E , y ∈ F . Dual processes can be
defined on common probability space so that X0 = x, Y0 = y P-a.s. and
∀t ∈ [0, T ] :
EH(x, YT ) = EH(Xt , YT −t ) = EH(XT , y ).
= weak pathwise duality. Often, processes can be coupled in such a way that
∀t ∈ [0, T ] :
H(x, YT ) = H(Xt , YT −t ) = H(XT , y )
P-a.s.
= strong pathwise duality. Examples:
I
H(x, y ) = 1(x ≤ y ) on totally ordered spaces Clifford, Sudbury ’85:
∀t ∈ [0, T ] :
x ≤ YT ⇔ Xt ≤ YT −t ⇔ XT ≤ y .
P-a.s.
I
Interacting particle systems with “graphical representations”.
I
Stochastic recursions: common “driving sequence” (Wn ), recurrence
relations
Xn+1 = f (Xn , Wn ), Yn+1 = g (Yn , WT −n ).
Ex. random walks, Wn = “up” or “down” Lindley ’52, books by
Feller, Asmussen.
Duality with respect to a measue
Discrete state spaces: if µ(x) > 0 for all x and (Pt ) and (Qt ) are time reversals
1
of each other wrt µ, then H(x, y ) = µ(x)
δx,y is a duality function. What about
continuous state spaces?
Definition: µ σ-finite measure on E = F . Processes (Xt ), (Yt ) with
semi-groups (Pt ), (Qt ) and resolvents (Rλ ), (R̂λ ) are in duality wrt µ if
I
I
For all λ, x, y , Rλ (x, ·) and R̂λ (x, ·) are abs. cont. wrt. µ.
R
R
For all λ and f , g ≥ 0, E (Pt f )g dµ = E f (Qt g )dµ.
Theorem Let (Pt ), (Qt ) be in duality wrt µ. Write
Rλ (x, dy ) = rλ (x, y )µ(dy ),
R̂λ (x, dy ) = rλ (y , x)µ(dy )
Functions y →
7 rλ (x, y ) and x 7→ rλ (x, y ) λ-excessive wrt (Rλ ) resp. (R̂λ ).
Then for all λ > 0, rλ (x, y ) is a duality function for (Xt ), (Yt ).
Blumenthal, Getoor: Markov processes and potential theory.
Duality of linear operators
Question: Relation with notions of duality in functional analysis?
Natural framework: dual pairs / duality wrt bilinear form.
Def.: V , W vector spaces. Bilinear form h·, ·i : V × W → R called
non-degenerate if right null space is trivial
NR = {w ∈ W | ∀v ∈ V : hv , w i = 0} = {0}
and left null space is trivial. (V , W , h·, ·i) is a dual pair.
Linear operators S : V → V , T : W → W are dual wrt h·, ·i if
∀v ∈ V ∀w ∈ W :
hSv , w i = hv , Tw i.
Notion useful in study of weak topologies, locally convex vector spaces. s
Application to duality of Markov processes
I
M(E ) = finite signed Borel measures on E .
I
Bilinear form associated with H:
Z
BH : M(E ) × M(F ) → R,
BH (µ, ν) :=
H(x, y )µ(dx)ν(dy ).
E ×F
I
(Pt∗ µ)(A) = (µPt )(A) =
R
E
µ(dx)Pt (x, A).
Observation: Markov semi-groups (Pt ) and (Qt ) dual wrt H iff
∀t > 0 ∀µ ∈ M(E ) ∀ν ∈ M(F ) :
BH (Pt∗ µ, ν) = BH (µ, Qt∗ ν).
H duality function for (Pt ), (Qt ) ⇔ (Pt∗ ), (Qt∗ ) dual wrt bilinear form BH .
Question: Given: (Pt ) and H, existence of a dual Markov semi-group (Qt )?
Uniqueness?
Uniqueness
Functional analysis fact: The dual of a given operator with respect to a
non-degenerate bilinear form is unique.
Consequence: if BH is non-degenerate then the dual, if it exists, is unique.
Applies to a lot of common duality functions:
I
Siegmund dual: H(x, y ) = 1(x ≤ y ), x, y ∈ R.
I
Moment dual: H(x, n) = x n , x ∈ [0, 1], n ∈ N0 .
I
Laplace dual: H(x, λ) = exp(−λx), λ, x ∈ [0, ∞).
Uniqueness in moment problems, injectivity of the Laplace transform.
Remark: Connection with kernel identities used in integrable quantum systems
(Toda, Calogero-Moser systems) related to Doob h-transforms / non-colliding
processes. Non-degeneracy does not come for free.
Ruijsenaars ’13 On positive Hilbet-Schmidt operators
Existence
Functional analysis fact: Every bounded operator in a Hilbert space has a
unique adjoint. Every bounded operator in a Banach space has a unique dual
operator. But: for general dual pairs, a given operator need not have a dual
(need continuity in suitable weak topology...)
Consequence: semi-group (Pt∗ ) need not have a dual semi-group of operators
Tt : M(F ) → M(F ).
Additional problem: dual operator semi-group need
R not come from a Markov
semi-group – need to know whether (Tt ν)(A) = F ν(dy )Qt (y , B) for Markov
semi-group (Qt ).
Existence and uniqueness criteria available for Siegmund duality in totally
ordered state spaces (stochastic monotonicity). Generalization?
Key: invariance of convex sets Möhle.
Existence, continued
M(F ) = signed Borel measures, M1,+ (F ) = probability measures. Images
nZ
o
V=
H(·, y )ν(dy ) | ν ∈ M(F ) ⊂ L∞ (E )
F
nZ
o
V1,+ =
H(·, y )ν(dy ) | ν ∈ M1,+ (F ) ⊂ V.
F
Analogous subsets W1,+ ⊂ W ⊂ L∞ (F ).
Finite state spaces: linear / convex combinations of rows / columns of H.
Proposition Let E and F be countable, discrete state spaces and P a E × E
stochastic matrix. Then
P
1. PH = HQ T has a solution Q = (Q(x, y ))x,y ∈F with z |Q(y , z)| < ∞ for
all y iff V is invariant.
2. Q can be chosen as a stochastic matrix iff V1,+ is invariant.
Siegmund duality H(x, y ) = 1(x ≤ y ) in totally ordered state spaces: V1,+ =
monotone increasing functions. Recover known relationship Siegmund duality /
stochastic monotonicity.
Existence, continued
In continuous time and space, more delicate – Chapman-Kolmogorov and
measurability issues, theorems for non-degenerate H given by Möhle.
Convex combination of the columns of H:
nZ
o
V1,+ =
H(·, y )ν(dy ) | ν ∈ M1,+ (F ) ⊂ L∞ (E ).
F
Theorem E , F locally compact, H : E × F → R continuous, non-degenerate.
Then
I
(Pt ) has a H-dual Markov semi-group if and only if V1,+ is invariant.
I
The dual (Qt ) is unique.
I
The dual is a Feller semi-group if and only if (Pt ) is.
Example: moment duality.
Spectrum
Finite state spaces: if Pt H = HQtT and H is invertible, then Pt and Qt have
the same eigenvalues.
In infinite state spaces, non-degenerate duals need not have the same spectrum.
Unless processes are reversible!
Theorem: (Pt ), (Qt ) dual with non-degenerate duality function H. Assume
that (Xt ) and (Yt ) have symmetrizing measures µ and ν. Then (Pt ) and (Qt ),
as operators in L2 (E , µ) and L2 (F , ν), are unitarily equivalent. In particular,
same spectrum.
Convex geometry & cone dual: definitions
Klebaner, Rösler, Sagitov ’07, Möhle ’11:
duality wrt a convex set rather than a function.
Simplex in Rn : non-empty convex, compact set such that every point in C is a
unique convex combination of its extremal points exC .
Choquet simplex: generalization to locally convex vector spaces; bijection
simplex ↔ probability measures on exC , unique integral representation.
Characterization via partial order defined by cone {λf | f ∈ C , λ ≥ 0}.
Proposition BH non-degenerate, H ∈ C0 (E × F ), F compact ⇒ V1,+ ⊂ C0 (E )
compact (topology of uniform convergence), Choquet simplex.
Def. C ⊂ L∞ (E ) simplex, invariant under (Pt ). Cone dual is process (Rt ) with
state space exC , duality fct
E × exC → R,
(x, e) 7→ e(x).
Observation: for non-degenerate BH , bijection F → exC , y 7→ H(·, y ), cone
dual and usual dual can be identified.
Cone dual vs. usual dual. Lifts and intertwining
More interesting: BH degenerate but V1,+ simplex. For y ∈ F , let Π(y , ·) be
the unique probability measure on exV1,+ such that
Z
H(·, y ) =
Π(y , de)e(·).
exV1,+
Transition kernel from F to “smaller” space exV1,+ .
Theorem H ∈ C0 (E × F ), F compact, V1,+ simplex, invariant under (Pt ).
1. The cone dual (Rt ) exists and is unique. It is defined on exV1,+ (Borel
σ-algebra, uniform topology); exV1,+ is Polish.
2. (Pt ) has at least one dual Markov semi-group (Qt ).
3. (Qt ) is dual to (Pt ) iff for all t, y , B
Z
Z
Qt (y , dy 0 )Π(y 0 , B) =
F
Π(y , de)Rt (e, B).
exV1,+
Intertwining relation. Usual duals are lifts of the cone dual.
In terms of processes: cone dual (Zt ), usual dual (Yt ) satisfy
P(Zt ∈ B | Yt = y ) = Π(y , B).
Quantum mechanics
Definition: H self-adjoint operator (Hamiltonian) in Hilbert space H . (Pt )
Markov semi-group in Polish state space E , symmetrizing σ-finite measure µ,
U : H → L2 (E , µ) unitary. Call ((Pt ), E , µ, U) a stochastic representation of H
if
∀t > 0 : exp(−tH) = U ∗ Pt U.
Examples: Ornstein-Uhlenbeck process vs. harmonic oscillator, simple
symmetric exclusion process vs. Heisenberg spin chain.
Earlier theorem, rephrased If two reversible processes with non-degenerate
duality function are associated with Hamiltonians H1 and H2 , then H1 and H2
are unitarily equivalent.
Strategy:
I
interpret infinitesimal generator L as quantum Hamiltonian H.
I
Interacting particle systems / quantum many-body systems:
Break down transitions into birth and death of particles / creation and
annihilation operators.
I
Look for different representations of the relevant operator algebra.
I
Often, this yields unitary equivalences and interesting dualities.
More flexible application: relax requirement of self-adjointness, reversibility.
Giardina’s talk!
Further aspects & conclusion
I
Generalized definition allowing for Feynman-Kac corrections.
I
Solutions of martingale problems.
I
Dualities and scaling limits.
Swart ’06, Alkemper, Hutzenthaler ’07,
J., Kurt ’12
I
Intertwining of Markov processes
Diaconis, Fill ’90, Carmona, Petit, Yor ’98; Huillet, Martinez ’11.
I
Duality for interacting particle systems as Fourier transforms on groups
Holley, Stroock ’79.
Conclusion: A lot of structure to explore.
Invariant subsets for Siegmund and moment duality
Convex subset V1,+ ⊂ L∞ (E ):
H(x, y )
1(x ≤ y )
x ∈E
x ∈R
y ∈F
y ∈R
xn
x ∈ [0, 1]
n ∈ N0
R
V1,+ = { F H(·, y )ν(dy ) | ν ∈ M1,+ (F )}
monotone increasing, right-continuous
functions f , lim−∞ f = 0, lim+∞ f = 1
absolutely monotone functions f w. f (1) = 1
Linear subspace V ⊂ L∞ (E ):
H(x, y )
χ(x ≤ y )
x ∈E
x ∈R
y ∈F
y ∈R
xn
x ∈ [0, 1]
n ∈ N0
R
V = { F H(·, y )ν(dy ) | ν ∈ M(F )}
bounded, right-continuous functions f with
bounded variation, lim−∞ f = 0,
lim+∞ f ∈ R
functions f ∈ C ([0, 1]) with analytic extension
F to the complex open unit disk, F in positive
Wiener algebra (⊂ H ∞ Hardy space).