Download Mikael Petersson Perturbed discrete time stochastic models

Perturbed discrete time
stochastic models
Mikael Petersson
Perturbed discrete time
stochastic models
Mikael Petersson
Abstract
In this thesis, nonlinearly perturbed stochastic models in discrete time are considered. We give algorithms for construction of asymptotic expansions with
respect to the perturbation parameter for various quantities of interest. In particular, asymptotic expansions are given for solutions of renewal equations,
quasi-stationary distributions for semi-Markov processes, and ruin probabilities for risk processes.
Keywords: Renewal equation, Perturbation, Asymptotic expansion, Regenerative process, Risk process, Semi-Markov process, Markov chain, Quasistationary distribution, Ruin probability, First hitting time, Solidarity property.
c Mikael
Petersson, Stockholm 2016
ISBN 978-91-7649-422-6
Printed in Sweden by Holmbergs, Malmö 2016
Distributor: Department of Mathematics, Stockholm University
List of Papers
The following papers, referred to in the text by their Roman numerals, are
included in this thesis.
I Silvestrov, D., Petersson, M. (2013). Exponential expansions for perturbed discrete time renewal equations. In: Frenkel, I., Karagrigoriou,
A., Lisnianski, A., Kleyner, A. (eds) Applied Reliability Engineering
and Risk Analysis: Probabilistic Models and Statistical Inference. Chapter 23, Wiley, Chichester, 349–362.
II Petersson, M. (2013). Quasi-stationary distributions for perturbed discrete time regenerative processes. Teor. Ǐmovirn. Mat. Stat., 89, 140–
155. (Also in Theory Probab. Math. Statist., 89, 2014, 153–168.)
III Petersson, M. (2014). Asymptotics of ruin probabilities for perturbed
discrete time risk processes. In: Silvestrov, D., Martin-Löf, A. (eds)
Modern Problems in Insurance Mathematics. Chapter 7, EAA Series,
Springer, Cham, 95–112.
IV Petersson, M. (2015). Quasi-stationary asymptotics for perturbed semiMarkov processes in discrete time. Research Report 2015:2, Department
of Mathematics, Stockholm University, 36 pp. and arXiv:1603.05889.
(Under revision for Methodology and Computing in Applied Probability.)
V Petersson, M. (2016). Asymptotic expansions for moment functionals of
perturbed discrete time semi-Markov processes. arXiv:1603.05891, 23
pp. (Accepted for publication in: Rančić, M., Silvestrov, S. (eds) Engineering Mathematics and Algebraic, Analysis and Stochastic Structures
for Networks, Data Classification and Optimization. Springer Proceedings in Mathematics and Statistics.)
VI Petersson, M. (2016). Asymptotics for quasi-stationary distributions of
perturbed discrete time semi-Markov processes. arXiv:1603.05895, 22
pp. (Accepted for publication in: Rančić, M., Silvestrov, S. (eds) Engineering Mathematics and Algebraic, Analysis and Stochastic Structures
for Networks, Data Classification and Optimization. Springer Proceedings in Mathematics and Statistics.)
Reprints were made with permission from the publishers.
Author’s contribution (in case of joint paper)
Paper I is based on the ideas of D. Silvestrov. The mathematical realization
was made by M. Petersson. The writing of the manuscript was done jointly.
Other sources containing parts of the thesis
Part A of the present thesis includes a modified and extended variant of the
corresponding part in the following Licentiate thesis which also contains Papers I–III:
• Petersson, M. (2013). Asymptotic expansions for perturbed discrete
time renewal equations. Licentiate Thesis, Stockholm University.
A shorter version of Paper IV can be found in:
• Petersson, M. (2015). Exponential expansions for perturbed discrete
time semi-Markov processes. In: Proceedings of the 16th Conference
of the Applied Stochastic Models and Data Analysis (ASMDA) International Society, Piraeus, Greece, 14 pp.
http://www.asmda.es/images/1_P-SG_ASMDA2015_Proceedings.pdf
Earlier report versions of Papers I and II can be found in:
• Petersson, M., Silvestrov, D. (2012). Asymptotic expansions for perturbed discrete time renewal equations and regenerative processes. Research Report 2012:12, Department of Mathematics, Stockholm University, 34 pp.
A combined report version of Papers V and VI can be found in:
• Petersson, M. (2015). Asymptotic expansions for quasi-stationary distributions of perturbed discrete time semi-Markov processes. Research
Report 2015:12, Department of Mathematics, Stockholm University, 39
pp.
Other publications
The following paper was written during the time of the author’s PhD studies,
but is not included in the thesis:
• Silvestrov, D., Petersson, M., Hössjer, O. (2016). Nonlinearly perturbed
birth-death-type models. Research Report 2016:6, Department of Mathematics, Stockholm University, 63 pp. and arXiv:1604.02295.
Contents
Abstract
iv
List of Papers
v
Part A
9
1
Introduction
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Summary of Papers . . . . . . . . . . . . . . . . . . . . . . .
9
9
13
2
Mathematical Introduction
2.1 Discrete Time Renewal Equations . .
2.2 Discrete Time Regenerative Processes
2.3 Regenerative Stopping Times . . . . .
2.4 Quasi-Stationary Distributions . . . .
2.5 Discrete Time Semi-Markov Processes
2.6 Discrete Time Risk Processes . . . . .
.
.
.
.
.
.
15
15
16
16
18
18
20
.
.
.
.
22
22
24
26
28
4
Examples
4.1 An Epidemic SIS Model . . . . . . . . . . . . . . . . . . . .
4.2 The Wright-Fisher Model with Selection . . . . . . . . . . . .
4.3 An Alternating Semi-Markov Process . . . . . . . . . . . . .
30
30
32
33
5
Conclusion
34
3
Summary of Main Results
3.1 Perturbed Renewal Equations . . .
3.2 Perturbed Regenerative Processes
3.3 Perturbed Semi-Markov Processes
3.4 Perturbed Risk Processes . . . . .
Sammanfattning
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
37
Tack
39
References
41
Part B
47
Papers I–VI
The papers are not included in the electronic version of the thesis.
Part A
1. Introduction
Let us begin by outlining the structure of Part A. In Section 1.1, we give a
background of the problem studied in the present thesis by reviewing the literature of closely related research areas. A short summary of each of the papers
included in the thesis is given in Section 1.2. Then, in Section 2, we give
a mathematical introduction where we define the most important models for
our studies. In Section 3, we summarize the main results in a rather informal
manner without going too deeply into mathematical details. Some examples
of more specific models where our results can be used are discussed in Section
4. Finally, in Section 5, we make some concluding remarks and suggest some
directions for future studies.
1.1
Background
Renewal equations, in discrete or continuous time, arise in many applications
such as risk theory, queuing systems, reliability models, population dynamics,
and epidemic models. Often some quantity of interest related to these models
is the solution of a renewal equation and the values of this quantity for large
time values are objects of interest.
The basic limit theorem for the discrete time renewal equation is given in
Erdös, Feller and Pollard (1949) and Feller (1950). This result gives conditions
under which the solution of the renewal equation converges as time tends to
infinity and gives an expression for the limit. The corresponding result for
continuous time renewal equations is given in its final form in Feller (1966,
1971).
One of the main reasons for the importance of the renewal equation is that
one-dimensional distributions of regenerative processes satisfy renewal equations. This type of processes was introduced by Smith (1955) and cover a large
class of interesting processes, in particular, Markov chains and semi-Markov
processes with discrete state spaces. Using the fact that such processes regen9
erate at return times into some fixed state, we can use the renewal theorem to
obtain ergodic theorems. Moreover, using the method of artificial regeneration developed in Kovalenko (1977), Athreya and Ney (1978), and Nummelin
(1978) it is sometimes possible to apply the tools of renewal theory to obtain
ergodic theorems for Markov processes with general state spaces.
A class of regenerative processes which will be given much attention in
this thesis is discrete time semi-Markov processes on a finite state space. For
an introduction to the theory of such processes we refer to Barbu, Boussemart,
and Limnios (2004) and Barbu and Limnios (2008).
A subject of special interest in our work is the long-time behavior of regenerative processes which describe stochastic systems with finite lifetimes.
Usually in this case, the lifetime of the system is the first time the process hits
some special absorbing subset of the state space. This means that the stationary distribution will be concentrated on this absorbing subset. However, if the
time before this happens is long, one can often observe something that resembles a stationary distribution over the non-absorbing states before absorption
takes place. This type of behavior is called a quasi-stationary phenomenon.
The extensive research related to quasi-stationary phenomena started in the
1960’s where the papers by Vere-Jones (1962), Kingman (1963), Darroch and
Seneta (1965), and Seneta and Vere-Jones (1966) are examples of some important early works on Markov chains. Extensions to semi-Markov processes
are presented in Cheong (1968, 1970) and Flaspohler and Holmes (1972). Additional references to works in this area can be found in, for example, the book
by Collet, Martínez and San Martín (2013). A survey on quasi-stationary distributions for models with discrete state spaces are given by van Doorn and
Pollett (2013).
A classical application of the renewal theorem is to approximate the ruin
probability of a Cramér-Lundberg risk model which describes the evolution
of capital of an insurance company. This is a continuous time model which
was originally studied by Lundberg (1903, 1926, 1932) and Cramér (1930,
1955) without the use of renewal theory. The main result of these studies is
the Cramér-Lundberg approximation which gives the limit of a normalized
ruin probability as the initial capital tends to infinity. Later, Feller (1966)
proved this result using the renewal theorem. The discrete time analogue of
the Cramér-Lundberg model, which is often called the compound binomial
model in the literature, was introduced by Gerber (1988) and further studied
by, for example, Shiu (1989) and Willmot (1993). This model can also be interpreted in terms of the number of customers in a queuing system. We refer
to Li, Lu and Garrido (2009) for a review of the literature related to discrete
time risk models.
Over the years, renewal theory has been generalized in several directions.
10
A generalization which deals with perturbed renewal equations plays an important role for the results of this thesis. A perturbed renewal equation arises when
a probability model, where some related quantity satisfies a renewal equation,
also depends on a small perturbation parameter. In this case, one is often interested in the asymptotic behavior for large time values and small values of the
perturbation parameter.
The studies of the perturbed renewal equation started with the works of
Silvestrov (1976, 1978, 1979). In these papers, the basic limit theorem was
generalized for perturbed continuous time renewal equations. Later, starting
with Silvestrov (1995), this theory was further developed by the construction
of exponential asymptotic expansions for the solution of the renewal equation.
This provides a framework for analyzing stochastic systems with nonlinear
perturbations as illustrated by Gyllenberg and Silvestrov (1999, 2000a, b). The
book by Gyllenberg and Silvestrov (2008) collects the results in this line of research until this year together with some new results. This book also contains
an extensive bibliography of works in related areas. Further generalizations are
investigated by Englund (2001) and Ni (2011, 2014) who consider perturbations of a non-polynomial type. Related work has also been done by Blanchet
and Zwart (2010) who gives asymptotic expansions for a perturbed renewal
equation under conditions where some moments of the distribution generating
the renewal equation are allowed to be infinite.
The corresponding studies for perturbed discrete time renewal equations
started in Gyllenberg and Silvestrov (1994) where the theory was used to describe a quasi-stationary behavior for a meta-population model. Asymptotic
expansions for the discrete time model was first studied by Englund and Silvestrov (1997). The perturbed discrete time renewal equation has also been
considered in Silvestrov (2000) and Englund (2001).
In the literature cited above, the perturbed renewal equation and its applications have received much more attention for the continuous time case compared to the discrete time case. However, the discrete time case is interesting
in its own right with its own special features, and methods used for continuous time models do not necessarily translate directly into discrete time models.
Also, sometimes a discrete time model gives a more intuitive interpretation, for
example, when some stochastic system is only observed at given time points,
such as days or months. Furthermore, in some situations, it may be an advantage to use a discrete approximation of a continuous time model, see, for
instance, Mode and Pickens (1988), Dickson, Egídio dos Reis, and Waters
(1995), and Cossette, Landriault, and Marceau (2004).
The aim of the present thesis is to thoroughly investigate the asymptotic
behavior of the perturbed discrete time renewal equation and to apply this theory in order to study different asymptotic properties of perturbed discrete time
11
regenerative processes, risk processes, Markov chains, and semi-Markov processes.
There exists a large set of literature where different types of perturbations
have been studied for Markov chains and semi-Markov processes under various assumptions. Let us first review the literature concerned with perturbed
discrete time Markov chains on a finite state space. To a large extent, this area
of research emerged from the studies of large stochastic systems with weak
interactions. In these studies, it is assumed that the system can be partitioned
into a finite number of subsystems where interactions within each subsystem
are strong while interactions between different subsystems are weak. This
idea was formalized by Simon and Ando (1961) who considered the weak interaction link between the subsystems as a perturbation parameter. In this paper, it was noted that equilibrium within each subsystem was achieved much
faster than for the whole system. In particular, it was proposed that the stationary distribution for the whole system could be approximated by using local
calculations for each subsystem separately combined with a coupling matrix
relating the different subsystems to each other. This method is often called
the aggregation technique. This idea and several generalizations of it have
later been employed to find approximations of stationary distributions, see, for
example, Pervozvanskiı̆ and Smirnov (1974), Courtois and Louchard (1976),
Delebecque (1983), Courtois and Semal (1984), and Vantilborgh (1985).
In the works mentioned above, the main objective is to find computationally tractable methods for approximating the stationary distribution of a
Markov chain with a large state space. However, these studies also stimulated
research concerned with other aspects of stationary distributions for perturbed
Markov chains. For example, the sensitivity of the stationary distribution subject to small perturbations in the transition probabilities has been studied by
Schweitzer (1968), Stewart (1991), and Hunter (2005) and a perturbation technique is used to calculate the exact stationary distribution in Hunter (1991).
Attention has also been given to several other types of characteristics. For
example, first hitting times are studied in Latouche and Louchard (1978), Latouche (1991), and Hassin and Haviv (1992). The fundamental matrix and the
closely related deviation matrix are considered in Lasserre (1994), Avrachenkov
and Lasserre (1999), and Avrachenkov and Haviv (2004). Multi-step transition
probabilities are objects of interest in Gaı̆tsgori and Pervozvanskiı̆ (1975) and
Yin and Zhang (2003).
The book by Avrachenkov, Filar, and Howlett (2013) contains a chapter
devoted to perturbed discrete time Markov chains on a finite state space where
this line of research is discussed thoroughly and several results from the above
mentioned papers are presented.
Finally, let us mention some further examples of works where perturbed
12
Markov chain and semi-Markov processes are investigated. Continuous time
Markov chains on a finite state space are considered in Phillips and Kokotovic
(1981), Coderch, Willsky, Sastry, and Castanon (1983), Rohlicek and Willsky
(1988), and Yin and Zhang (1998). Continuous time semi-Markov processes
on a finite state space are investigated thoroughly in Gyllenberg and Silvestrov
(1999, 2000b, 2008) and Silverstrov, D. and Silvestrov, S. (2015). Some results for Markov chains and semi-Markov processes on a countable or general
measurable state space are given in Kartashov (1996), Altman, Avrachenkov
and Núñez-Queija (2004), and Koroliuk and Limnios (2005, 2010).
1.2
Summary of Papers
We now give a short summary of each of the six papers contained in the present
thesis. In order to avoid repetitions, let us mention here that for all asymptotic
expansions obtained in the papers, we also give explicit recursive formulas for
calculating its coefficients.
Paper I
In this paper, we study the asymptotic behavior of the solution of a perturbed
discrete time renewal equation. The main result gives asymptotic expansions
of exponential type for this solution. We discuss how the results can be applied to obtain different types of ergodic theorems for regenerative processes
with regenerative stopping times. Furthermore, examples related to queuing
systems and risk processes are given. To make the paper more self-readable,
we also include a derivation of the renewal equation for the ruin probability of
a discrete time risk process.
Paper II
In this paper, perturbed discrete time regenerative processes with regenerative
stopping time are considered. The regenerative stopping time is interpreted as
the lifetime of some stochastic system. For each fixed value of the perturbation parameter, we can find quasi-stationary distributions for such processes by
applying the renewal theorem. The main result of the paper is an asymptotic
power series expansion for the quasi-stationary distribution with respect to the
perturbation parameter. The results are illustrated by explicit calculations for
an alternating regenerative process. We also discuss applications of the results
for asymptotic analysis of ruin probabilities.
13
Paper III
In this paper, we investigate the asymptotic behavior of the infinite time horizon ruin probability of a perturbed discrete time risk process. By applying the
results obtained in Paper I, an exponential asymptotic expansion for the ruin
probability is obtained. We then further generalize this result by also constructing an asymptotic expansion for the renewal limit considered as a function of
the perturbation parameter. This gives approximations of the ruin probability which have zero asymptotic relative error under some balancing condition
on the rate at which the initial capital tends to infinity and the perturbation
parameter tends to zero.
Paper IV
In this paper, we study perturbed discrete time semi-Markov processes on a
state space which consists of one finite communicating class and, in addition,
one absorbing state. Our main interest is the asymptotic behavior of the joint
probability of the position of the process and the event that the process has not
yet been absorbed. By applying the results of Paper I, we obtain exponential
asymptotic expansions for these probabilities. As an intermediate result, which
is interesting in its own right, we construct asymptotic expansions for moment
functionals of first hitting times.
Paper V
In this paper, we consider the same model as in Paper IV. Our interest here
lies in certain moment functionals of mixed power-exponential type which are
important for our studies of quasi-stationary distributions. Under some conditions, these moment functionals can be expanded in asymptotic power series
with respect to the perturbation parameter. The main results show how the
coefficients in these expansions can be computed from the coefficients in the
expansions of certain moment functionals of transition probabilities.
Paper VI
In this paper, the asymptotic expansions for moment functionals obtained in
Paper V play a central role. Using these expansions we construct asymptotic power series expansions for quasi-stationary distributions of perturbed
discrete time semi-Markov processes. This is done by applying the technique
developed for perturbed regenerative processes in Paper II. As a particularly
important special case, the results can be used for discrete time Markov chains.
In order to demonstrate this, we also include an illustrative numerical example.
14
2. Mathematical Introduction
In this section, we define discrete time renewal equations and describe some of
its applications which are relevant for the theoretical results presented in this
thesis. In order to keep the presentation on an appropriate but not too technical
level, we allow ourselves to make some small simplifications at some places.
2.1
Discrete Time Renewal Equations
Let q(n), n = 0, 1, . . . , be a sequence of real valued numbers and let f (n), n =
0, 1, . . . , be a sequence of non-negative real valued numbers satisfying F(∞) ≤
1 and f (0) < F(∞), where F(∞) = ∑∞
n=0 f (n). Consider the following recursive relation,
n
x(n) = q(n) + ∑ x(n − k) f (k), n = 0, 1, . . .
(2.1)
k=0
Relation (2.1) is called a discrete time renewal equation.
The renewal equation is important for its applications in probability theory.
Often some characteristic of a probability model is the solution of a renewal
equation. Examples of this are given in what follows.
The quantity f = 1 − F(∞) is called the defect of f (n). If f = 0, then
f (n) is the probability distribution of a random variable. In the case f > 0, we
can interpret f (n) as the probability distribution of a random variable which
takes the value ∞ with probability f . If f = 0, then f (n) is called a proper
distribution and if f > 0 it is called an improper distribution. Furthermore,
we call the corresponding renewal equation proper or improper depending on
whether f = 0 or f > 0.
The classical discrete time renewal theorem is an important asymptotic
result which states that if f (n) is a non-periodic proper distribution with finite
∞
expectation m = ∑∞
n=0 n f (n) and ∑n=0 |q(n)| < ∞, then
x(n) →
1 ∞
∑ q(k) < ∞, as n → ∞.
m k=0
This result and various modifications of it can often be applied to obtain
different types of limit theorems for stochastic processes. In particular, for a
discrete time Markov-type process, the one-dimensional distribution at time n
usually satisfies a renewal equation. This makes the renewal theorem useful in
order to obtain ergodic theorems for such processes.
15
2.2
Discrete Time Regenerative Processes
In this section we define regenerative processes in discrete time and show how
they are related to the renewal equation.
Let Z(n), n = 0, 1, . . . , be a stochastic process taking values in some set
X. For simplicity, let us assume that X = {0, 1, 2, . . .} is a countable set. Let
0 = τ0 < τ1 < τ2 < · · · , be a sequence of proper random variables defined
on the same probability space as Z(n). The process Z(n), n = 0, 1, . . . , is a
regenerative process with regeneration times τ0 , τ1 , . . . , if the following holds:
(a) For every k = 0, 1, . . . , the process Z(τk + n), n = 0, 1, . . . , and the random sequence τk+n − τk , n = 0, 1, . . . , are independent of the random sequence Z(n ∧ (τk − 1)), n = 0, 1, . . . , and the random variables τ0 , . . . , τk .
(b) The joint finite-dimensional distributions of the process Z(τk + n), n =
0, 1, . . . , and the random sequence τk+n − τk , n = 0, 1, . . . , are the same
for every k = 0, 1, . . .
Intuitively, a regenerative process is a process such that there exist random
times where the future of the process is independent of the past and the processes starting from these random times are probabilistic copies of the process
starting from time zero.
It can be shown that for any i ∈ X, the probabilities Qi (n) = P{Z(n) = i}
satisfy the renewal equation
n
Qi (n) = qi (n) + ∑ Qi (n − k) f (k), n = 0, 1, . . . ,
k=0
where
qi (n) = P{Z(n) = i, τ1 > n},
f (k) = P{τ1 = k}.
A homogeneous Markov chain on a discrete state space with one class of
communicating states is one example of a regenerative process. In this case,
the regeneration times are successive return times to the initial state. This is
illustrated in Figure 2.1. Semi-Markov processes, which will be discussed in
Section 2.5, constitute another important class of regenerative processes.
2.3
Regenerative Stopping Times
It is often of interest to analyze the long-time behavior of a process which
terminates at some point, but the expected time until this happens is large. For
example, in a population dynamics model we might know that the population
eventually will go extinct, but that it is expected that the population will persist
16
Z(n)
5
4
3
2
1
n
τ0
τ1
τ2
τ3
Figure 2.1: Trajectory of a regenerative process.
for a long time. For the model of regenerative processes this would mean that
τn = ∞ for some n. But such a process is not a regenerative process according
to the definition given in Section 2.2. In this section, we describe how this
problem still can be analyzed by renewal theory by introducing a special kind
of random variable for the lifetime of the process.
Let µ be a random variable defined on the same probability space as the
regenerative process Z(n) and taking values in the set {1, 2, . . . , ∞}. We say
that µ is a regenerative stopping time associated with the regenerative process
Z(n) if the probabilities Pi (n) = P{Z(n) = i, µ > n}, n = 0, 1, . . . , satisfy the
renewal equation
n
Pi (n) = qi (n) + ∑ Pi (n − k) f (k), n = 0, 1, . . . ,
(2.2)
k=0
where
qi (n) = P{Z(n) = i, µ > n, τ1 > n},
f (k) = P{τ1 = k, µ > τ1 }.
This means that the regenerative stopping time possesses a memoryless
property at times of regeneration. Also note that f = P{µ ≤ τ1 }, that is, the
defect of f (k) is equal to the stopping probability in one regeneration period.
A typical example of a regenerative stopping time is the first time the process hits some special subset of the state space. For instance, the stopping time
can be the first hitting time of some particular fixed state or the first time the
process exceeds some fixed level.
We can use a regenerative stopping time µ to describe the random lifetime
of a stochastic system. For example, µ can be the time of extinction of some
population or the time of extinction of an epidemic.
17
2.4
Quasi-Stationary Distributions
When describing the long-time behavior of a stochastic process, it is often of
interest to characterize a stationary distribution. For a stochastic process describing a system with a finite lifetime, such a distribution will be degenerated.
However, before the lifetime of the system goes to an end, one can often observe something that resembles a stationary distribution.
Consider the model of regenerative processes with regenerative stopping
time where the stopping time µ represents the lifetime of some stochastic system. If we expect the lifetime to be large, it can be of interest to determine a
distribution πi , i ∈ X, and some conditions under which we have
P{Z(n) = i | µ > n} → πi , as n → ∞, i ∈ X.
Such a distribution is called a quasi-stationary distribution.
2.5
Discrete Time Semi-Markov Processes
In this section, we define discrete time semi-Markov processes on a finite state
space.
In order to define a semi-Markov process, we first introduce a special type
of Markov chain. Let (ηn , κn ), n = 0, 1, . . . , be a homogeneous discrete time
Markov chain on the state space X × N, where X = {0, 1, . . . , N} and N =
{1, 2, . . .}. Furthermore, let us assume that its transition probabilities
Qi j (k) = P{ηn+1 = j, κn+1 = k | ηn = i, κn = l}, k, l ∈ N, i, j ∈ X,
do not depend on the current state of the second component. Such a process is
called a discrete time Markov renewal process.
Let us also define ν(n) = max{k ≥ 0 : τ(k) ≤ n}, where τ(0) = 0 and
τ(n) = κ1 + · · · + κn , for n ≥ 1. Now, we define
ξ (n) = ην(n) , n = 0, 1, . . .
The process ξ (n) is called a discrete time semi-Markov process with initial
distribution Qi = P{η0 = i} and transition probabilities Qi j (n). By definition,
κn are the times between successive moments of jumps, τ(n) are the moments
of the jumps, and ν(n) are the number of jumps in the interval [0, n]. Furthermore, the process ηn , n = 0, 1, . . . , is a homogeneous Markov chain on the state
space X with transition probabilities
pi j = ∑∞
k=1 Qi j (k), i, j ∈ X,
18
ξ (n)
(η0 , κ0 ) = (3, 0)
(η1 , κ1 ) = (4, 3)
(η2 , κ2 ) = (1, 6)
(η3 , κ3 ) = (1, 2)
(η4 , κ4 ) = (2, 4)
4
..
.
3
..
.
2
1
n
5
κ1
10
κ2
κ3
15
κ4
Figure 2.2: Trajectory of a semi-Markov process.
and it is called the embedded Markov chain.
An illustration of a semi-Markov process is given in Figure 2.2.
Let us now define random variables called first hitting times. The first
hitting time of state j ∈ X for the embedded Markov chain ηn is defined by
ν j = min{n ∈ N : ηn = j}.
The first hitting time of state j ∈ X for the semi-Markov process ξ (n) is
defined by
νj
µ j = τ(ν j ) = ∑k=1
κk .
Under some conditions, a semi-Markov process is a special case of a regenerative process. Indeed, let us assume that the state space X is a communicating class for the embedded Markov chain ηn and that the initial distribution
of ξ (n) is concentrated at some state i ∈ X. Then ξ (n) is a regenerative process
and successive return times to the initial state i serve as regeneration times.
In the present thesis, we are interested in the behavior of the semi-Markov
process ξ (n) before it hits state 0 for the first time when starting from some
state i 6= 0. Therefore, the following probabilities play an important role,
Pi j (n) = Pi {ξ (n) = j, µ0 > n}, n = 0, 1, . . . , i, j 6= 0,
where Pi denotes conditional probabilities given that {η0 = i}.
19
It can be shown that the probabilities Pi j (n) satisfy the following renewal
equation,
n
Pi j (n) = hi j (n) + ∑ Pi j (n − k)gi (k), n = 0, 1, . . . , i, j 6= 0,
(2.3)
k=0
where
hi j (n) = Pi {ξ (n) = j, µ0 > n, µi > n},
gi (k) = Pi {µi = k, ν0 > νi }.
Thus, µ0 is a regenerative stopping time. Also note that the defect of the
distribution gi (k) is given by gi = Pi {ν0 ≤ νi }, that is, the defect is equal to the
probability of hitting state 0 before returning to state i.
Let us also mention that a Markov chain is a special case of a semi-Markov
process. Indeed, let ηn , n = 0, 1, . . . , be a homogeneous Markov chain on the
state space X, with transition probabilities
pi j = P{ηn+1 = j | ηn = i}, i, j ∈ X.
Let us define the transition probabilities of the Markov renewal process as
follows,
pi j n = 1, i, j ∈ X,
Qi j (n) =
0
otherwise.
Then, by definition, ξ (n) = ηn .
2.6
Discrete Time Risk Processes
In this section, we describe a model which has applications in, for example,
queuing theory and risk theory. In queuing theory, it can be used to describe
the number of customers in a system where customers arrive individually but
are served in groups. In risk theory, it can be used to model the evolution
of capital of an insurance company. In this section, we shall use the latter
interpretation.
Let X1 , X2 , . . . , be a sequence of random variables which are non-negative,
integer-valued, independent, and identically distributed. For a non-negative
integer u, let
n
Zu (0) = u,
and
Zu (n) = u + n − ∑ Xn , n = 1, 2, . . .
(2.4)
k=1
We call the process defined by relation (2.4) a discrete time risk process.
We can interpret u as a starting capital and Xn as the claim costs at time n,
20
Zu (n)
u
n
Figure 2.3: Trajectory of a risk process.
where Xn = 0 means that there are no claims at time n. In this model, capital is measured in units equivalent to expected premiums per time unit. An
illustration of such a process is given in Figure 2.3 where the downward jumps
correspond to claim arrivals.
It is convenient to characterize the risk process by
p = P{X1 > 0} and
g(n) = P{X1 = n | X1 > 0}, n = 0, 1, . . . ,
which we call the claim probability and the claim size distribution, respectively. We also define the expected claim size by µ = ∑∞
n=0 ng(n) and the loading rate by α = pµ.
An object of interest is the probability that Zu (n) ever falls below zero
given an initial capital u. This probability is called the ruin probability and is
defined by
Ψ(u) = P{min Zu (n) < 0}, u = 0, 1, . . .
n≥0
If α ≥ 1, it can be shown that Ψ(u) = 1 for all u ≥ 0. If α ≤ 1, it can
be shown (see Section 23.5 in Paper I) that the ruin probability satisfies the
renewal equation
u
Ψ(u) = q(u) + ∑ Ψ(u − k) f (k), u = 0, 1, . . . ,
(2.5)
k=0
where
k
∞
q(u) = p
∑
(1 − G(k)),
f (k) = p(1 − G(k)),
G(k) =
k=u+1
∑ g(n).
n=0
Note here that the defect of f (n) takes the form f = 1 − α.
21
As an example, it can be interesting to note that in the case where the claim
size distribution is concentrated at 2, the model reduces to the model for the
classical gambler’s ruin problem.
3. Summary of Main Results
In this section, we summarize the main results of the present thesis. The results
will be presented in a somewhat informal manner and we will not go into
details about the conditions under which the results hold but only mention the
most important assumptions. For details, we refer to the papers I–VI.
3.1
Perturbed Renewal Equations
For every ε ≥ 0, let q(ε) (n), n = 0, 1, . . . , be a sequence of real-valued numbers and let f (ε) (n), n = 0, 1, . . ., be a probability distribution which may be
improper. Assume that q(ε) (n) and f (ε) (n) converge pointwise (i.e. for every n = 0, 1, . . .) to q(0) (n) and f (0) (n), respectively, as ε → 0. Consider the
following perturbed discrete time renewal equation,
n
x(ε) (n) = q(ε) (n) + ∑ x(ε) (n − k) f (ε) (k), n = 0, 1, . . .
(3.1)
k=0
We are interested in the asymptotic behavior of the solution x(ε) (n) of
equation (3.1) as n → ∞ and ε → 0. It turns out that the defects f (ε) =
(ε) (n) have a large influence on the structure of this asymptotic be1 − ∑∞
n=0 f
havior. We can separate between three different cases:
(a) Stationary case: f (ε) = 0 for all ε ≥ 0.
(b) Pseudo-stationary case: 0 < f (ε) → 0 as ε → 0.
(c) Quasi-stationary case: f (ε) > 0 for all ε ≥ 0.
These three cases have different interpretations depending on the model
under consideration, as we will see below.
Let us define the following functions,
22
r ρn (ε)
φ (ε) (ρ, r) = ∑∞
(n), ρ ∈ R, r = 0, 1, . . . ,
n=0 n e f
(3.2)
r ρn (ε)
ω (ε) (ρ, r) = ∑∞
n=0 n e q (n), ρ ∈ R, r = 0, 1, . . .
(3.3)
In what follows, we will refer to these quantities as mixed power-exponential
moment functionals. In particular, note that φ (ε) (ρ, 0) is the moment generating function for the (possibly improper) distribution f (ε) (n).
Of crucial importance for our results is the solution, with respect to ρ, of
the following equation,
φ (ε) (ρ, 0) = 1.
(3.4)
We call (3.4) the characteristic equation. If it is assumed that φ (ε) (δ , 0) ∈
(1, ∞) for some δ > 0, then there exists a unique non-negative solution ρ (ε) of
this equation.
The solution of the characteristic equation is important since it enables us
to transform improper renewal equations into proper ones, which are mathematically simpler to handle. Indeed, multiplying both sides of equation (3.1)
(ε)
with eρ n we obtain a renewal equation where the distribution is given by
(ε)
eρ n f (ε) (n). Since ρ (ε) is the solution of the characteristic equation, it follows from (3.2) and (3.4) that this is a proper distribution.
In Paper I, it is shown that for any non-negative integer valued function
n(ε) , such that n(ε) → ∞ as ε → 0, we have
eρ
(ε) n(ε)
x(ε) (n(ε) ) → x̃(0) , as ε → 0,
where
x̃(0) =
(3.5)
ω (0) (ρ (0) , 0)
.
φ (0) (ρ (0) , 1)
A good property of relation (3.5) is that it allows n → ∞ and ε → 0 in an
arbitrary manner. However, a serious drawback is that the quantity ρ (ε) is only
given implicitly as the solution of the nonlinear equation (3.4). In order to get
more explicit asymptotic relations, we can construct an asymptotic expansion
for ρ (ε) . In order to do this, we need to impose a stronger perturbation condition on the distributions f (ε) (n). Specifically, it is assumed that for some k ≥ 1,
we have for r = 0, . . . , k,
φ (ε) (ρ (0) , r) = φ (0) (ρ (0) , r) + a1,r ε + · · · + ak−r,r ε k−r + o(ε k−r ).
(3.6)
Then, as is shown in Paper I, the root of the characteristic equation has the
following asymptotic expansion,
ρ (ε) = ρ (0) + c1 ε + · · · + ck ε k + o(ε k ),
(3.7)
where the coefficients c1 , . . . , ck can be calculated from recursive formulas
which are rational functions of the coefficients in the expansions given by
equation (3.6).
23
The first coefficient ρ (0) in the asymptotic expansion (3.7) is the limit of
ρ (ε) as ε → 0. The second coefficient c1 is a measure of the sensitivity of
ρ (ε) with respect to small perturbations from ε = 0. The higher order coefficients c2 , . . . , ck can be useful in order to improve numerical approximations of
ρ (ε) for values of ε which are not small enough for higher order terms in the
expansion to be neglected.
Now, let us assume that n(ε) is a non-negative integer valued function of ε
such that
(3.8)
ε r n(ε) → λr ∈ [0, ∞), for some 1 ≤ r ≤ k.
Using the expansion (3.7) in relation (3.5) and assuming that the balancing
condition (3.8) holds, we get the following asymptotic expansion of exponential type,
x(ε) (n(ε) )
→ e−cr λr x̃(0) , as ε → 0.
exp(−(ρ (0) + c1 ε + · · · + cr−1 ε r−1 )n(ε) )
(3.9)
Relation (3.9) is the main asymptotic result of Paper I.
3.2
Perturbed Regenerative Processes
For every ε ≥ 0, let Z (ε) (n), n = 0, 1, . . ., be a discrete time regenerative process
(ε)
(ε)
with regeneration times 0 = τ0 < τ1 < · · · , and regenerative stopping time
µ (ε) , as defined in Sections 2.2 and 2.3. However, let us now assume that the
process takes values in a general measurable state space (X, Γ).
Now, let
(ε)
q(ε) (n, A) = P{Z (ε) (n) ∈ A, µ (ε) > n, τ1 > n}, n = 0, 1, . . . , A ∈ Γ,
(ε)
(ε)
f (ε) (n) = P{τ1 = n, µ (ε) > τ1 }, n = 0, 1, . . .
Assume that the functions f (ε) (n) and q(ε) (n, A) converge pointwise to
and q(0) (n, A), respectively, as ε → 0, at least for all sets A in some
subset Γ0 ⊆ Γ. In this case, we get a perturbed version of the renewal equation
(2.2), that is,
f (0) (n)
n
P(ε) (n, A) = q(ε) (n, A) + ∑ P(ε) (n − k, A) f (ε) (k), n = 0, 1, . . . ,
(3.10)
k=0
where
P(ε) (n, A) = P{Z (ε) (n) ∈ A, µ (ε) > n}, n = 0, 1, . . .
Our interest here is to study the asymptotic behavior of the probabilities
as n → ∞ and ε → 0. This asymptotic behavior is different depending on which of the cases (a)–(c) described in Section 3.1 that holds. In the
P(ε) (n, A)
24
stationary case (a) we have that the regenerative stopping time µ (ε) = ∞ almost
surely for all ε ≥ 0. In the pseudo-stationary case (b) we have that µ (ε) → ∞
in probability as ε → 0. In the quasi-stationary case (c), the stopping times
µ (ε) are stochastically bounded as ε → 0.
Conditions imposed for regenerative processes are formulated in terms of
the functions q(ε) (n, A) and f (ε) (n). This means that the results obtained for
perturbed renewal equations in Paper I are directly applicable for regenerative
processes. In particular, we get an asymptotic relation corresponding to (3.9)
for the probabilities P(ε) (n, A).
Mixed power-exponential moment functionals for the functions f (ε) (n)
and q(ε) (n, A) in the renewal equation (3.10) are defined in analogy with (3.2)
and (3.3) and are denoted by φ (ε) (ρ, r) and ω (ε) (ρ, r, A), respectively. Let us
also define
ω (ε) (ρ (ε) , 0, A)
π̃ (ε) (A) =
, A ∈ Γ, ε ≥ 0.
(3.11)
φ (ε) (ρ (ε) , 1)
Then, if n(ε) → ∞ as ε → 0 in such a way that the balancing relation (3.8)
holds, then we have the following exponential asymptotic expansion,
P{Z (ε) (n(ε) ) ∈ A, µ (ε) > n(ε) }
→ e−cr λr π̃ (0) (A), as ε → 0.
exp(−(ρ (0) + c1 ε + · · · + cr−1 ε r−1 )n(ε) )
In the context of regenerative processes, it is also of interest to consider
the renewal limit (3.11) as a function of ε. This is because normalized renewal
limits can be interpreted as quasi-stationary probabilities for regenerative processes. Indeed, for a fixed ε ≥ 0, the classical discrete time renewal theorem,
applied to a transformed version of (3.10), gives the following limit,
eρ
(ε) n
P(ε) (n, A) → π̃ (ε) (A) as n → ∞, A ∈ Γ.
(3.12)
Using (3.12) and the definition of conditional probability, it follows that
P{Z (ε) (n) ∈ A | µ (ε) > n} → π (ε) (A), as n → ∞, A ∈ Γ,
where
π (ε) (A) =
π̃ (ε) (A)
ω (ε) (ρ (ε) , 0, A)
=
.
π̃ (ε) (X) ω (ε) (ρ (ε) , 0, X)
(3.13)
Formula (3.13) can be used to obtain asymptotic power-series expansions
for quasi-stationary probabilities of perturbed regenerative processes. In order
to achieve this, we need to assume that (3.6) holds together with an additional
perturbation condition on the functions q(ε) (n, A).
25
Assume that for some k ≥ 0, we have the following expansions for r =
0, . . . , k, A ∈ Γ0 ,
ω (ε) (ρ (0) , r, A)
= ω (0) (ρ (0) , r, A) + b1,r (A)ε + · · · + bk−r,r (A)ε k−r + o(ε k−r ).
(3.14)
Then, we can construct the following asymptotic expansions,
π (ε) (A) = π (0) (A) + d1 (A)ε + · · · + dk (A)ε k + o(ε k ), A ∈ Γ0 ,
(3.15)
where the coefficients d1 (A), . . . , dk (A) can be calculated from recursive formulas which are rational functions of the coefficients in the expansions given
by equations (3.6) and (3.14). This is the main result of Paper II.
3.3
Perturbed Semi-Markov Processes
For every ε ≥ 0, let ξ (ε) (n), n = 0, 1, . . . , be a discrete time semi-Markov pro(ε)
cess on the state space X = {0, 1, . . . , N}, with transition probabilities Qi j (n),
n = 1, 2, . . . , i, j ∈ X, as defined in Section 2.5.
(ε)
(0)
Assume that Qi j (n) converges pointwise to Qi j (n) as ε → 0. Furthermore, we assume that the states {1, . . . , N} is a communicating class of states
for all sufficiently small ε.
In this case, we get the following perturbed version of the renewal equation
(2.3),
(ε)
(ε)
n
(ε)
(ε)
Pi j (n) = hi j (n) + ∑ Pi j (n − k)gi (k), n = 0, 1, . . . , i, j 6= 0,
(3.16)
k=0
where
(ε)
(ε)
(ε)
(ε)
Pi j (n) = Pi {ξ (ε) (n) = j, µ0 > n},
(ε)
hi j (n) = Pi {ξ (ε) (n) = j, µ0 > n, µi
(ε)
gi (k)
=
(ε)
Pi {µi
= k,
(ε)
ν0
>
> n},
(ε)
νi }.
We are interested in the asymptotic behavior, as n → ∞ and ε → 0, of the
(ε)
probabilities Pi j (n). Again, the asymptotic behavior is different depending
on which of the cases (a)–(c) described in Section 3.1 that is considered. The
(ε)
first hitting times µ0 of state 0 can be given the same interpretations as the
regenerative stopping times in Section 3.2. This means that in the stationary
case (a) transitions into state 0 are not possible for any ε ≥ 0. In the pseudostationary case (b), transitions into state 0 are possible for ε > 0 but not for
26
ε = 0. Finally, in the quasi-stationary case (c), transitions into state 0 are
possible for all ε ≥ 0.
Using the renewal equation (3.16), we can apply the results obtained for
perturbed renewal equations in Paper I to get an asymptotic relation corre(ε)
sponding to (3.9) for the probabilities Pi j (n).
(ε)
(ε)
Mixed power-exponential moments for the functions gi (n) and hi j (n) in
the renewal equation (3.16) are defined in analogy with (3.2) and (3.3) and are
(ε)
(ε)
denoted by φi (ρ, r) and ωi j (ρ, r), respectively.
If n(ε) → ∞ as ε → 0 in such a way that the balancing relation (3.8) holds,
then we have the following exponential asymptotic expansion,
(ε)
Pi {ξ (ε) (n(ε) ) = j, µ0 > n(ε) }
(0)
→ e−cr λr π̃i j , as ε → 0,
(0)
(ε)
r−1
exp(−(ρ + c1 ε + · · · + cr−1 ε )n )
(3.17)
where
(0)
(0)
π̃i j =
ωi j (ρ (0) , 0)
(0)
φi (ρ (0) , 1)
.
(0)
In the pseudo-stationary case, the limit constants π̃i j do not depend on the
initial state i and are given by the stationary probabilities of the limiting semiMarkov process. This is the main result of Paper IV.
Let us make some comments about this result. First, we note that since the
(ε)
characteristic equation now is given by φi (ρ, 0) = 1, it first appears as the
coefficients in the expansion of ρ (ε) in relation (3.17) should also depend on i.
(ε)
However, under conditions imposed on exponential moments of φi (ρ, 0) in
our papers, the root of the characteristic equation is independent of i. This is
shown both in Paper IV and V.
Then, let us also note that in order to compute the coefficients c1 , . . . , cr
in (3.17) we need the coefficients in expansions corresponding to (3.6) for the
(ε)
moment functionals φi (ρ (0) , r) for some i 6= 0 which we can choose arbitrarily. However, perturbation conditions for semi-Markov processes are more
naturally formulated in terms of the following moment functionals of transition
probabilities,
(ε)
(ε)
r ρn
pi j (ρ, r) = ∑∞
n=0 n e Qi j (n), ρ ∈ R, r = 0, 1, . . . , i, j ∈ X.
Our perturbation condition now has the following form, for r = 0, . . . , k,
(ε)
pi j (ρ (0) , r)
(0)
= pi j (ρ (0) , r) + ei j [1, r]ε + · · · + ei j [k − r, r]ε k−r + o(ε k−r ).
(3.18)
27
Thus, in order to apply the results obtained for perturbed renewal equations, we need to be able to compute the coefficients in asymptotic expan(ε)
sions of φi (ρ (0) , r) as functions of the coefficients in the expansions given by
(ε)
(ε)
(3.18). This can be done by first linking pi j (ρ, r) and φi (ρ, r) via systems
of linear equations which, for r = 0, 1, . . . , are connected recursively. Then, using these systems and arithmetic rules for asymptotic expansions of solutions
for systems of linear equations, we can construct the desired expansions for
(ε)
φi (ρ (0) , r). In Paper IV, it is shown in detail how this is done.
We also consider quasi-stationary distributions for semi-Markov processes,
that is, distributions independent of the initial state i 6= 0 which for each fixed
ε ≥ 0 satisfy the following relation,
(ε)
(ε)
π j = lim Pi {ξ (ε) (n) = j | µ0 > n}, j = 1, . . . , N.
n→∞
Using the same type of renewal arguments as we did for regenerative processes in Section 3.2, it can be shown that the quasi-stationary distribution is
given by the following formula,
(ε)
(ε)
πj
=
ωi j (ρ (ε) , 0)
(ε)
∑k6=0 ωik (ρ (ε) , 0)
, j = 1, . . . , N,
where i 6= 0 can be chosen arbitrarily. Using this formula, asymptotic expan(ε)
sions for π j can be obtained by using the techniques developed for regenerative processes. In order to do this, we need the coefficients in expansions
(ε)
corresponding to (3.6) and (3.14) for the moment functionals φi (ρ (0) , r) and
(ε)
ωi j (ρ (0) , r). Using the technique based on recursive systems of linear equations mentioned above, these expansions can be computed from the coefficients in the expansions given by (3.18). We give a detailed description of how
this is done in Paper V.
For the quasi-stationary distribution, we get the following asymptotic power
series expansion,
(ε)
(0)
π j = π j + d1, j ε + · · · + dk, j ε k + o(ε k ), j = 1, . . . , N,
(3.19)
where the coefficients d1, j , . . . , dk, j can be computed from explicit recursive
formulas which are rational functions of the coefficients in the expansions
given by equation (3.18). This is the main result of Paper VI.
3.4
Perturbed Risk Processes
(ε)
For every ε ≥ 0, let Zu (n), n = 0, 1, . . . , be a discrete time risk process with
claim probability p(ε) and claim size distribution g(ε) (n), as defined in Section
2.6.
28
Assume that p(ε) → p(0) as ε → 0 and that g(ε) (n) converge pointwise to
g(0) (n) as ε → 0.
In this case, the ruin probability satisfies a perturbed version of the renewal
equation (2.5), that is,
u
Ψ(ε) (u) = q(ε) (u) + ∑ Ψ(ε) (u − k) f (ε) (k), u = 0, 1, . . . ,
(3.20)
k=0
where
(ε)
Ψ(ε) (u) = P{min Zu (n) < 0},
n≥0
(ε)
q(ε) (u) = p(ε) ∑∞
k=u+1 (1 − G (k)),
f (ε) (k) = p(ε) (1 − G(ε) (k)),
G(ε) (k) = ∑kn=0 g(ε) (n).
We are interested in the asymptotic behavior, as u → ∞ and ε → 0, of the
ruin probabilities Ψ(ε) (u). Also in this case, the scenarios (a)–(c) described in
Section 3.1 have different interpretations. Here, the stationary case (a) is trivial
and corresponds to the situation where ruin is certain to occur for all ε ≥ 0. In
the pseudo-stationary case (b), the ruin probability is strictly less than one for
ε > 0 but ruin occurs with probability one for the limiting model where ε = 0.
Finally, in the quasi-stationary case (c), ultimate survival is possible for all
ε ≥ 0.
We can apply the results obtained for perturbed renewal equations in Paper
I to get an asymptotic relation for the ruin probability corresponding to (3.9).
Mixed power-exponential moment functionals of f (ε) (n) and q(ε) (n) are
denoted by φ (ε) (ρ, r) and ω (ε) (ρ, r), respectively, as in equations (3.2) and
(3.3). Let us define
ω (ε) (ρ (ε) , 0)
π̃ (ε) = (ε) (ε) , ε ≥ 0.
φ (ρ , 1)
Assume that u(ε) → ∞ as ε → 0 in such a way that
ε r u(ε) → λr ∈ [0, ∞), for some 1 ≤ r ≤ k.
(3.21)
Then, we have the following exponential asymptotic expansion,
Ψ(ε) (u(ε) )
→ e−cr λr π̃ (0) , as ε → 0,
exp(−(ρ (0) + c1 ε + · · · + cr−1 ε r−1 )u(ε) )
(3.22)
where c1 , . . . , cr can be computed from recursive formulas which are rational
functions of the coefficients in the expansions of φ (ε) (ρ (0) , r).
29
For a fixed ε ≥ 0, the classical discrete time renewal theorem, applied to a
transformed version of (3.20), implies the following discrete time analogue of
the Cramér-Lundberg approximation,
eρ
(ε) u
Ψ(ε) (u) → π̃ (ε) , as u → ∞.
This motivates that relation (3.22) can be improved by also taking the asymptotics of π̃ (ε) as ε → 0 into account. In order to do this, we can construct an
asymptotic expansion π̃ (ε) = π̃ (0) + d1 ε + · · · + dl ε l + o(ε l ). Under additional
perturbation conditions imposed on the moment functionals ω (ε) (ρ (0) , r), this
can be done by applying the same type of technique as used for regenerative
processes in Paper II. Using this asymptotic expansion in relation (3.22) and
taking the balancing relation (3.21) into account, we get the following asymptotic relation,
b (ε) (u(ε) ) → 1, as ε → 0,
Ψ(ε) (u(ε) )/Ψ
(3.23)
r,l
where
b (ε) (u) =
Ψ
r,l
π̃ (0) + d1 ε + · · · + dl ε l
.
exp{(ρ (0) + c1 ε + · · · + cr ε r )u}
b (ε) (u) have zero asympRelation (3.23) shows that the approximations Ψ
r,l
totic relative error under the balancing condition (3.21). These approximations
can be seen as generalizations of discrete time analogues of both the CramérLundberg approximation and the diffusion approximation for ruin probabilities. This is the main result of Paper III.
4. Examples
In this section, we present some examples of perturbed discrete time models
where the results of the present thesis can be used. In particular, these examples show that even simple linear perturbations of model parameters may lead
to nonlinearly perturbed Markov chains or semi-Markov processes of the type
studied in this thesis.
4.1
An Epidemic SIS Model
In this example, we consider a model for the spread of an infectious disease in
a closed homogeneously mixed population of size N, which is a variant of the
classical Reed-Frost stochastic epidemic model. The difference is that an infected individual who recover gets susceptible again. We follow the definition
given by Longini Jr. (1980).
30
The model under consideration is a discrete time model where each time
unit is called a period of infectiousness. During one such period, any individual in the population is either infected or susceptible. It is assumed that the
number of infected and susceptible individuals at time n + 1 only depends on
the corresponding numbers at time n. At time n, each of the infected individuals, independently of one another, has infectious contact with each of the
susceptible individuals in the population, independently, with probability p.
The infected individuals at time n + 1 are those susceptible individuals which
got infected at time n. The individuals that were infectious at time n are susceptible at time n + 1.
The evolution of the number of infected individuals can be described by a
discrete time Markov chain. Let In denote the number of infected individuals
at time n. Notice that a susceptible individual escapes infection if and only
if he/she avoids infectious contact with all infected individuals. If In = i, this
happens with probability (1− p)i . Since all susceptible individuals are infected
independently of one another, this means that if In = i then In+1 ∼ Bin(N −
i, 1 − (1 − p)i ). Thus, the Markov chain has one-step transition probabilities
pi j = P{In+1 = j | In = i} given by
pi j =
N−i
j
0
(1 − (1 − p)i ) j (1 − p)i(N−i− j) 0 ≤ j ≤ N − i,
otherwise.
(4.1)
Here, state N can be excluded since unless we start with all individuals infected, this state cannot be reached. Then, the Markov chain described above
has one class of transient communicating states {1, 2, . . . , N − 1} and one absorbing state 0.
If the population size N is not too small, then the epidemic can persist for
a long time even for small values of the individual infectious probability p. In
this case, the quasi-stationary distribution can be used to describe the number
of infected individuals in the long run before the epidemic disappears.
The results of the present thesis can be used to study the sensitivity of the
quasi-stationary distribution subject to small changes in the infectious probability p. For example, let us assume that p = p(ε) = p0 + ε, for some reference
value p0 > 0. Then, the Markov chain In will have the same class structure as
described above for all sufficiently small ε. Furthermore, we see from (4.1)
that the transition probabilities will be nonlinear functions of ε, so this leads
to the model of a nonlinearly perturbed Markov chain. Since the transition
probabilities can be expanded in power series with respect to ε, we can construct asymptotic expansions for the quasi-stationary distribution by applying
the results of Paper VI.
31
4.2
The Wright-Fisher Model with Selection
In this example, we consider a classical model from population genetics which
describes the genetic drift of a certain gene type in a population with nonoverlapping generations.
Let N be an even positive integer and consider a one-sex population of N/2
individuals, each carrying two copies of a certain gene type. This gene type
exists in two different variants called alleles which we denote by A1 and A2 . It
is assumed that the allele frequency in generation n + 1 only depends on the
allele frequency in generation n. The alleles in a new generation are formed by
sampling N times with replacement from the alleles in the previous generation.
The model described above is called the Wright-Fisher model. This model
and several variants of it have attracted a lot of attention over the years, see,
for instance, the books by Ewens (2004) and Durrett (2008). Here, a slight
extension which allows us to include selection in the model will be considered.
Assume that allele A1 has a selective advantage s, meaning that a sampled
allele enters the new generation with probabilities proportional to 1 + s and 1
for A1 and A2 , respectively. Then, the probability that a sampled allele that
enters the new generation is of type A1 , given that there are i alleles of type A1
in the previous generation, is equal to
pi =
(1 + s)i
, 0 ≤ i ≤ N.
(1 + s)i + N − i
(4.2)
Now, let ηn , n = 0, 1, . . . , be the number of gene copies with allele A1 in
generation n. It follows from the description above that ηn is a discrete time
Markov chain on the state space X = {0, 1, . . . , N} with one-step transition
probabilities given by
N j
pi j =
p (1 − pi )N− j , i, j ∈ X.
(4.3)
j i
The Markov chain described above have one transient class of communicating states {1, . . . , N − 1} and two absorbing states 0 and N. The absorbing
states correspond to a fixation of one of the two alleles. However, if N is large
and s is small, the time to absorption may be very long. In this case, the quasistationary distribution can be used to describe the frequency of A1 alleles in
the long run before fixation.
Using the results of this thesis, we can study the sensitivity of the quasistationary distribution subject to small perturbations of the selection parameter.
Here, a natural choice is to consider the selection parameter itself as the perturbation parameter, that is, s = s(ε) = ε. Then, the Markov chain ηn will have
the same class structure as described above for all sufficiently small ε. Let us
32
note here that we have two absorbing states while our results are formulated
for processes with one absorbing state. However, in this case we can easily construct a modified process with one absorbing state which has the same
quasi-stationary distribution as the process described above by merging 0 and
N into one state.
From (4.2) and (4.3) we see that the perturbed transition probabilities are
nonlinear functions of ε, so this is another example of a nonlinearly perturbed
Markov chain. Furthermore, these transition probabilities can be expanded in
power series with respect to ε, so we can apply the results obtained in this
thesis to construct asymptotic expansions for the quasi-stationary distribution.
4.3
An Alternating Semi-Markov Process
In this example, we consider a simple reliability model describing the working
status of a repairable system which eventually will have a fatal non-repairable
failure. We refer to Barbu, Boussemart, and Limnios (2004) and Barbu and
Limnios (2008) for an introduction to reliability theory of discrete time semiMarkov systems.
Let us consider a stochastic process which starts in state 2 and stays there
for a random time with distribution f2 (n). After this, the process goes to state
1 and stays there for a random time with distribution f1 (n). Then, with some
small probability p, the process is absorbed in state 0, or, with probability
1 − p, the process starts over in state 2.
Such a process can, for example, be interpreted as the working status of a
machine which is successively being repaired after breakdowns. In this case,
state 2 means that the machine is functioning without problems. In state 1
the machine is not working but can possibly be repaired, while state 0 means
that the machine is broken and cannot be repaired. We assume that in each
discrete time step, the machine is in either of these three states. Furthermore,
f2 (n) is the distribution of the time between repair and breakdown, f1 (n) is the
distribution of the time it takes to locate the error after a breakdown, and p is
the probability of a fatal non-repairable failure.
This can be modeled by a discrete time semi-Markov process ξ (n), n =
0, 1, . . . , on the state space X = {0, 1, 2} with transition probabilities given by

f2 (n)




 (1 − p) f1 (n)
p f1 (n)
Qi j (n) =


χ(n
= 1)



0
i = 2, j = 1,
i = 1, j = 2,
i = 1, j = 0,
i = 0, j = 0,
otherwise.
33
Note that if p = 0, then {1, 2} is a closed communicating class and 0 is
an isolated state. If p > 0, then we have one class of transient communicating
states {1, 2} and one absorbing state 0.
This example is considered in detail in Paper II where we present explicit formulas for calculating asymptotic expansions for quasi-stationary distributions when the probability p and the distributions f1 (n) and f2 (n) are
perturbed. Here, higher order perturbation conditions are formulated for the
probability p and mixed power-exponential moment functionals ψi (ρ, r) =
r ρn
∑∞
n=0 n e f i (n), i = 1, 2.
Now, suppose that the sojourn time distributions f1 (n) and f2 (n) belong to
some parametric class of distributions. If some parameters in these distributions are estimated by some statistical procedure, then it may be of interest to
know how small perturbations from the estimated values affects some quantity
of interest, for example, the stationary or quasi-stationary distribution.
For the sake of simplicity, let us assume that f1 (n) and f2 (n) are shifted
Poisson distributions with parameters λ1 > 0 and λ2 > 0, i.e.,
fi (n) =
λin−1 −λi
e , n = 1, 2, . . . , i = 1, 2.
(n − 1)!
Even though this might not be realistic in the context of failure and repairing
times, Poisson distributed sojourn times may have applications in other areas.
The moment generating functions for the sojourn time distributions are
given by
ψi (ρ) = exp(ρ + λi (eρ − 1)), ρ ∈ R, i = 1, 2.
(4.4)
Mixed power-exponential moment functionals ψi (ρ, r) for fi (n) are obtained
by differentiating ψi (ρ).
Now, let us assume that the parameters λ1 and λ2 are linear functions of
(ε)
(ε)
ε, that is λ1 = a0 + a1 ε and λ2 = b0 + b1 ε. In this case, we see from (4.4)
that the mixed power-exponential moment functionals ψi (ρ, r) will be nonlinear functions of ε, so we get a model which is a nonlinearly perturbed semiMarkov process. Furthermore, these moment functionals will have asymptotic
power series expansions with respect to ε, so our results can be used to construct asymptotic expansions for stationary distributions (if p = 0) or quasistationary distributions (if p > 0).
Another possibility in this example is to simply take the probability of a
fatal error as the perturbation parameter, that is, p = p(ε) = ε. This provides
an example of the pseudo-stationary case.
34
5. Conclusion
This thesis presents a number of asymptotic results for perturbed discrete time
stochastic models. In particular, models describing stochastic systems with a
finite lifetime are objects of special interest. The results are obtained by first
developing methods for constructing asymptotic expansions for solutions of
perturbed discrete time renewal equations. These methods are then applied
and further developed in the context of regenerative processes, risk processes,
Markov chains, and semi-Markov processes.
Let us briefly comment on three important features of our results. First
of all, throughout the thesis, we focus exclusively on processes with discrete
time. As pointed out in the introduction, discrete time models are interesting
in their own rights and appear naturally in applications where some quantity is
measured only at specific time points. Several of the results in this thesis were
previously only available for continuous time models.
Another feature is that the characteristics of the processes we study are
allowed to be nonlinear functions of the perturbation parameter. This is important since even simple linear perturbations of model parameters often lead
to nonlinearly perturbed Markov chains and semi-Markov processes, as illustrated in Section 4.
The third feature we would like to mention is that all our main asymptotic
results are given in the form of higher order asymptotic expansions. This may
be useful since higher order coefficients can improve approximations in numerical computations if the perturbation parameter is not small enough for a
linear approximation to be satisfactory.
Let us finally mention some possible directions for future studies. All results presented in this thesis are given for a very general class of stochastic
processes. Indeed, it would also be interesting to study more specific types
of processes which belong to the framework developed here. In such cases,
perturbation conditions are usually formulated for some local characteristics
of the model under consideration. Then, in order to apply the results of the
present thesis, we first need to find the coefficients for the perturbation conditions formulated for moment functionals of transition probabilities. In some
cases, this may be a very difficult step.
Another interesting area for future studies are numerical investigations.
The formulas for the coefficients in our asymptotic expansions are very involved and are given in a recursive form. Only in very special cases it is
possible to find explicit formulas. Thus, numerical studies are natural complements to our analytical results in order to understand the asymptotic properties. Moreover, investigations of this type will certainly lead to new interesting
questions. Some numerical experiments for some closely related models have
35
been done, for instance, by Ni (2011, 2014) and Silvestrov, Petersson, and
Hössjer (2016).
Another potential continuation of the studies in this thesis is to consider
Markov chains and semi-Markov processes with a countable or general measurable state space. In the proofs of our results, the assumption of a finite
state space plays a central role. A relaxation of this assumption will make the
problem substantially more complicated. An interesting question is if we still,
with the results of Paper II as a framework, can obtain higher order asymptotic expansions of stationary and quasi-stationary distributions, at least for
some models having a special structure. For example, a first step could be
to consider a Markov chain or semi-Markov process of birth-death type on a
countable state space.
36
Sammanfattning
Inom sannolikhetsteorin används matematiska modeller för att beskriva olika
händelser som innehåller slump och därmed inte kan förutsägas exakt. Speciellt så används stokastiska processer för att beskriva hur tillståndet av någon
typ av system utvecklas med tiden. Exempel på stokastiska processer är utvecklingen av antalet smittade i en sjukdomsepedemi eller hur förekomsten av
en genvariant varierar mellan olika generationer inom en viss population.
I denna avhandling fokuserar vi främst på en klass av stokastiska processer som kallas regenerativa processer, där speciellt viktiga delklasser utgörs av
Markovkedjor och semi-Markovprocesser. Kännetecknande för denna typ av
processer är att det existerar ett tillstånd sådant att när processen återkommer
till detta tillstånd så är processens framtida utveckling oberoende av vad som
hänt tidigare och har samma egenskaper som den hade från början. Processen
säges regenerera vid de tidpunkter då den återvänder till detta tillstånd. Denna egenskap gör att regenerativa processer lämpar sig väl för att analyseras
matematiskt.
En ytterligare komponent i de processer som behandlas i denna avhandling är att definitionen av dem även innehåller en så kallad störningsparameter.
Detta är en typ av parameter vars värde i någon mening kan antas vara ett litet positivt tal. Till exempel så kan denna parameter vara sannolikheten att en
genvariant muterar. Ett annat exempel är att störningsparametern kan beskriva
en osäkerhet för andra parametrar som vi bara vet ett approximativt värde på.
Den centrala frågan i denna avhandling är hur störningsparametern påverkar olika mått som beskriver processens egenskaper över lång tid. Eftersom relationen mellan störningsparametern och de mått vi är intresserade av är mycket komplicerad så är det nästan aldrig möjligt att hitta ett explicit teoretiskt
samband som beskriver denna relation. Det som då kan göras är att konstruera
approximativa samband mellan störningsparametern och de olika måtten av intresse. De samband som konstrueras i denna avhandling kallas för asymptotiska utvecklingar och ges av det exakta värdet på måttet när störningsparametern
är lika med noll, plus en summa av ett antal korrektionstermer.
För stokastiska processer kan man bland annat skilja mellan om de utvecklas i diskret eller kontinuerlig tid. Att en process utvecklas i diskret tid betyder
37
att den bara observeras för givna tidpunkter, till exempel dagar, månader eller
generationer, medan en process i kontinuerlig tid observeras för alla tidpunkter. Denna skillnad gör att den matematiska teorin för stokastiska processer ofta
skiljer sig åt beroende på om tiden antas vara diskret eller kontinuerlig. Den
teori som används i denna avhandling har tidigare främst utvecklats och applicerats for stokastiska processer i kontinuerlig tid. Denna avhandling bidrar
med en omfattande studie av denna teori och dess tillämpningar för stokastiska
processer i diskret tid.
38
Tack
Denna avhandling representerar slutpunkten av mina fem år som doktorand
vid Stockholms universitet, juli 2011–juni 2016. Jag vill härmed tacka alla
som på ett eller annat sätt har bidragit positivt till denna period av mitt liv.
Ett speciellt stort tack vill jag rikta till min handledare Dmitrii Silvestrov som
har bidragit med ytterst värdefull hjälp och uppmuntran under mina år som
doktorand. Min bihandledare Ola Hössjer förtjänar också ett stort tack för att
han alltid har ställt upp när jag har haft frågor eller funderingar. Vidare vill
jag speciellt tacka Jens Malmros, som jag har delat kontor med sedan hösten
2012, för att han har varit en exceptionellt bra arbetskollega och vän. Ett stort
tack också till övriga kollegor, både tidigare och nuvarande, på avdelningen
för matematisk statistik för att ni har bidragit med mycket glädje genom åren.
Jag vill också passa på att rikta ett varmt tack till övriga vänner och bekanta,
speciellt David, Persa, Prashant, Robert och övriga deltagare vid de mycket
trevliga Champions League-kvällarna. Slutligen vill jag tacka min närmaste
familj, mamma Birgitta, Magnus, Tina och John som betyder mycket för mig,
samt min pappa Ingemar (1923–2004) som jag inte har glömt.
Solliden, Rimforsa,
5 april 2016.
Mikael Petersson
39
40
References
[1] Altman, E., Avrachenkov, K.E., Núñez-Queija, R. (2004). Perturbation analysis for denumerable Markov chains with application to queueing models. Adv.
Appl. Prob., 36, 839–853.
[2] Athreya, K.B., Ney, P. (1978). A new approach to the limit theory of recurrent
Markov chains. Trans. Amer. Math. Soc., 245, 493–501.
[3] Avrachenkov, K.E., Filar, J.A., Howlett, P.G. (2013). Analytic Perturbation
Theory and Its Applications. SIAM, Philadelphia, xii+372 pp.
[4] Avrachenkov, K.E., Haviv, M. (2004). The first Laurent series coefficients for
singularly perturbed stochastic matrices. Linear Algebra Appl., 386, 243–259.
[5] Avrachenkov, K.E., Lasserre, J.B. (1999). The fundamental matrix of singularly
perturbed Markov chains. Adv. Appl. Prob., 31, 679–697.
[6] Barbu, V.S., Boussemart, M., Limnios, N. (2004). Discrete-time semi-Markov
model for reliability and survival analysis. Comm. Statist. Theory Methods, 33,
2833–2868.
[7] Barbu, V.S., Limnios, N (2008). Semi-Markov Chains and Hidden SemiMarkov Models Toward Applications. Lecture Notes in Statistics, 191,
Springer, New York, xiv+226 pp.
[8] Blanchet, J., Zwart, B. (2010). Asymptotic expansions of defective renewal
equations with applications to perturbed risk models and processor sharing
queues. Math. Meth. Oper. Res., 72, 311–326.
[9] Cheong, C.K. (1968). Ergodic and ratio limit theorems for α-recurrent semiMarkov processes. Z. Wahrsch. Verw. Gebiete, 9, 270–286.
[10] Cheong, C.K. (1970). Quasi-stationary distributions in semi-Markov processes.
J. Appl. Prob., 7, 388–399.
[11] Coderch, M., Willsky, A.S., Sastry, S.S., Castanon, D.A. (1983). Hierarchical
aggregation of singularly perturbed finite state Markov processes. Stochastics,
8, 259–289.
41
[12] Collet, P., Martínez, S., San Martín, J. (2013). Quasi-Stationary Distributions. Markov chains, Diffusions and Dynamical Systems. Springer, Heidelberg,
xvi+280 pp.
[13] Cossette, H., Landriault, D., Marceau, É. (2004). Compound binomial risk
model in a markovian environment. Insur. Math. Econ., 35, 425–443.
[14] Courtois, P.J., Louchard, G. (1976). Approximation of eigencharacteristics in
nearly-completely decomposable stochastic systems. Stoch. Process. Appl., 4,
283–296.
[15] Courtois, P.J., Semal, P. (1984). Bounds for the positive eigenvectors of nonnegative matrices and for their approximations by decomposition. J. Assoc.
Comput. Mach., 31, 804–825.
[16] Cramér, H. (1930). On the Mathematical Theory of Risk. Skandia Jubilee Volume, Stockholm, 84 pp.
[17] Cramér, H. (1955). Collective Risk Theory. Skandia Jubilee Volume, Stockholm.
[18] Darroch, J.N., Seneta, E. (1965). On quasi-stationary distributions in absorbing
discrete-time finite Markov chains. J. Appl. Prob., 2, 88–100.
[19] Delebecque, F. (1983). A reduction process for perturbed Markov chains. SIAM
J. Appl. Math., 43, 325–350.
[20] Dickson, D.C.M., Egídio dos Reis, A.D., Waters, H.R. (1995). Some stable
algorithms in ruin theory and their applications. Astin Bull., 25(2), 153–175.
[21] van Doorn, E.A., Pollett, P.K. (2013). Quasi-stationary distributions for
discrete-state models. Eur. J. Oper. Res., 230, 1–14.
[22] Durrett, R. (2008). Probability Models for DNA Sequence Evolution. 2nd ed.,
Springer, New York, xii+431 pp.
[23] Englund, E. (2001). Nonlinearly perturbed renewal equations with applications.
PhD Thesis, Umeå University.
[24] Englund, E., Silvestrov, D.S. (1997). Mixed large deviation and ergodic theorems for regenerative processes with discrete time. In: Jagers, P., Kulldorff,
G., Portenko, N., Silvestrov, D. (eds) Proceedings of the Second ScandinavianUkrainian Conference in Mathematical Statistics, vol. I, Umeå. Theory Stoch.
Process., 3(19), no. 1–2, 164–176.
[25] Erdös, P., Feller, W., Pollard, H. (1949). A property of power series with positive coefficients. Bull. Amer. Math. Soc., 55(2), 201–204.
[26] Ewens, W. (2004). Mathematical Population Genetics. 2nd ed., Springer, New
York, xx+418 pp.
42
[27] Feller, W. (1950, 1957, 1968). An Introduction to Probability Theory and Its
Applications. Vol. I, 3rd ed., New York, Wiley, xviii+509 pp.
[28] Feller, W. (1966, 1971). An Introduction to Probability Theory and Its Applications. Vol. II, 2nd ed., New York, Wiley, 669 pp.
[29] Flaspohler, D.C., Holmes, P.T. (1972). Additional quasi-stationary distributions
for semi-Markov processes. J. Appl. Prob., 9, 671–676.
[30] Gaı̆tsgori, V.G., Pervozvanskiı̆, A.A. (1975). Aggregation of states in a Markov
chain with weak interaction. Cybernetics, 11, 441–450.
[31] Gerber, H.U. (1988). Mathematical fun with the compound binomial process.
Astin Bull., 18(2), 161–168.
[32] Gyllenberg, M., Silvestrov, D.S. (1994). Quasi-stationary distributions of a
stochastic metapopulation model. J. Math. Biol., 33, 35–70.
[33] Gyllenberg, M., Silvestrov, D.S. (1999). Quasi-stationary phenomena for semiMarkov processes. In: Janssen, J., Limnios, N. (eds) Semi-Markov Models and
Applications. Kluwer, Dordrecht, 33–60.
[34] Gyllenberg, M., Silvestrov, D.S. (2000a). Cramér-Lundberg approximation for
nonlinearly perturbed risk processes. Insur. Math. Econ., 26, 75–90.
[35] Gyllenberg, M., Silvestrov, D.S. (2000b). Nonlinearly perturbed regenerative
processes and pseudo-stationary phenomena for stochastic systems. Stoch. Process. Appl., 86, 1–27.
[36] Gyllenberg, M., Silvestrov, D.S. (2008). Quasi-Stationary Phenomena in Nonlinearly Perturbed Stochastic Systems. De Gruyter Expositions in Mathematics,
44, Walter de Gruyter, Berlin, ix+579 pp.
[37] Hassin, R., Haviv, M. (1992). Mean passage times and nearly uncoupled
Markov chains. SIAM J. Discrete Math., 5(3), 386–397.
[38] Hunter, J.J. (1991). The computation of stationary distributions of Markov
chains through perturbations. J. Appl. Math. Stoch. Anal., 4, 29–46.
[39] Hunter, J.J. (2005). Stationary distributions and mean first passage times of
perturbed Markov chains. Linear Algebra Appl., 410, 217–243.
[40] Kartashov, N.V. (1996). Strong Stable Markov Chains. VSP, Utrecht and
TBiMC Scientific Publishers, Kiev, 138 pp.
[41] Kingman, J.F.C. (1963). The exponential decay of Markov transition probabilities. Proc. London Math. Soc., 13, 337–358.
[42] Koroliuk, V.S., Limnios, N. (2005). Stochastic Systems in Merging Phase
Space. World Scientific, Singapore, xv+331 pp.
43
[43] Koroliuk, V.S., Limnios, N. (2010). Reliability of semi-Markov systems with
asymptotic merging phase space. In: Rykov, V.V, Balakrishnan, N., Nikulin,
M.S. (eds) Mathematical and Statistical Models and Methods in Reliability.
Birkhäuser, Boston, 3–18.
[44] Kovalenko, I.N. (1977). Limit theorems for reliability theory. Cybernetics,
13(6), 902–914.
[45] Lasserre, J.B. (1994). A formula for singular perturbations of Markov chains.
J. Appl. Prob., 31, 829–833.
[46] Latouche, G. (1991). First passage times in nearly decomposable Markov
chains. In: Stewart, W.J. (ed) Numerical Solution of Markov Chains. Probability: Pure and Applied, 8. Marcel Dekker, New York, 401–411.
[47] Latouche, G., Louchard, G. (1978). Return times in nearly-completely decomposable stochastic processes. J. Appl. Prob., 15, 251–267.
[48] Li, S., Lu, Y., Garrido, J. (2009). A review of discrete-time risk models. Rev. R.
Acad. Cien. Serie A. Mat., 103(2), 321–337.
[49] Longini Jr., I.M. (1980). A chain binomial model of endemicity. Math. Biosci.,
50, 85–93.
[50] Lundberg, F. (1903). I. Approximerad framställning av sannolikhetsfunktionen.
II. Återförsäkring av kollektivrisker. Almqvist & Wiksell, Uppsala, 53 pp.
[51] Lundberg, F. (1926). Försäkringsteknisk riskutjämning. F. Englunds boktryckeri AB, Stockholm, 103 pp.
[52] Lundberg, F. (1932). Some supplementary researches on the collective risk theory. Skandinavisk Aktuarietidskrift, 15, 137–158.
[53] Mode, C.J., Pickens, G.T. (1988). Computational methods for renewal theory
and semi-Markov processes with illustrative examples. The American Statistician, 42, 143–152.
[54] Ni, Y. (2011). Nonlinearly perturbed renewal equations: asymptotic results and
applications. PhD Thesis, Mälardalen University, School of Education, Culture
and Communication, Västerås.
[55] Ni, Y. (2014). Exponential asymptotical expansions for ruin probability in a
classical risk process with non-polynomial perturbations. In: Silvestrov D.,
Martin-Löf, A. (eds) Modern Problems in Insurance Mathematics. EAA Series, Springer, Cham, 69–93.
[56] Nummelin, E. (1978). A splitting technique for Harris recurrent Markov chains.
Z. Wahrsch. Verw. Gebiete, 43, 309–318.
[57] Pervozvanskiı̆, A.A., Smirnov, I.N. (1974). Stationary-state evaluation for a
complex system with slowly varying couplings. Cybernetics, 10, 603–611.
44
[58] Phillips, R.G., Kokotovic, P.V. (1981). A singular perturbation approach to
modeling and control of Markov chains. IEEE Trans. Automat. Control, 26,
1087–1094.
[59] Rohlicek, J.R., Willsky, A.S. (1988). The reduction of perturbed Markov generators: An algorithm exposing the role of transient states. J. Assoc. Comput.
Mach., 35, 675–696.
[60] Schweitzer, P.J. (1968). Perturbation theory and finite Markov chains. J. Appl.
Prob., 5, 401–413.
[61] Seneta E., Vere-Jones, D. (1966). On quasi-stationary distributions in discretetime Markov chains with a denumerable infinity of states. J. Appl. Prob., 3,
403–434.
[62] Shiu, E.S.W. (1989). The probability of eventual ruin in the compound binomial
model. Astin Bull., 19(2), 179–190.
[63] Silvestrov, D.S. (1976). A generalization of the renewal theorem. Dokl. Akad.
Nauk. Ukr. SSR, A(11), 978–982.
[64] Silvestrov, D.S. (1978). The renewal theorem in a series scheme 1. Theory
Probab. Math. Statist., 18, 155–172.
[65] Silvestrov, D.S. (1979). The renewal theorem in a series scheme 2. Theory
Probab. Math. Statist., 20, 113–130.
[66] Silvestrov, D.S. (1995). Exponential asymptotic for perturbed renewal equations. Theory Probab. Math. Statist., 52, 153–162.
[67] Silvestrov, D.S. (2000). Nonlinearly perturbed Markov chains and large deviations for lifetime functionals. In: Limnios, N., Nikulin, M. (eds) Recent Advances in Reliability Theory: Methodology, Practice and Inference. Birkhäuser,
Boston, 135–144.
[68] Silvestrov, D., Petersson, M., Hössjer, O. (2016). Nonlinearly perturbed birthdeath-type models. Research Report 2016:6, Department of Mathematics,
Stockholm University, 63 pp. and arXiv:1604.02295.
[69] Silvestrov, D., Silvestrov S. (2015). Asymptotic expansions for stationary distributions of perturbed semi-Markov processes. Research Report
2015:9, Department of Mathematics, Stockholm University, 75 pp. and
arXiv:1603.03891.
[70] Simon, H.A., Ando, A. (1961). Aggregation of variables in dynamic systems.
Econometrica, 29, 111–138.
[71] Smith, W.L. (1955). Regenerative stochastic processes. Proc. Roy. Soc. London
Ser. A., 232, 6–31.
45
[72] Stewart, G.W. (1991). On the sensitivity of nearly uncoupled Markov chains.
In: Stewart, W.J. (ed) Numerical Solution of Markov Chains. Probability: Pure
and Applied, 8. Marcel Dekker, New York, 105–119.
[73] Vantilborgh, H. (1985). Aggregation with an error of O(ε 2 ). J. Assoc. Comput.
Mach., 32, 162–190.
[74] Vere-Jones, D. (1962). Geometric ergodicity in denumerable Markov chains.
Quart. J. Math., 13, 7–28.
[75] Willmot, G.E. (1993). Ruin probabilities in the compound binomial model. Insur. Math. Econ., 12, 133–142.
[76] Yin, G., Zhang, Q. (1998). Continuous-Time Markov Chains and Applications.
A Singular Perturbation Approach. Applications of Mathematics, 37, Springer,
New York, xv+349 pp.
[77] Yin, G., Zhang, Q. (2003). Discrete-time singularly perturbed Markov chains.
In: Yao, D.D., Zhang, H., Zhou, X.Y. (eds) Stochastic Modelling and Optimization. Springer, New York, 1–42.
46
Part B
Papers I–VI
47
48