Download Renewal Processes - Eaton.math.rpi.edu

Renewal Processes
Friday, April 25, 2014
1:59 PM
Homework 4 due Wednesday, May 7.
Reading:
• Karlin and Taylor Ch. 5
• Resnick Ch. 3
The Poisson point process that we developed last time has two important extensions for more
advanced applications:
• "marked Poisson point process": also generate some random variables with each Poisson
point.
• clumped Poisson point process: Each Poisson point actually corresponds to a random
number of agents
A renewal process is a point process in time (in one dimension) which has the property that
the length of the intervals between successive incidents are independently distributed.
The one-dimensional Poisson point process is a special case of a
renewal process where the common probability distribution of the is
exponential.
• A delayed renewal process is a slight extension of a renewal
process that allows to have a different probability distribution
than
.
Stoch14 Page 1
than
.
The reason this point process is called a renewal process is that we can
think of it as follows: Follow time continuously forward, and every time
an incident occurs, the stochastic process starts afresh.
• The independence of the interincident times therefore means
that every time an incident happens, the distribution of future
incidents is independent of the distribution of past incidents, given
that we know an incident has just occurred. Note this is not quite
Markov because this decoupling of future and past only occurs at
the times of an incident, not at arbitrary times t. In particular, it is
not in general true that the amount of time one expects to wait
until the next incident happens is independent of the amount of
time one has waited since the last incident. (If this were the case,
then we'd have a 1-D Poisson point process because exponential
distribution is the only memoryless distribution.)
Example applications of renewal processes:
• Equipment replacement
• The times of action potentials (spikes) produced by a neuron
(interspike interval distributions)
• Generic way to define a renewal process associated to a Markov
process by triggering an incident whenever the Markov chain visits
a specified state or group of states.
• i.e., biophysical chemistry, neuronal networks
• Spatial renewal processes for i.e., traffic following
One can also use the idea of
renewal processes:Poisson point processes::semi-Markov processes::continuous-time
Markov chains. That is a semi-Markov process is a discrete-state stochastic process that
generalizes a CTMC only in that the probability distribution for the amount of time spent
in a state ( can have arbitrary probability distribution (but is still independent of each
other).
Stoch14 Page 2
other).
• Queueing theory with nonexponential times for processing requests
• Branching processes with age considerations so that the times of branching are not
exponentially distributed
• Epidemiological models where the amount of time spent in the infective stage is not
exponentially distributed
The theory of semi-Markov processes is however, somewhat more complicated and we
won't treat it here.
Mathematical Specification of a Renewal Process
The key defining object is the cumulative distribution function (CDF) for the interincident
time:
.
When the interincident time intervals have a continuous distribution, then we can
equivalently refer to the PDF
for the interincident times, and the CDF and PDF are
related as follows:
The reason why renewal process works with CDFs is that they
generalize to interincident times that might not be purely continuously distributed,
i.e., an equipment replacement schedule where equipment is replaced
when it breaks or when a fixed time interval has elapsed, whatever
happens first. This technically doesn't have a PDF (but you can write
the PDF in terms of delta-functions for the discrete component), but
the CDF is perfectly well-behaved.
For the Poisson point process, the interincident PDF would be:
Stoch14 Page 3
The interincident CDF describes the "dynamics" of a renewal process.
Initial condition? Pure renewal process treats
in the same way as
if an incident happened at t=0. Otherwise, for a delayed renewal
process, the initial condition is given by a separate CDF for .
Key Random Variables Associated to Renewal Process
• Associated counting process
○
: counts the number of incidents that
have occurred before time t.
• Current life : amount of time between t and the previous
incident
• Residual life
amount of time between t and the next incident
• Total life
: amount of time between the previous and next
incident whose interval covers the time t.
One of the goals of the theory of renewal processes is to derive the
probability distributions for these random variables from the given
interincident CDF. There is a machinery for doing this, but to warm up
with the concepts, we'll just use direct calculation to compute these
probability distributions for Poisson point process. These will reveal the
important Poisson paradox.
The counting process N(t) associated to the Poisson point process is the
Poisson counting process, and we know from our previous calculations
in CTMC theory, that probability distribution for
is given by a
Stoch14 Page 4
in CTMC theory, that probability distribution for
Poisson distribution with mean
is given by a
The residual life is also easy to compute. Because of memoryless
property of the exponential distribution, which characterizes the
interincident time intervals of a Poisson point process, the residual life
has the same probability distribution as the interincident time
probability distribution. We continue to refer to CDFs for proper
generality.
Current life : Because the Markov property for the associated Poisson counting process is
past/future symmetric, the probability distribution of the current life would be the same as for
the residual life, except for the introduction of the initialization of the system at time 0. This
only modifies the probability distribution as follows: If no incident has happened in the
interval [0,t], then we set the current life
Therefore, we obtain:
The total life
has a probability distribution that can be computed from the
probability distributions of and because these random variables are independent for
Poisson point process (future and past conditionally independent of present.) The result can
Stoch14 Page 5
Poisson point process (future and past conditionally independent of present.) The result can
be computed by convolution of the probability distributions; get something that is
like.
But let's focus on a more interesting characteristic of this total life that we can approach
without having to do this calculation.
To do this, we'll introduce a technical device of Stieltjes integration that is often used in
renewal process theory.
Given a piecewise continuous function
the Riemann-Stieltjes integral:
where
and a function
of bounded variation we define
is some partition of the real line.
Any bounded monotone function has bounded variation and that is the
only kind of F we will be dealing with.
Once we allow ourselves to use Stieltjes integrals, we have a general
way for computing means of random variables that need not be
continuous or discrete:
where
is the CDF of Y.
This gives the formulas for discrete and continuous random variables as
special cases. (To see that the continuous random variable works, note
that in this case
when the derivative
exists).
Means of nonnegative random variables have a simpler expression in
terms of the CDF:
Stoch14 Page 6
terms of the CDF:
If
,
(an ordinary Riemann integral).
Proof:
Stoch14 Page 7