Download RENEWAL THEORY 1.1. Example: A Dice

RENEWAL THEORY
1. RENEWAL PROCESSES
1.1. Example: A Dice-Rolling Problem. In board games like Monopoly, Clue, Parcheesi, and so
on, players take turns rolling a pair of dice and then moving tokens on a board according to rules
that depend on the dice rolls. The rules of these games are all somewhat complicated, but in
most of the games, in most circumstances, the number of spaces a player moves his/her token is
the sum of the numbers showing on the dice.
Consider a simplified board game in which, on each turn, a player rolls a single fair die and
then moves his/her token ahead X spaces, where X is the number showing on the die. Assume
that the board has K spaces arranged in a cycle, so that after a token is moved ahead a total of K
spaces it returns to its original position. Assign the spaces labels 0, 1, 2, . . . , K − 1 in the order they
are arranged around the cycle, with label 0 assigned to the space where the tokens are placed
at the beginning of the game (in Monopoly, this would be the space marked “Go”). Then the
movement of a player’s token may be described as follows: after n turns the token will be on the
space labelled S n mod K , where
(1)
Sn =
n
X
Xi
i =1
and X 1 , X 2 , . . . are the dice rolls obtained on successive turns.
Problem 0: What is the probability that on the tenth time around the board a player’s token lands
on the space labelled K − 1?
Anyone who has played Monopoly will know that this particular space is “Boardwalk”, and
that if another player owns it then it is a bad thing to land on it, especially late in the game.
Of course, the answer to Problem 0 isn’t especially relevant to Monopoly, because the rules of
movement aren’t the simple rules we have imposed above; in fact, the problem is really more
interesting in Monopoly, because there it turns out that the probability of landing on Boardwalk
isn’t quite the same as that of landing on (say) Park Place (space K − 3). But we have to start
somewhere, so Problem 0 is what we’ll consider. Here is a more general problem, formulated in
a more convenient way:
Problem 1: Fix an integer x ≥ 0. What is the probability u x that S n = x for some n?
Observe that when x = 10K − 1, this reduces to Problem 0.
There is a related issue in Monopoly that is of some interest: when you pass Go, there is a
block of spaces (Baltic and Mediterranean Avenues) that you might want to avoid if you don’t
own them. So what is the probability that on the tenth trip around the board you manage to
jump over them altogether (that is, the chance that you don’t land on any of the spaces labelled
1,2,3)? Here is a more general formulation:
1
2
RENEWAL THEORY
Problem 2: Fix t ≥ 0, and define the first-passage and overshoot random variables by
(2)
τ(t ) = min{n ≥ 0 : S n > t }
(3)
R(t ) = S τ(t ) − t
and
What is the probability distribution of R(t )?
1.2. Ordinary Renewal Processes. Problems 1 and 2 above are prototypical problems in renewal
theory. In general, a renewal process (or more precisely, an ordinary renewal process) is the increasing sequence of random nonnegative numbers 0,S 1 ,S 2 , . . . gotten by adding i.i.d. nonnegative random variables X 1 , X 2 , . . . , as in equation (1). The individual terms S n of this sequence are
called renewals, or sometimes occurrences. With each renewal process is associated a renewal
counting process N (t ) that tracks the total number of renewals (not including the initial occurrence) to date: the random variable N (t ) is defined by
(4)
N (t ) = max{n : S n ≤ t } = τ(t ) − 1
where τ(t ) is the first passage time defined by (2) above.
In many instances (but not the dice-rolling example) the renewals have a natural interpretation as the times at which some irregularly occurring event take place. (For example, imagine
that X 1 , X 2 , . . . are the lifetimes of replaceable components in a system, such as light bulbs in a
light socket. If the first component is inserted at time 0, then the renewals S n are the times at
which components must be replaced.) Thus, it is traditional in the subject to refer to the random
variables X i as interoccurrence times, and their common distribution as the interoccurrence time
distribution. The random variables
(5)
A t = A(t ) = t − S N (t ) ,
R t = R(t ) = S τ(t ) − t ,
and
L t = L(t ) = S τ(t ) − S N (t )
are usually called the age, residual lifetime, and total lifetime random variables (although in certain applications it is also common to refer to R(t ) as the overshoot). Note that L(t ) = A(t )+ R(t ).
1.3. Delayed Renewal Processes. It isn’t always reasonable to insist that the first renewal occurs
at time S 0 = 0. For instance, in applications where the occurences are at times when a component of a system must be replaced, one might well be interested in situations where there is
already a working component in place at time 0. For this reason, we define a delayed renewal
process to be a sequence S 0 ,S 1 ,S 2 , . . . where
(6)
Sn = S0 +
n
X
Xj ,
j =1
and (i) the interoccurence times X 1 , X 2 , . . . are positive and i.i.d., as in an ordinary renewal process, and (ii) the initial delay S 0 ≥ 0 is independent of the interoccurence times X i . Notice that
the distribution of the initial delay random variable S 0 is not required to be the same as that of the
interoccurence time random variables X i . The age, residual lifetime, and total lifetime random
variables for a delayed renewal process are defined in exactly the same manner as for ordinary
renewal processes (equations (5) above).
RENEWAL THEORY
3
1.4. There’s a renewal process in my Markov chain. Let {Yn }n≥0 be an irreducible, recurrent
Markov chain on a finite or countable state space Y . Fix a state y ∗ , and let 0 < τ1 < τ2 < · · · be
the times of successive visits to y ∗ . Since the chain is recurrent, all of these times are finite w.p.1.
What’s more, the sequence {τm }m ≥1 constitutes a (delayed) renewal process. This follows from
the (strong) Markov property, because at every visit to y ∗ the Markov chain “starts afresh”, and
so the time until the next return (a) is independent of everything that happened earlier, and (b)
has the same distribution as all other return times (except possibly the first). Let { f k }k ≥1 be the
distribution of the times τm +1 − τm between successive visits, and let µ be its mean. Then by
basic Markov chain theory, the Markov chain Yn is positive recurrent if and only if µ < ∞, and if
µ < ∞ then the stationary probability of state y ∗ is
π(y ∗ ) = 1/µ.
1.5. Poisson and Bernoulli Processes. Two special cases are especially important. When the
interoccurrence time distribution is the exponential distribution with mean 1/λ, then the corresponding renewal process is the Poisson process with intensity λ. When the interoccurrence time
distribution is the geometric distribution with parameter p , that is,
P{X i = k } = p (1 − p )k −1
for k = 1, 2, . . . ,
then the corresponding renewal process is the Bernoulli process with success parameter p . Many
interesting problems that are quite difficult for renewal processes in general have simple solutions for Poisson and Bernoulli processes. For instance, in general the distribution of the count
random variable N (t ) does not have a simple closed form, but for a Poisson process it has the
Poisson distribution with mean λt . Similarly, the age and residual lifetime random variables
A(t ) and R(t ) generally have distributions that do not depend on the interoccurrence time distribution in a simple way, but for the Poisson process they are, for any fixed t , just independent
exponential random variables with mean 1/λ. These properties follow easily from the memoryless property of the exponential distribution. Renewal processes are important because in many
systems the times between successive renewals do not have memoryless distributions.
Exercise: What are the distributions of A t , R t , and L t for a Bernoulli process?
1.6. Renewal Processes: Arithmetic and Nonarithmetic. Renewal theory breaks into two parallel streams, one for discrete time, the other for continuous time. These are called the arithmetic
and nonarithmetic cases, respectively: A renewal process is said to be nonarithmetic if the distribution of the interoccurrence time random variables is not fully supported by an arithmetic
progression h, 2h, 3h, . . . . Otherwise, it is said to be arithmetic, and the maximal value of h > 0
such that the interoccurrence time distribution is fully supported by hZ is called the span. For
simplicity, assume henceforth that in the arithmetic case the span is h = 1, and also, for delayed
arithmetic renewal processes, that the initial delay random variable S 0 is integer-valued.
1.7. Arithmetic Renewal Processes: Associated Markov Chains. If {S n }n ≥0 is an arithmetic renewal process with span 1 then the discrete-time processes A n and R n are Markov chains.
Exercise: Prove this, and prove also that the vector-valued process (A n , R n ) is a Markov chain.
Let’s focus on the residual lifetime process R n . It might seem that the natural state space for
this process is N = {1, 2, ...}. However, if the distribution of the increments f (k ) = f k = P{X 1 = k }
puts no mass on the integers > M then all of these will be inaccessible from state 1. So when
the distribution { f k }k ≥1 has finite support, it’s more natural to let the state space be [m ] :=
4
RENEWAL THEORY
{1, 2, . . . , m } where m is the largest integer such that f m > 0. With this convention, the Markov
chain R n is irreducible. Its transition probabilities are
p (x , x − 1) = 1
if x ≥ 2;
p (1, x ) = f x
for x = 1, 2, ....
This makes it clear (if it wasn’t already) that R n is a recurrent Markov chain, because from any
initial state x ≥ 1 there is P x −probability one that R n will visit state 1. Furthermore, the expected
return time to state 1 is
∞
X
(x − 1)f x = µ − 1,
x =1
where µ is the mean of the increment distribution. Thus, the Markov chain is positive recurrent
if and only if µ < ∞.
Assume then that µ < ∞. Since the Markov chain R n is irreducible and positive recurrent it
has a unique stationary distribution πx . This must satisfy the equations
πx = πx +1 + π1 f x
for all x ≥ 1.
These are easy to solve: the solution is
X
1X
(7)
πx =
f k /π1 =
fk
µ
k ≥x
for all x ≥ 1.
k ≥x
Exercise: Why is 1/µ the correct normalizing factor?
Exercise: What is the stationary distribution of the Markov chain A n ? How about (A n , R n )?
2. THE RENEWAL THEOREM
2.1. Renewal Theorem: Statement. Consider now an arithmetic renewal process {S n }. Let {u m }m ≥0
be the associated renewal measure, that is, u m is the probability that the random walk {S n } ever
visits the point m . Since the random walk makes only positive steps, it can only visit a point m
at most once, so by the law of total probability,
(8)
um =
∞
X
P{S n = m }.
n =0
The cornerstone of renewal theory (in the arithmetic case) is the following theorem, due to
FELLER, ERDÖS, & POLLARD, which asserts that the sequence u m converges to a constant.
Feller-Erdös-Pollard Theorem . Assume that the step distribution of the renewal process is not
fully concentrated on any proper additive subgroup of the integers.1 Then
(9)
lim u m = 1/µ,
m →∞
where µ is the mean of the step distribution.
This gives a partial answer to the dice-rolling problem (Problem 1) of section 1.1: It implies
that if x is large, the chance that S n = x for some n is about 1/µ = 1/(3.5) = 2/7. Later we will see
how to get much more precise numerical approximations.
1Equivalently, there is no integer h ≥ 2 such that the step distribution is concentrated on the set hZ of integer
multiples of h.
RENEWAL THEORY
5
Proof of the Feller-Erdös-Pollard Theorem. Consider the residual lifetime chain R n . Every time
that R n = 1 there will be a renewal at time n + 1; conversely, every renewal (except the one at
m = 0) must be preceded by a visit to 1 by R n . Hence,
u0 = 1
and u m = P{R m −1 = 1} for m = 2, 3, ....
By Kolmogorov’s convergence theorem for positive recurrent Markov chains, in the case µ <
∞ the probability that R m −1 = 1 converges as m → ∞ to the steady-state probability π1 . In
section 1.7 we saw that π1 = 1/µ. Similarly, Kolmogorov’s theorem for null recurrent Markov
chains implies that when µ = ∞ the probability P{R m −1 = 1} converges to 0 = 1/µ.
Feller, Erdös, and Pollard seem not to have realized at the time they wrote their paper that
their theorem was essentially a trivial consequence of Kolmogorov’s theorem, which had been
proved over 15 years earlier (I think). They gave two different analytical proofs2, both harder
than that above. It was pointed out somewhat later that Kolmogorov implies F-E-P, and also that
F-E-P implies Kolmogorov’s theorem (by using the embedded renewal processes exhibited in
section 1.4 above). In section 3 below, I’ll give yet another analytic proof, based on generating
functions, for the special case where the step distribution has finite support.
2.2. The Renewal Equation. The usefulness of the Feller-Erdös-Pollard theorem derives partly
from its connection with another basic theorem called the Key Renewal Theorem (see below)
which describes the asymptotic behavior of solutions to the Renewal Equation. The Renewal
Equation is a convolution equation relating bounded sequences {z (m )}m ≥0 and {b (m )}m ≥0 of
real numbers:
Renewal Equation, First Form:
(10)
z (m ) = b (m ) +
m
X
f (k )z (m − k ).
k =1
Here f (k ) = f k is the interoccurrence time distribution for the renewal process. There is an
equivalent way of writing the Renewal Equation that is more suggestive of how it actually arises
in practice. Set z (m ) = b (m ) = 0 for m < 0; then the upper limit k = m − 1 in the sum in the
Renewal Equation may be changed to m = ∞ without affecting its value. The Renewal Equation
may now be written as follows, with X 1 representing the first interoccurrence time:
Renewal Equation, Second Form:
(11)
z (m ) = b (m ) + E z (m − X 1 ).
As you will come to appreciate, renewal equations crop up all over the place. In many circumstances, the sequence z (m ) is some scalar function of time whose behavior is of some interest;
the renewal equation is gotten by conditioning on the value of the first interoccurrence time. In
carrying out this conditioning, it is crucial to realize that the sequence S ∗1 ,S ∗2 , . . . defined by
S ∗n = S n − X 1 =
n
X
Xj
j =2
is itself a renewal process, independent of X 1 , and with the same interoccurrence time distribution f (x ).
2I know of at least four other proofs of a more probabilistic nature.
6
RENEWAL THEORY
Example 1: The Renewal Measure. The renewal measure is the sequence u (k ) = probability that
there is a renewal at time k . If k = 0, then u (k ) = 1, because in an ordinary renewal process there
is always a renewal at time 0. Suppose that k ≥ 1: then in order that there be a renewal at k , on
the event that X 1 = m , it must be the case that the renewal process S ∗1 ,S ∗2 , . . . has an occurrence
at time k − m . Thus, the renewal measure satisfies the Renewal Equation
u (k ) = δ0 (k ) + E u (k − S 1 ),
(12)
where δ0 (·) is the Kronecker delta function. Here is a formal proof:
u (k ) =
∞
X
P{S n = k }
n =0
= P{S 0 = k } +
= δ0 (k ) +
= δ0 (k ) +
= δ0 (k ) +
= δ0 (k ) +
∞
X
P{S n = k }
n =1
∞ X
∞
X
n =1 x =1
∞ X
∞
X
n =1 x =1
∞
X
P{X 1 = x and S ∗n = k − x }
P{X 1 = x }P{S ∗n = k − x }
f (x )
x =1
∞
X
∞
X
n =1
P{S ∗n = k − x }
f (x )u (k − x )
x =1
= δ0 (k ) + E u (k − X 1 )
The change in the order of summation on the fifth line is justified because all the terms of the
double sum are nonnegative.
Example 2: The Total Lifetime Distribution. Let L(m ) = A(m ) + R(m ) be the total lifetime of the
component in use at time m . (Here A(m ) and R(m ) are the age and residual lifetime random
variables, respectively.) Fix r ≥ 1, and set z (m ) = P{L(m ) = r }. Then z satisfies the Renewal
Equation (11) with
b (m ) = f (r ) for m = 0, 1, 2, . . . , r − 1
(13)
=0
for m ≥ r
EXERCISE: Derive this.
2.3. Solution of the Renewal Equation. Consider the Renewal Equation in its second form z (m ) =
b (m )+ E z (m −X 1 ) where by convention z (m ) = 0 for all negative values of m . Since the function
z (·) appears on the right side as well as on the left, it is possible to resubstitute on the right. This
leads to a sequence of equivalent equations:
z (m ) = b (m ) + E z (m − X 1 )
= b (m ) + E b (m − X 1 ) + E z (m − X 1 − X 2 )
= b (m ) + E b (m − X 1 ) + E b (m − X 1 − X 2 ) + E z (m − X 1 − X 2 − X 3 )
RENEWAL THEORY
7
and so on. After m iterations, there is no further change (because S m +1 > m and z (l ) = 0 for all
negative integers l ), and the right side no longer involves z . Thus, it is possible to solve for z in
terms of the sequences b and p :
z (m ) =
=
=
=
∞
X
E b (m − S n )
n =0
∞ X
∞
X
n =0 x =0
∞ X
∞
X
b (m − x )P{S n = x }
b (m − x )P{S n = x }
x =0 n =0
∞
X
b (m − x )u (x ).
x =0
Note that only finitely many terms in the series are nonzero, so the interchange of summations
is justified. Thus, the solution to the Renewal Equation is the convolution of the sequence b (m )
with the renewal measure:
(14)
z (m ) =
∞
X
b (m − x )u (x )
x =0
2.4. The Key Renewal Theorem. The formula (14) and the Feller-Erdös-Pollard theorem now
combine to give the asymptotic behavior (as m → ∞) of the solution z .
Key Renewal Theorem (Lattice Case) . Let z (m ) be the solution to the Renewal Equation (11). If
the sequence b (m ) is absolutely summable, then
(15)
lim z (m ) = µ−1
m →∞
∞
X
b (k ).
k =0
Proof. The formula (14) may be rewritten as
(16)
z (m ) =
∞
X
b (k )u (m − k )
k =0
For each fixed k , the sequence u (m − k ) → µ−1 as m → ∞, by the Feller-Erdös-Pollard theorem.
Thus, as m → ∞, the k th term of the series (16) converges to b (k )/µ. Moreover, because u (m −
k ) ≤ 1, the k th term is bounded in absolute value by |b (k )|. By hypothesis, this sequence is
summable, so the Dominated Convergence Theorem implies that the series converges as m → ∞
to the right side of (15).
Example 4: Limiting Residual Lifetime Distribution. For each fixed r ≥ 1, the sequence z (m ) =
P{R(m ) = r } satisfies the renewal equation
(17)
z (m ) = P{X 1 = m + r } +
m
X
k =1
z (m − k )P{X 1 = k } = f m +r +
m
X
k =1
z (m − k )f k .
8
RENEWAL THEORY
This reduces to (11) with b (m ) = f (m + r ). The sequence b (m ) is summable, because µ = E X 1 <
∞ (why?). Therefore, the Key Renewal Theorem implies that for each r = 1, 2, 3, . . . ,
(18)
lim P{R(m ) = r } = µ−1
m →∞
∞
X
f (k + r ) = µ−1 P{X 1 ≥ r }.
k =0
This could also be deduced from the convergence theorem for Markov chains, using the fact that
the sequence R m is an irreducible, positive recurrent Markov chain with stationary distribution
(7).
Example 5: Limiting Total Lifetime Distribution. Recall (Example 3 above) that the sequence
z (m ) = P{L(m ) = r } satisfies the Renewal Equation (11) with b (m ) defined by (13). Only finitely
many terms of the sequence b (m ) are
P nonzero, and so the summability hypothesis of the Key
Renewal Theorem is satisfied. Since k ≥0 b (m ) = r f (r ), it follows from (15) that
Corollary 1.
(19)
lim P{L(m ) = r } = r f (r )/µ.
m →∞
Example 6: A renewal equation in disguise. Discrete convolution equations arise in many parts
of probability and applied mathematics, but often with a “kernel” that isn’t a proper probability distribution. It is important to realize that such equations can be converted to (standard)
renewal equations by the device known as exponential tilting. Here is a simple example.
Consider the Fibonacci sequence 1,1,2,3,5,8,. . . . This is the sequence a n defined by the recursion
a n +2 = a n +1 + a n
(20)
and the initial conditions a 0 = a 1 = 1. To convert the recursion to a renewal equation, multiply
a n by a geometric sequence:
z n = θ −n a n
for some value θ > 0. Also, set z n = 0 for n ≤ −1. Then (20) is equivalent to the equation
(21)
z n = z n −1 θ 1 + z n−2 θ 2 + δ0,n
where δi ,j is the Kronecker delta function. (The delta function makes up for the fact that the
original Fibonacci recursion (20) does not by itself specify a 0 and a 1 .) Equation (21) is a renewal
equation for any value of θ > 0 such that θ 1 + θ2 = 1, because then we can set f 1 = θ 1 , f 2 = θ 2 ,
and f m = 0 for m ≥ 3. Is there such a value of θ ? Yes: it’s the golden ratio
p
−1 + 5
.
θ=
2
(The other root is negative, so we can’t use it to produce a renewal equation from (20).) The Key
Renewal Theorem implies that
lim z n = lim θ −n a n = 1/(θ + 2θ 2 ).
n →∞
n→∞
Thus, the Fibonacci sequence grows at an exponential rate, and the rate is the inverse of the
golden ratio.
RENEWAL THEORY
9
3. GENERATING FUNCTIONS AND RENEWAL THEORY
3.1. Generating Functions. The Renewal Theorem tells us how the age and residual lifetime
distributions behave at large times, and similarly the Key Renewal Theorem tells us how the
solution of a renewal equation behaves at infinity. In certain cases, exact calculations can be
done. These are usually done using generating functions.
Definition 1. The generating function of a sequence {a n }n ≥0 of real (or complex) numbers is the
function A(z ) defined by the power series
A(z ) =
(22)
∞
X
anzn.
n =0
Observe that for an arbitrary sequence {a n } the series (22) need not converge for all complex
values of the argumetn z . In fact, for some sequences the series (22) diverges for every z except
z = 0: this is the case, for instance, if a n = n n . But for many sequences of interest, there will
exist a positive number R such that, for all complex numbers z such that |z | < R, the series (22)
converges absolutely. In such cases, the generating function A(z ) is said to have positive radius of
convergence. The generating functions in all of the problems considered in these notes will have
positive radius of convergence. Notice that if the entries of the sequence a n are probabilities,
that is, if 0 ≤ a n ≤ 1 for all n, then the series (22) converges absolutely for all z such that |z | < 1.
If the generating function A(z ) has positive radius of convergence then, at least in principal,
all information about the sequence {a n } is encapsulated in the generating function A(z ). In
particular, each coefficient a n can be recovered from the function A(z ), since n!a n is the nth
derivative of A(z ) at z = 0. Other information may also be recovered from the generating function: for example, if the sequence {a n } is a discrete probability density, then its mean may be
obtained by evaluating A 0 (z ) at z = 1, and all of the higher moments may be recovered from the
higher derivatives of A(z ) at z = 1.
A crucially important property of generating functions is the multiplication law: The generating function of the convolution of two sequences is the product of their generating functions. This is the basis of most uses of generating functions in random walk theory, and all of
the examples considered below. The calculation that establishes this property is spelled out in
section ??. Note that for probability generating functions, this fact is a consequence of the multiplication law for expectations of independent random variables: If X and Y are independent,
nonnegative-integer valued random variables, then
E z X +Y = E z X E z Y .
(23)
3.2. The Renewal Equation.
Let { f k }k ≥1 be a probability distribution on the positive integers
P
with finite mean µ = k f k , and let X 1 , X 2 , . . . be a sequence of independent, identically distributed random variables with common discrete distribution { f k }. Define the renewal sequence
associated to the distribution { f k } to be the sequence
(24)
u m = P{S n = m for some n ≥ 0}
=
∞
X
P{S n = m }
n=0
where S n = X 1 + X 2 + · · · + X n (and S 0 = 0). note that u 0 = 1 A simple linear recursion for the
sequence u m , called the renewal equation, may be obtained by conditioning on the first step
10
RENEWAL THEORY
S1 = X 1:
(25)
um =
m
X
f k u m −k + δ0 (m )
k =1
where δ0 is the Kronecker delta (1 at 0; and 0 elsewhere).
The renewal equation is a particularly simple kind of recursive relation: the right side is just
the convolution of the sequences f k and u m . The appearance of a convolution should always
suggest the use of some kind of generating function or transform (Fourier or Laplace), because
these always convert convolution to multiplication. Let’s try it: define generating functions
∞
X
(26)
U (z ) =
um z m
and
(27)
F (z ) =
m =0
∞
X
fkzk.
k =1
Observe that if you multiply the renewal equation (25) by z m and sum over m then the left side
becomes U (z ), so
∞ X
m
X
(28)
U (z ) = 1 +
f k u m −k z m
=1+
m =0 k =1
∞ X
∞
X
f k z k u m −k z m −k
k =1 m =k
= 1 + F (z )U (z ).
Thus, we have a simple functional equation relating the generating functions U (z ) and F (z ). It
may be solved for U (z ) in terms of F (z ):
(29)
U (z ) =
1
1 − F (z )
3.3. Partial Fraction Decompositions. Formula (29) tells us how the generating function of the
renewal sequence is related to the probability generating function of the steps X j . Extracting useful information from this relation is, in general, a difficult analytical task. However, in the special
case where the probability distribution { f k } has finite support, the method of partial fraction decomposition provides an effective method for recovering the terms u m of the renewal sequence.
Observe that when the probability distribution { f k } has finite support, its generating function
F (z ) is a polynomial, and so in this case the generating function U (z ) is a rational function.3
The strategy behind the method of partial fraction decomposition rests on the fact that a simple pole may be expanded as a geometric series: in particular, for |z | < 1,
∞
X
−1
(30)
(1 − z ) =
zn.
n =0
Differentiating with respect to z repeatedly gives a formula for a pole of order k + 1:
∞ X
n n −k
−k −1
(31)
(1 − z )
=
z
.
k
n =k
3A rational function is the ratio of two polynomials.
RENEWAL THEORY
11
Suppose now that we could write the generating function U (z ) as a sum of poles C /(1−(z /ζ))k +1
(such a sum is called a partial fraction decomposition). Then each of the poles could be expanded
in a series of type (30) or (31), and so the coefficients of U (z ) could be obtained by adding the
corresponding coefficients in the series expansions for the poles.
Example: Consider the probability distribution f 1 = f 2 = 1/2. The generating function F is given
by F (z ) = (z + z 2 )/2. The problem is to obtain a partial fraction decomposition for (1 − F (z ))−1 .
To do this, observe that at every pole z = ζ the function 1 − F (z ) must take the value 0. Thus, we
look for potential poles at the zeros of the polynomial 1 − F (z ). In the case under consideration,
the polynomial is quadratic, with roots ζ1 = 1 and ζ2 = −2. Since each of these is a simple root
both poles should be simple; thus, we should try
C1
C2
1
=
+
.
1 − (z + z 2 )/2 1 − z 1 + (z /2)
The values of C 1 and C 2 can be gotten either by adding the fractions and seeing what works or by
differentiating both sides and seeing what happens at each of the two poles. The upshot is that
C 1 = 2/3 and C 2 = 1/3. Thus,
(32)
U (z ) =
1
2/3
1/3
=
+
.
1 − F (z ) 1 − z 1 + (z /2)
We can now read off the renewal sequence u m by expanding the two poles in geometric series:
(33)
um =
2 1
+ (−2)m .
3 3
There are several things worth noting. First, the renewal sequence u m has limit 2/3. This
equals 1/µ, where µ = 3/2 is the mean of the distribution { f k }. We should be reassured by this,
because it is what the Feller-Erdös-Pollard Renewal Theorem predicts the limit should be. Second, the remainder term (1/3)(−2)−m decays exponentially in m . This is always the case for
distributions { f k } with finite support. It is not always the case for arbitrary distributions { f k },
however.
Problem 1. Recall that the Fibonacci sequence 1, 1, 2, 3, 5, 8, . . . is the sequence a n such that a 1 =
a 2 = 1 and such that
a m +2 = a m + a m +1 .
(A) Find a functional equation for the generating function of the Fibonacci sequence. (B) Use the
method of partial fractions to deduce an exact formula for the terms of the Fibonacci sequence
in terms of the golden ratio.
3.4. Step Distributions with Finite Support. Assume now that the step distribution { f k } has finite support, is nontrivial (that is, does not assign probability 1 to a single point) and is nonlattice
(that is, it does not
P give probability 1 to a proper arithmetic progression). Then the generating
function F (z ) = f k z k is a polynomial of degree at least two. By the Fundamental Theorem of
Algebra, 1 − F (z ) may be written as a product of linear factors:
(34)
1 − F (z ) = C
K
Y
(1 − z /ζ j )
j =1
The coefficients ζ j in this expansion are the (possibly complex) roots of the polynomial equation
F (z ) = 1. Since the coefficients f k of F (z ) are real, the roots of F (z ) = 1 come in conjugate pairs;
thus, it is only necessary to find the roots in the upper half plane (that is, those with nonnegative
12
RENEWAL THEORY
imaginary part). In practice, it is usually necessary to solve for these roots numerically. The
following proposition states that none of the roots is inside the unit circle in the complex plane.
Lemma 2. If the step distribution { f k } is nontrivial, nonlattice, and has finite support, then the
polynomial 1−F (z ) has a simple root at ζ1 = 1, and all other roots ζ j satisfy the inequality |ζ j | > 1.
Proof. It is clear that ζ1 = 1 is a root, since F (z ) is a probability generating function. To see that
ζ1 = 1 is a simple root (that is, occurs only once in the product (34)), note that if it were a multiple
root then it would have to be a root of the derivative F 0 (z ) (since the factor (1 − z ) would occur
at least twice in the product (34)). If this were the case, then F 0 (1) = 0 would be the mean of the
probability distribution { f k }. But since this distribution has support {1, 2, 3 · · · }, its mean is at
least 1.
In order that ζ be a root of 1 − F (z ) it must be the case that F (ζ) = 1. Since F (z ) is a probability
generating function, this can only happen if |ζ| ≥ 1. Thus, to complete the proof we must show
that there are no roots of modulus one other than ζ = 1. Suppose, then, that ζ = e i θ is such that
F (ζ) = 1, equivalently,
X
f k e i θ k = 1.
Then for every k such that f k > 0 it must be that e i θ k = 1. This implies that θ is an integer
multiple of 2π/k , and that this is true for every k such that f k > 0. Since the distribution { f k }
is nonlattice, the greatest common divisor of the integers k such that f k > 0 is 1. Hence, θ is an
integer multiple of 2π, and so ζ = 1.
Corollary 3. If the step distribution { f k } is nontrivial, nonlattice, and has finite support, then
(35)
R
X
Cr
1
1
=
+
1 − F (z ) µ(1 − z ) r =1 (1 − z /ζr )k r
where µ is the mean of the distribution { f k } amd the poles ζr are all of modulus strictly greater
than 1.
Proof. The only thing that remains to be proved is that the simple pole at 1 has residue 1/µ. To
see this, multiply both sides of equation (35) by 1 − z :
X
Cr
1−z
= C + (1 − z )
.
1 − F (z )
(1 − z /ζr )k r
r
Now take the limit of both sides as z → 1−: the limit of the right side is clearly C , and the limit of
the left side is 1/µ, because µ is the derivative of F (z ) at z = 1. Hence, C = 1.
Corollary 4. If the step distribution { f k } is nontrivial, nonlattice, and has finite support, then
(36)
lim u m = 1/µ,
m →∞
and the remainder decays exponentially as m → ∞.
Remark. The last corollary is a special case of the Feller-Erdös-Pollard Renewal Theorem. This
theorem asserts that (9) is true under the weaker hypothesis that the step distribution is nontrivial and nonlattice. Various proofs of the Feller-Erdös-Pollard theorem are known, some of
which exploit the relation (29). Partial fraction decomposition does not work in the general case,
though.
RENEWAL THEORY
13
4. RENEWAL THEORY: NONLATTICE CASE
Renewal theory in the nonlattice case is technically more complicated, but the main results
are very similar to those in the lattice case once the obvious adjustments for continuous time
(e.g., changing sums to integrals) are made. Since many of the arguments are parallel to those in
the lattice case, we shall merely record the main results without proofs.
A nonlattice distribution is one which is not lattice, that is, does not attach all of its probability
to an arithmetic progression hZ. Probability distributions with continuous densities f (x ) for
x > 0 are nonlattice, but so are lots of other distributions that do not have densities, including
some discrete ones. An example that will turn out to be interesting is the discrete distribution
that assigns mass f k to the point 2−k for k = 0, 1, 2, 3, . . . , with f k > 0 for infinitely many values of
k (note that if f k > 0 for only finitely many k then the distribution would be lattice). Throughout
this section we assume that 0,S 1 ,S 2 , . . . is a renewal process with interoccurrence distribution F
satisfying
Assumption 1. The interoccurrence time distribution F is supported by the positive real numbers and is nonlattice.
R
Assumption 2. The interoccurrence time distribution has finite mean µ = x d F (x ).
Renewal Measure: In the nonlattice case, the renewal measure is defined by its cumulative mass
function
X
(37)
U (t ) = 1 + E N (t ) =
P{S n ≤ t }
n =0
Blackwell’s Theorem . For each positive real number h > 0,
(38)
lim (U (t + h) − U (t )) = h/µ.
t →∞
This is the natural analogue of the Feller-Erdös-Pollard Theorem for nonlattice renewal processes. As in the lattice case, the practical importance of the Blackwell Theorem stems from
its connection with a Key Renewal Theorem that describes the asymptotic behavior of solutions
to the Renewal Equation. The Renewal Equation in the nonlattice case is a convollution equation relating two (measurable) functions z (t ) and b (t ) that are defined for all real t , and satisfy
z (t ) = b (t ) = 0 for all t < 0.
Renewal Equation:
t
Z
(39)
z (t ) = b (t ) + E z (t − X 1 ) = b (t ) +
z (t − s ) d F (s ).
0
Technical Note: The notation d F denotes integration with respect to the interoccurrence
disR
tribution. If this distribution has a density f (t ) then d F (t ) = f (t )d t . Otherwise, d F must be
interpreted as a Lebesgue-Stieltjes integral. If you have never heard of these, don’t worry — all
the important examples in this course will either be purely discrete or will have a density.
Just as in the lattice case, the renewal equation has a unique solution z (t ) provided b (t ) is
bounded. There is an explicit formula for this:
14
RENEWAL THEORY
z (t ) =
(40)
∞
X
∞
Z
E b (t − S n ) =
b (t − s ) d U (s ).
s =0
n =0
The Key Renewal Theorem for the nonlattice case differs from the corresponding theorem for
the lattice case in an important technical respect. In the lattice case, it was necessary only to
assume that the sequence b (m ) be absolutely summable. Thus, it is natural to expect that in
the nonlattice case it should be enough to assume that the function b (t ) be L 1 , that is, absolutely
integrable. However, this isn’t so! The right condition turns out to be direct Riemann integrability,
which is defined as follows.
Definition 2. Let b (t ) be a real-valued function of t ∈ R such that b (t ) = 0 for all t < 0. Then b (t )
is said to be directly Riemann integrable if
Z
∞
∞
X
X
(41)
lim
a
inf
b (t ) = lim
a
sup
b (t ) = b (t ) d t .
a →0+
n=1
(n −1)a ≤t ≤na
a →0+
n =1
(n−1)a ≤t ≤na
Notice that the sums would be Riemann sums if not for the infinite limit of summation. If a
function is directly Riemann integrable, then it is Lebesgue integrable (if you don’t know what
this is, don’t worry about it), but the converse is not true. In practice, one rarely needs to deal
with the condition (41), because there are relatively simple sufficient conditons that guarantees
direct Riemann integrability:
Lemma R5. If b (t ) is nonnegative and nonincreasing on [0, ∞), and if the Lebesgue (or Riemann)
∞
integral 0 b (t ) d t < ∞, then b (t ) is directly Riemann integrable.
Lemma 6. If b (t ) has compact support and is Riemann integrable then it is directly Riemann
integrable.
The proofs are omitted but are not extremely difficult.
Key Renewal Theorem (Nonlattice Case) . Assume that the interoccurrence time distribution is
nonlattice and has finite mean µ > 0. If b (t ) is directly Riemann integrable then the solution z (t )
of the renewal equation (39) satisfies
Z∞
(42)
lim z (t ) = µ−1
t →∞
b (t ) d t .
0