Download 7 recurrence relations

Recursion
• Review of recursive functions
• Recursive definitions
– In the case of defining a set of strings, compare with
regular expressions
• Later: Solving recurrence relations
– The purpose is to convert a recursive definition of a
sequence of numbers into an explicit definition.
– Simple 1st and 2nd order
– Verifying explicit formula matches recursive rule
– Inhomogeneous recurrence relations
Recursive functions
• A function that calls itself
• To make it work, there should also be a base case to
exit the recursion.
• Factorial
– fact(0) = 1
– fact(n) = n * fact(n – 1) for n  1
• Fibonacci
– F(1) = 1, and F(2) = 1
– F(n) = F(n – 1) + F(n – 2) for n  3
– Question: Can you write a formula for F(n+4) in terms of
just F(n) and F(n+1)?
More examples
• Chebyshev (Чебышёв) polynomials
– T0(x) = 1
– T1(x) = x
– Tn(x) = 2x Tn–1 (x) – Tn–2 (x)
• Pascal’s triangle / binomial coefficients
– How should we define this one?
r=0
r=1
r=2
r=3
n=0
1
n=1
1
1
n=2
1
2
1
n=3
1
3
3
1
n=4
1
4
6
4
r=4
1
Recursive definition
• How do we write a recursive definition, e.g. for a list
of numbers?
• Often, it’s sufficient to do…
– Base case(s): Write the rule for the first value.
– Recursive rule: How to go from one value to the next?
• Example: 4, 7, 10, 13, 16, …
a1 = 4, and for n >= 2, an = an–1 + 3
• Example: 5, 10, 20, 40, 80, …
a1 = 5, and for n >= 2, an = 2 an–1
• Example: 7, –9, 11, –13, 15, –17, 19, –21, …
a1 = 7, and for n >= 2, an = (–1)n+1 (|an–1|+ 2)
For strings
• Often a set of strings is introduced this way. 
• Skill: given a verbal description of a set of strings,
write a precise definition
• Base case(s): think about the shortest string that
satisfies the definition
• Recursive case(s): how do we build larger strings?
• *** See handout
• Given a recursive definition, we can prove using
induction that an essential property (e.g. even
length) is satisfied.
Regular expressions (review)
• Often, a set of strings does not have to be defined recursively.
• A regular expression gives the essential format of a string in
the language, using these operators:
– Concatenation
– +, which means “or”
– *, which means 0 or more instances of
• Examples… (can you convert these to recursive definitions?)
– 001(0+1)*
– (0+1)0(0+1)*
– 0(0+1)*1
– 0*10* + 1*01*
– ((0+1)1)*
Recurrence relations
• A sequence of numbers defined recursively
• Often we need to “solve” a recurrence: Simplify a
recursive formula into an explicit one.
– Why? Somebody might ask for the 100th element!
• Examples:
– a1 = 3 and an = an–1 + 5
– a0 = 1 and an = 2 an–1
• If we expand the recursive rule a couple of times, we
see a simple pattern emerge, and we can write an
explicit formula. 
Simple types
• Let’s begin by considering 3 kinds of recurrences:
1. Adding a constant
a1 = 6
an = an–1 + 4
2. Multiplying by a constant
a1 = 5
an = 3 an–1
3. Both
a1 = 7
an = 2 an–1 + 1
• The first two are simple sequences. The third one
takes some work.
Geometric series
• Before we dive into an example, let’s review
geometric series.
• Let r = common ratio; n = # terms; f = first term
• Finite number of terms
Sum = f * (rn – 1) / (r – 1)
Example: 5 + 10 + 20 + 40 + 80 + 160.
sum = 5 * (26 – 1) / (2 – 1) = 5 * 63 = 315.
• Infinite number of terms
Sum = f / (1 – r)
Only works if –1 < r < 1. Otherwise sum diverges.
Example of 3rd type
• a0 = 1 and an = 3 an–1 + 5
• Expand the recursive definition to detect a pattern.
a1 = 3a0 + 5
a2 = 3a1 + 5 = 3(3a0 + 5) + 5 = 9a0 + 3 * 5 + 5
a3 = 3a2 + 5 = 3(9a0 + 3*5 + 5) + 5 = 27a0 + 9*5 + 3*5 + 5
a4 = 3a3 + 5 = 3(27a0 + 9*5 + 3*5 + 5) + 5
= 81a0 + 27*5 + 9*5 + 3*5 + 5 = 34a0 + 5*(33 + 32 +31 +30)
So, an = 3na0 + 5*(3n – 1)/2
• Since a0 = 1, we conclude: an = (7/2) 3n – (5/2)
• If we didn’t know value of a0, we would leave it in
our answer. Doing so is not considered recursion.
Additional types
• Here are some more interesting recurrence relation
types
4. Based on the previous two terms
a1 = 2, a2 = 1, an = 6 a n – 1 + 7 a n – 2
– Note the two base cases!
5. Previous two terms, plus an independent term
a1 = 3, a2 = 0, an = 7 a n – 1 + 8 a n – 2 + 4n
– This is called an “inhomogeneous” recurrence relation, and
we’ll do these last.
Remarks
• In general, recurrence relations are difficult to solve,
because we have to expand and look for a pattern.
• In some cases, there is a much more straightforward
approach:
Type
First order
Second order
Form of recursive rule Form of solution
an = K an – 1
___ K n
an = K an – 1 + L an – 2
___ p n + ___ q n
• In second order, we create a quadratic equation
using K and L. Let p and q be the roots.
Solving 2nd order
• First, set equal to zero, and solve the corresponding
quadratic equation with roots p and q.
• Form of the solution is an = c1 p n + c2 q n, where c1
and c2 are determined by the base cases.
– If there is a double root, insert factor n in one term.
• Try these examples
a0 = 7 a1 = 1
an = 5 an – 1 – 6 an – 2
a0 = 1 a1 = 1
an = 3 an – 1 + 4 an – 2
a0 = 2 a1 = 5
an = 7 an – 1 – 12 an – 2
a0 = 1 a1 = 1
an = 6 an – 1 – 9 an – 2
a0 = 4 a1 = 10 an = 6 an – 1 – 8 an – 2
Verifying
• Now that we can solve a recurrence, it would be nice
to check our answer for all n. Use induction. 
• We have 2 ways to write a function: recursive and
explicit. Pn is the statement that n  0, the two
formulas are equivalent.
• First step: Usually there are 2 base cases, e.g. n = 0
and n = 1. Verify them individually:
– recursive(0) = explicit(0)
– recursive(1) = explicit(1)
Verifying (2)
• Next, assume Pk is true, which says the recursive and
explicit are equal all the way up to n = k. In other
words, we can say: recursive(k) = explicit(k)
• Next, we consider the next term of the sequence,
and write out the formula for recursive(k+1).
– It should look like this:
recursive(k+1) = … recursive(k) + … recursive(k–1)
– On the right hand side, replace recursive calls to k and k –
1 with explicit calls to k and k – 1.
recursive(k+1) = … explicit(k) + … explicit(k–1)
– Finally, simplify the right side until it looks like explicit(k+1).
Then your final equation is Pk+1.
Verifying (3)
• Let’s try one we just solved. We’d like to verify that an = 20(2n) –
13(3n) solves the recurrence a0 = 7, a1 = 1, an = 5 an–1 – 6 an–2
• Begin with base cases.  Plug 0 and 1 into explicit formula to see
if they match base case values 7 and 1.
• Next, assume that for some k, the explicit formula is correct. In
other words:
ak = 5 ak – 1 – 6 ak – 2 = 20(2k) – 13(3k)
• Let’s compute the recursive formula for k+1, and substitute the
explicit formula where we can.
ak+1 = 5 ak – 6 ak – 1
= 5[20(2k) – 13(3k)] – 6[20(2k – 1) – 13(3k – 1)]
= 100(2k) – 65(3k) – 120(2k – 1) + 78(3k – 1)
= 100(2k) – 65(3k) – 60(2k) + 26(3k)
= 40(2k) – 39(3k) = 20(2k+1) – 13(3k+1)
Note this last equation is Pk+1.
Non-homogeneous case
• More about recurrence relations
– i.e. recursively defined sequence of values
– “solving” one means re-write as explicit formula
– Purposes: explicit may be more efficient to evaluate;
useful in analysis of an algorithm
• Two kinds
– Homogeneous: In the recursive rule, all terms refer to the
sequence (e.g. an or an-1 or an-2) √
– Non-homogeneous: Some terms do not refer to the
sequence (e.g. 15, 3n2, 2n)
• Today we will solve non-homogeneous problems, since this is
the more general case.
What they look like
• Here is a non-homogeneous recurrence relation:
an = 2an-1 – an-2 + 7
• We would also specify base cases, e.g. a2 and a3.
• We customarily write all of the sequence terms on
the left, to obtain:
an – 2an-1 + an-2 = 7
• Notice the right side is not 0. If it were 0, this would
just be a homogeneous equation, which we know
how to solve already.
• Note: no limit on the number of terms on left.
Usually 2, 3, or 4.
Procedure
1. Find a particular solution to the recurrence.
a. Examine right side of equation to deduce the general
form of the solution, e.g. 2nd-degree polynomial. This will
be some function of n.
b. Express an , an-1 , an-2 using this general form by plugging
in n, (n – 1), (n – 2), respectively.
c. Substitute into LHS of equation
d. Determine values of constant coefficients.
2. Find a homogeneous solution to the recurrence.
a. Solve the parallel (characteristic) equation. Roots
become exponentials. E.g. x = 2 means we need 2n.
b. Write general form of homogeneous solution, and add
the particular solution you found in #1.
c. Use base cases to determine coefficient values.
Example #1
•
•
•
•
•
an – 5an-1 + 6an-2 = 1
a0 = 3/2
a1 = 9/2
The right side of equation is a constant, so this is the
form of our particular solution for an. Let it be c.
In this case, if an = c, then an-1 = c and an-2 = c also.
Re-write the equation, substituting:
c – 5c + 6c = 1
We need to determine c. We get 2c = 1  c = 1/2.
So, our particular solution is just an = 1/2. What does
this mean? It tells us by how much our solution
differs from the homogeneous situation.
continued
•
•
•
•
•
an – 5an-1 + 6an-2 = 1
a0 = 3/2
a1 = 9/2
We now need a homogeneous solution.
The characteristic equation is x2 – 5x + 6 = 0. Roots
are x = 2 and 3.
Homogeneous solution has form c1 2n + c2 3n.
Let’s add (particular) + (homogeneous):
an = c1 2n + c2 3n + 1/2
Use base cases to determine c1 and c2, and we’re
done. Solution is an = – 2n + (2) 3n + 1/2
Particular forms
• The first step is to determine the format of the particular
solution. Based on the right side of the recurrence equation.
RHS
Root of Char.
Polynomial
Form of particular soln
Polynomial
1 is NOT a root
Polynomial (of same degree)
Polynomial
1 is a root, r times
Polynomial * nr
k cn
c is NOT a root
a cn
k cn
c is a root, r times
a n r cn
Polynomial * k cn c is NOT a root
Polynomial * cn
Polynomial * k cn c is a root, r times
Polynomial * nr cn
Example forms
RHS
Form of particular soln
7
5n2
(18) 3n, char roots are 5, 2
(n+1) 3n, char roots are 5, 2
c
an2 + bn + c
a 3n
(an + b) 3n
(12) 3n, char roots are 5, 3
(20) 3n, char roots are 3, 3
(n+1) 2n, char roots are 2, 2
an 3n
an2 3n
(an + b) n2 2n
• Special rule: if 1 is a characteristic root, you need to multiply
the particular form by n. If 1 is a root of multiplicity r, multiply
by nr.
Example #2
an + 5an-1 + 6an-2 = 42*4n
a2 = 278
• Form of particular solution is c*4n.
• In this case, we have
an = c * 4n
an-1 = c * 4n – 1
a3 = 962
an-2 = c * 4n – 2
• Substitute into equation, to obtain:
(c * 4n) + 5(c * 4n – 1) + 6(c * 4n – 2) = 42*4n
c * 4n + (5/4)c * 4n + 6/16c * 4n = 42 *4n
c (1 + 5/4 + 3/8) = 42  c = 16
• So, our particular solution is an = 16 * 4n.
continued
•
•
•
•
•
•
an + 5an-1 + 6an-2 = 42*4n
a2 = 278
a3 = 962
Next, we need homogeneous solution.
Characteristic equation is x2 + 5x + 6 = 0. Roots are
–2 and –3.
Homogeneous solution has form c1 (–2)n + c2 (–3)n.
Let’s add (particular) + (homogeneous):
an = c1 (–2)n + c2 (–3)n + 16*4n
Use base cases to determine values of c1 and c2.
It turns out c1 = 1 and c2 = 2.
Example #3
•
•
•
•
an – an-1 = 2n – 2
a2 = 4
The RHS is 2n – 2, so ordinarily this means the form
of the particular solution is (an + b). However, 1 is a
characteristic root, so need to multiply by n.
This means an = an2 + bn, and an-1 = a(n–1)2 + b(n–1)
Rewrite equation:
(an2+bn) – (a(n – 1)2+b(n – 1)) = 2n – 2
2an – a + b = 2n – 2
 2a = 2, and (–a + b) = –2  a = 1 and b = –1
This means our particular solution is an = n2 – n.
continued
•
•
•
•
•
an – an-1 = 2n – 2
a2 = 4
Now we need a homogeneous solution.
x–1=0x=1
Its general form is c*1n or simply c.
To solve for c, we need (particular) + (homogeneous)
an = n2 – n + c
Substitute a2 = 4, and we obtain 4 = 22 – 2 + c
c = 2, so our total solution is an = n2 – n + 2.
Homework
• The “nine rings puzzle” is an ancient Chinese game. The goal
is to remove 9 rings from a handle according to certain rules.
The problem can generalize to n rings.
• The puzzle asks: how many moves are necessary, in terms of
n, the initial number of rings? The various solutions are 1, 2,
5, 10, 21, 42,… The current number equals the previous
number plus twice the number before that, plus 1.
1. Write this sequence as a recurrence relation. Use the first
two terms as base cases, a1 = 1 and a2 = 2.
2. Solve the recurrence.
3. Briefly check your solution by verifying a3 is computed
correctly by your explicit solution.