Download Recurrence Relation

Recurrence Relation
Outline
What
is a recurrence relation ?
Solving
linear recurrence relations
Divide-and-conquer
relations
algorithms and recurrence
What is a recurrence relation ?
Definition
A recurrence relation for the sequence {an} is an
equation that expresses an in terms of one or more
of the previous terms of the sequence, namely a0,
a1, …, an-1, for all integers n n0, where n0 is a nonnegative integer.
A sequence is called a solution of a recurrence
relation if its terms satisfy the recurrence relation.
Example: Given the recurrence relation an=2an-1 -an-2.
3, 5, 7, 9, 11, … satisfies the recurrence relation.
2, 3, 4, 5, 6, … also satisfies the recurrence relation.
Example: Compound interest
Let Pn be the amount of money in the account after
n years. If you initially deposit Pn=1000 in your
account and the bank gives k % per year with
interest compounded annually.
Pn = (1+k/100) Pn-1
P1 = (1+k/100) P0
= (1+k/100) P0
P2 = (1+k/100) P1 = (1+k/100)2 P0
P3 = (1+k/100) P2 = (1+k/100)3 P0
Pn = (1+k/100) Pn-1 = (1+k/100)n P0
Example: Fibonacci numbers
A
pair of rabbits is placed on an island. After 2
months old, a pair of rabbits produces another pair
each month. How many rabbits are there after n
months?
month
1-month
rabbits
>=2-month
rabbits
Total
fn
1
1
0
1
2
0
1
1
3
1
1
2
4
1
2
3
5
2
3
5
6
3
5
8
7
5
8
13
8
8
13
21
fn =fn-1 + fn-2
Tower of Hanoi
Example: Tower of Hanoi
H(1) = 1
Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1
H(n) = 2H(n-1) + 1
x +1 = (2n-1 + 2n-2 + … + 4 + 2 + 1) +1
H(n) = 2(2H(n-2) +1) +1
x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2
= 4H(n-2) + 2 + 1
x + 1 = 2n-1 + 2n-2 + … + 4 + 4
= 4(2(H(n-3) + 1) + 2 + 1
x + 1 = 2n-1 + 2n-2 + … + 8
= 8H(n-3) + 4 + 2 + 1
x + 1 = 2n
= 8(2H(n-4) +1) + 4 + 2 + 1
= 16H(n-4) + 8 + 4 + 2 + 1
= 2n-1 + 2n-2 + 2n-3 + … + 1
= 2n - 1
x = 2n - 1
Linear recurrence relation
Homogeneous and non-homogeneous
What is a linear recurrence relation ?
Linear recurrence relation of degree k
an= c1an-1 + c2an-2 +…+ ckan-k + F(n)
ci
is a constant, for i = 1, 2, …, k
ck 
0
Linear homogeneous recurrence relation
F(n)
=0
Linear non-homogeneous recurrence relation
F(n)
0
Examples
Linear homogeneous
Linear non-homogeneous
 an
= 1.2 an-1
: degree 1
 an
= an-1 + 2n
 fn
= fn-1 + fn-2 : degree 2
 hn
= 2hn-1 + 1
 an
= 3an-3
 an
= 3an-1 + n
: degree 3
Non-linear homogeneous
Non-linear non-homogeneous
 an
= a2n-1 + an-2
 an
= a2n-1 + 2n
 an
= nan-1 - 2an-2
 an
= n2 an-1 + n
Solving linear homogeneous
recurrence relation
Theorem
sequence {an} is a solution of the recurrence
relation an = c1 an-1 + c2 an-2 iff an = 1 rn1 + 2 rn2 ,
where :
The
• r1 and r2 are two distinct roots of r2 - c1r - c2 = 0,
• a0 = 1 + 2
• a1 = 1 r1 + 2 r2
• [ 1 = (a1 – a0 r2)/(r1 - r2 )
2 = (a0 r1 –a1)/(r1 - r2 ) ]
Informal proof
Recurrence relation: an = c1 an-1 + c2 an-2
Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.
Let  and  be constants.

Show if an = 1 rn1 + 2 rn2, the sequence {an} is a
solution of the recurrence relation.
if the sequence {an} is a solution of the
recurrence relation, then an = 1 rn1 + 2 rn2.
Show
Informal proof
Show if an = 1 rn1 + 2 rn2, the sequence {an} is a
solution of the recurrence relation.
Let an = c1 an-1 + c2 an-2 be a recurrence relation.
Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.
Let  and  be constants.
Since r1 and r2 are roots of r2 - c1r - c2= 0,
r12 - c1r1 - c2 = 0 (r12 = c1r1 + c2) and r22 - c1r2 - c2= 0 (r22
= c1r2 + c2).
Informal proof
r12 = c1r1 + c2 and r22 = c1r2 + c2.
c1an-1 + c2an-2
= c1(1r1n-1+2r2n-1) + c2(1r1n-2+2r2n-2)
= c11r1n-1+ c1 2r2n-1 + c21r1n-2+ c2 2r2n-2
= c11r1n-1+ c21r1n-2 +c1 2r2n-1 + c2 2r2n-2
= 1r1n-2 (c1r1 + c2) + 2r2n-2 (c1r2 + c2)
= 1r1n-2 r12 + 2r2n-2 r22 = 1 r1n + 2 r2n = an
Informal proof
if the sequence {an} is a solution of the
recurrence relation, then there are 1 and 2 such that
an = 1 rn1 + 2 rn2.
 Show
 From
the initial conditions a0 and a1 ,
a0 = 1 + 2 and
a1 = 1 r1 + 2 r2
2 = a 0 - 1
1r1 = a1 - 2r2 = a1 - (a0-1) r2 = a1 – a0r2+ 1r2
1 (r1 - r2) = a1 – a0r2 >> 1 = (a1 – a0r2)/(r1 - r2)
2 =a0-1 =a0-(a1–a0r2)/(r1-r2) = a0r1-a0r2-a1+a0r2 /(r1-r2)
2 = (a0r1-a1) / (r1-r2)
Example
What is the solution of an = an-1 + 2an-2 with a0=2 a1=7 ?
We have c1 = 1 and c2 = 2.
The characteristic equation is r2 - r - 2 = 0, with roots
r1= 2 and r2= -1 .
1 = (a1 – a0r2)/(r1 - r2) = (7+2)/(2+1) = 3
2 = (a0r1 – a1)/(r1- r2) = (4-7)/(2+1) = -1
an = 1 rn1 + 2 rn2 = 32n - (-1)n
Example: Fibonacci number
What is the solution of an = an-1 + an-2 with a0=0 a1=1 ?
We have c1 = 1 and c2 = 1.
The characteristic equation is r2 - r - 1 = 0, with roots
r1= (1+5)/2 and r2= (1-5)/2 .
1 = (a1 – a0r2)/(r1 - r2) = (1-0)/5 = 1/5
2 = (a0r1 – a1)/(r1- r2) = (0-1)/5 = -1/5
an = 1 rn1 + 2 rn2 = ((1+5)/2)n/5 - ((1-5)/2)n/5
an = ( (1+5)n - (1-5)n )/(52n)
Theorem
sequence {an} is a solution of the recurrence
relation an = c1 an-1 + c2 an-2 iff an = 1rn0 + 2nrn0 ,
where :
The
• r0 is the only root of r2 - c1r - c2 = 0,
• a0 = 1
• a1 = 1 r0 + 2 r0 = r0(1 + 2)
• [ 1 = a0
2 = a1/r0 – a0 ]
Example
What is the solution of an= 6an-1 -9an-2 with a0=1 a1=6 ?
We have c1 = 6 and c2 = -9.
The characteristic equation is r2 - 6r + 9 = 0, with
roots r0= 3.
1 = a0 = 1
2 = a1/r0 – a0 = 6/3-1 = 1
an = 1rn0 + 2nrn0 = 3n + n 3n = 3n (1+ n).
Theorem
The
sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff an =
1rn1 + 2rn2 +…+ k rnk , where :
• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…ck= 0,
• a0 = 1 + 2 +…+ k
• a1 = 1 r1 + 2 r2 +…+ k rk
• a2 = 1 r21 + 2 r22 +…+ k r2k
•…
Theorem
The
sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff an =
1rn1 + 2rn2 +…+ k rnk , where :
• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…ck= 0,
• a0 = 1 + 2 +…+ k
• a1 = 1 r1 + 2 r2 +…+ k rk
• a2 = 1 r21 + 2 r22 +…+ k r2k
•…
Theorem
The
sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff
an = (1,0 + 1,1n + …+ 1,m1nm1-1 ) rn1 +
(2,0 + 2,1n + …+ 2,m2nm2-1 ) rn2 +…+
(t,0 + t,1n + …+ t,mtnmt-1 ) rnt , where :
• r1, r2 ,…, rt are t distinct roots of rk -c1rk-1 - c2rk-2 -…ck= 0, with multiplicities m1, m2 ,…, mt
Example
What is the solution of an=6an-1-11an-2+6an-3 with a0=2,
a1=5, a2=15 ?
We have c1 = 6, c2 = -11 and c3 = 6.
The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with
roots r1= 1, r2= 2 and r3= 3 .
a 0 = 1 +  2 + 3
a1 = 1 r1 + 2 r2 + 3 r3 = 1 + 22 + 33
a2 = 1 r12 + 2 r22 + 3 r32 = 1 + 42 + 93
1 = 1, 2 = -1, 3 = 2
an = 1 – 2n + 23n
Solving linear non-homogeneous
recurrence relation
Theorem
If the sequence {an(p)} is a particular solution of the
recurrence relation an = c1 an-1 + c2 an-2 +…+ ck
an-k + F(n) then every solution is of the form {an(p) +
an(h)} , where {an(h)} is a solution of the associated
homogeneous recurrence relation an = c1 an-1 + c2
an-2 +…+ ck an-k.
Example
What are the solutions of an= 3an-1+ 2n with a1=3 ?
We have c1 = 3.
The associated characteristic eqn is r-3=0, with root r = 3.
n
Then, a(h)
n = 3 .
Let pn = cn +d.
Then, from an= 3an-1+ 2n, cn +d = 3(c(n-1)+d ) + 2n.
Thus, c = -1 d = -3/2, and a(p)
n = -n -3/2
(p)
n
an= a(h)
n + an = 3 -n -3/2.
From a1=
3 = 3 -1 - 3/2,  = 11/6.
The solution is an= (11/6)3n -n -3/2.
Example: summation from 1 to n
What are the solutions of an= an-1+ n with a1=1 ?
We have c1 = 1.
The associated characteristic eqn is r-1=0, with root r = 1.
n
Then, a(h)
n = 1 =  .
Let pn = n(cn +d) = cn2 +dn.
Then, from an= an-1+ n, cn2 +dn = c(n-1)2+d (n-1) + n.
Thus, cn2 +dn = cn2-2cn+c +d n - d + n , c-d + n(1-2c)=0.
c-d =0 and 1-2c=0. That is, c=d=1/2.
(p)
an= a(h)
n + an =  + n(n+1)/2.
The solution is an= n(n+1)/2.
From a1=
1 =  +1,  = 0.
Divide-and-conquer Algorithms
Divide-and-Conquer Concept
n/b
X1
n/b
X2
n/b
XX3
...
n/b
Xa
n elements
To solve a problem of size n, we solve a subproblems of size n/b, and combine solutions from
sub-problems.
Binary Search
search for an element a from a sorted list X of
size n elements.
To
X1
m
X
• If a=m, then stop
• If a<m, then search for a from X1.
• If a>m, then search for a from X2.
X2
Fast multiplication of large integers
a
= 2n A1 + A0.
10010
A1
b = 2n B1 + B0.
00011
A0
10000
B1
10011
B0
a b = 22nA1B1 + 2n(A0B1+ A1B0) +A0 B0.
a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.
a b = 22n A1B1 + 2nA1B1 + 2nA1 B0 –2nA0B0 +2nA0B1
– 2nA1B1 + 2n A0 B0 + A0 B0.
Recurrence Relations
In Divide-and-conquer algorithms
Divide-and-Conquer Recurrence Relations
n/b
X1
n/b
X2
n/b
XX3
...
n/b
Xa
n elements
Let f(n) be the number of operations required to
solve a problem of size n.
f(n) = a f(n/b) + g(n).
Binary Search
search for an element a from a sorted list X of
size n elements.
m
X
X1
X2
To
• If a=m, then stop
• If a<m, then search for a from X1.
• If a>m, then search for a from X2.
Let f(n) be the # of comparison in binary search within n
elements.
f(n) = f(n/2) + 2.
Fast multiplication of large integers
a
= 2n A1 + A0.
10010
A1
b = 2n B1 + B0.
00011
A0
10000
B1
10011
B0
a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.
Let f(n) be the number of operations needed to
multiply two n-bit integers.
f(2n) = 3f(n) + Cn.
Theorem
Let f be an increasing function that satisfies the
recurrence relation
f(n) = a f(n/b) + c
whenever n is divisible by b, b is an integer greater
than 1, a1, c is a positive real number.
When n=bk, f(n) = t nlogba + s, for some t and s.
When a>1, f(n) is O(nlogba ).
When a=1, f(n) is O(log n).
Examples
Binary Search:
f(n) = f(n/2) + 2.
Because a=1, f(n) is O(log n).
Find Max. :
f(n) = 2f(n/2) + 2.
Because a>1, f(n) is O(nlog11 ) = O(n) .
Master Theorem
Let f be an increasing function that satisfies the
recurrence relation
f(n) = a f(n/b) + cnd
whenever n=bk for some positive integer k, b is an
integer greater than 1, a1, c is a positive real
number, d is a non-negative real number.
When a<bd, f(n) is O(nd ).
When a=bd, f(n) is O(nd log n).
When a>bd, f(n) is O(nlogba).
Examples
Merge sort:
f(n) = 2f(n/2) + n.
a = 2, b = 2, c = 1, d = 1
Because a=bd, f(n) is O(n log n).
Integer multiplication:
f(n) = 3f(n/2) + Cn
a = 3, b = 2, c = C, d = 1
Because a>bd, f(n) is O(nlog23).