Download Homogeneous Solutions

Recurrence Relations and Recursive
Algorithms
For a numeric function (a0, a1, a2, ..., ar, ...), an equation
relating ar, for any r, to one or more of the ai’s, i < r, is called a
recurrence relation.
 A recurrence relation is also called a difference equation.

 Boundary conditions: The value of function at one or more
points is given.
 A recurrence relation together with an appropriate set of
boundary conditions can describe a numeric function.
 The numeric function is also referred to as the solution of the
recurrence relation.
 Unfortunately, no general method of solution for handling all
recurrence relations is known.

kth-order linear recurrence relation with constant coefficients :
A recurrence relation of the form
C0ar+ C1ar-1+ C2ar-2+...+ Ck ar-k = f(r),
(*)
where Ci’s are constants, C0  0 and Ck  0.
Example :
1st-order: 2ar + 3ar-1 = 2r
2nd-order: 3ar - 5ar-1 + 2ar-2= r2 + 5
1
 Case I: If k consecutive values of the numeric function a, am-k,
am-k+1,..., am-1 are known for some m, the value of am can be
calculated
am=-1/C0[C1am-1+ C2am-2+...+ Ckam-k - f(m)].
am+1:
am+1=-1/C0[C1am+ C2am-1+...+ Ckam-k+1 - f(m+1)]
the values of am+2, am+3,...can be computed similarly.
am-k-1:
am-k-1=-1/Ck[C0am-1+ C1am-2+...+ Ck-1am-k - f(m-1)]
am-k-2:
am-k-2=-1/Ck[C0am-2+ C1am-3+...+ Ck-1am-k-1 - f(m-2)]
The value of am-k-3, am-k-4,...can be computed similarly.
 Case II: Fewer than k values of the numeric function will not be
sufficient to determine the numeric function uniquely.
 Case III: More than k values of the numeric function might
make it possible for the existence of a numeric function that
satisfies the recurrence relation and the given boundary
condition.
2
Homogeneous Solutions

The solution of a linear difference equation with constant
coefficients is the sum of two parts, the homogeneous solution,
which satisfies the difference equation when the right-hand side
of the equation is set to 0, and the particular solution, which
satisfies the difference equation with f(r) on the right-hand side.
 Total solution = homogeneous solution + particular solution.
homogeneous solution: a(h)=(a0(h), a1(h),..., ar(h),...)
particular solution: a(p)=(a0(p), a1(p),..., ar(p),...).
Since
C0ar(h)+ C1ar-1(h)+...+ Ckar-k(h) = 0
and
C0ar(p)+ C1ar-1(p)+...+ Ckar-k(p) = f(r)
We have
C0(ar(h)+ar(p))+ C1(ar-1(h)+ar-1(p))+...+ Ck(ar-k(h)+ar-k(p))= f(r)
The total solution, a = a(h)+a(p) satisfies the difference
equation.

characteristic root solution: A homogeneous solution of a
linear difference equation with constant coefficients is of the
r
form Aα1 , whereα1 is called a characteristic root and A is a
constant determined by the boundary conditions.
r
 characteristic equation: Substituting Aα1 for ar, we have

r
r-1
r-2
r-k
C0 Aα + C1 Aα + C2 Aα +...+ Ck Aα = 0
3
It can be simplified to
r
r-1
r-2
r-k
C0α + C1α + C2α +...+ Ckα = 0
r
Ifα1 is one of the roots of the equation, Aα1 is a homogeneous
solution to the difference equation.

multiple roots: Letα1 be a root of multiplicity m. The
corresponding homogeneous solution becomes
(A1rm-1+ A2rm-2+ ...+ Am-2r2+ Am-1r1+ Am)α1r,
where the Ai’s are constants to be determined by the boundary
conditions.
 Am-1rα1r is also a homogeneous solution: Recall thatα1 not
only is a root of the equation
C0αr+ C1αr-1+ C2αr-2+...+ Ckαr-k = 0
(**)
But also is a root of the derivative equation of (**),
C0 rαr-1+ C1 (r-1)αr-2+ C2 (r-2)αr-3+...+ Ck (r-k)αr-k-1 = 0 (***)
Becauseα1 is a multiple root of (**).
Multiplying (***) by Am-1α and replacingαbyα1,
we obtain
C0 Am-1rα1+C1 Am-1(r-1)α1r-1+C2 Am-1(r-2)α1r-2+ ...+
Ck Am-1(r-k)α1r-k = 0
4
Particular Solutions

There is no general procedure for determining the particular
solution of a difference equation.
Example :
Consider the difference equation
ar - 5ar-1 + 6ar-2= 3r2
(*)
we assume that the general form of the particular solution is
P1r2 + P2r + P3
Substituting it into the left-hand side of (*), we obtain
P1r2 + P2r + P3 + 5P1(r – 1)2 + 5P2(r – 1) + 5P3 + 6P1(r – 2)2
+ 6P2(r – 2) + 6P3
Which simplifies to
12P1r2 – (34P1 – 12P2)r + (29P1 - 17P2 + 12P3) (**)
Comparing (**) with the right-hand side of (*), we obtain
12P1 = 3
34P1 – 12P2 = 0
29P1 - 17P2 + 12P3 = 0,
which yield
P1 = 1/4, P2 = 17/24, P3 = 115/288

In general, when f(r) is of the form of a polynomial of degree t
in r
F1rt + F2rt-1 + ... + Ftr +Ft+1.
The corresponding particular solution will be of the form
P1rt + P2rt-1 + ... + Ptr +Pt+1.
5
Total Solutions

For a kth-order difference equation, the k undetermined
coefficients A1, A2,..., Ak in the homogenous solution can be
determined by the boundary conditions, aro, aro+1,...aro+k-1,for
any r0.
Suppose the characteristic roots of the difference equation are
all distinct. The total solution is of the form
r
r
r
ar= A1α1 + A2α2 +...+ Akαk +p(r)
where p(r) is the particular solution. Thus, for r = r0, r0+1,...,
r0+k-1,we have the system of linear equations:
r0
r0
r0
ar0= A1α1 + A2α2 +...+ Akαk +p(r0)
r0+1
r0+1
r0+1
ar0+1= A1α1 + A2α2 +...+ Akαk +p(r0+1)
........................
+ A2α2r0+k-1+...+ Akαkr0+k-1+p(r0+k-1)
r0+k-1
1
ar0+k-1= A1α
These k linear equation can be solved for A1, A2,..., Ak.
6
Solution by the Method of Generating Functions

Consider the recurrence realtion
ar = 3ar-1 + 2 r  1
With the boundary condition a0 = 1.
Multiplying both sides by zr and summing for all r, r  1,



r 1
r 1
r 1
 ar z r  3 ar 1 z r  2 z r

a z
r 1
r
r
 A( z )  a 0


r 1
r 1
 ar 1 z r  z  ar 1 z r 1  zA( z )
We obtain
A( z )  a0  3zA( z ) 
A( z ) 
2z
1 z
1 z
2
1


(1  3z )(1  z ) 1  3z 1  z
Consequently, we have
ar = 23r – 1, r  0.
7
Sorting Algorithms

Recursive specification of BUBBLESORT: Let S(x1,x2,...,xn)
denote BUBBLESORT algorithm that sorts the numbers in
registers x1, x2,..., xn in ascending order, let M(x1, x2,..., xn)
denote LARGEST2 algorithm that places in register xn the
largest number among the numbers in registers x1, x2, ..., xn. We
have
S(x1, x2,..., xn)  M(x1, x2,..., xn) S(x1, x2,..., xn-1)
The symbol “” means “is defined to be”.
 M(x1 x2,..., xn) can be defined as
M(x1, x2,..., xn)  A(x1, x2)A(x2, x3) ... A(xn-2, xn-1)A(xn-1, xn).
S(x1, x2,..., xn) can be rewritten as
S(x1, x2,..., xn)  A(x1, x2)A(x2, x3) ... A(xn-2, xn-1)A(xn-1, xn)
S(x1, x2,..., xn-1).
With the boundary condition
S(x1, x2)  A(x1, x2)


Let an denote the number of comparison steps the bobble sort
algorithm takes to sort n numbers. We have,
an = (n-1) + an-1,
using the boundary condition a1 = 0, we obtain
an = n(n-1)/2.
8

Bose-Nelson algorithm, (T(x1, x2,..., xn), n = 2r): Let P[(x1, x2,...,
xn), (xn+1, xn+2,..., x2n)] denote an algorithm that will arrange the
number in registers x1, x2,..., x2n in ascending order giving that
the numbers x1, x2,..., xn have already been arranged in
ascending order, as have the numbers in registers xn+1, xn+2,...,
x2n.
T(x1, x2,..., x2n)  T(x1, x2,..., xn) T(xn+1, xn+2,..., x2n)
P[(x1, x2,..., xn), (xn+1, xn+2,..., x2n)] (*)
The algorithm P[(x1, x2,..., xn), (xn+1, xn+2,..., x2n)] can be defined
recursively as:
P[(x1,..., xn), (xn+1,..., x2n)]
 P[(x1,..., xn/2), (xn+1,..., x3n/2)]
P[(xn/2+1,..., xn), (x3n/2+1,..., x2n)]
P[(xn/2+1,..., xn), (xn+1,..., x3n/2)]
(**)
1
(n/2) (n/2)+1
n n+1
(3n/2) (3n/2)+1
2n
We can express T(x1, x2,..., xn),n = 2r,as a sequence of
procedures of the forms T(xI, xj) and P[(xi), (xj)],
both of T(xi, xj) and P[(xi), (xj)] are equal to A(xi, xj).
 Let cr denote the number of comparison steps that P[(x1, x2,...,
x2rr), (x2rr+1, x2rr+2,..., x2rr+r)] takes. According to the relation in
(**), we have cr = 3cr-1. Solving it with the boundary condition
c0 = 1,we obtain cr = 3r.
9
 Let dr denote the number of comparison steps the procedure
T(x1, x2,..., xn) takes. According to the relation in (*),we have
dr = 2dr-1+cr-1
or
dr =2dr-1+3r-1
we obtain
dr = B2r+3r
From the boundary condition d0 = 0, we have B = -1. Thus,
Dr = 3r - 2r.

odd-even merge algorithm, U(x1, x2,..., xn): Let
U(x1, x2,..., x2n)  U(x1, x2,..., xn) U(xn+1, xn+2,..., x2n)
B[(x1, x2, ..., xn), (xn+1, xn+2,..., x2n)],
where the algorithm B is defined to be
B[(x1, x2, x3, x4, ..., x2n), (y1, y2, y3, y4, ..., y2n)]
 B[(x1, x3, x5, ..., x2n-1), (y1, y3, y5, ..., x2n-1)]
B[(x2, x4, x6, ..., x2n), (y2, y4, y6, ..., x2n)]
A(x2, x3) A(x4, x5) A(x6, x7) ... A(x2n, y1)
A(y2,y3) A(y4,y5)... A(y2n-2,y2n-1)
(*)
 To verify the validity of (*), we want first to show that after the
execution of B[(x1, x3, x5, ..., x2n-1), (y1, y3, y5, ..., x2n-1)] and
B[(x2, x4, x6, ..., x2n), (y2, y4, y6, ..., x2n)]
1. The number in x1 is the smallest of the 4n numbers.
2. The number in y2n is the largest of the 4n numbers.
3. The two numbers in x2i and x2i+1,1≦ i ≦ n-1,are the 2ith
and the (2i+1)st smallest of the 4n numbers, the two
numbers in x2n and y1 are the 2nth and (2n+1)st smallest of
the 4n numbers, and the two numbers in y2i and y2i+1, 1≦ i
≦ n - 1, are the (2n +2i)th and (2n+2i+1)st smallest of the
4n numbers.
10
If this is indeed the case, the comparison A(y2, y3) A(y4, y5) ...
A(y2n-2, y2n-1) will yield a sorted sequence of 4n numbers.
 Let cr denote the number of comparison steps the algorithm
B[(x1, x2, x3, x4, ..., x2n), (y1, y2, y3, y4, ..., yn)] takes. According
to the relation in (*), we have
cr = 2cr-1 + (2r-1)
with the boundary condition c0 = 1, we obtain
cr = r2r +1.
 Let dr denote the number of comparison steps the algorithm
U(x1, x2, ..., xn) takes. According to the relation in (*), we have
dr = 2dr-1 + cr-1
or
dr = 2dr-1+(r-1)2r-1 +1
with boundary condition d0 = 0, we obtain
dr = (r2-r+4)2r-2 – 1.
11
Matrix Multiplication Algorithms

Given two n n matrices A and B, we can partition them into
(n/2)  (n/2) submatrices:
a
A   11
a21
a12 
b11
B

b
a22 

 21
b12 
b22 

The product AB can be expressed as
c 
c
C  AB   11 12 
c21 c22 
Where a11, a12, a21, a22, b11, b12, b21, b22, c11, c12, c21, c22 are all
(n/2) (n/2) submatrices. We can then compute
m1=(a11+ a22) (b11+ b22)
m2=(a12 - a22) (b21 + b22)
m3=(a11 - a21) (b11 + b12)
m4=(a11 + a12) b22
m5=(a21 + a22) b11
m6=a11(b12 - b22)
m7=a22(b21 - b11)
Finally, we can compute
c11=m1 + m2 - m4 + m7
c12=m4 + m6
c21=m5 + m7
c22=m1 - m3 - m5 + m6.
12

The total cost of multiplying A and B is 7 matrix
multiplications and 18 matrix additions.
 If we use the classical method to carry out multiplication of
(n/2)(n/2) matrices, the total number of arithmetic operations
will be
2
  n 3  n  2 
7
11
n
7  2       18     n3  n 2
4
4
 2  
 2
  2 
Let fr denote the total number of arithmetic operations required
to multiply two 2r2r matrices. We have
fr = 7 fr-1+ 18 ·22r-2
r ≧ 1
with the boundary condition f0 = 1, we obtain
fr = 7 ·7r - (18/3) ·4r
Thus, for n = 2r
fr = 7 ·7lgn - (18/3) ·n2 = 7n2.81 - (18/3) ·n2.
 Since the time complexity of the algorithm presented above is
O (n2.81), the time complexity of the matrix multiplication
problem is O(n2.81).
13