Download CPS130, Lecture 1: Introduction to Algorithms

9/2/2002
CPS130, Lecture 2; Sums, Function Growth, L’Hospital’s Rule
1. Introduction
To facilitate our analyses of algorithms to come, we collect and discuss here some tools
we will use. In particular we discuss some important sums and also present a framework
which allows for functions to be compared in terms of their “growth.” In a detailed
example, we will show that the function h(x) = ln(x)/xp  0 as x   for p>0.
We first pause to think about the function f(x) = xp. If p = 0, f=1 for all x, and if p<0,
f = 1/x-p so nothing is lost by only considering p>0. If p is a positive integer, say p = k, f
is easy to understand and f maps [0,  ) to [0,  ) in a 1-1 manner; that is, given any y
there is a unique x>0 such that y = xk. In fact, for y>0, we call this x the positive kth root y
or x = k y or x = y1/k. Interchanging the immediately previous x and y to y = x1/k explains
our f for those p = 1/k; to summarize x1/k = y such that yk = x, and we have extended our
understanding of xp for all k to all 1/k. And the extension to any positive rational number
p = n/m is even easier since then f = (x1/m)n; we take the nth power of the mth root of x.
Now take out your scientific calculator or open matlab or click up PowerToy Calculator
(which you can get free from Microsoft and which I have been using for all the inserted
graphs so far) and compute:
2
2 = 1.632526919438152844773495381024
Very impressive considering that we have shown in Lecture 1 that 2 is irrational.
How does powertoy do it? Because powertoy has a deep understanding of the
exponential function!
Recall from Lecture 1:
Notice that ln(x) and ex are inverse functions since
eln(x)  x for x  (0,) and ln(ex )  x for all real x.
The key relationship between e and ln is that
(LGEX)
y = ex if and only if (iff) x = ln(y) , y>0.
So somewhat like xk above, ex maps ( ,  ) to (0,  ) in a 1-1 manner. From above
2
x p  eln(x )  epln(x) . So to compute 2 , Jerry Gremlin, inside of powertoy, looks at the
graph of ex. He goes to the y  2 line and finds where it intersects the graph, goes
p
vertically down to find the x-value which is ln( 2) (he could also have used ln( 2) =
1
ln(2)/2 ). Then he computes ex, where x=
2 (x-value), maybe using the power series of

L1: e x   (x n / n!) .
n 0
We could have used other bases here than loge. A lot of work has been done to compute
this type of function efficiently. And it generalizes in a natural way, xr, r rational. To get
an even deeper understanding of f, we would need to study its further extension as a
function of a complex variable.
2. Sums
n
Problem: evaluate S=
i
i 1
1
Solution 1: note also that S   i ; hence,
in
n
(S1)
S  (1/ 2) (n  1)  n(n  1) / 2 .
j1
Solution 2: proof by induction.
Base case: true for k=1 (obvious).
Assume true for k=n (call the sums Sk) and show this implies true for k = (n+1).
Sn 1  Sn  (n  1) = (n(n+1)/2) + (n+1) = (n+1)(n+2)/2,
where the induction hypothesis (k=n case) is applied following the second “=”.
n
Next we derive “tight” bound on the sums Spn   i p . First consider the function
i 1
f(x) = xp. In each subinterval [i, i+1] of [0,n] for 1  i  n  1 , ip is bounded below and
above as
(x  1)p  i p  x p .
Now using the “area under the curve” property of the integral calculus,
we see that
2
n
n
1
1
p
p
p
 (x  1) dx  Sn   x dx ;
Evaluating the integrals gives
((n  1) p 1 /(p  1))  Spn  (n p 1  1) /(p  1) .
We are beginning to see that S pn is sandwiched between functions of n that grow as np+1.
To see this clearly we manipulate (work this out and ask in recitation) the above
inequalities as
(1  1/ n) p 1  ((p  1) / n p 1 )Spn  (1  1/ n p 1 ) .
Now take n>2; then
(SW)
c1n p 1  Spn  c 2 n p 1 ,
where c1  (1/ 2) p 1 /(p  1) and c2  1/(p  1) . Since we think of p as fixed as n increases
(but BONUS for thinking about the situation when they both increase), we will say below
that (SW) means
(S2)
Spn  (n p 1 ) .
Our final interesting sum starts with :
n
( x i )(1  x)  1  x n 1
i 0
which is easily seen because most of the terms multiplying 1 cancel with the terms
multiplying x. Now if |x|<1 and we take n   , we obtain

(S3)
x
i
 1/(1  x) .
i 0
Example 1: take x = ½; then 1 + ½ + ¼ + 1/8 + … = 2.
Example 2: take x = - ½; then 1 – ½ + ¼ - 1/8 + … = 2/3
Example 3: HOMEWORK; add up the previous two sums and use (S3) to interpret.
3
3. Growth of Functions
This section is mostly definitions and examples; see large, lecture 2 and text, section 3.1.
These references assume that the functions to be considered map the natural numbers to
the natural numbers for n large enough (n>n0). This in not a necessary restriction but one
quite applicable to the types of complexity functions we will consider. If we want to
consider real valued functions we can usually replace n by x and may want to use
absolute values sometimes.
O, , and  notations
(BigO)
f(n) = O(g(n)) iff there exists a constant c1>0 and an integer n0 such that
0  f (n)  c1g(n) for all n  n 0 ;
(Big  )
f(n) = (g(n)) iff there exists a constant c2>0 and an integer n0 such that
0  c2g(n)  f (n) for all n  n 0 ;
(Big  )
f(n) = (g(n)) iff there exist positive constants c1 and c2 such that
0  c2g(n)  f (n)  c1g(n) for all n  n 0 .
o,  notations (f and positive for n>n0)
(little o)
f(n) = o(g(n)) iff lim (f(n)/g(n)) = 0;
(little  )
f(n) =  (g(n)) iff lim (g(n)/f(n)) = 0.
n 
n 
Now go back to our derivation of (SW) and hence (S2); the key was in dividing by np+1
and playing with the inequalities. That approach usually works.
Example 4: p(n) = n 2 / 3  29n  (n 2 )
Let q(n) = p(n)/n2 = 1/3 –29/n. Clearly q(n)  1/3. For the other bound we must deal
with the " " sign. We must make n>87 just to get q>0. Take n0 =100. Then
.04  q(n)  1/ 3 . The difference between .04 and 1/3 is irritating; we can make these
numbers close by increasing n, but remember in this course, n will be the size of realistic
input.
Example 5, large L2, p. 6: 6n lg(n)  n lg 2 (n)  (n lg(n))
4
This derivation is fine but implicitly assumes that h(n)= lg(n) / n is a decreasing
function of n, which is true but maybe not obvious at this point. In fact, we will see that
h(n)  0 as n   ; i.e., h = o(1).
There are a lot of things to prove in this section if we want to do everything carefully. We
will indicate some and do others if we need them later. Sometimes we need to use
calculus to establish “basic facts” as in the following example. You should also review
L’Hopital’s rule, “order of magnitude”, infinite sums and series.
m
Example 6: p m (n)   a i n i  (n m ) , am>0, pm(n)>0 for n>0.
i 1
m 1
m 1
i 1
1
Let r(n) = pm/nm and q(n)   a i n i . Then r(n)  a m   a i = c1 (defines c1 and is true
n  1). Now am/2  am – q(n)/nm  r(n) for some n  n0 since q(n)/nm decreases
monotonically to zero. Thus take c2 = am/2.
Example 7: ln(n)/np = o(1), p>0
Well now we are getting back to the large Example 5 above in a way. There p =1/2.
h = ln(x)/x1/2 ; x from 1 to 20 space 1, y from –1 to 1 space .1
First let’s look at dh / dx  h . We recall (!)
h  (x p 1/ x  px p1  ln(x)) / x 2p  (1  p ln(x)) / x p1 .
5
First h(x0) =0 when x0 = e1/p, which is what the figure shows for p = ½ since e2 =7.38…;
there h is a maximum and from then on it’s downhill. Now this is good enough to show
h = O(1), but we want more. How do we know h  0 as n   . The easiest way is to use
L’Hopital’s rule; we will review this below.
First let’s do it directly for p = 1/2 by recalling from L1 that
x
h(x) = (  1/ s ds) / x1/ 2 and note that
1
x
x
1
1
h(x) = (1/x1/4) (  (1/(x1/ 4s)) ds)  (1/x1/4) (  1/ s 5/4 ds) since (1/x)1/4 < (1/s)1/4 for s in
[1,x]. Hence
h(x)  4[1 - 1/x1/4]/ x1/4  0 as x   .
Now let’s do the general case using the discussion of section 1. For any p>0,
ln(x)/xp = ln(x)/epln(x) = (1/p)(y/ey) where y = p ln(x). Since p>0 is fixed
y   as x   . Clearly y/ey  0 as x   (since also y  ) by looking at the power
series for ey. So ln(x)/xp = o(1); note that this is true for other bases as we have discussed
in L1.
Example 8: PpLq(n) =
l
mp
p 0
q 0
 (a p n p  bq,p (lg(n))q ) = (nl (log(n))ml ) ; a l  0 ; b ml ,l  0 for all l
(notice that ml = 0 is allowed).
This is a generalization of Example 6 and uses the result of Example 7.
Extra Credit for working this out.
n
Example 9: H n  1/ n  (ln(n))
i 1
x
Do as Homework; use ln(x) = (  1/ s ds) and bounds from integration as in the
1
derivation (SW) above.
6
4. L’Hospital’s Rule
Here is a statement of the form of L’Hospital’s Rule we will find most convenient,
adapted from Calculus with analytic geometry, R. Johnson and F. Kiokemeister, 2nd
Ed., 1962, sections 14.3 and 14.4.
Theorem (x   form of L’Hospital’s Rule)
Let f and g be differentiable functions with g(x)  0 on some interval  x 0 ,  . If
lim f (x)  lim g(x)  0 or lim f (x)   and lim g(x)   , then
x 
x 
x 
x 
f (x)
f (x)
 lim
x  g(x)
x  g(x)
lim
provided the second limit exists or is infinite.
Example 10 (Ex 7 revisited, for p>0)
ln(x)
1/ x
1
 lim p1  lim p  0 .
p
x  x
x  px
x  px
lim
Notice that you still need to know that xp   as x   , which follows, as in Example
7, from the (e, ln) definition of xp and the properties of ex.
7
Betsy Helping in Lecture Preparation
8