Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
9/2/2002 CPS130, Lecture 2; Sums, Function Growth, L’Hospital’s Rule 1. Introduction To facilitate our analyses of algorithms to come, we collect and discuss here some tools we will use. In particular we discuss some important sums and also present a framework which allows for functions to be compared in terms of their “growth.” In a detailed example, we will show that the function h(x) = ln(x)/xp 0 as x for p>0. We first pause to think about the function f(x) = xp. If p = 0, f=1 for all x, and if p<0, f = 1/x-p so nothing is lost by only considering p>0. If p is a positive integer, say p = k, f is easy to understand and f maps [0, ) to [0, ) in a 1-1 manner; that is, given any y there is a unique x>0 such that y = xk. In fact, for y>0, we call this x the positive kth root y or x = k y or x = y1/k. Interchanging the immediately previous x and y to y = x1/k explains our f for those p = 1/k; to summarize x1/k = y such that yk = x, and we have extended our understanding of xp for all k to all 1/k. And the extension to any positive rational number p = n/m is even easier since then f = (x1/m)n; we take the nth power of the mth root of x. Now take out your scientific calculator or open matlab or click up PowerToy Calculator (which you can get free from Microsoft and which I have been using for all the inserted graphs so far) and compute: 2 2 = 1.632526919438152844773495381024 Very impressive considering that we have shown in Lecture 1 that 2 is irrational. How does powertoy do it? Because powertoy has a deep understanding of the exponential function! Recall from Lecture 1: Notice that ln(x) and ex are inverse functions since eln(x) x for x (0,) and ln(ex ) x for all real x. The key relationship between e and ln is that (LGEX) y = ex if and only if (iff) x = ln(y) , y>0. So somewhat like xk above, ex maps ( , ) to (0, ) in a 1-1 manner. From above 2 x p eln(x ) epln(x) . So to compute 2 , Jerry Gremlin, inside of powertoy, looks at the graph of ex. He goes to the y 2 line and finds where it intersects the graph, goes p vertically down to find the x-value which is ln( 2) (he could also have used ln( 2) = 1 ln(2)/2 ). Then he computes ex, where x= 2 (x-value), maybe using the power series of L1: e x (x n / n!) . n 0 We could have used other bases here than loge. A lot of work has been done to compute this type of function efficiently. And it generalizes in a natural way, xr, r rational. To get an even deeper understanding of f, we would need to study its further extension as a function of a complex variable. 2. Sums n Problem: evaluate S= i i 1 1 Solution 1: note also that S i ; hence, in n (S1) S (1/ 2) (n 1) n(n 1) / 2 . j1 Solution 2: proof by induction. Base case: true for k=1 (obvious). Assume true for k=n (call the sums Sk) and show this implies true for k = (n+1). Sn 1 Sn (n 1) = (n(n+1)/2) + (n+1) = (n+1)(n+2)/2, where the induction hypothesis (k=n case) is applied following the second “=”. n Next we derive “tight” bound on the sums Spn i p . First consider the function i 1 f(x) = xp. In each subinterval [i, i+1] of [0,n] for 1 i n 1 , ip is bounded below and above as (x 1)p i p x p . Now using the “area under the curve” property of the integral calculus, we see that 2 n n 1 1 p p p (x 1) dx Sn x dx ; Evaluating the integrals gives ((n 1) p 1 /(p 1)) Spn (n p 1 1) /(p 1) . We are beginning to see that S pn is sandwiched between functions of n that grow as np+1. To see this clearly we manipulate (work this out and ask in recitation) the above inequalities as (1 1/ n) p 1 ((p 1) / n p 1 )Spn (1 1/ n p 1 ) . Now take n>2; then (SW) c1n p 1 Spn c 2 n p 1 , where c1 (1/ 2) p 1 /(p 1) and c2 1/(p 1) . Since we think of p as fixed as n increases (but BONUS for thinking about the situation when they both increase), we will say below that (SW) means (S2) Spn (n p 1 ) . Our final interesting sum starts with : n ( x i )(1 x) 1 x n 1 i 0 which is easily seen because most of the terms multiplying 1 cancel with the terms multiplying x. Now if |x|<1 and we take n , we obtain (S3) x i 1/(1 x) . i 0 Example 1: take x = ½; then 1 + ½ + ¼ + 1/8 + … = 2. Example 2: take x = - ½; then 1 – ½ + ¼ - 1/8 + … = 2/3 Example 3: HOMEWORK; add up the previous two sums and use (S3) to interpret. 3 3. Growth of Functions This section is mostly definitions and examples; see large, lecture 2 and text, section 3.1. These references assume that the functions to be considered map the natural numbers to the natural numbers for n large enough (n>n0). This in not a necessary restriction but one quite applicable to the types of complexity functions we will consider. If we want to consider real valued functions we can usually replace n by x and may want to use absolute values sometimes. O, , and notations (BigO) f(n) = O(g(n)) iff there exists a constant c1>0 and an integer n0 such that 0 f (n) c1g(n) for all n n 0 ; (Big ) f(n) = (g(n)) iff there exists a constant c2>0 and an integer n0 such that 0 c2g(n) f (n) for all n n 0 ; (Big ) f(n) = (g(n)) iff there exist positive constants c1 and c2 such that 0 c2g(n) f (n) c1g(n) for all n n 0 . o, notations (f and positive for n>n0) (little o) f(n) = o(g(n)) iff lim (f(n)/g(n)) = 0; (little ) f(n) = (g(n)) iff lim (g(n)/f(n)) = 0. n n Now go back to our derivation of (SW) and hence (S2); the key was in dividing by np+1 and playing with the inequalities. That approach usually works. Example 4: p(n) = n 2 / 3 29n (n 2 ) Let q(n) = p(n)/n2 = 1/3 –29/n. Clearly q(n) 1/3. For the other bound we must deal with the " " sign. We must make n>87 just to get q>0. Take n0 =100. Then .04 q(n) 1/ 3 . The difference between .04 and 1/3 is irritating; we can make these numbers close by increasing n, but remember in this course, n will be the size of realistic input. Example 5, large L2, p. 6: 6n lg(n) n lg 2 (n) (n lg(n)) 4 This derivation is fine but implicitly assumes that h(n)= lg(n) / n is a decreasing function of n, which is true but maybe not obvious at this point. In fact, we will see that h(n) 0 as n ; i.e., h = o(1). There are a lot of things to prove in this section if we want to do everything carefully. We will indicate some and do others if we need them later. Sometimes we need to use calculus to establish “basic facts” as in the following example. You should also review L’Hopital’s rule, “order of magnitude”, infinite sums and series. m Example 6: p m (n) a i n i (n m ) , am>0, pm(n)>0 for n>0. i 1 m 1 m 1 i 1 1 Let r(n) = pm/nm and q(n) a i n i . Then r(n) a m a i = c1 (defines c1 and is true n 1). Now am/2 am – q(n)/nm r(n) for some n n0 since q(n)/nm decreases monotonically to zero. Thus take c2 = am/2. Example 7: ln(n)/np = o(1), p>0 Well now we are getting back to the large Example 5 above in a way. There p =1/2. h = ln(x)/x1/2 ; x from 1 to 20 space 1, y from –1 to 1 space .1 First let’s look at dh / dx h . We recall (!) h (x p 1/ x px p1 ln(x)) / x 2p (1 p ln(x)) / x p1 . 5 First h(x0) =0 when x0 = e1/p, which is what the figure shows for p = ½ since e2 =7.38…; there h is a maximum and from then on it’s downhill. Now this is good enough to show h = O(1), but we want more. How do we know h 0 as n . The easiest way is to use L’Hopital’s rule; we will review this below. First let’s do it directly for p = 1/2 by recalling from L1 that x h(x) = ( 1/ s ds) / x1/ 2 and note that 1 x x 1 1 h(x) = (1/x1/4) ( (1/(x1/ 4s)) ds) (1/x1/4) ( 1/ s 5/4 ds) since (1/x)1/4 < (1/s)1/4 for s in [1,x]. Hence h(x) 4[1 - 1/x1/4]/ x1/4 0 as x . Now let’s do the general case using the discussion of section 1. For any p>0, ln(x)/xp = ln(x)/epln(x) = (1/p)(y/ey) where y = p ln(x). Since p>0 is fixed y as x . Clearly y/ey 0 as x (since also y ) by looking at the power series for ey. So ln(x)/xp = o(1); note that this is true for other bases as we have discussed in L1. Example 8: PpLq(n) = l mp p 0 q 0 (a p n p bq,p (lg(n))q ) = (nl (log(n))ml ) ; a l 0 ; b ml ,l 0 for all l (notice that ml = 0 is allowed). This is a generalization of Example 6 and uses the result of Example 7. Extra Credit for working this out. n Example 9: H n 1/ n (ln(n)) i 1 x Do as Homework; use ln(x) = ( 1/ s ds) and bounds from integration as in the 1 derivation (SW) above. 6 4. L’Hospital’s Rule Here is a statement of the form of L’Hospital’s Rule we will find most convenient, adapted from Calculus with analytic geometry, R. Johnson and F. Kiokemeister, 2nd Ed., 1962, sections 14.3 and 14.4. Theorem (x form of L’Hospital’s Rule) Let f and g be differentiable functions with g(x) 0 on some interval x 0 , . If lim f (x) lim g(x) 0 or lim f (x) and lim g(x) , then x x x x f (x) f (x) lim x g(x) x g(x) lim provided the second limit exists or is infinite. Example 10 (Ex 7 revisited, for p>0) ln(x) 1/ x 1 lim p1 lim p 0 . p x x x px x px lim Notice that you still need to know that xp as x , which follows, as in Example 7, from the (e, ln) definition of xp and the properties of ex. 7 Betsy Helping in Lecture Preparation 8