Download 1 THE GROWTH OF FUNCTIONS Logarithms Review Basic Rules

Logarithms Review
logb x = y ⇔ by = x
THE GROWTH OF FUNCTIONS
log10100 = 2 ⇔ 102 = 100
Rosen 6th Ed., 3.2, 3.3
log216 = 4 ⇔ 24 = 16
log2(4*8) = log232 = 5
log2(4*8) = log24 + log28 = 2+3 = 5
Basic Rules of Logarithms
logz (xy)
logz (x/y)
logz (xy)
If x = y
log2(16/4) = log24 = 2
log2(16/4) = log216 – log24 = 4 – 2 = 2
= logz (x) + logz (y)
= logz (x) - logz (y)
= ylogz (x)
then logz (x) = logz (y)
log243 = log264 = 6
log243 = 3log24 = 3*2 = 6
How to quantify growth as n gets larger?
Growth
•  If f is a function from Z or R to R, how can
we quantify the rate of growth and compare
rates of growth of different functions?
•  Possible problem: Whether f(n) or g(n) is
larger at any point may depend on value of
n.
For example: 100n > n2 if n < 100
1200
1000
800
f(n)
600
400
200
0
-200 0
2
4
6
n
8
10
12
1
Big-O Notation
Show that 3x3+2x2+7x+9 is O(x3)
•  Let f and g be functions from the set of integers or
the set of real numbers to the set of real numbers.
We say that f(x) is O(g(x) if there are constants
C∈N and k∈R such that
|f(x)| ≤ C|g(x)| whenever x > k.
•  We say “f(x) is big-oh of g(x)”.
•  The intuitive meaning is that as x gets large, the
values of f(x) are no larger than a constant times
the values of g(x), or f(x) is growing no faster than
g(x).
25000
20000
21x3
Proof: We must show that ∃ constants C∈N
and k∈R such that |3x3+2x2+7x+9| ≤ C|x3|
whenever x > k.
Choose k = 1 then
3x3+2x2+7x+9 ≤ 3x3+2x3+7x3+9x3 = 21x3
So let C = 21.
Then 3x3+2x2+7x+9 ≤ 21x3 when x ≥ 1.
x
21x3
x3
21x3
3x3+2x2+7x+9
3x3+2x2+7x+9
x3
1
21
1
6.875
5
21
5.597
3.752
10
21
6.404
3.279
20
21
6.734
3.119
30
21
6.83
3.075
40
21
6.875
3.055
100
21
6.952
3.021
15000
10000
5000
3x3+2x2+7x+9
x3
0
1
2
3
4
5
6
7
8
9
10
General Rules
•  Multiplication by a constant does not change the
rate of growth. If f(n) = kg(n) where k is a
constant, then f is O(g) and g is O(f).
•  Addition of smaller terms does not change the rate
of growth. If f(n) = g(n) + smaller terms, then f is
O(g) and g is O(f).
Ex.: f(n) = 4n6 + 3n5 + 100n2 + 2 is O(n6).
General Rules (cont.)
•  If f1(x) is O(g1(x)) and f2(x) is O(g2(x)),
then f1(x)f2(x) is O(g1(x)g2(x)).
•  Examples:
10xlog2x is O(xlog2x)
n!6n3 is O(n!n3)
2
Show that n! is O(nn)
Examples
•  f(x) = 10 is O(1)
•  f(x) = x2 + x + 1 is O(x2)
•  f(x) = 2x5 + 100 x3 + xlogx is O(x5)
•  f(x) = 2n + n10 is O(2n)
How would you prove that 2n is bigger than
n10?
Proof: We must show that ∃ constants C∈N
and k∈R such that |n!| ≤ C|nn| whenever n >
k.
n! = n(n-1)(n-2)(n-3)…(3)(2)(1)
≤ n(n)(n)(n)…(n)(n)(n)
n times
=nn
So choose k = 0 and C = 1
Prove that n10 is O(2n )
Example: Not Big-Oh
Proof: We must show that ∃ constants C∈N and k∈R
such that |n10| ≤ C|2n| whenever n > k.
Take log2 of both expressions.
log2n10 = 10log2n
log22n = nlog22 =n
When is 10log2n < n? or n/log2n > 10?
2/1 = 2,
4/2 = 2,
8/3 ≈ 2.67, 16/4 = 4
32/5 = 6.2,
64/6 ≈ 10.67
For n = 64, 264 > 6410. So, if we choose then k = 64, C =
1, then |n10| ≤ 1*|2n| whenever n > 64.
•  Order matters in big-oh. Sometimes f is
O(g) and g is O(f), but in general big-oh is
not symmetric.
Consider f(n) = 4n and g(n) = n2. f is O(g).
•  Can we prove that g is O(f)? Formally, ∃
constants C∈N and k∈R such that |n2| ≤ C|
4n| whenever n > k.
•  No. To prove, must show that negation is
true. ∀C∈N, ∀k∈R, ∃n>k such that n2 >
4Cn.
∀C∈N, ∀k∈R, ∃n>k such that n2 > 4Cn.
10000
9000
8000
7000
n*n
6000
4n
5000
5*4n
4000
10*4*n
3000
20*4*n
2000
1000
•  To prove that negation is true, start with
arbitrary C and k. Must show/construct an
n>k such that n2 > 4nC
•  Easy to satisfy n > k, then
•  To satisfy n2>4nc, divide both sides by n to
get n>4C. Pick n = max(4C+1,k)
0
0
20
40
60
80
100
3
Is 2n O(n!)?
Is 2n O(n!)?
We must show that ∃ constants C∈N and k∈Z
such that |2n| ≤ C|n!| whenever n > k.
2n = 2(2)(2)…(2)(2)(2) n times
≤ n(n-1)(n-2)…(3)(2)(1) =N! if n = 4
So let C = 1 and k = 3.
Note that we could also choose k = 1 and C =
2
Since
|20| ≤ 2*|0!| = 2
|21| ≤ 2*|1!| = 2
|22| ≤ 2*|2!| = 4
|23| ≤ 2*|3!|= 12
Is f(x)=(x2+1)/(x+1) O(x)?
Hierarchy of functions
We must show that ∃ constants C∈N and
k∈R such that |f(x)| ≤ C|x| whenever x > k.
x 2 + 1 x 2 − 1 + 2 ( x − 1)( x + 1) + 2
=
=
=
x +1
x +1
x +1
2
x −1+
<x
When x > 1 (Why?)
x +1
n√n nlog n n
√n
2
n
2
1
n!
n2
n3
Therefore let k=1, C = 1
3√n
log2n
nn
|(x2+1)/(x+1)| ≤ |x| when x > 1
Hierarchy of functions
1
1, log2n
n√n nlog n
2
n2
n3
Hierarchy of functions
n!
n
2n
3√n
√n
nn
log2n
n√n nlog n
2
n2
n3
n!
n
2n
3√n
√n
nn
4
Hierarchy of functions
1, log2n, 3√n
n√n nlog n
2
n2
n!
Hierarchy of functions
1, log2n, 3√n, √n
n
2n
n√n nlog n
2
√n
nn
n3
n2
n!
Hierarchy of functions
n√n
2n
nn
n3
n2
1, log2n, 3√n, √n, n, nlog2n, n√n
n3
n!
2n
nn
Hierarchy of functions
1, log2n, 3√n, √n, n, nlog2n, n√n, n2
2n
nn
n!
n3
Hierarchy of functions
n2
nn
1, log2n, 3√n, √n, n, nlog2n
n√n nlog n
2
n2
2n
n3
Hierarchy of functions
1, log2n, 3√n, √n, n
n!
n
n!
n3
2n
nn
5
Hierarchy of functions
1, log2n, 3√n, √n, n, nlog2n, n√n, n2, n3
n!
2n
nn
Hierarchy of functions
1, log2n, 3√n, √n, n, nlog2n, n√n, n2, n3, 2n, n!, nn
Each one is Big-Oh of any function to its right
Hierarchy of functions
1, log2n, 3√n, √n, n, nlog2n, n√n, n2, n3, 2n
n!
nn
Prove that log10x is O(log2x)
First we will prove the following lemma:
Lemma: log10x = clog2x where c is a
constant.
Proof:
Let y = log2x.
Then 2y = x → log102y = log10x.
log102y = ylog102 = log10x. But since y =
log2x, this means that
log2xlog102 = log10x. Therefore c = log102
To Prove that log10x is O(log2x)
Prove log(n!) is O(nlogn)
We must show that ∃ constants C∈N and
k∈R such that |log10x| ≤ C|log2x| whenever
x > k.
From the lemma log2xlog102 = log10x so
choose C = log102, k=1
We must show that ∃ constants C∈N and
k∈R such that |logn!| ≤ C|nlogn| whenever
x > k.
We know that n! ≤ nn so
log(n!) ≤ log(nn)= nlogn
So choose k = 1, C = 1
6
We care about the running time of
computer programs!
Running time of a program depends on factors
such as:
1.  The input to the program
2.  The quality of the code generated by the
compiler.
3.  The nature and speed of the instructions on
the machine used to execute the program.
4.  The time complexity of the algorithm
underlying the program.
Why do we care about growth of
functions?
Example: Find the location of a particular
element, k, in a sequential list of n elements.
Time Complexity
•  The amount of time required for an
algorithm to solve a problem.
•  Since different computers run at different
speeds we actually describe time
complexity in terms of the number of
operations required to solve a problem as a
function of the size of the input.
•  T(n) = time complexity of a problem of size
n.
Big-O Notation for Time Complexity
•  We say that the time complexity of a function,
T(n) is O(g(n)) if there are constants C∈N and
k∈N such that
|T(n)| ≤ C|g(n)| whenever n > k.
This is exactly the same thing as the growth of
functions definition with the assumption that the
input is a natural number.
•  Best case complexity
–  We find k in the first location we check.
•  Worst case complexity
–  We find k in the last (nth ) location we check.
•  Average case complexity
–  We find k after searching n/2 locations.
Assume T(n) is the number of operations needed to solve a
problem and each operation takes 10-6 seconds.
n→ 20
50
100
500
1000
T(n)↓
1000n
.02 sec.
.05 sec.
0.1 sec.
0.5 sec.
1 sec.
500nlogn
.04 sec.
.14 sec.
.33 sec.
2.2 sec.
5 sec.
50n2
.02 sec.
.13 sec.
.5 sec.
12.5 sec.
50 sec.
10n3
.08 sec.
1.25 sec.
10 sec.
21 min.
2.8 hrs.
nlogn
4 sec.
1.1 hrs.
224 days
5*1010
centuries
*
2n
1.05 sec.
36 years
4*1014
centuries
*
*
7