Download 3.2 - The Growth of Functions

3.2 - The Growth of Functions
Introduction
In this section we will introduce the notation used to estimate the number of steps needed by an algorithm.
Big-O Notation
Definition 1
Let f and g be functions from the set of integers or the set or real numbers to the set of real numbers. We
way that f HxL is OHgHxLL if there exist constants C and k such that † f HxL§ § C †gHxL§ whenever x > k.
The constants C and k in the definition above are called witnesses to the relationship f HxL is OHgHxLL. We need to find
only one pair of witnesses for this relationship (though there are really infinitely many pairs).
ü Example 3.2.1
Determine whether each of these functions is OIx2 M.
(a)
f HxL = 17 x + 11
(b)
f HxL = x2 + 1000
2
Lecture_03_02.nb
In the relationship “ f HxL is OIx2 M”, we can replace x2 by any function with larger values than x2 (such as OIx2 + 10 xM
or OIx3 M). In part (b) of the previous example, we have two functions such that f HxL is OHgHxLL and gHxL is OH f HxLL. We
say that two functions f HxL and gHxL that satisfy both of these big-Oh relationships are of the same order. Keep in
mind, though we write f HxL = OHgHxLL, what we really mean is f HxL œ OHgHxLL. Also, if f HxL is OHgHxLL and hHxL is some
function such that †gHxL§ < †hHxL§ for all x > k, then f HxL is OHhHxLL. When big-O notation is used, the function g in the
relationship f HxL is OHgHxLL is chosen to be as small as possible. For the most part, we will deal with functions that
only take on positive values, so we will not use the absolute values when we work with big-O.
ü Example 3.2.2
Determine whether the function f HxL =
x4
2
is OIx2 M.
Big-O Estimates for Some Important Functions
Theorem 1
Let f HxL = an xn + an-1 xn-1 + ∫ + a1 x + a0 , where a0 , a1 , . . . , an-1 , an are real numbers. Then f HxL is
n
OHx L.
ü Example 3.2.3
Give a big-O estimate for the factorial function (where n ∫ nHn - 1L ∫2 ÿ 1 for any positive integer) and the logarithm
of the factorial function.
We will use a set of reference functions to estimate the number of operations an algorithm preforms in the next section.
This set of reference function includes: 1, log n, , n log n, n2 , 2n , n!. We can show that each function in this list is
smaller than the preceding function by showing limnض
tions in the list.
fn HxL
fn+1 HxL
= 0 where fn HxL and fn+1 HxL are two consecutive func-
Lecture_03_02.nb
3
From Theorem 1 we know that if f HnL is a polynomial of degree d, then f HnL is OInd M. If d > c > 1, then nc is OInd M. It
can be shown (see textbook Example 7) that logb n is OHnL for b > 1, and so for b > 1, c and d positive, Hlogb nLc is
OInd M. We can also show (see textbook Example 7) that n is OH2n L, so more generally, if d is positive and b > 1, then
OHbn L. Also, if c > b > 1, then bn is OHcn L.
The Growth of Combinations of Functions
Theorem 2
Suppose that f1 HxL is OHg1 HxLL and f2 HxL is OHg2 HxLL. Then H f1 + f2 L HxL is OHmax 8g1 HxL, g2 HxL<L.
Corollary 1
Suppose that f1 HxL and f2 HxL are both OHgHxLL. Then H f1 + f2 L HxL is OHgHxLL.
Theorem 3
Suppose that f1 HxL is OHg1 HxLL and f2 HxL is OHg2 HxLL. Then H f1 ÿ f2 L HxL is OHg1 HxLÿ g2 HxLL.
ü Example 3.2.4
Determine where f HxL = dxt `xp is OIx2 M.
ü Example 3.2.5
Give a big-O estimate for the function I2n + n2 M In3 + 3 nM. For the function g in the estimate that f HxL is OHgHxLL, use a
simple function g of the smallest order.
4
Lecture_03_02.nb
Big-Omega and Big-Theta Notation
Big-O notation provides an upper bound, but not a lower bound. Big-Omega (big W) will a lower bound estimate.
Finally, if we want to give both an upper and lower bound on the size of a function f, we will use Big-Theta (Big-Q).
Definition 2
Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say
that f HxL is WHgHxLL if there exist constants C and k such that † f HxL§ ¥ C †gHxL§ whenever x > k.
ü Example 3.2.6
Determine whether the function f HxL = x2 + 1000 is WIx2 M.
Definition 3
Let f and g be functions from the set of integers or the set of real numbers to the set of real numbers. We say
that f HxL is QHgHxLL if f HxL is OHgHxLL and f HxL is WHgHxLL. When f HxL is QHgHxLL, we say that f is big-Theta of gHxL”
and we also say that f HxL is of order gHxL.
ü Example 3.2.7
Show f HxL = x2 + 1000 is QIx2 M.
Theorem 4
Let f HxL = an xn + an-1 xn-1 + ∫ + a1 x + a0 , where a0 , a1 , . . . , an-1 , an are real numbers with an ∫ 0. Then
f HxL is of order xn .
ü Example 3.2.8
Give the order of each polynomial.
(a)
2 x2 + x - 7
(b)
x5 - 7 x2 + 8 x - 1