Download 2 k+1

Efficiency of Algorithms:
Analyzing algorithm segments,
Insertion sort algorithm
Logarithmic Orders
Binary Search Algorithm
1
Computing an Order
of an Algorithm Segment
• Consider the following algorithm segment:
max:=a[1]; min:=b[1]
for i:=2 to n
if max < a[i] then max:=a[i] ;
if min > b[i] then min:=b[i]
next i
if min ≥ max then med:=(min+max)/2
• Analysis: Number of loop iterations: n-1
Comparisons in each iteration: 2
Thus, elementary operations in the loop: 2(n-1)
Operations after the loop: 3
Total number of operations in the segment: 2(n-1)+3 = 2n+1
• Order of the segment: O(n)
2
The Insertion Sort Algorithm
• Insertion sort arranges the elements of one-dimensional array
a[1], …, a[n] into increasing order.
• for i:=2 to n
(insert a[i] into the sorted sequence a[1],…,a[i-1])
key:=a[i]
for j:=1 to i-1
if key<a[j] then goto (*)
next j
(*) (shift the values of a[j],…,a[i-1] to a[j+1],…,a[i])
for k:=i to j+1
a[k]:=a[k-1]
next k
a[j]:=key
next i
3
The Insertion Sort Algorithm:
Example
• Arrange array <8, 2, 7, 9, 1, 4> into increasing order.
8
2
7
9
1
4
2
8
7
9
1
4
2
7
8
9
1
4
2
7
8
9
1
4
1
2
7
8
9
4
1
2
4
7
8
9
4
The Insertion Sort Algorithm:
Analysis
• For each i (outer loop),
maximum number of elementary operations is i – 1 .
• Thus, the total number of elementary operations:
1+2+…+(n-1) = n(n-1)/2 = .5n2 - .5n
• The insertion sort algorithm is O(n2) .
5
Logarithmic function
• Definition: The logarithmic function with base b
(b>0, b1)
is the following function from R+ to R:
logb(x) = the exponent to which b must
raised to obtain x .
Symbolically, logbx = y  by = x .
• Property: If the base b>1,
then the logarithmic function is increasing:
if x1<x2 , then logb(x1) < logb(x2) .
Note: Logarithmic function grows very slowly,
e.g., log2(1,024)=10, log2(1,048,576)=20 .
6
A property of
logarithmic function
• Proposition 1: If k is an integer
and x is a real number with 2k  x < 2k+1,
then log2x = k .
• Proof:
2k  x < 2k+1
log2(2k)  log2(x) < log2(2k+1) (since log2x increasing)
k  log2(x) < k+1
(by definition of log2x)
log2x = k
(by definition of floor function)
■
An application of logarithms
• Question: Given a positive integer n,
how many binary digits are needed to represent n?
• Solution: Binary representation of n: 1ck-1ck-2…c2c1c0
which corresponds to n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0
Since ci  1,
n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0
 2k + 2k-1 + … + 22 + 2 + 1
= (2k+1-1) / (2-1) (as a sum of geometric sequence)
= 2k+1-1 < 2k+1
(1)
On the other hand,
n = 2k + ck-1∙2k-1 + … + c2∙22 + c1∙2 + c0 ≥ 2k
(2)
Combining (1) and (2):
2k  n < 2k+1
(3)
log2n
and the number of binary digits is log2n + 1.
Based on (3) and Proposition 1, k =
8
Exponential and Logarithmic Orders
 For all real numbers b and r with b>1 and r>0
and for all sufficiently large values of x,
 logbx  xr;
( which implies that logbx is O(xr) )
 xr  bx
( which implies that xr is O(bx) )
 For all real numbers b with b>1
and for all sufficiently large values of x,
x  x logbx  x2
(which implies that x is O(x logbx) and x logbx is O(x2) )
9
Binary Search Algorithm
• The algorithm searches for an element x
in an ascending array of elements a[1],…,a[n] .
• Algorithm body:
index:=0, bot:=1, top:=n
while (top ≥ bot and index=0)
mid :=  (bot+top) / 2
if a[mid] = x then index := mid
if a[mid] > x
then top := mid-1
else bot := mid+1
end while
Output: index
(If index has the value 0 then x is not in the array;
otherwise, index gives the index of the array where x is located.)
Binary Search Algorithm: Example
• Suppose a[1]=Amy, a[2]=Bob, a[3]=Dave, a[4]=Erin,
a[5]=Jeff, a[6]=John, a[7]=Larry, a[8]=Mary,
a[9]=Mike, a[10]=Sam, a[11]=Steve, a[12]=Tom.
(sorted in alphabetical order)
• Search for x=Erin.
• The table tracing the binary search algorithm:
index
bot
0
1
1
4
top
12
5
5
6
3
mid
4
4
11
The Efficiency of
the Binary Search Algorithm
• At each iteration,
the length of the new subarray to be searched
is approximately half of the previous one.
• If n = 2k+m, where 0  m < 2k,
then n can be split approximately in half k times.
• Since 2k  n < 2k+1, then k = log2n (by proposition 1)
• Thus, the number of iterations of the while loop
in a worst-case execution of the algorithm is log2n+1.
• The number of operations in each loop is constant
(doesn’t increase with n).
• Thus, the binary search algorithm is O(log2n) .
12