Download Chapter 1: Introduction

The Design & Analysis of the
algorithms
Lecture 1. 2010.. by me M. Sakalli
You must have learnt these today..

Why analysis?..

What parameters to observe? Slight # 6..

Methodological differences to achieve better outcomes,
there Euclid’s algorithm with three methods.

Asymptotic order of growth.

Examples with insertion and selection sorting
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-1
Finite number of steps.
An algorithm: A sequence of unambiguous – well defined
procedures, instructions for solving a problem.
For a desired output.
For given range of input values
Execution must be completed in a finite amount of time.
Problem, relating input parameters with certain
state parameters, analytic rendering..
algorithm
input
“computer”
output
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-2
Important points to keep in mind

Reduce ambiguity
• Keep the code simple, clean with well-defined and
correct steps .
• Specify the range of inputs applicable.

Investigate other approaches solving the problem
leading to determine if other efficient algorithms
possible.

Theoretically:
• Prove its correctness..
• Efficiency: Theoretical and Empirical analysis
• Its optimality
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-3
Efficiency.

Complexity
• Time efficiency: Estimation in the asymptotic sense Big O, omega,
theta. In comparison two also you have a hidden constant.
• Space efficiency.
• The methods applied, recursive, parallel.
• Desired scalability: Various range of inputs and the size and
dimension of the problem under consideration. Examples: Video
sequences. ROI.


Computational Model in terms of an abstract computer: A
Turing machine.
http://en.wikipedia.org/wiki/Analysis_of_algorithms.
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-4
Historical Perspective
…

Muhammad ibn Musa al-Khwarizmi – 9th century
mathematician
www.lib.virginia.edu/science/parshall/khwariz.html

…
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-5
Analysis means:
 evaluate the costs, time and space costs, and manage the
resources and the methods..
 A generic RAM model of computation in which instructions
are executed consecutively, but not concurrently or in
parallel.
Running time analysis: The # of the primitive operations
executed for every line of the code ci (defined as machine
independent as possible).
 Asymptotic analysis
 Ignore machine-dependent constants
 Look at computational growth of T(n) as n→∞, n: input size
that determines the number of iterations:
Relative speed (in the same machine)
Absolute speed (between computers)
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-6
Euclid’s Algorithm
Problem definition: gcd(m,n) of two nonnegative, not both
zero integers m and n, m > n
Examples: gcd(60,24) = 12, gcd(60,0) = 60, gcd(0,0) = ?
Euclid’s algorithm is based on repeated application of
equality
gcd(m,n) = gcd(n, m mod n)
until the second number reaches to 0.
Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
r0 = m, r1 = n,
ri-1 = ri qi + ri+1,
0 < ri+1 < ri , 1i<t,
…
rt-1 = rt qt + 0
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-7
Step 1 If (n == 0 or m==n), return m and stop;
otherwise go to Step 2
Step 2 Divide m by n and assign the value of the remainder to r
Step 3 Assign the value of n to m and the value of r to n. Go to
Step 1.
while n ≠ 0 do
{ r ← m mod n
m← n
n←r }
return m
The upper bound of i iterations: ilog(n)+1 where  is
(1+sqrt(5))/2. !!! Not sure on this.. I need to check..
i = O(log(max(m, n))), since ri+1  ri-1/2.
The Lower bound, (log(max(m, n))),
(log(max(m, n))).
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-8
Proof of Correctness of the Euclid’s algorithm


Step 1, If n divides m, then gcd(m,n)=n
Step 2, gcd(mn,)=gcd(n, m mod(n)).
• If gcd(m, n) divides n, this implies that gcd must be  n:
gcd(m, n)n. n divides m and n, implies that n must be
smaller than the gcd of the pair {m,n}, ngcd(m, n)
• If m=n*b+r, for r, b integer numbers, then gcd(m, n) =
gcd(n, r). Every common divisor of m and n, also divides r.
• Proof: m=cp, n=cq, c(p-qb)=r, therefore g(m,n), divides r,
so that this yields g(m,n)gcd(n,r).
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-9
Other methods for computing gcd(m,n)
Consecutive integer checking algorithm, not a good way, it
checks all the ..
Step 1 Assign the value of min{m,n} to t
Step 2 Divide m by t. If the remainder is 0, go to Step 3;
otherwise, go to Step 4
Step 3 Divide n by t. If the remainder is 0, return t and stop;
otherwise, go to Step 4
Step 4 Decrease t by 1 and go to Step 2
Exhaustive??.. Very slow even if zero inputs would be checked..
O(min(m, n)).
(min(m, n)), when gcd(m,n) =1.
(1) for each operation,
Overall complexity is (min(m, n))
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-10
Other methods for gcd(m,n) [cont.]
Middle-school procedure
Step 1
Step 2
Step 3
Step 4
Find the prime factorization of m
Find the prime factorization of n
Find all the common prime factors
Compute the product of all the common prime factors
and return it as gcd(m,n)
Is this an algorithm?
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-11
Sieve of Eratosthenes
Input: Integer n ≥ 2
Output: List of primes less than or equal to n, sift out the numbers that are not.
for p ← 2 to n do A[p] ← p
for p ← 2 to n do
if A[p]  0 //p hasn’t been previously eliminated from the list
j ← p* p
while j ≤ n do
A[j] ← 0 // mark element as eliminated
j←j+p
Ex: 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
2 3
2 3
2 3
5
5
5
7
7
7
9
11
11
11
13
13
13
15
17
17
17
19
19
19
21
23
23
23
25
25
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-12
Asymptotic order of growth
A way of comparing functions that ignores constant factors and
small input sizes

O(g(n)): class of functions f(n) that grow no faster than g(n)

Θ(g(n)): class of functions f(n) that grow at same rate as g(n)

Ω(g(n)): class of functions f(n) that grow at least as fast as g(n)
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-13
Establishing order of growth using the definition
Definition: f(n) is in O(g(n)), O(g(n)), if order of growth of f(n) ≤ order of
growth of g(n) (within constant multiple),
i.e., there exist a positive constant c, c  N, and non-negative integer n0
such that
f(n) ≤ c g(n) for every n ≥ n0
f(n) is o(g(n)), if f(n) ≤ (1/c) g(n) for every n ≥ n0 which means f grows
strictly!! more slowly than any arbitrarily small constant of g. ???
f(n) is (g(n)), c  N, f(n) ≥ c g(n) for n ≥ n0
f(n) is (n) if f(n) is both O(n) and (n)
Examples:
 10n is O(cn2), c ≥ 10,
• since 10n ≤ 10n2 for n ≥ 1 or 10n ≤ n2 for n ≥ 10

5n+20 is O(cn), for all n>0, c>=25,
• since 5n+20 ≤ 5n+20n ≤ 25n, or c>=10 for n ≥ 4
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-14
O, Ω, Θ
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-15
Example of computational problem: sorting

Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
• Problem: Reorder <a´1, a´2, …, a´n> in ascending or
descending order.
• Output desired: a´i ≤ a´j , for i < j or i > j

Instance: The sequence <5, 3, 2, 8, 3>

Algorithms:
• Selection sort
• Insertion sort
• Merge sort
• (many others)
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-16
Selection Sort

Input: array a[1],…,a[n]
Output for example: array a sorted in non-decreasing order
*** Scanning elements of unsorted part, n-1, n-2, .. Swapping.
(n-1)*n/2=theta(n^2). Independent from the input.
 Algorithm: (Insertion in place)

for i=1 to n
swap a[i] with smallest of a[i],…a[n]
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-17
Insertion-Sort and an idea of runtime analysis
Input size is n = length[A], and tj times line-4 executed
1: for j ← 2 to n do
2: key ← A[j]
//Insert A[j] into the sorted sequence A[1 . . . j − 1].
3: i ← j - 1
4: while i > 0 and A[i] > key do
5:
A[i+1] ← A[i]
6:
i ← i-1
-4:
end while
7: A[i+1] ← key
-1: end for
The total runtime for is T(n)
The Best Case c1n + c2(n-1)+c3(n-1)+ c4(n-1)+c7(n-1)
= an + b..
Times
n
n-1
n-1
Σj=2:n tj
Σj=2:n (tj-1)
Σj=2:n (tj-1)
n-1
n-1
n
Cost
c1
c2
c3
c4
c5
c6
0
c7
0
= c1n + c2(n-1)+c3(n-1)+
c4Σj=2:n tj + c5 Σj=2:n (tj-1)
+ c6 Σj=2:n (tj-1)+ c7(n-1)
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-18
Insertion-Sort Analysis continued
For the worst case run time.. If the array A is sorted in reverse order,
then.. while loop,
Σj=2:n tj = Σj=2:n j = Σj=2:n (n(n-1)/2) -1
Σj=2:n (tj-1) = Σj=2:n (j-1) = Σj=2:n n(n-1)/2
T(n) = (c4 + c5 + c6)n2/2 + (c1 + c2 + c3 - (c4 - c5 - c6)/2 + c7 )n +
(c2 + c3 + c4 + c7)
= An2 + Bn + C
Average case runtime: Assume that about half of the subarray is out of
order, then tj = j/2, which will lead a similar kind quadratic function
M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-19
The methods

Brute force
Divide and conquer
Decrease and conquer
Transform and conquer
Greedy approach
Dynamic programming
Iterative improvement

Backtracking

Branch and bound

Randomized algorithms






M, Sakalli, CS246 Design & Analysis of Algorithms, Lecture Notes
1-20