Download Lecture 11: Algorithms - United International College

Discrete Structures
Lecture 11: Algorithms
Miss, Yanyan,Ji
United International College
Thanks to Professor Michael Hvidsten
Objective and Outline
• Objective:
Construct and analysis algorithm
• Outline:
 Algorithm definition and properties
Using Pseudocode describe sorting and search
algorithm
Algorithm analysis: Big oh, Big Theta, Big Omega
Algorithms
• Definition: An algorithm is a finite set of precise instructions
for performing a computation or for solving a problem.
• Example: Find the largest (maximum) number in a list.
• Algorithm: (in Pseudocde)
procedure max(a1, a2 , a3 , …, an: integers)
max = a1
for i = 2 to n
if max < ai then max = ai
return max
Algorithm Properties
• All algorithms share these properties:
–
–
–
–
Input: The set of input value(s) for the algorithm
Output: Value(s) that form the solution to the problem
Well-Defined: The steps of the algorithm must be precise
Correctness: An algorithm should produce the correct
input for a given set of input values
– Finiteness: An algorithm should produce an output after a
finite number of steps
– Effectiveness and Generality: The algorithm should be
applicable for all possible input values.
Algorithms
• Example: Find the maximum number in a list.
procedure max(a1, a2 , a3 , …, an: integers)
max = a1
for i = 2 to n
if max < ai then max = ai
return max
• Does this algorithm have all of the properties listed
above?
Solution
• Input :a sequence of integers
• Output: the largest integer in the sequence
• Definiteness: only assignments, a finite loop, and condition
statements occur.
• Correctness: initial value of max is the first term of the sequence, as
successive terms of the sequence are examined. max is updated to
the value of a term if the term exceeds the maximum of the terms
previously examined.
• Finiteness: it terminates after all the integers in the sequence have
been examined.
• Effectiveness: the algorithm can be carried out in a finite amount of
time since each step is either a comparison or an assignment.
• Generality: it can be used to find the maximum of any finite
sequence of integers.
Algorithm Properties
• Note: An algorithm can be considered as a function that takes
an input and returns an output.
• Pseudocode: Provides an intermediate step between an
English language description and a implementation in a
programming language. We want to use instructions
resembling those of a programming language. We don’t use
any particular language (like C or Java) to describe the steps in
an algorithm, since we want to be able to implement the
algorithm in every language.
Search Algorithms
• General Problem: Locate an element x in a list of
elements a1, a2 , a3 , …, an.
• Example: Locate a word in a dictionary.
• Linear Search Algorithm:
Compare x with a1. If x = a1, return 1. Otherwise, compare x
with a2. If x = a2, return 2. Continue in this process until we
either find x, then the solution will be the location of x or
we exhaust the list (in which case we return 0).
Linear Search
• Linear Search Algorithm: (Pseudocode)
procedure linear search (x: integer,
a1,a2,a3,…, an: distinct integers)
i=1
while (i ≤ n and x ≠ ai )
i = i+1
if i ≤ n then return i
else return 0
Binary Search
• Binary Search Algorithm:
If the terms in the list are in order of increasing size,
a1 ≤ a2 ≤ a3 ≤ … ≤ an we can use the following procedure:
Compare x to the middle term of the list am where m =
⌊(n+1)/2⌋ . If x > am, continue searching in the second half
of the list. Otherwise, continue searching in the first half of
the list.
In either case, we now have a new list to search of half the
size as the original. Use the same procedure on this new
list, on so on.
Binary Search
• Binary Search Algorithm:
i = left endpoint of search interval
j = right endpoint of search interval
Start: i = 1, j = n, x = (term to find)
While i<j do the following:
Compute middle index of list, m = ⌊(i+j)/2⌋
If x > am, search in second half of list (am+1, am+2 , … aj)
and set i = m+1
Else search in first half of list (ai, ai+1 , … am) and set j = m
Binary Search
Example: Find index of 13 in (1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16)
Start: i=1, j=12, x = 13
(1) m = ⌊(i+j)/2 =13/2⌋ = 6 and am = 7. Since x > 7, we set i =
m+1 = 7 and continue searching in (8 10 12 13 15 16).
[i=7, j=12]
(2) m = ⌊(i+j)/2⌋ = 9 and am = 12. Since x > 12, we set i = m+1 =
10 and continue searching in (13 15 16). [i=10, j=12]
(3) m = ⌊22/2⌋ = 11 and am = 15. Since x < 15, set j= m = 11 and
continue searching in (13 15). [i=10, j=11]
(4) m = ⌊21/2⌋ =10 and am = 13. Since x = 13, set j=m=10. We are
done, since i=j=10.
Return 10.
Binary Search
• Binary Search Algorithm: (Pseudocode)
procedure Binary search(x: integer,
a1,a2,a3,…, an: increasing integers)
i = 1 (left endpoint of search interval)
j = 1 (right endpoint of search interval)
while (i < j)
begin
m = ⌊(i+j)/2⌋
if x > am then i = m+1
else j = m
end
if x = ai return i
else return 0
public class BinarySearch {
public static int search(int x, int[] list) {
int i=0, j = list.length-1, m =0;
while (i < j) {
m = (int)Math.floor( ((double)(i+j)) / 2.0);
System.out.println("i = " + i + ", j = " + j + ", m = " + m);
System.out.println(" x = " + x + ", list[m]= " + list[m]);
if(x > list[m]) i = m+1;
else j = m;
}
System.out.println("last step: i = " + i + ", j = " + j + ", list[i] = " + list[i]);
if(x == list[i]) return i;
else return 0;
}
public static void main(String arg[]) {
int list[] = {1,2,3,5,6,7,8,10,12,13,15,16};
int x = 13;
int m = BinarySearch.search(x,list);
System.out.println("index of x in list = " + m);
}
}
Java compiling
• Third, we would type “javac BinarySearch.java” to
compile our java program. If successful, a file called
“BinarySearch.class” is created. This is the bytecode
for our program.
• Lastly, to run the program type “java BinarySearch”
Java Compiling
Question
Question
Class exercise:
List all the steps used to search for 9 in the sequence
1,3,4,5,6,8,9,11 using
(a) a linear search
(b) a binary search
Answer:
(a) i=1,i=2,i=3,i=4,i=5,i=6,i=7 So Location=7
(b) i=1,j=8,m=4,i=5,j=8,m=6,i=7,j=8,m=7,j=7,location=7
Sorting
• Input sequence <a1,a2,….,an>
• Output sequence<a1’,a2’,…,an’>
such that:
a1’<=a2’<=a3’…<=an’
or
a1’>=a2’>=a3’…>=an’
Sorting
• Bubble sort is one of the simplest sorting algorithms,
but not one of the most efficient. The smaller
elements “bubble” to the top ,the larger elements
“sink” to the bottom.
• The Bubble sort algorithm:
Procedure bubblesort(a1,…,an :real numbers with
n>=2)
for i:=1 to n-1
for j=1 to n-I
if aj > aj+1 then interchange aj and aj+1
{a1,…,an is in increasing order}
Sorting
• Use the bubble sort to put 3,2,4,1,5 into increasing
order.
Sorting
• Insertion sort is a simple sorting algorithm, a comparison sort
in which the sorted array is built one entry at a time.
• The insertion sort algorithm:
Procedure insertion sort(A[1:n])
for j:=2 to n
do key=A [j]
i=j-1;
while i>0 and A [i]>key
do A [i+1]:=A [i]
i:=i-1
end
A [i+1]:=key
End {A [n] are sorted}
j
i
key
Sorting
• Insertion Example:
824936
284936
248936
248936
234896
2 3 4 6 8 9 done
Question
• Class Exercise:
Sorting the sequence numbers: 6,2,3,1,5,4 using
(a) Bubble sort
(b) Insertion sort
Algorithm Analysis
• Why need algorithm analysis ?
 Writing a working program is not good enough
 The program may be inefficient!
 If the program is run on a large data set, then the
running time becomes an issue
The growth of functions
• Without worry about the hardware and software used to
implement an algorithm.
• Assume the different operations used in an algorithm take the
same time, which simplifier the analysis.
• Determine whether it is practical to use a particular algorithm
to solve a problem as the size of the input increase
• Compare two algorithms to determine which is more efficient
as the size of input grows.
Big-O Notation
• Big-O notation is used extensively to estimate the
number of operations an algorithm as its input grows.
• Let f and g be functions from the set of integers or
the set of real numbers to the set of real numbers.
•  c , n0 > 0 such that f(N)  c g(N) when N  n0
• g(N) is an upper bound on f(N)
• f(x) is big –O of g(x)
Big-O Notation
• F(x)=x2+2x+1 is O(x2)
• We observe that we can readily estimate of f(x) when
x>1 because x<x2 and 1<x2 when x>1.
• x2<=x2+2x+1<=x2+2x2+x2=4x2
• Consequently, we can take c=4 and n0=1
Big-Omega
• Let f and g be functions from the set of integers or
the set of real numbers to the set of real numbers.
•  c , n0 > 0 such that f(N)  c g(N) when N  n0
• g(N) is an lower bound on f(N)
• f(x) is big – of g(x)
Big-Theta
• Let f and g be functions from the set of integers or
the set of real numbers to the set of real numbers.
• f(N) = (g(N)) iff
f(N) = O(g(N)) and f(N) = (g(N))
• g(N) is both an upper and
a lower bound on the size
of f(N)
Big-Theta
• Show that 3x2+8x log x is (x2)
0  8x log x  8 x2, it follows that 3x2+8x log x
11 x2 for x>1. Consequently, 3x2+8x log x is O(X2).
Clearly. X2 is O(3x2+8x log x ).
So, 3x2+8x log x is (x2)
Exercise
Class Exercise:
Using the definition of Big-O notation to proof:
(1) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)).
Then (f1+f2)(x) is O(max(|g1(x)|,|g2(x)|).
(2) Suppose that f1(x) is O(g1(x)) and f2x is O(g2(x)).
Then (f1f2)(x) is O(g1(x)g2(x)).
Complexity of Algorithm
• Best Case
– constraints on the input, other than size, resulting
in the fastest possible running time.
• Worst Case
– constraints on the input, other than size, resulting
in the slowest possible running time.
• Average Case
– average running time over every possible type of
input (usually involve probabilities of different
types of input).
Three Analyses of Insertion Sorting
Best Case
A[1] ≤ A[2] ≤ A[3] ≤ ··· ≤ A[n]
The number of comparisons needed is equal to
Worst Case
A[1] ≥ A[2] ≥ A[3] ≥ ··· ≥ A[n]
The number of comparisons needed is equal to
Average Case
Θ(n2) assuming that each of the n! instances are equally like
Complexity of Algorithm
Commonly Used Terminology for the Complexity of
Algorithm
Complexity
Terminology
Θ(1)
Constant Complexity
Θ(log n)
Logarithmic Complexity
Θ(n)
Linear Complexity
Θ(n log n)
N log N complexity
Θ(n b)
Polynomial Complexity
Θ(b n)
Exponential Complexity
Θ(n!)
Factorial Complexity