* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download LEC01 - aiub study guide
Survey
Document related concepts
Computational complexity theory wikipedia , lookup
Cryptanalysis wikipedia , lookup
Genetic algorithm wikipedia , lookup
Theoretical computer science wikipedia , lookup
Recursion (computer science) wikipedia , lookup
Fisher–Yates shuffle wikipedia , lookup
Algorithm characterizations wikipedia , lookup
Post-quantum cryptography wikipedia , lookup
Factorization of polynomials over finite fields wikipedia , lookup
Pattern recognition wikipedia , lookup
Transcript
CSC2105: Algorithms Mashiour Rahman [email protected] American International University Bangladesh Literature Introduction to Algorithms, Second Edition, Thomas H. Cormen, Charle E. Leiserson, Ronald L. Rivest, Clifford Stein (CLRS). Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj Sahni, Sanguthevar Rajasekaran (HSR). Helpful link for Problem Solving : http://acm.uva.es/problemset/ CSC2105 Mashiour Rahman 2 The Goals of this Course The main things we will learn in this course: To think algorithmically and get the spirit of how algorithms are designed. To get to know a toolbox of classical algorithms. To learn a number of algorithm design techniques (such as divide-and-conquer). To reason (in a precise and formal way) about the efficiency and the correctness of algorithms. CSC2105 Mashiour Rahman 3 General Algorithms are first solved on paper and later keyed in on the computer. The most important thing is to be simple and precise. During lectures: CSC2105 Interaction is welcome; ask questions. Additional explanations and examples if desired. Speed up/slow down the progress. Mashiour Rahman 4 Prerequisites Introduction to programming Data types, operations Conditional statements Loops Procedures and functions C/C++/Java Computer lab (edit, compile, execute) CSC2105 Mashiour Rahman 5 History Name: Persian mathematician Mohammed alKhowarizmi, in Latin became Algorismus. First algorithm: Euclidean Algorithm, greatest common divisor, 400-300 B.C. 19th century – Charles Babbage, Ada Lovelace. 20th century – Alan Turing, Alonzo Church, John von Neumann. CSC2105 Mashiour Rahman 6 Data Structures and Algorithms Data structure Algorithm Organization of data to solve the problem at hand. Outline, the essence of a computational procedure, step-by-step instructions. Program CSC2105 Implementation of an algorithm in some programming language. Mashiour Rahman 7 Overall Picture Using a computer to help solve problems. CSC2105 Precisely specify the problem. Designing programs architecture algorithms Writing programs Verifying (testing) programs Data Structure and Algorithm Design Goals Correctness Efficiency Implementation Goals Reusability Robustness Adaptability Mashiour Rahman 8 Overall Picture/2 This course is not about: Programming languages Computer architecture Software architecture Software design and implementation principles Issues concerning small and large scale programming. We will only touch upon the theory of complexity and computability. CSC2105 Mashiour Rahman 9 Algorithmic problem Specification of input Infinite number of input instances satisfying the specification. For example: A sorted, non-decreasing sequence of natural numbers. The sequence is of non-zero, finite length: CSC2105 ? Specification of output as a function of input 1, 20, 908, 909, 100000, 1000000000. 3. Mashiour Rahman 10 Algorithmic Solution Input instance, adhering to the specification CSC2105 Algorithm Output related to the input as required Algorithm describes actions on the input instance. There may be many correct algorithms for the same algorithmic problem. Mashiour Rahman 11 Definition of an Algorithm An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Properties: Precision Determinism Finiteness CSC2105 Mashiour Rahman Efficiency Correctness Generality 12 How to Develop an Algorithm Precisely define the problem. Precisely specify the input and output. Consider all cases. Come up with a simple plan to solve the problem at hand. The plan is language independent. The precise problem specification influences the plan. Turn the plan into an implementation CSC2105 The problem representation (data structure) influences the implementation. Mashiour Rahman 13 Preconditions, Postconditions It is important to specify the preconditions and the postconditions of algorithms: CSC2105 INPUT: precise specifications of what the algorithm gets as an input. OUTPUT: precise specifications of what the algorithm produces as an output, and how this relates to the input. The handling of special cases of the input should be described. Mashiour Rahman 14 Analysis of Algorithms Efficiency: Running time Space used Efficiency as a function of the input size: CSC2105 Number of data elements (numbers, points). The number of bits of an input number . Mashiour Rahman 15 The RAM model It is important to choose the level of detail. The RAM model: Instructions (each taking constant time), we usually choose one type of instruction as a characteristic operation that is counted: CSC2105 Arithmetic (add, subtract, multiply, etc.) Data movement (assign) Control flow (branch, subroutine call, return) Comparison Data types – integers, characters, and floats Mashiour Rahman 16 Analysis of Insertion Sort Time to compute the running time as a function of the input size (exact analysis). for j := 2 to n do key := A[j] // Insert A[j] into A[1..j-1] i := j-1 while i>0 and A[i]>key do A[i+1]:=A[i] i-A[i+1]:=key CSC2105 Mashiour Rahman cost c1 c2 0 c3 c4 c5 c6 c7 times n n-1 n-1 n-1 n t (t 1) jn = 2 j (t 1) j =2 j j nj = 2 n-1 17 Analysis of Insertion Sort/2 The running time of an algorithm is the sum of the running times of each state-ment. A statement with cost c that is executed n times contributes c*n to the running time. The total running time T(n) of insertion sort is T(n) = c1*n + c2(n-1) + c3(n-1) + c4 j = 2 t j+ n n ( t 1) c5 j =2 j + c6 j =2 (t j 1)+ c7(n-1) CSC2105 n Mashiour Rahman 18 Analysis of Insertion Sort/3 Often the performance depends on the details of the input (not only the length n). This is modeled by tj. In the case of insertion sort the time tj depends on the original sorting of the input array. CSC2105 Mashiour Rahman 19 Performance Analysis Often it is sufficient to count the number of iterations of the core (innermost) part. No distinction between comparisons, assignments, etc (that means roughly the same cost for all of them). Gives precise enough results. In some cases the cost of selected operations dominates all other costs. CSC2105 Disk I/O versus RAM operations. Database systems. Mashiour Rahman 20 Best/Worst/Average Case Analyzing insertion sort’s CSC2105 n j =2 (t j 1) Best case: elements already sorted, tj=1, running time = n-1, i.e., linear time. Worst case: elements are sorted in inverse order, tj=j-1, running time = (n2-n)/2, i.e., quadratic time. Average case: tj=j/2, running time = (n2+n-2)/4, i.e., quadratic time. Mashiour Rahman 21 Best/Worst/Average Case/3 For inputs of all sizes: worst-case average-case Running time 6n 5n best-case 4n 3n 2n 1n 1 CSC2105 2 3 4 5 6 7 8 9 10 11 12 ….. InputMashiour instance size Rahman 22 Best/Worst/Average Case/4 Worst case is usually used: CSC2105 It is an upper-bound. In certain application domains (e.g., air traffic control, surgery) knowing the worst-case time complexity is of crucial importance. For some algorithms worst case occurs fairly oftenThe average case is often as bad as the worst case. Finding the average case can be very difficult. Mashiour Rahman 23 Analysis of Linear Search INPUT: A[1..n] – a sorted array of integers, q – an integer. OUTPUT: an index j such that A[j] = q. NIL if "j (1jn): A[j] q j := 1 while j n and A[j] q do j++ if j n then return j else return NIL Worst case running time: n Average case running time: n/2 Best case running time: 0 CSC2105 Mashiour Rahman 24 Binary Search Idea: Have a left and right bound. Elements to the right of r are bigger than the search element. Equivalent for l. In each step reduce the range of the search space by half. INPUT: A[1..n] – a sorted array of (increasing) integers, q – an integer. OUTPUT: an index j such that A[j] = q. NIL, if "j (1jn): A[j] q l := 1; r := n do m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= r return NIL CSC2105 Mashiour Rahman 25 Analysis of Binary Search How many times the loop is executed? With each execution the difference between l and r is cut in half. CSC2105 Initially the difference is n. The loop stops when the difference becomes 0 (less than 1) . How many times do you have to cut n in half to get 0? log n – better than the brute-force approach of linear search (n). Mashiour Rahman 26 Linear Search vs Binary Search Costs of linear search: n Costs of binary search: log(n) Should we care? Phone book with 200’000 entries: CSC2105 n = 200’000 log n = log 200’000 = 17.6 Mashiour Rahman 27 Asymptotic Analysis Goal: to simplify the analysis of the running time by getting rid of details, which are affected by specific implementation and hardware “rounding” of numbers: 1,000,001 1,000,000 “rounding” of functions: 3n2 n2 Capturing the essence: how the running time of an algorithm increases with the size of the input in the limit. CSC2105 Asymptotically more efficient algorithms are best for all but small inputs Mashiour Rahman 28 Asymptotic Notation The “big-Oh” O-Notation asymptotic upper bound f(n) = O(g(n)), if there exists constants c>0 and n0>0, s.t. f(n) c g(n) for n n0 f(n) and g(n) are functions over non-negative integers Used for worst-case analysis CSC2105 Mashiour Rahman c g ( n) f (n ) Running Time n0 Input Size 29 Asymptotic Notation/2 The “big-Omega” WNotation asymptotic lower bound f(n) = W(g(n)) if there exists constants c>0 and n0>0, s.t. c g(n) f(n) for n n0 Used to describe best-case running times or lower bounds of algorithmic problems. E.g., lower-bound of searching in an unsorted array is W(n). CSC2105 Mashiour Rahman f (n ) c g ( n) Running Time n0 Input Size 30 Asymptotic Notation/3 Simple Rule: Drop lower order terms and constant factors. 50 n log n is O(n log n) 7n - 3 is O(n) 8n2 log n + 5n2 + n is O(n2 log n) Note: Although (50 n log n) is O(n5), it is expected that an approximation is of the smallest possible order. CSC2105 Mashiour Rahman 31 Asymptotic Notation/4 The “big-Theta” QNotation asymptoticly tight bound f(n) = Q(g(n)) if there exists constants c1>0, c2>0, and n0>0, s.t. for n n0 c1 g(n) f(n) c2 g(n) f(n) = Q(g(n)) if and only if f(n) = O(g(n)) and f(n) = W(g(n)) O(f(n)) is often abused instead of Q(f(n)) CSC2105 Mashiour Rahman c 2 g (n ) f (n ) Running Time c 1 g (n ) n0 Input Size 32 A Quick Math Refresher Arithmetic progression n i = 1 2 3 ... n = i =0 n(1 n) 2 Geometric progression given an integer n0 and a real number 0<a1 n 1 1 a ai = 1 a a 2 ... a n = 1 a i =0 n CSC2105 geometric progressions exhibit exponential growth Mashiour Rahman 33 Miscellaneous Manipulating logarithms: a log b = log b / log a log ab = b log a Manipulating summations: ca = c a j (aj bj) = j aj j bj j CSC2105 j j j Mashiour Rahman 34 Summations The running time of insertion sort is determined by a nested loop. for j := 2 to n key := A[j] i := j-1 while i>0 and A[i]>key A[i+1] := A[i] i := i-1 A[i+1] := key Nested loops correspond to summations: n j =2 ( j 1) CSC2105 Mashiour Rahman 35 Proof by Induction We want to show that property P is true for all integers n n0. Basis: prove that P is true for n0. Inductive step: prove that if P is true for all k such that n0 k n – 1 then P is also true for n. n n(n 1) Example S ( n) = i = for n 1 2 i =0 1 Basis S (1) = i = i =0 CSC2105 1(1 1) 2 Mashiour Rahman 36 Proof by Induction/2 Inductive Step k (k 1) S (k ) = i = for 1 k n 1 2 i =0 k n n 1 i =0 i =0 S (n) = i = i n =S (n 1) n = (n 1 1) ( n 2 n 2n) = (n 1) n= = 2 2 n(n 1) = 2 CSC2105 Mashiour Rahman 37 Correctness of Algorithms An algorithm is correct if for any legal input it terminates and produces the desired output. Automatic proof of correctness is not possible. There are practical techniques and rigorous formalisms that help to reason about the correctness of (parts of) algorithms. CSC2105 Mashiour Rahman 38 Partial and Total Correctness Partial correctness IF this point is reached, Any legal input Algorithm THEN this is the desired output Output Total correctness INDEED this point is reached, AND this is the desired output Any legal input CSC2105 Algorithm Mashiour Rahman Output 39 Assertions To prove partial correctness we associate a number of assertions (statements about the state of the execution) with specific checkpoints in the algorithm. E.g., A[1], …, A[j] form an increasing sequence Preconditions – assertions that must be valid before the execution of an algorithm or a subroutine (INPUT). Postconditions – assertions that must be valid after the execution of an algorithm or a subroutine (OUTPUT). CSC2105 Mashiour Rahman 40 Pre/post-conditions Example: Write a pseudocode algorithm to find the two smallest numbers in a sequence of numbers (given as an array). INPUT: an array of integers A[1..n], n > 0 OUTPUT: (m1, m2) such that CSC2105 m1<m2 and for each i[1..n]: m1 A[i] and, if A[i] m1, then m2 A[i]. m2 = m1 = A[1] if "j,i[1..n]: A[i]=A[j] Mashiour Rahman 41 Loop Invariants Invariants: assertions that are valid any time they are reached (many times during the execution of an algorithm, e.g., in loops) We must show three things about loop invariants: CSC2105 Initialization: it is true prior to the first iteration. Maintenance: if it is true before an iteration, then it is true after the iteration. Termination: when a loop terminates the invariant gives a useful property to show the correctness of the algorithm Mashiour Rahman 42 Example: Binary Search/1 We want to show that q is not in A if NIL is returned. Invariant: "i[1..l-1]: A[i]<q (Ia) "i[r+1..n]: A[i]>q (Ib) l := 1 r := n do m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= r return NIL Initialization: l = 1, r = n the invariant holds because there are no elements to the left of l or to the right of r. l=1 yields "j,i [1..0]: A[i]<q this holds because [1..0] is empty r=n yields "j,i [n+1..n]: A[i]>q this holds because [n+1..n] is empty CSC2105 Mashiour Rahman 43 Example: Binary Search/2 Invariant: "i[1..l-1]: A[i]<q (Ia) "i[r+1..n]: A[i]>q (Ib) l := 1 r := n do m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= r return NIL Maintenance: l, r, m = (l+r)/2 A[m]!=q & A[m]>q, r=m-1, A sorted implies "k[r+1..n]: A[k]>q (Ib) A[m]!=q & A[m]<q, l=m+1, A sorted implies "k[1..l-1]: A[k]<q (Ib) CSC2105 Mashiour Rahman 44 Example: Binary Search/3 Invariant: "i[1..l-1]: A[i]<q (Ia) "i[r+1..n]: A[i]>q (Ib) Termination: l, r, l<=r Two cases: l := 1 r := n do m := (l+r)/2 if A[m] = q then return m else if A[m] > q then r := m-1 else l := m+1 while l <= r return NIL l:=m+1 we get (l+r)/2 +1 > l r:=m-1 we get (l+r)/2 -1 < r The range gets smaller during each iteration and the loop will terminate when l<=r no longer holds. CSC2105 Mashiour Rahman 45 Example: Insertion Sort/1 for j := 2 to n do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-A[i+1] := key Invariant: outside while loop A[1...j-1] is sorted A[1...j-1] Aorig inside while loop: A[1...i], key, A[i+1…j-1] A[1...i] is sorted A[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1 CSC2105 Mashiour Rahman 46 Example: Insertion Sort/2 outside while loop A[1...j-1] is sorted A[1...j-1] Aorig inside while loop: A[1...i], key, A[i+1…j-1] A[1...i] is sorted A[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1 for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-A[i+1] := key Initialization: j=2: the invariant holds, A[1…1] is trivially sorted. i=j-1: A[1...j-1], key, A[j…j-1] where key=A[j] A[j…j-1] is empty (and thus trivially sorted) A[1…j-1] is sorted (invariant of outer loop) CSC2105 Mashiour Rahman 47 Example: Insertion Sort/2 outside while loop A[1…j-1] is sorted A[1...j-1] Aorig inside while loop: A[1...i], key, A[i+1…j-1] A[1...i] is sorted A[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1 for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-A[i+1] := key Maintenance: (A[1…j-1] sorted) + (insert A[j]) A[1…j] sorted). A[1...i-1], key, A[i,i+1…j-1] satisfies conditions because of condition A[i]>key and A[1...j-1] being sorted. CSC2105 Mashiour Rahman 48 Example: Insertion Sort/2 outside while loop A[1…j-1] is sorted A[1...j-1] Aorig inside while loop: A[1...i], key, A[i+1…j-1] A[1...i] is sorted A[i+1…j-1] is sorted A[k] > key, i+1<=k<=j-1 for j := 2 to length(A)do key := A[j] i := j-1 while i>0 and A[i]>key do A[i+1] := A[i] i-A[i+1] := key Termination: main loop, j=n+1: A[1…n] sorted. A[i]key: (A[1...i], key, A[i+1…j-1]) = A[1…j-1] is sorted i=0: (key, A[1…j-1]) = A[1…j-1] is sorted. CSC2105 Mashiour Rahman 49 Recursion An object is recursive if it contains itself as part of it, or it is defined in terms of itself. Factorial: n! CSC2105 How do you compute 10!? n! = 1 * 2 * 3 *...* n n! = n * (n-1)! Mashiour Rahman 50 Factorial Function Pseudocode of factorial: fac1 INPUT: n – a natural number. OUTPUT: n! (factorial of n) fac1(n) if n < 2 then return 1 else return n * fac1(n-1) A recursive procedure includes a CSC2105 Termination condition (determines when and how to stop the recursion). One (or more) recursive calls. Mashiour Rahman 51 Tracing the Execution 6 fac(3) 3 * fac(2) 2 fac(2) 2 * fac(1) 1 fac(1) 1 CSC2105 Mashiour Rahman 52 Bookkeeping The computer maintains an activation stack for active procedure calls (-> compiler construction). Example for fac(5). CSC2105 fac(1) 1 fac(2) 2*fac(1) 1 fac(3) 3*fac(2) 2 fac(4) 4*fac(3) 6 fac(5) 5*fac(4) 24 120 Mashiour Rahman 53 Variants of Factorial fac2 INPUT: n – a natural number. OUTPUT: n! (factorial of n) fac2(n) if n < 2 then return 1 return n * fac2(n-1) fac3 INPUT: n – a natural number. OUTPUT: n! (factorial of n) fac3(n) return n * fac3(n-1) if n 2 then return 1 CSC2105 Mashiour Rahman 54 Analysis of the solutions fac2 is correct The return statement in the if clause terminates the function and, thus, the entire recursion. fac3 is incorrect CSC2105 Infinite recursion. The termination condition is never reached. Mashiour Rahman 55 Fibonacci Numbers Definition fib(1) = 1 fib(2) = 1 fib(n) = fib(n-1) + fib(n-2), n>2 Numbers in the series: 1, 1, 2, 3, 5, 8, 13, 21, 34, ... CSC2105 Mashiour Rahman 56 Fibonacci Implementation fib INPUT: n – a natural number larger than 0. OUTPUT: fib(n), the nth Fibonacci number. fib(n) if n 2 then return 1 else return fib(n-1) + fib(n-2) Multiple recursive calls are possible. CSC2105 Mashiour Rahman 57 Fibonacci Implementation/2 int fib(int i) { if (i <= 2) { return 1;} else { return fib(i-1) + fib(i-2); } } int main() { printf(“Fibonacci of 5 is %d\n", fac(5)); } CSC2105 Mashiour Rahman 58 Tracing fib(4) 3 fib(4) fib(3) + fib(2) 1 2 fib(3) fib(2) + fib(1) fib(2) 1 1 1 fib(2) 1 CSC2105 fib(1) 1 Mashiour Rahman 59 Bookkeeping Activation stack for fib(4). CSC2105 fib(1) fib(2) 1 fib(2) fib(3) 1 fib(2) 1 + fib(1) 1 fib(4) fib(3) 2 + fib(2) 1 3 Mashiour Rahman 60 Mutual Recursion Recursion does not always occur because a procedure calls itself. Mutual recursion occurs if two procedures call each other. A B CSC2105 Mashiour Rahman 61 Mutual Recursion Example Problem: Determine whether a natural number is even. Definition of even: CSC2105 0 is even N is odd if N-1 is even N is even if N-1 is odd Mashiour Rahman 62 Implementation of even even INPUT: n – a natural number. OUTPUT: true if n is even; false otherwise odd(n) if n = 0 then return TRUE return !even(n-1) even(n) if n = 0 then return TRUE else return !odd(n-1) Can it be used to determine whether a number is odd? CSC2105 Mashiour Rahman 63 Is Recursion Necessary? Theory: You can always resort to iteration and explicitly maintain a recursion stack. Practice: Recursion is elegant and in some cases the best solution by far. In the previous examples recursion was never appropriate since there exist simple iterative solutions. Recursion is more expensive than corresponding iterative solutions since bookkeeping is necessary. CSC2105 Mashiour Rahman 64 Sorting Sorting is a classical and important algorithmic problem. We look at sorting arrays (in contrast to files, which restrict random access). A key constraint is the efficient management of the space In-place sorting algorithms The efficiency comparison is based on the number of comparisons (C) and the number of movements (M). CSC2105 Mashiour Rahman 65 Sorting Simple sorting methods use roughly n * n comparisons Insertion sort Selection sort Bubble sort Fast sorting methods use roughly n * log n comparisons. CSC2105 Merge sort Heap sort Quicksort Mashiour Rahman 66 Example 2: Sorting INPUT OUTPUT sequence of n numbers a permutation of the input sequence of numbers a1, a2, a3,….,an 2 5 4 10 b1,b2,b3,….,bn Sort 2 7 4 5 7 10 Correctness (requirements for the output) For any given input the algorithm halts with the output: • b1 < b2 < b3 < …. < bn • b1, b2, b3, …., bn is a permutation of a1, a2, a3,….,an CSC2105 Mashiour Rahman 67 Insertion Sort A 3 4 6 1 9 i Strategy • Start with one sorted card. • Insert an unsorted card at the correct position in the sorted part. • Continue until all unsorted cards are inserted/sorted. CSC2105 8 7 2 5 1 j n A Mashiour Rahman 44 44 12 12 12 12 06 06 55 55 44 42 42 18 12 12 12 12 55 44 44 42 18 18 42 42 42 55 55 44 42 42 94 94 94 94 94 55 44 44 18 18 18 18 18 94 55 55 06 06 06 06 06 06 94 67 67 67 67 67 67 67 67 94 68 Insertion Sort/2 INPUT: A[1..n] – an array of integers OUTPUT: a permutation of A such that A[1] A[2] …A[n] for j := 2 to n do key := A[j] j i := j-1 while i > 0 and A[i] > key do A[i+1] := A[i]; i-A[j+1] := key n j =2 The number of comparisons during the jth iteration is CSC2105 at least 1: Cmin = j =21 = n-1 n at most j-1: Cmax = j =2 j 1 = (n*n-n-2)/2 n Mashiour Rahman 69 Insertion Sort/3 The number of comparisons during the jth iteration is: j / 2 = (n*n+n–2)/4 j/2 in average: Cavg = The number of movements is Ci+1: n j =2 2 = 2*(n-1), j / 2 1= (n*n+5n-6)/4 Mavg = Mmax = j = (n*n+n-2)/2 n Mmin = j =2 n j =2 n j =2 CSC2105 Mashiour Rahman 70 Selection Sort A 1 2 3 4 1 5 7 8 9 6 j n i Strategy • Start empty handed. • Enlarge the sorted part by switching the first element of the unsorted part with the smallest element of the unsorted part. • Continue until the unsorted part consists of one element only. CSC2105 Mashiour Rahman A 44 06 06 06 06 06 06 06 55 55 12 12 12 12 12 12 12 12 55 18 18 18 18 18 42 42 42 42 42 42 42 42 94 94 94 94 94 44 44 44 18 18 18 55 55 55 55 55 06 44 44 44 44 94 94 67 67 67 67 67 67 67 67 94 71 Selection Sort/2 INPUT: A[1..n] – an array of integers OUTPUT: a permutation of A such that A[1] A[2] …A[n] for j := 1 to n-1 do key := A[j]; ptr := j for i := j+1 to n do if A[i] < key then ptr := i; key := A[i]; A[ptr] := A[j]; A[j] := key The number of comparisons is indepen-dent of the original ordering (this is less natural behavior than insertion sort): j = (n*n-n)/2 C = n 1 j =1 CSC2105 Mashiour Rahman 72 Selection Sort/3 The number of movements is: Mmax = Mmin = n 1 j =1 n 1 j =1 CSC2105 3 = 3*(n-1) j 3= (n*n–n)/4 + 3*(n-1) Mashiour Rahman 73 Bubble Sort A 1 2 3 1 Strategy • Start from the back and compare pairs of adjacent elements. • Switch the elements if the larger comes before the smaller. • In each step the smallest element of the unsorted part is moved to the beginning of the unsorted part and the sorted part grows by one. CSC2105 4 5 7 9 8 6 j A 44 06 06 06 06 06 06 06 Mashiour Rahman i 55 44 12 12 12 12 12 12 n 12 55 44 18 18 18 18 18 42 12 55 44 42 42 42 42 94 42 18 55 44 44 44 44 18 94 42 42 55 55 55 55 06 18 94 67 67 67 67 67 67 67 67 94 94 94 94 94 74 Bubble Sort/2 INPUT: A[1..n] – an array of integers OUTPUT: a permutation of A such that A[1] A[2] …A[n] for j := 2 to n do for i := n to j do if A[i-1] < A[i] then key := A[i-1]; A[i-1] := A[i]; A[i]:=key The number of comparisons is indepen-dent of the original ordering: CSC2105 C = j = 2 j 1= (n*n-n)/2 n Mashiour Rahman 75 Bubble Sort/3 The number of movements is: Mmin = 0 Mavg = Mmax = n j =2 n CSC2105 j =2 3( j 1) = 3*n*(n-1)/2 3( j 1) / 2= Mashiour Rahman 3*n*(n-1)/4 76 CSC2105 Mashiour Rahman 77 CSC2105 Mashiour Rahman 78 CSC2105 Mashiour Rahman 79 CSC2105 Mashiour Rahman 80 The divide- and- conquer design paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions. CSC2105 Mashiour Rahman 81 The divide- and- conquer design paradigm Example: merge sort 1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Linear- time merge. Recurrence for merge sort T( n) = 2 T( n/ 2) # subproblems + subproblem size O( n) work dividing and combining CSC2105 Mashiour Rahman 82 The divide- and- conquer design paradigm Binary search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Recurrence for binary search T(n) = 1 T(n/2) + Θ(1) # subproblems subproblem work size dividing and CSC2105 Mashiour Rahman 83 The divide- and- conquer design paradigm CSC2105 Mashiour Rahman 84 The divide- and- conquer design paradigm Conclusion • Divide and conquer is just one of several powerful techniques for algorithm design. • Divide- and- conquer algorithms can be analyzed using recurrences and the master method (so practice this math). • Can lead to more efficient algorithms CSC2105 Mashiour Rahman 85 CSC2105 Mashiour Rahman 86 CSC2105 Mashiour Rahman 87 CSC2105 Mashiour Rahman 88 CSC2105 Mashiour Rahman 89 CSC2105 Mashiour Rahman 90 CSC2105 Mashiour Rahman 91 CSC2105 Mashiour Rahman 92 CSC2105 Mashiour Rahman 93 CSC2105 Mashiour Rahman 94 CSC2105 Mashiour Rahman 95 CSC2105 Mashiour Rahman 96 CSC2105 Mashiour Rahman 97 CSC2105 Mashiour Rahman 98 CSC2105 Mashiour Rahman 99 CSC2105 Mashiour Rahman 100 CSC2105 Mashiour Rahman 101 CSC2105 Mashiour Rahman 102 CSC2105 Mashiour Rahman 103 CSC2105 Mashiour Rahman 104