Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Reg. No:……………………………. KIGALI INSTITUTE OF SCIENCE AND TECHNOLOGY INSTITUT DES SCIENCES ET TECHNOLOGIE Avenue de l'Armée, B.P. 3900 Kigali, Rwanda INSTITUTE EXAMINATIONS – ACADEMIC YEAR 2013 END OF SEMESTER EXAMINATION: MAIN EXAM FACULTY OF SCIENCE DEPARTMENT OF APPLIED MATHEMATICS FOURTH YEAR SEMESTER 2(Year 4 Math - Statistics, Year 4 Pure Math) ALGORITHMS DESIGN AND ANALYSIS [MAT 3426] DATE: / /2013 TIME: 3 HOURS MAXIMUM MARKS = 60 INSTRUCTIONS 1. This paper contains Four (4) questions. 2. Answer One compulsory question section A and any two (2) questions in section B 3. The compulsory question mark is 20 Marks and each question of section B is 20 Marks 4. No written materials allowed. 5. Do not forget to write your Registration Number. 6. Do not write any answers on this question paper. SECTION A Question 1: a) Discuss the relationship between Design and Analysis of algorithms? [10 Marks] b) Demonstrate the Asymptotic Bounds in terms of Θ-Notation, Ο-Notation and Ω-Notation [10 Marks] SECTION B Question1: Using Asymptotic analysis, demonstrate the running times of insertion sort algorithm are: - Wost case: T(n)= Θ (n2) -Best case : T(n)= Θ (n) [20 Marks] Question2: Using Asymptotic analysis, demonstrate the running times of Merge sort algorithm are: - Wost case: T(n)= Θ (nlogn) -Best case : T(n)= Θ (1) [20 Marks] Question3: Write the pseadocode of Insertion sort algorithm and show its implementation using Java. [20 Marks] Question4: a)Discuss about Dynamic algorithms and their characteristics b) Analyze the following algorithm to determine its running time. [8Marks] [12Marks] MAIN EXAM MARKING SCHEME ALGORITHMS DESIGN AND ANALYSIS [MAT 3426] Question 1: a) Discuss the relationship between Design and Analysis of algorithms? [10 Marks] The “design” pertain to i. The description of algorithm at an abstract level by means of a pseudo language, and ii. Proof of correctness that is, the algorithm solves the given problem in all cases. 2. The “analysis” deals with performance evaluation (complexity analysis). Two important ways to characterize the effectiveness of an algorithm are its space complexity and time complexity. Time complexity of an algorithm concerns determining an expression of the number of steps needed as a function of the problem size. The performance evaluation of an algorithm is obtained by totaling the number of occurrences of each operation when running the algorithm. The performance of an algorithm is evaluated as a function of the input size n and is to be considered modulo a multiplicative constant. b) Demonstrate the Asymptotic Bounds in terms of Θ-Notation, Ο-Notation and Ω-Notation [10 Marks] The following notations are commonly use notations in performance analysis and used to characterize the complexity of an algorithm. Θ-Notation (Same order) This notation bounds a function to within constant factors. We say f(n) = Θ(g(n)) if there exist positive constants n0, c1 and c2 such that to the right of n0 the value of f(n) always lies between c1 g(n) and c2 g(n) inclusive. In the set notation, we write as follows: Θ(g(n)) = {f(n) : there exist positive constants c1, c1, and n0 such that 0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n) for all n ≥ n0} We say that is g(n) an asymptotically tight bound for f(n). Graphically, for all values of n to the right of n0, the value of f(n) lies at or above c1 g(n) and at or below c2 g(n). In other words, for all n ≥ n0, the function f(n) is equal to g(n) to within a constant factor. We say that g(n) is an asymptotically tight bound for f(n). In the set terminology, f(n) is said to be a member of the set Θ(g(n)) of functions. In other words, because O(g(n)) is a set, we could write f(n) ∈ Θ(g(n)) to indicate that f(n) is a member of Θ(g(n)). Instead, we write f(n) = Θ(g(n)) to express the same notation. Historically, this notation is "f(n) = Θ(g(n))" although the idea that f(n) is equal to something called Θ(g(n)) is misleading. Example: n2/2 − 2n = (n2), with c1 = 1/4, c2 = 1/2, and n0 = 8. SECTION B Question1: Using Asymptotic analysis, demonstrate the running times of insertion sort algorithm are: - Wost case: T(n)= Θ (n2) -Best case : T(n)= Θ (n) [20 Marks] Algorithm: Insertion Sort We use a procedure INSERTION_SORT. It takes as parameters an array A[1.. n] and the length n of the array. The array A is sorted in place: the numbers are rearranged within the array, with at most a constant number outside the array at any time. INSERTION_SORT (A) 1. 2. 3. 4. 5. 6. 7. 8. FOR j ← 2 TO length[A] DO key ← A[j] {Put A[j] into the sorted sequence A[1 . . j − 1]} i←j−1 WHILE i > 0 and A[i] > key DO A[i +1] ← A[i] i←i−1 A[i + 1] ← key Example: Following figure (from CLRS) shows the operation of INSERTION-SORT on the array A= (5, 2, 4, 6, 1, 3). Each part shows what happens for a particular iteration with the value of j indicated. j indexes the "current card" being inserted into the hand. Read the figure row by row. Elements to the left of A[j] that are greater than A[j] move one position to the right, and A[j] moves into the evacuated position. Analysis Since the running time of an algorithm on a particular input is the number of steps executed, we must define "step" independent of machine. We say that a statement that takes ci steps to execute and executed n times contributes cin to the total running time of the algorithm. To compute the running time, T(n), we sum the products of the cost and times column. That is, the running time of the algorithm is the sum of running times for each statement executed. So, we have T(n) = c1n + c2 (n − 1) + 0 (n − 1) + c4 (n − 1) + c5 ∑2 ≤ j ≤ n ( tj )+ c6 ∑2 ≤ j ≤ n (tj − 1) + c7 ∑2 ≤ j ≤ n (tj − 1) + c8 (n − 1) In the above equation we supposed that tj be the number of times the while-loop (in line 5) is executed for that value of j. Note that the value of j runs from 2 to (n − 1). We have T(n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑2 ≤ j ≤ n ( tj )+ c6 ∑2 ≤ j ≤ n (tj − 1) + c7 ∑2 ≤ j ≤ n (tj − 1) + c8 (n − 1) Equation (1) Best-Case The best case occurs if the array is already sorted. For each j = 2, 3, ..., n, we find that A[i] less than or equal to the key when i has its initial value of (j − 1). In other words, when i = j −1, always find the key A[i] upon the first time the WHILE loop is run. Therefore, tj = 1 for j = 2, 3, ..., n and the best-case running time can be computed using equation (1) as follows: T(n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑2 ≤ j ≤ n (1) + c6 ∑2 ≤ j ≤ n (1 − 1) + c7 ∑2 ≤ j ≤ n (1 − 1) + c8 (n − 1) T(n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 (n − 1) + c8 (n − 1) T(n) = (c1 + c2 + c4 + c5 + c8 ) n + (c2 + c4 + c5 + c8) This running time can be expressed as an + b for constants a and b that depend on the statement costs ci. Therefore, T(n) it is a linear function of n. The punch line here is that the while-loop in line 5 executed only once for each j. This happens if given array A is already sorted. T(n) = an + b = Θ (n) It is a linear function of n. Worst-Case The worst-case occurs if the array is sorted in reverse order i.e., in decreasing order. In the reverse order, we always find that A[i] is greater than the key in the while-loop test. So, we must compare each element A[j] with each element in the entire sorted subarray A[1 .. j − 1] and so tj = j for j = 2, 3, ..., n. Equivalently, we can say that since the while-loop exits because i reaches to 0, there is one additional test after (j − 1) tests. Therefore, tj = j for j = 2, 3, ..., n and the worst-case running time can be computed using equation (1) as follows: T(n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑2 ≤ j ≤ n ( j ) + c6 ∑2 ≤ j ≤ n(j − 1) + c7 ∑2 ≤ j ≤ n(j − 1) + c8 (n − 1) And using the summations in CLRS on page 27, we have T(n) = c1n + c2 (n − 1) + c4 (n − 1) + c5 ∑2 ≤ j ≤ n [n(n +1)/2 + 1] + c6 ∑2 ≤ j ≤ n [n(n − 1)/2] + c7 ∑2 ≤ j ≤ n [n(n − 1)/2] + c8 (n − 1) T(n) = (c5/2 + c6/2 + c7/2) n2 + (c1 + c2 + c4 + c5/2 − c6/2 − c7/2 + c8) n − (c2 + c4 + c5 + c8) This running time can be expressed as (an2 + bn + c) for constants a, b, and c that again depend on the statement costs ci. Therefore, T(n) is a quadratic function of n. Here the punch line is that the worst-case occurs, when line 5 executed j times for each j. This can happens if array A starts out in reverse order T(n) = an2 + bn + c = Θ (n2) It is a quadratic function of n. Question2: Using Asymptotic analysis, demonstrate the running times of Merge sort algorithm are: - Wost case: T(n)= Θ (nlogn) -Best case : T(n)= Θ (1) [20 Marks] Analyzing Merge Sort For simplicity, assume that n is a power of 2 so that each divide step yields two subproblems, both of size exactly n/2. The base case occurs when n = 1. When n ≥ 2, time for merge sort steps: Divide: Just compute q as the average of p and r, which takes constant time i.e. Θ(1). Conquer: Recursively solve 2 subproblems, each of size n/2, which is 2T(n/2). Combine: MERGE on an n-element subarray takes Θ(n) time. Summed together they give a function that is linear in n, which is Θ(n). Therefore, the recurrence for merge sort running time is Solving the Merge Sort Recurrence By the master theorem in CLRS-Chapter 4 (page 73), we can show that this recurrence has the solution T(n) = Θ(n lg n). Reminder: lg n stands for log2 n. Compared to insertion sort [Θ(n2) worst-case time], merge sort is faster. Trading a factor of n for a factor of lg n is a good deal. On small inputs, insertion sort may be faster. But for large enough inputs, merge sort will always be faster, because its running time grows more slowly than insertion sorts. Recursion Tree We can understand how to solve the merge-sort recurrence without the master theorem. There is a drawing of recursion tree on page 35 in CLRS, which shows successive expansions of the recurrence. The following figure (Figure 2.5b in CLRS) shows that for the original problem, we have a cost of cn, plus the two subproblems, each costing T (n/2). The following figure (Figure 2.5c in CLRS) shows that for each of the size-n/2 subproblems, we have a cost of cn/2, plus two subproblems, each costing T (n/4). The following figure (Figure: 2.5d in CLRS) tells to continue expanding until the problem sizes get down to 1. In the above recursion tree, each level has cost cn. The top level has cost cn. The next level down has 2 subproblems, each contributing cost cn/2. The next level has 4 subproblems, each contributing cost cn/4. Each time we go down one level, the number of subproblems doubles but the cost per subproblem halves. Therefore, cost per level stays the same. The height of this recursion tree is lg n and there are lg n + 1 levels. Mathematical Induction We use induction on the size of a given subproblem n. Base case: n = 1 Implies that there is 1 level, and lg 1 + 1 = 0 + 1 = 1. Inductive Step Our inductive hypothesis is that a tree for a problem size of 2i has lg 2i + 1 = i +1 levels. Because we assume that the problem size is a power of 2, the next problem size up after 2i is 2i + 1. A tree for a problem size of 2i + 1 has one more level than the size-2i tree implying i + 2 levels. Since lg 2i + 1 = i + 2, we are done with the inductive argument. Total cost is sum of costs at each level of the tree. Since we have lg n +1 levels, each costing cn, the total cost is cn lg n + cn. Ignore low-order term of cn and constant coefÞcient c, and we have, Θ(n lg n) which is the desired result. Question3: Write the pseadocode of Insertion sort algorithm and show its implementation using Java. [20 Marks] Pseudocode INSERTION_SORT (A) 1. 2. 3. 4. 5. 6. 7. 8. FOR j ← 2 TO length[A] DO key ← A[j] {Put A[j] into the sorted sequence A[1 . . j − 1]} i←j−1 WHILE i > 0 and A[i] > key DO A[i +1] ← A[i] i←i−1 A[i + 1] ← key Implementation void insertionSort(int numbers[], int array_size) { int i, j, index; for (i = 1; i < array_size; i++) { index = numbers[i]; j = i; while ((j > 0) && (numbers[j − 1] > index)) { numbers[j] = numbers[j − 1]; j = j − 1; } numbers[j] = index; } } Question4: a) Briefly discuss about Dynamic algorithms and their characteristics [5 Marks] -Dynamic programming is a stage-wise search method suitable for optimization problems whose solutions may be viewed as the result of a sequence of decisions. -The dynamic programming technique is related to divide-and-conquer, in the sense that it breaks problem down into smaller problems and it solves recursively. Its caracteristics include: 1. Substructure , 2. Table-Structure 3. Bottom-up Computation [1 Marks] [1 Marks] [1 Marks] b) Analyze the following algorithm to determine its running time. [12 Marks] for (i = 0; i < N; i++) { for (j = i+1; j < N; j++) { sequence of statements } } Now we can't just multiply the number of iterations of the outer loop times the number of iterations of the inner loop, because the inner loop has a different number of iterations each time. So let's think about how many iterations that inner loop has. That information is given in the following table: Value of i Number of iterations of inner loop 0 N 1 N-1 2 N-2 ... ... N-2 2 N-1 1 So we can see that the total number of times the sequence of statements executes is: N + N-1 + N-2 + ... + 3 + 2 + 1. We've seen that formula before: the total is O(N2).