Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Sorting Algorithms Mohammad Asad Abbasi Lecture 8 1 EXCHANGE SORT (Basic Ideas) The exchange sort is almost similar as the bubble sort. The difference between these two sorts is the manner in which they compare the elements. The exchange sort compares the first element with each following element of the array, making any necessary swaps. 2 Pass # 1 index 1 3 4SORT 5 (Example) SHELL 0 1 2 84 69 76 i j 2 84 69 i 3 84 i 4 86 i 5 94 i 86 94 91 J = i+1 = 1 76 86 94 91 j 69 76 swap 69 86 94 91 j 76 84 84 i=0 J=3 94 91 j 76 i=0 J=2 swap 69 i=0 86 i=0 J=4 91 i=0 j J=5 for (i=0; i< (numLength -1); i+ { for(j = (i+1); j < numLen { if (num[i] < num[j]) { temp= num[i]; num[i] = num[j] num[j] = temp; } } } 3 EXCHANGE SORT (Example) 4 Exchange Sort Algorithm 1. Compare the first pair of numbers (positions 0 and 1) and reverse them if they are not in the correct order. 2. Repeat for the next pair (positions 0 and 2). 3. Continue the process until all pairs have been checked. 4. Repeat steps 1 through 3 for positions 0 through n - 1 to i (for i = 1, 2, 3, ...) until no pairs remain to be checked. 5. The list is now sorted. 5 Exchange Sort Function for Descending Order void ExchangeSort() { int i, j; int temp; // holding variable int numLength = num.length( ); for (i=0; i< (numLength -1); i++) // element to be compared { for(j = (i+1); j < numLength; j++) // rest of the elements { if (num[i] < num[j]) // descending order { temp= num[i]; // swap num[i] = num[j]; num[j] = temp; } } 6 Sorting by Exchange: Quick Sort Divide-and-Conquer Problem Divide P1 P2 P3 Pn Conquer Combine Solution 7 QUICK SORT (Basic Ideas) (Another divide-and-conquer algorithm) Pick an element, say P (the pivot) Re-arrange the elements into 3 sub-blocks, 1. Those less than or equal to (<=) P (the left-block S1) 2. P (the only element ¡n the middle-block) 3. Those greater than or equal to (>=) P (the right-block S2) Repeat the process recursively for the left- and right- sub-blocks. Return (qu¡cksort(S1), P,quicksort(S2)}. That is the results of quicksort(S1), followed by P, followed by the results of quicksort(S2) 8 QUICK SORT (Basic Ideas) The main idea is to find the “right” position for the pivot element P. After each “pass”, the pivot element, P, should be “in place”. Eventually, the elements are sorted since each pass puts at least one element (i.e., P) into its final position. 9 QUICK SORT (Basic Ideas) 10 QUICK SORT (Basic Ideas) 11 QUICK SORT (Example 1) 12 QUICK SORT (Example 2) 55 88 22 99 44 11 66 77 33 55 33 11 22 44 99 66 33 22 11 77 99 44 88 22 66 88 11 77 66 77 13 66 77 88 Example 3 14 Example 3 15 Example 4 16 ALGORITHM Let A be a linear array of n elements A (1), A (2), A (3)......A (n), low represents the lower bound pointer and up represents the upper bound pointer. Key represents the first element of the array, which is going to become the middle element of the sub-arrays. 1. Input n number of elements in an array A 2. Initialize low = 2, up = n , key = A[(low + up)/2] 3. Repeat through step 8 while (low < = up) 4. Repeat step 5 while(A [low] > key) 5. low = low + 1 6. Repeat step 7 while(A [up] < key) 7. up = up–1 8. If (low < = up) (a) Swap = A [low] (b) A [low] = A [up] (c) A [up] = swap (d) low=low+1 (e) up=up–1 9. If (1 < up) Quick sort (A, 1, up) 10. If (low < n) Quick sort (A, low, n) 11. Exit 17 QUICK SORT (Program) int partition(int arr[], int left, int right) { int i = left, j = right; int tmp; int pivot = arr[(left + right) / 2]; while (i <= j) { while (arr[i] < pivot) i++; while (arr[j] > pivot) j--; if (i <= j) { tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp; i++; j--; } }; return i; } 18 QUICK SORT (Program) void quickSort (int arr[], int left, int right) { int index = partition(arr, left, right); if (left < index - 1) quickSort(arr, left, index - 1); if (index < right) quickSort(arr, index, right); } 19 Running time analysis The advantage of this quick sort is that we can sort “in-place”, i.e., without the need for a temporary buffer depending on the size of the inputs.(merge sort) Partitioning Step: Time Complexity is θ(n). Recall that quick sort involves partitioning, and 2 recursive calls. Thus, giving the basic quick sort relation: T(n) = θ(n)+ T(i) + T(n-i-1) = cn+ T(i) + T(n-i-1) where I is the size of the first sub-block after partitioning. 22 We shall take T(0) = T(1) = 1 as the initial conditions. Running time analysis Worst-Case (Data is sorted already) When the pivot is the smallest (or largest) element at partitioning on a block of size n, the result yields one empty sub-block, one element (pivot) in the “correct” place and one sub-block of size (n-1) takes θ(n) times. Recurrence Equation: T(1) = 1 T(n) = T(n-1) + cn Solution: θ(n2) Worse than Mergesort!!! 23 Running time analysis Best case: The pivot is in the middle (median) (at each partition step), i.e. after each partitioning, on a block of size n, the result yields two sub-blocks of approximately equal size and the pivot element in the “middle” position takes n data comparisons. Recurrence Equation becomes T(1) = 1 T(n) = 2T(n/2) + cn Solution: θ(n logn) Comparable to Merge sort!! 24 So the trick is to select a good pivot Different ways to select a good pivot. First element Last element Median-of-three elements Pick three elements, and find the median x of these elements. Use that median as the pivot. Random element Randomly pick a element as a pivot. 25 RADIX SORT Radix sort or bucket sort is a method that can be used to sort a list of numbers by its base. If we want to sort list of English words, where radix or base is 26, then 26 buckets are used to sort the words. 26 RADIX SORT EXAMPLE 27 RADIX SORT EXAMPLE (1st Pass) 28 RADIX SORT EXAMPLE (2nd Pass) 29 RADIX SORT EXAMPLE (3rd Pass) 30 ALGORITHM Let A be a linear array of n elements A [1], A [2], A [3],...... A [n]. Digit is the total number of digits in the largest element in array A. 1. Input n number of elements in an array A. 2. Find the total number of Digits in the largest element in the array. 3. Initialize i = 1 and repeat the steps 4 and 5 until (i <= Digit). 4. Initialize the buckets j = 0 and repeat the steps (a) until ( j < n) (a) Compare ith position of each element of the array with bucket number and place it in the corresponding bucket. 5. Read the element(s) of the bucket from 0th bucket to 9th bucket and from first position to higher one to generate new array A. 6. Display the sorted array A. 7. Exit. 31 Radix Sort Program 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. // C++ implementation of Radix Sort #include<iostream> using namespace std; // A utility function to get maximum value in arr[] int getMax(int arr[], int n) { int mx = arr[0]; for (int i = 1; i < n; i++) if (arr[i] > mx) mx = arr[i]; return mx; } 32 Radix Sort Program 14. // A function to do counting sort of arr[] according to the digit represented by exp. 15. void countSort(int arr[], int n, int exp) 16. { 17. int output[n]; // output array 18. int i, count[10] = {0}; 19. 20. // Store count of occurrences in count[] 21. for (i = 0; i < n; i++) 22. count[ (arr[i]/exp)%10 ]++; 23. 24. // Change count[i] so that count[i] now contains actual position of 25. // this digit in output[] 26. for (i = 1; i < 10; i++) 33 27. count[i] += count[i - 1]; Radix Sort Program 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. } // Build the output array for (i = n - 1; i >= 0; i--) { output[count[ (arr[i]/exp)%10 ] - 1] = arr[i]; count[ (arr[i]/exp)%10 ]--; } // Copy the output array to arr[], so that arr[] now // contains sorted numbers according to current digit for (i = 0; i < n; i++) arr[i] = output[i]; 34 Radix Sort Program 40. // The main function to that sorts arr[] of size n using Radix Sort 41. void radixsort(int arr[], int n) 42. { 43. // Find the maximum number to know number of digits 44. int m = getMax(arr, n); 45. 46. // Do counting sort for every digit. Note that instead of passing digit 47. // number, exp is passed. exp is 10^i where i is current digit number 48. for (int exp = 1; m/exp > 0; exp *= 10) 49. countSort(arr, n, exp); 50. } 35 Advantages and Disadvantages Advantages Radix and bucket sorts are stable, preserving existing order of equal keys. They work in linear time, unlike most other sorts. In other words, they do not bog down when large numbers of items need to be sorted. Most sorts run in O(n log n) or O(n^2) time. The time to sort per item is constant, as no comparisons among items are made. With other sorts, the time to sort per time increases with the number of items. Radix sort is particularly efficient when you have large numbers of records to sort with short keys. Drawbacks Radix and bucket sorts do not work well when keys are very long, as the total sorting time is proportional to key length and to the number of items to sort. They are not “in-place”, using more working memory than a traditional sort. 36 Running time analysis of Radix sort How many passes? How much work per pass? Total time? Conclusion Not truly linear if K is large. In practice Radix Sort only good for large number of items, relatively small keys Hard on the cache, vs. MergeSort/QuickSort 37 Running time analysis of Radix sort Time requirement for the radix sorting method depends on the number of digits and the elements in the array. WORST CASE f (n) = O(n2) BEST CASE f (n) = O(n logn) AVERAGE CASE f (n) = O(n log n) 38 Cocktail/Shaker/Bidirectional Bubble Sort Shaker sort (cocktail sort, shake sort) is a stable sorting algorithm with quadratic asymptotic complexity. Shaker sort is a bidirectional version of bubble sort. 39 Cocktail/Shaker/Bidirectional Bubble Sort Each iteration of the algorithm is broken up into two stages: The first stage loops through the data set from bottom to top, just like the Bubble Sort. During the loop, adjacent items are compared. If at any point the value on the left is greater than the value on the right, the items are swapped. At the end of the first iteration, the largest number will reside at the end of the set. The second stage loops through the data set in the opposite direction – starting from the item just before the most recently sorted item, and moving back 40towards the start of the list. Again, adjacent items are swapped if required. Cocktail Sort Example 1 41 Cocktail Sort Example 2 Sorting the list 6, 7, 1, 3, 2, 4: Pass 1: 42 Cocktail Sort Example 2 Pass 2: 43 Cocktail Sort (Program) 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. static void CocktailSortBasic(int[] dataSet) { bool swapped = false; int start = 0; int end = dataSet.Length - 1; do { // make sure we reset the swapped flag on entering the loop, because it might be true from a previous iteration. swapped = false; // loop from bottom to top just like we do with the bubble sort for (int i = start; i < end; ++i) { if (dataSet[i] > dataSet[i + 1]) { Swap(dataSet, i, i + 1); swapped = true; } } // if nothing moved, then we're sorted. if (!swapped) { 44 break; } Cocktail Sort (Program) 24. 25. // otherwise, reset the swapped flag so that it can be used in the next stage swapped = false; 26. 27. //move the end point back by one, because we know that the item at the end is in its rightful spot --end; 28. 29. 30. 31. 32. 33. 34. 35. 36. // this time we loop from top to bottom, doing the same comparison as in the previous stage for (int i = end - 1; i >= start; --i) { if (dataSet[i] > dataSet[i + 1]) { Swap(dataSet, i, i + 1); swapped = true; } } 37. /* this time we increase the starting point, because the last stage would have moved the next smallest number to its rightful spot.*/ ++start; 45 } while (swapped); 38. 39. 40. } Difference with Bubble sort The difference between cocktail sort and bubble sort is that instead of repeatedly passing through the list from bottom to top, it passes alternately from bottom to top and then from top to bottom. The result is that it has a slightly better performance than bubble sort, because it sorts in both directions. (Bubble sort can only move items backwards one step per iteration.) Normally cocktail sort or shaker sort pass (one time in both directions) is counted as two bubble sort passes. In a typical implementation the cocktail sort is less than two times faster than a bubble sort implementation. Because you have to implement a loop in both directions that is changing 46 each pass it is slightly more difficult to implement. Sorting by Exchange: Shell Sort Sorting methods based on comparison: Comparisons and hence movements of data take place between adjacent entries only This leads to a number of redundant comparisons and data movements A mechanism should be followed with which the comparisons can take in long leaps instead of short Donald L. Shell (1959) Use increments: ht , ht 1 , ht 2 ,..., h1 47 Shell Sort Shell sort, also known as the diminishing increment sort, is one of the oldest sorting algorithms. It improves on insertion sort. Starts by comparing elements far apart, then elements less far apart, and finally comparing adjacent elements (effectively an insertion sort). By this stage the elements are sufficiently sorted that the running time of the final stage is much closer to O(N) than O(N2). 48 Shell Sort (Steps) Let A be a linear array of n numbers A [1], A [2], A [3], ...... A [n]. Step 1: The array is divided into k sub-arrays consisting of every kth element. Say k= 5, then five sub-array, each containing one fifth of the elements of the original array. Sub array 1 → A[0] A[5] A[10] Sub array 2 → A[1] A[6] A[11] Sub array 3 → A[2] A[7] A[12] Sub array 4 → A[3] A[8] A[13] Sub array 5 → A[4] A[9] A[14] Note : The ith element of the jth sub49array is located as A [(i–1) × k+j–1] Shell Sort (Steps) Step 2: After the first k sub array are sorted (usually by insertion sort) , a new smaller value of k is chosen and the array is again partitioned into a new set of sub arrays. 50 Shell Sort (Steps) Step 3: And the process is repeated with an even smaller value of k, so that A [1], A [2], A [3], ....... A [n] is sorted. 51 Shell Sort: Illustration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 21 72 43 94 85 16 67 38 59 91 32 73 24 45 56 60 This is the list after pass 1 1 2 3 4 5 6 7 Pass 1 with h = 7 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 21 59 43 32 73 16 45 38 60 91 94 85 24 67 56 72 1 2 3 4 5 Pass 2 with h = 5 52 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 21 59 43 32 73 16 45 38 60 91 94 85 24 67 56 72 1 2 3 4 5 Pass 2 with h = 5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 16 45 24 32 56 21 59 38 60 73 72 85 43 67 91 94 1 2 3 53 This is the list after pass 2 Shell Sort: Illustration 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 16 45 24 32 56 21 59 38 60 73 72 85 43 67 91 94 1 2 3 Pass 3 with h = 3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 16 38 21 32 45 24 43 56 60 59 67 85 73 72 91 94 1 Pass 4 with h = 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 16 21 24 32 38 43 45 56 59 60 67 72 73 85 91 94 Output list54 This is the list after pass 4 1 This is the list after pass 3 Shell Sort: Illustration Shell Sort: Illustration 55 ALGORITHM Let A be a linear array of n elements, A [1], A [2], A [3], ...... A[n] and Incr be an array of sequence of span to be incremented in each pass. X is the number of elements in the array Incr. Span is to store the span of the array from the array Incr. 1. Input n numbers of an array A 2. Initialise i = 0 and repeat through step 6 if (i < x) 3. Span = Incr[i] 4. Initialise j = span and repeat through step 6 if ( j < n) (a) Temp = A [ j ] 5. Initialise k = j-span and repeat through step 5 if (k > = 0) and (temp < A [k ]) (a) A [k + span] = A [k] 6. A [k + span] = temp 56 7. Exit Pass # 1 i= num / 2 ( i= 7/2 = 3) index 0 1 42 3 4 SORT 5 6 SHELL (Example) 1 2 33 23 k 74 44 67 49 K+i j=i J=3 K=j-i K=3-3=0 K+i = 3 2 42 33 23 74 k 3 42 33 44 42 33 33 49 67 49 K+I 23 74 44 k 4 42 67 23 23 K+i 74 44 k swap 49 44 67 49 K+i 67 74 for(i=num/2; i>0; i=i/2) { K=4-3=1 for(j=i; j<num; j++) { for(k=j-i; k>=0; k=k-i) J=5 { if(arr[k+i]>=arr[k]) K=5-3=2 break; else { J=6 tmp=arr[k]; K=6-3=3 arr[k]=arr[k+i]; arr[k+i]=tmp; } 57 } J=4 Pass # 2 i= i / 2 ( i= 3 / 2 = 1) index 1 0 1 2 42 33 23 k K+i 3 4 5 6 SHELL SORT (Example) 49 44 67 74 j=i J=1 K=j-i K=1-1=0 K+i = 1 2 3 4 5 33 42 23 k K+i 33 23 42 k K+i K=k-i 23 33 23 33 49 44 67 74 J=2 K=2-1= 1 49 44 67 74 42 49 44 67 74 k K+i 42 49 44 K K+i J=3 K=3-1=2 67 74 J=4 K=4-1=3 for(i=num/2; i>0; i=i/2) { for(j=i; j<num; j++) { for(k=j-i; k>=0; k=k-i) { if(arr[k+i]>=arr[k]) break; else { tmp=arr[k]; arr[k]=arr[k+i]; arr[k+i]=tmp; } 58 } Pass # 2 i= i / 2 ( i= 3 / 2 = 1) index 5 6 7 0 1 2 23 33 42 23 23 33 33 42 42 SHELL SORT (Example) 3 4 5 6 49 44 67 74 K K+i 44 49 67 K K+i 49 67 74 J=6 K K+i K=6-1=5 44 SORTED J=4 K=4-1=3 74 J=5 K=5-1=4 for(i=num/2; i>0; i=i/2) { for(j=i; j<num; j++) { for(k=j-i; k>=0; k=k-i) { if(arr[k+i]>=arr[k]) break; else { tmp=arr[k]; arr[k]=arr[k+i]; arr[k+i]=tmp; } 59 } #include<stdio.h> #include<conio.h> int main() { int arr[30]; int i,j,k,tmp,num; printf("Enter total no. of elements : "); scanf("%d", &num); for(k=0; k<num; k++) { printf("\nEnter %d number : ",k+1); scanf("%d",&arr[k]); } for(i=num/2; i>0; i=i/2) { for(j=i; j<num; j++) { for(k=j-i; k>=0; k=k-i) { if(arr[k+i]>=arr[k]) break; else { tmp=arr[k]; arr[k]=arr[k+i]; arr[k+i]=tmp; } } } SHELL SORT PROGRAM 60 SHELL SORT PROGRAM printf("For vlue of increment %d = \n\n", i); for(j=0; j<num; j++) { printf("%d\t",arr[j]); } printf("\n\n"); } printf("\t**** Shell Sorting ****\n"); for(k=0; k<num; k++) printf("%d\t",arr[k]); getch(); return 0; } 61 The Complexity If an appropriate sequence of increments is classified, then the order of the shell sort is: f (n) = O(n (log n)2) 62 Shell Sort In the Sample, arbitrarily chose sub array sizes of 1, 3 and 5. In reality that increment isn’t ideal, but then it isn’t clear what increment is. From empirical studies, a suggested increment follows this pattern; h1 = 1 hi+1 = 3hi+1 until ht+2 ≥n What sequence will this give? Issues in Shell Sort Algorithm to be used to sort subsequences in shell sort Straight insertion sort Shell sort is better than the insertion sort Lower number of passes than n number of passes in insertion sort Deciding the values of increments Several choices have been made 64 Summary of Sorting Algorithms Algorithm selection-sort insertion-sort heap-sort merge-sort Time Notes O(n2) slow in-place for small data sets (< 1K) O(n2) slow in-place for small data sets (< 1K) O(n log n) fast in-place for large data sets (1K — 1M) O(n log n) fast sequential data access for huge data sets (> 1M) 65 MERGE SORT Merge sort is based on the divide-and-conquer paradigm. Its worst-case running time has a lower order of growth than insertion sort. 66 Divide and Conquer Recursive in structure Divide the problem into sub-problems that are similar to the original but smaller in size Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner. Combine the solutions to create a solution to the original problem Divide-and-conquer Technique a problem of size n subproblem 1 of size n/2 a solution to subproblem 1 subproblem 2 of size n/2 a solution to subproblem 2 a solution to 68 the original problem An Example: Merge Sort Sorting Problem: Sort a sequence of n elements into non-decreasing order. Divide: Divide the n-element sequence to be sorted into two subsequences of n/2 elements each Conquer: Sort the two subsequences recursively using merge sort. Combine: Merge the two sorted subsequences to produce the sorted answer. Merge Sort – Example Original Sequence 18 26 32 6 Sorted Sequence 43 15 9 1 1 6 9 26 15 18 26 1 9 18 26 32 6 43 15 9 1 6 18 18 26 32 6 43 15 9 1 18 26 6 32 15 18 26 32 6 43 15 9 1 26 32 6 43 18 26 32 6 43 15 9 1 18 32 32 43 15 43 43 43 1 9 15 9 1 MERGE SORT (STEPS) To sort A[p .. r]: 1. Divide Step If a given array A has zero or one element, simply return; it is already sorted. Otherwise, split A[p .. r] into two subarrays A[p .. q] and A[q + 1 .. r], each containing about half of the elements of A[p .. r]. That is, q is the halfway point of A[p .. r]. 2. Conquer Step Conquer by recursively sorting the two subarrays A[p .. q] and A[q + 1 .. r]. 3. Combine Step Combine the elements back in A[p .. r] by merging the two sorted subarrays A[p .. q] and A[q + 1 .. r] into a sorted sequence. To accomplish this step, we will define a procedure MERGE (A, p, q, r). Note that the recursion bottoms out when the subarray has just one element, so that it is trivially sorted. 72 MERGE SORT Conceptually, a merge sort works as follows: 1. Divide the unsorted list into n sub lists, each containing 1 element (a list of 1 element is considered sorted). 2. Repeatedly merge sub lists to produce new sorted sub lists until there is only 1 sub list remaining. This will be the sorted list. 73 Main idea: •Dividing is trivial •Merging is non-trivial Merge Sort Input O(lg n) steps (divide) How much work at every step? O(n) sub-problems O(lg n) steps (conquer) How much work at every step? 74 Merge sort Example 8 2 9 4 5 3 1 6 Divide Divide 8 2 Divide 1 element 8 2 Merge 8 2 9 4 9 4 9 2 8 Merge 2 4 8 9 Merge 5 3 1 6 4 4 1 6 5 3 9 5 3 1 3 5 1 3 5 6 1 2 3 4 5 6 8 9 6 1 6 MERGE SORT 76 MERGE SORT In a simple pseudocode form, the algorithm could look something like this: function merge_sort(m) 1. 2. 3. 4. 5. if length(m) ≤ 1 return m var list left, right, result var integer middle = length(m) / 2 for each x in m up to middle 1. add x to left 6. for each x in m after middle 1. add x to right 7. 8. 9. 10. left = merge_sort(left) right = merge_sort(right) result = merge(left, right) return result 77 MERGE SORT function merge (left, right) 1. var list result 2. while length(left) > 0 or length(right) > 0 1. if length(left) > 0 and length(right) > 1. if first(left) ≤ first(right) append first(left) to result left = rest(left) 2. 2. 1. 2. 3. 4. 5. Else append first(right) to result right = rest(right) else if length(left) > 0 append first(left) to result left = rest(left) else if length(right) > 0 append first(right) to result right = rest(right) end while return result 78