Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Algorithm Analysis • A data structure is a systematic way of organizing and accessing data, and an algorithm is a step-bystep procedure for performing some task in a finite amount of time. ACS-2947 Lecture 5 • These concepts are central to computing, but to be able to classify some data structures and algorithms as "good," we must have precise ways of analyzing them. Algorithm Analysis Running Time How “good” is a data structure or algorithm? • Most algorithms transform input objects into output objects • The running time of an algorithm typically grows with the input size Average case time is often difficult to determine Algorithm Analysis • The primary analysis tool that we will use will focus on involves characterizing the running times of algorithms and data structure operations, with space usage also being of interest à characterizing an algorithm’s running time as a function of the input size • But what is the proper way of measuring it? • • We focus on the worst case running time • Easier to analyze Focusing on the Worst-Case Input • Worst-case analysis is much easier than averagecase analysis • Requires only the ability to identify the worst-case input, which is often simple • • • Experimental Studies Typically leads to better algorithms • Write a program implementing the algorithm Run the program with inputs of varying size and composition • Use a method like • to get an accurate measure of the actual running time Plot the results Making the standard of success for an algorithm to perform well in the worst case necessarily requires that it will do well on every input System.currentTimeMillis() 1 Limitations of Experiments Theoretical Analysis • In order to compare two algorithms, the same hardware and software environments must be used • Allows us to evaluate the speed of an algorithm independent of the hardware/software environment • Results may not be indicative of the running time on other inputs not included in the experiment. • Uses a high-level description of the algorithm instead of an implementation • It is necessary to implement the algorithm, which may be difficult • Characterizes running time as a function of the input size, n. • Takes into account all possible inputs The 7 functions used in this class Primitive Operations • Constant f(n)=c • Logarithm f(n)=logbn • Linear f(n)=n • N-log-n f(n)=nlogn • • Quadratic f(n)=n2 • • Cubic f(n)=n3 • Exponential f(n)=bn • To analyze the running time of an algorithm without performing experiments, we perform an analysis of primitive operations: • • • • • Example: find the max element in array Assigning a value to a variable Following an object reference Performing an arithmetic operation Comparing two numbers Accessing a single element of an array by index Calling a method Returning from a method Asymptotic Behaviour int x = A[0]; for (int i = 1; i < n; i++){ if (A[i] >= x) { x = A[i]; } 2 2 2(n-1) 2(n-1) 2(n-1) f(n)=6n-2 • In complexity analysis we only care about what happens to this function as the program input n grows large } f(n)=4+3(2(n–1)) f(n)=6n-2 2 Asymptotic Behaviour f(n)=6n-2 Asymptotic Analysis • In algorithm analysis, we focus on the growth rate of the running time as a function of the input size n, taking a “big-picture” approach. • For example, it is often enough just to know that the running time of an algorithm grows proportionally ton. • • Set-up time may vary between languages and machines Programming languages may compile differently Efficiency and Big O Notation Efficiency and Big O Notation • In an asymptotic analysis, we care more about the order of magnitude of a function rather than the actual value of a function itself. • In addition, we don't know exactly how long each operation might actually take on a certain computer. • In terms of the abstract time of an algorithm, this should make some intuitive sense. • • We call it "abstract time" because we use "abstract" operations such as "number of math operations" in f(n), or "number of comparisons" in f(n). Intuitively, the order of magnitude appeals to our sense that n2 is a faster-growing function than a linear function like n. Big O Notation • Big O Notation Formal definition: • Let f (n) and g(n) be functions mapping positive integers to positive real numbers. • We say that f (n) is O (g(n)) if there is a real constant c > 0 and an integer constant n0 ≥ 1 such that# f (n) ≤ c · g (n) , for n ≥ n0 3 Big O Notation • Big O A way of analyzing how fast a program’s runtime grows asymptotically • E.g. as the size of your inputs increase towards infinity à How does the runtime of your program grow? In our example, 6n-2isO(n) • Runs in linear time WRT number of elements, n • The time required to traverse through the array is proportional to the number of elements Big 0 Operations for 10 “things” Operations for 100 “things” O(1) O(logn) O(n) O(nlogn) O(n2) O(n3) O(bn) 1 3 10 30 100 1000 1024 1 7 300 700 10000 1000000 2100 Big O Rules • Big O Rules • • • If f(n)is a polynomial of degree d, then f(n)is O(nd), i.e., 1. Drop lower-order terms 2. Drop constant factors Use the smallest possible class of functions • • Big O Notation Say “2n is O(n)” instead of “2n is O(2n)” • Use the simplest expression of the class • To describe the order of magnitude of a function, we use Big O notation. • If we had an algorithm that did 8n-2operations, the Big O notation would be O(n) • If we had an algorithm that did 5n4+3n3–2n2+2 operations, its Big O notation would be O(n4). In our example, our algorithm did 6n-2operations, so the Big O notation would be O(n). Say “3n+5is O(n)” instead of “3n+5is O(3n)” Examples Comparing Growth Rates f(n)=2n+12 O(n) f(n)=25 O(1) f(n)=n2+4n+6 O(n2) • Ideally, we would like data structure operations to run in times proportional to the constant or logarithm function, and we would like our algorithms to run in linear or n-log-n time. • Algorithms with quadratic or cubic running times are less practical, but algorithms with exponential running times are infeasible for all but the smallest sized inputs. f(n)=n3+100n+99 O(n3) f(n)=5n2+3nlogn+2n+5 O(n2) f(n)=3logn+2 O(logn) 4 Growth Rates O(1)– Constant time public boolean isFirstZero(int arr[]){ if (array[0] == 0) return true; else return false; } • O(logn)– Logarithmic Time • Any algorithm which cuts the problem in half each time is O(logn). • O(logn)operations run in logarithmic time - the operation will take longer as the input size increases, but once the input gets fairly large it won’t change enough to worry about. • If you double n, you have to spend an extra amount of time t to complete the task. If n doubles again, t won’t double, but will increase by a constant amount. • An example of an O(logn)operation is a binary search Binary Search public static boolean binarySearch (int[] data, int target, int low, int high) { if (low > high) return false; else{ int mid = (low + high) / 2; // Determine the middle element // Split the result in half and search again recursively until found if (target == data[mid]) return true; else if (target < data[mid]) return binarySearch(data, target, low, mid-1); else return binarySearch(data, target, mid+1, high); } } Will take the same time to execute, no matter how big arr is Binary Search O(n)– Linear time • O(n)means that for every element, you are doing a constant number of operations, such as comparing each element to a known value. • run in linear time - the larger the input, the longer it takes, in an even tradeoff. • • E.g. Every time you double n, the operation will take twice as long. An example of an O(n)operation is finding the largest value in an array 5 O(n)– Linear time • The linear function also represents the best running time we can hope to achieve for any algorithm that processes each of n objects that are not already in the computer’s memory, because reading in the n objects already requires n operations. int x = A[0]; for (int i = 1; i < n; i++){ if (A[i] >= x) { x = A[i]; } } O(nlogn)–Loglinear Time • O(nlogn)means that you’re performing an O(logn) operation for each item in your input. • Most (efficient) sort algorithms are an example of this. • O(nlogn)operations run in loglinear time increasing the input size hurts, but may still be manageable. Every time you double n, you spend twice as much time plus a little more. • Examples of O(nlogn)operations are quicksort (in the average and best case) and merge sort. O(n2)– Quadratic Time • O(n2)means that for every element, you do something with every other element, such as comparing them. • O(n2)operations run in quadratic time - the operation is only really practical up to a certain input size. Every time n doubles, the operation takes four times as long. • Examples of O(n2)operations are quicksort (in the worst case) and bubble sort. • The following function is an example of an O(n2) operation, where every element in an array is compared to every other element : public boolean containsDup (int array[]){ int i = 0; int j; while (i < array.length){ j = i + 1; while (j < array.length){ if (array[i] == array[j]) return true; j++; } i++; } return false; } Asymptotic Analysis • Suppose two algorithms solving the same problem are available: an algorithm A, which has a running time of O(n), and an algorithm B, which has a running time of O(n2). • Which algorithm is better? • We know that n is O(n2), which implies that algorithm A is asymptotically better than algorithm B, although for a small value of n, B may have a lower running time than A. 6