Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Data Structure and Algorithms Design BITS Pilani Pilani Campus Dr. Maheswari Karthikeyan Lecture1 Course Outline o Data structures: conceptual and concrete ways to organize data for efficient storage and efficient manipulation o Employment of this data structures in the design of efficient algorithms BITS Pilani, Pilani Campus Methodology o Algorithm analysis techniques o Algorithm design techniques o Implementation/analysis of data structures: • Lists, stacks, and queues • Trees • Hashing • Priority queues (heaps) • Sorting • Graphs BITS Pilani, Pilani Campus Why to Study Data Structures? Any organization has a collection of records that can be searched, processed in any order, or modified. o Data structures organize data: • Good choice: more efficient programs • Bad choice: poor program performance – The choice can make a difference between the program running in a few seconds or many day o Characteristics of a problem’s solution • efficient: if it solves problem within resource constraints – time – space • Cost: amount of resources a solution consumes BITS Pilani, Pilani Campus Costs and Benefits A data structure requires a certain amount of: • space for each data item it stores • time to perform a single basic operation • programming effort. BITS Pilani, Pilani Campus Selecting a Data Structure Select a data structure as follows: 1. Analyze the problem to determine the resource constraints a solution must meet. 2. Determine the basic operations that must be supported. Quantify the resource constraints for each operation. 3. Select the data structure that best meets these requirements. BITS Pilani, Pilani Campus Abstract Data Types (ADT) o A logical view of the data objects together with specifications of the operations required to create and manipulate them. • Describe an algorithm – pseudo-code • Describe a data structure – ADT o A data structure is the physical implementation of an ADT • Each ADT operation is implemented by one or more subroutines • Data structures are used to organize data in main memory BITS Pilani, Pilani Campus Data Type ADT: Type Operations Data Items: Logical Form Data Structure: Storage Space Subroutines Data Items: Physical Form BITS Pilani, Pilani Campus Problems, Algorithms and Programs Programmers deal with: – problems, – algorithms and – computer programs. BITS Pilani, Pilani Campus Problems, Algorithms and Programs Problem: a task to be performed. – Best thought of as inputs and matching outputs. – Problem definition should include constraints on the resources that may be consumed by any acceptable solution. Problems mathematical functions – A function is a matching between inputs (the domain) and outputs (the range). – An input to a function may be single number, or a collection of information. – The values making up an input are called the parameters of the function. – A particular input must always result in the same output every time the function is computed. BITS Pilani, Pilani Campus Problems, Algorithms and Programs Algorithm: a method or a process followed to solve a problem. – A recipe: The algorithm gives us a “recipe” for solving the problem by performing a series of steps, where each step is completely understood and can be implemented. An algorithm takes the input to a problem (function) and transforms it to the output. – A mapping of input to output. A problem can be solved by many algorithms. BITS Pilani, Pilani Campus Problems, Algorithms and Programs For example, the problem of sorting can be solved by the following algorithms: • • • • • • Insertion sort Bubble sort Selection sort Shellsort Mergesort Others BITS Pilani, Pilani Campus Problems, Algorithms and Programs An algorithm possesses the following properties: – It must be correct. – It must be composed of a series of concrete steps. – There can be no ambiguity as to which step will be performed next. – It must be composed of a finite number of steps. – It must terminate. A computer program is an instance, or concrete representation, for an algorithm in some programming language. BITS Pilani, Pilani Campus Algorithm Design Techniques The design of algorithms is also an important focus. Types of algorithms: • Greedy algorithms • Divide and Conquer • Dynamic programming • Randomized algorithms • Backtracking BITS Pilani, Pilani Campus Algorithm Analysis Predict the amount of resources required: memory: how much space is needed? computational time: how fast the algorithm runs? FACT: running time grows with the size of the input Input size (number of elements in the input) – Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary representation of the input, vertices and edges in a graph Def: Running time = the number of primitive operations (steps) executed before termination Running time is expressed as T(n) for some function T on input size n. BITS Pilani, Pilani Campus Algorithm Analysis Two approaches to obtaining running time: – Measuring under standard benchmark conditions. – Estimating the algorithms performance Estimation is based on: – The “size” of the input – The number of basic operations The time to complete a basic operation does not depend on the value of its operands. BITS Pilani, Pilani Campus Running Time (Example ) sum = 0; What is the running time for this code? BITS Pilani, Pilani Campus Running Time (Example ) Number of executions k 1 2 3 …. n j 1 1,2 1,2,3 … 1,2,. n # runs 1 2 3 … n BITS Pilani, Pilani Campus Running Time (Example) n # runs = 1 + 2 + 3 + 4 ..+ n = ∑ j j=1 n ∑ j = n(n+1)/2 = n2 j=1 T(n) = c1 + c2 (n+1) + c3(n2+ 1) + c4 (n2 ) = Order of n2 BITS Pilani, Pilani Campus Running Time (Example) What is the running time for the following codes? a) sum1 = 0; for (k=1; k<=n; k*=2) for (j=1; j<=n; j++) sum1++; b) sum2 = 0; for (k=1; k<=n; k*=2) for (j=1; j<=k; j++) sum2++; c) sum3 = 0; for (k=1; k<=n; k++) for (j=1; j<=n; j++) sum3++; BITS Pilani, Pilani Campus Running Time (Example a ) Number of executions k 1 2 4 …. n j 1,2,..n 1,2,..n 1,2..n … 1,2,. n # runs n n n … log n N x log N BITS Pilani, Pilani Campus Running Time (Example a) # runs = (1 + ..N) log n = log n log n ∑ n j=1 ∑ n = n log n j=1 T(n) = Order of n log n. BITS Pilani, Pilani Campus Running Time (Example b ) Number of executions k 1 2 4 …. n j 1 1,2 1,2,3,4 … 1,2,. n # runs 1 2 4 … log n 1 + 2 + 4 + 8 + 16 + …… n BITS Pilani, Pilani Campus Running Time (Example b) # runs = 1+2+4+8+16 ..+ n = 1 + 21 + 22 + 23 + 24 + …… + 2logn log n ∑ 2i = 2n-1 j=1 T(n) = Order of n. BITS Pilani, Pilani Campus Growth Rate The growth rate for an algorithm is the rate at which the cost of the algorithm grows as the size of the input grows. – – – – Linear Growth T(n) = n Quadratic Growth T(n) = n2 Exponential Growth T(n) = 2n Logarithmic Growth T(n) = n log n BITS Pilani, Pilani Campus Types of Analysis Not all inputs of a given size take the same time to run. • • • Best case Worst case Average case BITS Pilani, Pilani Campus Types of Analysis • The best case running time of an algorithm is the function defined by the minimum number of steps taken on any instance of size n. • The worst case running time of an algorithm is the function defined by the maximum number of steps taken on any instance of size n. • The average-case running time of an algorithm is the function defined by an average number of steps taken on any instance of size n. BITS Pilani, Pilani Campus Types of Analysis Worst case (e.g. numbers reversely ordered) – Provides an upper bound on running time – An absolute guarantee that the algorithm would not run longer, no matter what the inputs are Best case (e.g., numbers already ordered) – Input is the one for which the algorithm runs the fastest Average case (general case) – Provides a prediction about the running time – Assumes that the input is random BITS Pilani, Pilani Campus Asymptotic Notations A way to describe behavior of functions in the limit – How we indicate running times of algorithms – Describe the running time of an algorithm as n grows to O notation: asymptotic “less than”: f(n) “≤” g(n) notation: asymptotic “greater than”: f(n) “≥” g(n) notation: asymptotic “equality”: f(n) “=” g(n) BITS Pilani, Pilani Campus Asymptotic Notations Examples For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine which relationship is correct. – f(n) = log n2; g(n) = log n + 5 f(n) = (g(n)) – f(n) = n; g(n) = log n2 f(n) = (g(n)) – f(n) = log log n; g(n) = log n f(n) = O(g(n)) – f(n) = n; g(n) = log2 n f(n) = (g(n)) BITS Pilani, Pilani Campus Asymptotic notations O-notation • Intuitively: O(g(n)) = the set of functions with a smaller or same order of growth as g(n) BITS Pilani, Pilani Campus Examples 3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2 3n + 3 = O(n) ; 3n + 3 <= 4n for all n >= 3 100n + 6 = O(n) ; 100n + 6 <= 101n for all n >= 6 BITS Pilani, Pilani Campus Examples 3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2 3n + 3 = O(n) ; 3n + 3 <= 4n for all n >= 3 100n + 6 = O(n) ; 100n + 6 <= 101n for all n >= 6 10 n 2 + 4n + 2 = O(n2) 10 n 2 + 4n + 2 < = 11 n2 for n >= 5 BITS Pilani, Pilani Campus Asymptotic notations (cont.) -notation • Intuitively: (g(n)) = the set of functions with a larger or same order of growth as g(n) BITS Pilani, Pilani Campus Examples 3n + 2 = ? 3n + 3 = ? 100n + 6 = ? 3n + 2 >= 3n for all n >= 1 3n + 3 >= 3n for all n >= 1 100n + 6 >= 100n for all n >= 1 BITS Pilani, Pilani Campus Asymptotic notations (cont.) -notation • Intuitively (g(n)) = the set of functions with the same order of growth as g(n) BITS Pilani, Pilani Campus Topics Covered • Introduction to the course • Algorithm Analysis BITS Pilani, Pilani Campus