Download DSAD-_Notes_2nd_Feb_2013.ppt

Data Structure and
Algorithms Design
BITS Pilani
Pilani Campus
Dr. Maheswari Karthikeyan
Lecture1
Course Outline
o Data structures: conceptual and concrete ways to
organize data for efficient storage and efficient
manipulation
o Employment of this data structures in the design of
efficient algorithms
BITS Pilani, Pilani Campus
Methodology
o Algorithm analysis techniques
o Algorithm design techniques
o Implementation/analysis of data structures:
• Lists, stacks, and queues
• Trees
• Hashing
• Priority queues (heaps)
• Sorting
• Graphs
BITS Pilani, Pilani Campus
Why to Study Data Structures?
Any organization has a collection of records that can be
searched, processed in any order, or modified.
o Data structures organize data:
• Good choice: more efficient programs
• Bad choice: poor program performance
– The choice can make a difference between the program
running in a few seconds or many day
o Characteristics of a problem’s solution
• efficient: if it solves problem within resource constraints
– time
– space
• Cost: amount of resources a solution consumes
BITS Pilani, Pilani Campus
Costs and Benefits
A data structure requires a certain amount of:
• space for each data item it stores
• time to perform a single basic operation
• programming effort.
BITS Pilani, Pilani Campus
Selecting a Data Structure
Select a data structure as follows:
1. Analyze the problem to determine the resource
constraints a solution must meet.
2. Determine the basic operations that must be
supported. Quantify the resource constraints
for each operation.
3. Select the data structure that best meets these
requirements.
BITS Pilani, Pilani Campus
Abstract Data Types (ADT)
o A logical view of the data objects together with specifications
of the operations required to create and manipulate them.
• Describe an algorithm – pseudo-code
• Describe a data structure – ADT
o A data structure is the physical implementation of an ADT
• Each ADT operation is implemented by one or more
subroutines
• Data structures are used to organize data in main
memory
BITS Pilani, Pilani Campus
Data Type
ADT:
Type
Operations
Data Items:
Logical Form
Data Structure:
Storage Space
Subroutines
Data Items:
Physical Form
BITS Pilani, Pilani Campus
Problems, Algorithms and Programs
Programmers deal with:
– problems,
– algorithms and
– computer programs.
BITS Pilani, Pilani Campus
Problems, Algorithms and Programs
Problem: a task to be performed.
– Best thought of as inputs and matching outputs.
– Problem definition should include constraints on the resources that may be consumed
by any acceptable solution.
Problems  mathematical functions
– A function is a matching between inputs (the domain) and
outputs (the range).
– An input to a function may be single number, or a
collection of information.
– The values making up an input are called the parameters
of the function.
– A particular input must always result in the same output
every time the function is computed.
BITS Pilani, Pilani Campus
Problems, Algorithms and Programs
Algorithm: a method or a process followed to solve
a problem.
– A recipe: The algorithm gives us a “recipe” for solving
the problem by performing a series of steps, where
each step is completely understood and can be
implemented.
An algorithm takes the input to a problem
(function) and transforms it to the output.
– A mapping of input to output.
A problem can be solved by many algorithms.
BITS Pilani, Pilani Campus
Problems, Algorithms and Programs
For example, the problem of sorting can be solved by the
following algorithms:
•
•
•
•
•
•
Insertion sort
Bubble sort
Selection sort
Shellsort
Mergesort
Others
BITS Pilani, Pilani Campus
Problems, Algorithms and Programs
An algorithm possesses the following properties:
– It must be correct.
– It must be composed of a series of concrete steps.
– There can be no ambiguity as to which step will be performed
next.
– It must be composed of a finite number of steps.
– It must terminate.
A computer program is an instance, or concrete
representation, for an algorithm in some
programming language.
BITS Pilani, Pilani Campus
Algorithm Design Techniques
The design of algorithms is also an important
focus.
Types of algorithms:
• Greedy algorithms
• Divide and Conquer
• Dynamic programming
• Randomized algorithms
• Backtracking
BITS Pilani, Pilani Campus
Algorithm Analysis
Predict the amount of resources required:

memory: how much space is needed?

computational time: how fast the algorithm runs?
FACT: running time grows with the size of the input
Input size (number of elements in the input)
–
Size of an array, polynomial degree, # of elements in a matrix, # of bits
in the binary representation of the input, vertices and edges in a graph
Def: Running time = the number of primitive operations (steps) executed before
termination
Running time is expressed as T(n) for some function T on input size n.
BITS Pilani, Pilani Campus
Algorithm Analysis
Two approaches to obtaining running time:
– Measuring under standard benchmark conditions.
– Estimating the algorithms performance
Estimation is based on:
– The “size” of the input
– The number of basic operations
The time to complete a basic operation does not depend on
the value of its operands.
BITS Pilani, Pilani Campus
Running Time (Example )
sum = 0;
What is the running time for this code?
BITS Pilani, Pilani Campus
Running Time (Example )
Number of executions
k
1
2
3
….
n
j
1
1,2
1,2,3
…
1,2,. n
#
runs
1
2
3
…
n
BITS Pilani, Pilani Campus
Running Time (Example)
n
# runs = 1 + 2 + 3 + 4 ..+ n = ∑ j
j=1
n
∑ j =
n(n+1)/2
= n2
j=1
T(n) = c1 + c2 (n+1) + c3(n2+ 1) + c4 (n2 ) = Order of n2
BITS Pilani, Pilani Campus
Running Time (Example)
What is the running time for the following codes?
a)
sum1 = 0;
for (k=1; k<=n; k*=2)
for (j=1; j<=n; j++)
sum1++;
b)
sum2 = 0;
for (k=1; k<=n; k*=2)
for (j=1; j<=k; j++)
sum2++;
c) sum3 = 0;
for (k=1; k<=n; k++)
for (j=1; j<=n; j++)
sum3++;
BITS Pilani, Pilani Campus
Running Time (Example a )
Number of executions
k
1
2
4
….
n
j
1,2,..n
1,2,..n
1,2..n
…
1,2,. n
#
runs
n
n
n
…
log n
N x log N
BITS Pilani, Pilani Campus
Running Time (Example a)
# runs = (1 + ..N) log n =
log n
log n
∑ n
j=1
∑ n = n log n
j=1
T(n) = Order of n log n.
BITS Pilani, Pilani Campus
Running Time (Example b )
Number of executions
k
1
2
4
….
n
j
1
1,2
1,2,3,4
…
1,2,. n
#
runs
1
2
4
…
log n
1 + 2 + 4 + 8 + 16 + …… n
BITS Pilani, Pilani Campus
Running Time (Example b)
# runs = 1+2+4+8+16 ..+ n
= 1 + 21 + 22 + 23 + 24 + …… + 2logn
log n
∑ 2i =
2n-1
j=1
T(n) = Order of n.
BITS Pilani, Pilani Campus
Growth Rate
The growth rate for an algorithm is the rate at which the
cost of the algorithm grows as the size of the input
grows.
–
–
–
–
Linear Growth T(n) = n
Quadratic Growth T(n) = n2
Exponential Growth T(n) = 2n
Logarithmic Growth T(n) = n log n
BITS Pilani, Pilani Campus
Types of Analysis
Not all inputs of a given size take the same time to
run.
•
•
•
Best case
Worst case
Average case
BITS Pilani, Pilani Campus
Types of Analysis
• The best case running time of an algorithm is the
function defined by the minimum number of steps taken
on any instance of size n.
• The worst case running time of an algorithm is the
function defined by the maximum number of steps taken
on any instance of size n.
• The average-case running time of an algorithm is the
function defined by an average number of steps taken on
any instance of size n.
BITS Pilani, Pilani Campus
Types of Analysis
Worst case
(e.g. numbers reversely ordered)
– Provides an upper bound on running time
– An absolute guarantee that the algorithm would not
run longer, no matter what the inputs are
Best case
(e.g., numbers already ordered)
– Input is the one for which the algorithm runs the
fastest
Average case
(general case)
– Provides a prediction about the running time
– Assumes that the input is random
BITS Pilani, Pilani Campus
Asymptotic Notations
A way to describe behavior of functions in the limit
–
How we indicate running times of algorithms
–
Describe the running time of an algorithm as n grows to 
O notation: asymptotic “less than”:
f(n) “≤” g(n)
 notation: asymptotic “greater than”:
f(n) “≥” g(n)
 notation: asymptotic “equality”:
f(n) “=” g(n)
BITS Pilani, Pilani Campus
Asymptotic Notations Examples
For each of the following pairs of functions, either f(n) is
O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine
which relationship is correct.
– f(n) = log n2; g(n) = log n + 5
f(n) =  (g(n))
– f(n) = n; g(n) = log n2
f(n) = (g(n))
– f(n) = log log n; g(n) = log n
f(n) = O(g(n))
– f(n) = n; g(n) = log2 n
f(n) = (g(n))
BITS Pilani, Pilani Campus
Asymptotic notations
O-notation
• Intuitively: O(g(n)) = the set of
functions with a smaller or same
order of growth as g(n)
BITS Pilani, Pilani Campus
Examples
3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2
3n + 3 = O(n) ; 3n + 3 <= 4n for all n >= 3
100n + 6 = O(n) ; 100n + 6 <= 101n for all n >= 6
BITS Pilani, Pilani Campus
Examples
3n + 2 = O(n) ; 3n + 2 <= 4n for all n >= 2
3n + 3 = O(n) ; 3n + 3 <= 4n for all n >= 3
100n + 6 = O(n) ; 100n + 6 <= 101n for all n >= 6
10 n 2 + 4n + 2
= O(n2)
10 n 2 + 4n + 2 < = 11 n2 for n >= 5
BITS Pilani, Pilani Campus
Asymptotic notations (cont.)
-notation
• Intuitively: (g(n)) = the set of
functions with a larger or same order
of growth as g(n)
BITS Pilani, Pilani Campus
Examples
3n + 2 = ?
3n + 3 = ?
100n + 6 = ?
3n + 2 >= 3n for all n >= 1
3n + 3 >= 3n for all n >= 1
100n + 6 >= 100n for all n >= 1
BITS Pilani, Pilani Campus
Asymptotic notations (cont.)
-notation
• Intuitively (g(n)) = the set of
functions with the same order of
growth as g(n)
BITS Pilani, Pilani Campus
Topics Covered
• Introduction to the course
• Algorithm Analysis
BITS Pilani, Pilani Campus