Download FinalReview - Computer Science

UMass Lowell Computer Science 91.404
Analysis of Algorithms
Prof. Karen Daniels
Spring, 2001
Final Review
Mon. 5/14-Wed. 5/16
Overview of Next 2 Lectures
Review of some key course material
 Final Exam:

 Course
Grade
 Logistics, Coverage, Format
 Handout for basis of 40% of test

Course Evaluations
Review of Key Course Material
What’s It All About?

Algorithm:
 steps
for the computer to follow to
solve a problem

Problem Solving Goals:
 recognize
structure of some common
problems
 understand important characteristics of
algorithms to solve common problems
 select appropriate algorithm & data
structures to solve a problem
 tailor existing algorithms
 create new algorithms
Some Algorithm Application Areas
Robotics
Bioinformatics
Geographic
Information Systems
Design
Analyze
Telecommunications
Apply
Computer
Graphics
Medical Imaging
Astrophysics
Tools of the Trade

Algorithm Design Patterns such as:
 binary search
 divide-and-conquer

Data Structures such as:
 trees,

linked lists, hash tables, graphs
Theoretical Computer Science
principles such as:
 NP-completeness,
Summations
Growth of Functions
hardness
MATH
Probability
Proofs
Sets
Recurrences
Discrete Math Review
Chapters 1-6
Growth of Functions, Summations,
Recurrences, Sets, Counting, Probability
Topics

Discrete Math Review : Chapters 1-6






Solving Summations & Recurrences
Sets, Basic Tree & Graph concepts
Counting: Permutations/Combinations
Probability: Basics, including Expectation of a Random Variable
Proof Techniques: Induction
Basic Algorithm Analysis Techniques: Chapters 1-6





Asymptotic Growth of Functions
Types of Input: Best/Average/Worst
Bounds on Algorithm vs. Bounds on Problem
Algorithmic Paradigms/Design Patterns: Divide-and-Conquer
Analyze pseudocode running time to form summations &/or
recurrences
What are we measuring?

Some Analysis Criteria:

Scope



“Dimension”


Upper? Lower? Both?
Type of Input


Time Complexity? Space Complexity?
Type of Bound


The problem itself?
A particular algorithm that solves the problem?
Best-Case? Average-Case? Worst-Case?
Type of Implementation

Choice of Data Structure
Function Order of Growth
1
lglg(n)
lg(n)
n
n lg(n) n lg2(n)
n2
n5
know how to order functions asymptotically
(behavior as n becomes large)
O( ) upper bound
W( ) lower bound
Q( ) upper & lower bound
know how to use asymptotic complexity notation
to describe time or space complexity
2n
Types of Algorithmic Input
Best-Case Input: of all possible algorithm inputs of
size n, it generates the “best” result
for Time Complexity: “best” is smallest running time
Best-Case Input Produces Best-Case Running Time
provides a lower bound on the algorithm’s asymptotic running time
(subject to any implementation assumptions)
for Space Complexity: “best” is smallest storage
Average-Case Input
Worst-Case Input
these are defined similarly
Best-Case Time <= Average-Case Time <= Worst-Case Time
Bounding Algorithmic Time
(using cases)
Using “case” we can discuss lower and/or upper bounds on:
best-case running time or average-case running time or worst-case running time
1
lglg(n)
T(n) = W(1)
lg(n)
n
n lg(n) n lg2(n)
n2
very loose bounds are not very useful!
n5
2n
T(n) = O(2n)
Worst-Case time of T(n) = O(2n) tells us that worst-case inputs cause the algorithm to
take at most exponential time (i.e. exponential time is sufficient).
But, can the algorithm every really take exponential time? (i.e. is exponential time
necessary?)
If, for arbitrary n, we find a worst-case input that forces the algorithm to use exponential
time, then this tightens the lower bound on the worst-case running time. If we can force
the lower and upper bounds on the worst-case time to match, then we can say that, for
the worst-case running time, T(n) = Q(2n ) (i.e. we’ve found the minimum upper bound,
so the bound is tight.)
Bounding Algorithmic Time
(tightening bounds)
for example...
1
lglg(n)
TB(n) = W(1)
1st attempt
lg(n)
n
n lg(n) n lg2(n)
TB (n) = O(n)
1st attempt
TB(n) = Q(n)
2nd attempt
n2
n5
2n
n
TW (n) = W(n2) TW (n) = O(2 )
1st attempt 1st attempt
TW(n) = Q(n2)
2nd attempt
Algorithm Bounds
Here we denote best-case time by TB(n); worst-case time by TW(n)
Approach

Explore the problem to gain intuition:






Establish worst-case upper bound on the problem using
an algorithm



1
Describe it: What are the assumptions? (model of computation, etc...)
Has it already been solved?
Have similar problems been solved? (more on this later)
What does best-case input look like?
What does worst-case input look like?
Design a (simple) algorithm and find an upper bound on its worst-case
asymptotic running time; this tells us problem can be solved in a certain
amount of time. Algorithms taking more than this amount of time may exist,
but won’t help us.
Establish worst-case lower bound on the problem
Tighten each bound to form a worst-case “sandwich”
n
n2
n3
n4
n5
increasing worst-case asymptotic running time as a function of n
2n
Know the Difference!
Strong Bound: This
worst-case lower bound
on the problem holds for
every algorithm that
solves the problem and
abides by our problem’s
assumptions.
1
No algorithm for the problem
exists that can solve it for
worst-case inputs in less
than linear time .
Weak Bound: This worst-case
upper bound on the problem
comes from just considering
one algorithm. Other, less
efficient algorithms that solve
this problem might exist, but
we don’t care about them!
n5
n
worst-case bounds
on problem
An inefficient algorithm for
the problem might exist
that takes this much time,
but would not help us.
Both the upper and lower bounds
are probably loose (i.e. probably
can be tightened later on).
2n
Master Theorem
n
T (n)  aT ( )  f (n)
b
Master Theorem :
LetT (n)  aT ( n )  f (n) with a > 1 and b > 1 .
b
Then :
Use ratio test to
Case 1: If f(n) = O ( n (log b a) - e ) for some e > odistinguish
between cases:
then T ( n ) = Q ( n log b a )
f(n)/ n log b a
Case 2: If f (n) = Q (n log b a )
then T ( n ) = Q (n log b a * log n )
Look for
“polynomially
larger” dominance.
Case 3: If f ( n ) = W (n (log b a) + e ) for some e > o and if
a f( n/b) < c f ( n ) for some c < 1 , n > N0
then T ( n ) = Q ( f ( n ) )
Sorting
Chapters 7-10
Heapsort, Quicksort, LinearTime-Sorting,
Medians
Topics

Sorting: Chapters 7-10

Sorting Algorithms:




[Insertion & MergeSort from Chapters 1-6)], Heapsort, Quicksort,
LinearTime-Sorting, Medians
Comparison-Based Sorting and its lower bound
Breaking the lower bound using special assumptions
Tradeoffs: Selecting an appropriate sort for a given situation
 Time vs. Space Requirements
 Comparison-Based vs. Non-Comparison-Based
Comparison-Based Sorting
Time: BestCase
Algorithm:
InsertionSort
AverageCase
WorstCase
W(n lg n)
O(n2)
MergeSort
W(n lg n)
O(n lg n)
QuickSort
Q(n lg n)
HeapSort
W(n lg n)
O(n lg n)
Q(n2)
O(n lg n)
In algebraic decision tree model, comparison-based
sorting of n items requires W(n lg n) time.
To breaking the lower bound and obtain linear
time, forego direct value comparisons and/or
make stronger assumptions about input.
Data Structures
Chapters 11-14
Stacks, Queues, LinkedLists, Trees,
HashTables, Binary Search Trees, Balanced
Trees
Topics

Data Structures: Chapters 11-14

Abstract Data Types: their properties/invariants



Stacks, Queues, LinkedLists, (Heaps from Chapter 7), Trees, HashTables,
Binary Search Trees, Balanced (Red/Black) Trees
Implementation/Representation choices -> data structure
Dynamic Set Operations:

Query [does not change the data structure]


Manipulate: [can change data structure]



Search, Minimum, Maximum, Predecessor, Successor
Insert, Delete
Running Time & Space Requirements for Dynamic Set
Operations for each Data Structure
Tradeoffs: Selecting an appropriate data structure for a situation
 Time vs. Space Requirements
 Representation choices
 Which operations are crucial?
Advanced Techniques
Chapters 16-17
Dynamic Programming, Greedy
Algorithms
Topics

Advanced Techniques: Chapters 16-17

Algorithmic Paradigms/Design Patterns:





Using Dynamic Programming &/or Greedy Algorithms to
solve Optimization Problems



Divide-and-Conquer
Dynamic Programming
Greedy Algorithms
Brute-Force/Naive
Optimal Substructure
Greedy Choice Property: Locally optimal -> Globally optimal
Tradeoffs:


Selecting an appropriate paradigm to solve a problem
Tackling a problem using a sequence of paradigms:

Brute-Force (high running time) then improve...
Problem Characteristics
Divide-and-Conquer Dynamic Programming Greedy Algorithms
Modular
Independent pieces
Modular
Modular
Optimization
Optimization
Optimal substructure:
Optimal substructure:
optimal solution contains
optimal solution
optimal solutions to
contains optimal
subproblems
solutions to
subproblems
Overlapping subproblems
Greedy choice property:
locally optimal choices
lead to global
optimum
Graph Algorithms
Chapters 23-25
DFS/BFSTraversals, Topological Sort,
MinimumSpanningTrees, Shortest Paths
Topics

Graph Algorithms: Chapters 23-25




Undirected, Directed Graphs
Connected Components of an Undirected Graph
Representations: Adjacency Matrix, Adjacency List
Traversals: DFS and BFS









Differences in approach: DFS: LIFO/stack vs. BFS:FIFO/queue
Forest of spanning trees
Vertex coloring, Edge classification: tree, back, forward, cross
Shortest paths (BFS)
Topological Sort
Weighted Graphs
MinimumSpanningTrees: 2 different approaches
Shortest Paths: Single source: Dijkstra’s algorithm
Tradeoffs:


Representation Choice: Adjacency Matrix vs. Adjacency List
Traversal Choice: DFS or BFS
Traversals: DFS, BFS

DFS backtracks  visit most recently
discovered vertex  LIFO structure 
stack data structure

BFS  vertices close to v are visited before
those further away  FIFO structure 
queue data structure
FINAL EXAM
Logistics, Coverage, Format
Handout for basis of 40% of test
Course Grading
Homework 40%
 Exam 1 15% (closed book)
 Midterm 20% (open book)
 Final Exam 25% (open book)

Results are scaled if necessary.
Check grade status with us before final!
Final Exam: Logistics
Friday, 12/18
 Olsen 311: 8:00-11:00 a.m.
 Open book, open notes
 Closed computers, neighbors
 Cumulative
 Worth 25% of grade

Note change
from registrar’s
room number
Text/Chapter/Topic Coverage

Discrete Math Review & Basic Algorithm Analysis
Techniques : Chapters 1-6


Sorting: Chapters 7-10


Stacks, Queues, LinkedLists, Trees, HashTables, Binary Search
Trees, Balanced (Red/Black) Trees
Advanced Techniques: Chapters 16-17


Heapsort, Quicksort, LinearTime-Sorting, Medians
Data Structures: Chapters 11-14


Summations, Recurrences, Sets, Trees, Graph, Counting,
Probability, Growth of Functions, Divide-and-Conquer
Dynamic Programming, Greedy Algorithms
Graph Algorithms: Chapters 23-25

Traversal, MinimumSpanningTrees, Shortest Paths
Format

60%
Mixture of questions of the following types:
1) Multiple Choice
2) True/False
3) Short Answer
4) Analyze Pseudo-Code and/or Data Structure
5) Solve a Problem by Designing an Algorithm

40%




Select an appropriate paradigm/ design pattern
Select appropriate data structures
Write pseudo-code
Justify correctness
Analyze asymptotic complexity