Download Algorithm Analysis Algorithm Analysis Algorithm Analysis Running

Algorithm Analysis
• 
A data structure is a systematic way of organizing
and accessing data, and an algorithm is a step-bystep procedure for performing some task in a finite
amount of time. ACS-2947 Lecture 5
• 
These concepts are central to computing, but to be
able to classify some data structures and
algorithms as "good," we must have precise ways
of analyzing them. Algorithm Analysis Running Time
How “good” is a data structure or algorithm?
• 
Most algorithms transform input
objects into output objects
• 
The running time of an algorithm
typically grows with the input
size
Average case time is often
difficult to determine
Algorithm Analysis
• 
The primary analysis tool that we will use will focus
on involves characterizing the running times of
algorithms and data structure operations, with
space usage also being of interest
à  characterizing
an algorithm’s running time as a
function of the input size
• 
But what is the proper way of measuring it?
• 
• 
We focus on the worst case
running time
• 
Easier to analyze
Focusing on the Worst-Case Input
• 
Worst-case analysis is much easier than averagecase analysis • 
Requires only the ability to identify the worst-case
input, which is often simple
• 
• 
• 
Experimental Studies
Typically leads to better algorithms
• 
Write a program
implementing the
algorithm Run the program with
inputs of varying size and
composition • 
Use a method like
• 
to get an accurate
measure of the actual
running time Plot the results Making the standard of success for an algorithm to
perform well in the worst case necessarily requires
that it will do well on every input
System.currentTimeMillis()
1
Limitations of Experiments
Theoretical Analysis
• 
In order to compare two algorithms, the same
hardware and software environments must be used • 
Allows us to evaluate the speed of an algorithm
independent of the hardware/software environment • 
Results may not be indicative of the running time on
other inputs not included in the experiment. • 
Uses a high-level description of the algorithm
instead of an implementation • 
It is necessary to implement the algorithm, which
may be difficult • 
Characterizes running time as a function of the
input size, n. • 
Takes into account all possible inputs The 7 functions used in this class
Primitive Operations • 
Constant f(n)=c
• 
Logarithm f(n)=logbn
• 
Linear f(n)=n
• 
N-log-n f(n)=nlogn
• 
• 
Quadratic f(n)=n2
• 
• 
Cubic f(n)=n3
• 
Exponential f(n)=bn
• 
To analyze the running time of an algorithm without
performing experiments, we perform an analysis of
primitive operations: • 
• 
• 
• 
• 
Example: find the max element in array
Assigning a value to a variable
Following an object reference
Performing an arithmetic operation Comparing two numbers
Accessing a single element of an array by index
Calling a method
Returning from a method Asymptotic Behaviour
int x = A[0];
for (int i = 1; i < n; i++){
if (A[i] >= x) {
x = A[i];
}
2
2
2(n-1)
2(n-1)
2(n-1)
f(n)=6n-2
• 
In complexity analysis we only care about what
happens to this function as the program input n
grows large
}
f(n)=4+3(2(n–1))
f(n)=6n-2
2
Asymptotic Behaviour
f(n)=6n-2
Asymptotic Analysis
• 
In algorithm analysis, we focus on the growth rate
of the running time as a function of the input size n,
taking a “big-picture” approach. • 
For example, it is often enough just to know that
the running time of an algorithm grows
proportionally ton. • 
• 
Set-up time may vary between languages and
machines
Programming languages may compile differently
Efficiency and Big O Notation
Efficiency and Big O Notation
• 
In an asymptotic analysis, we care more about the
order of magnitude of a function rather than the
actual value of a function itself. • 
In addition, we don't know exactly how long each
operation might actually take on a certain
computer. • 
In terms of the abstract time of an algorithm, this
should make some intuitive sense. • 
• 
We call it "abstract time" because we use
"abstract" operations such as "number of math
operations" in f(n), or "number of comparisons" in
f(n). Intuitively, the order of magnitude appeals to our
sense that n2 is a faster-growing function than a
linear function like n. Big O Notation
• 
Big O Notation
Formal definition:
• 
Let f (n) and g(n) be functions mapping positive
integers to positive real numbers. • 
We say that f (n) is O (g(n)) if there is a real constant
c > 0 and an integer constant n0 ≥ 1 such that#
f (n) ≤ c · g (n) , for n ≥ n0
3
Big O Notation
• 
Big O
A way of analyzing how fast a program’s runtime grows
asymptotically
• 
E.g. as the size of your inputs increase towards infinity
à How does the runtime of your program grow?
In our example, 6n-2isO(n)
• 
Runs in linear time WRT number of elements, n
• 
The time required to traverse through the array is
proportional to the number of elements
Big 0
Operations for
10 “things”
Operations for
100 “things”
O(1)
O(logn)
O(n)
O(nlogn)
O(n2)
O(n3)
O(bn)
1
3
10
30
100
1000
1024
1
7
300
700
10000
1000000
2100
Big O Rules
• 
Big O Rules • 
• 
• 
If f(n)is a polynomial of degree d, then f(n)is O(nd),
i.e., 1. 
Drop lower-order terms 2. 
Drop constant factors Use the smallest possible class of functions • 
• 
Big O Notation
Say “2n is O(n)” instead of “2n is O(2n)” • 
Use the simplest expression of the class
• 
To describe the order of magnitude of a function,
we use Big O notation. • 
If we had an algorithm that did 8n-2operations, the
Big O notation would be O(n)
• 
If we had an algorithm that did 5n4+3n3–2n2+2
operations, its Big O notation would be O(n4).
In our example, our algorithm did 6n-2operations,
so the Big O notation would be O(n). Say “3n+5is O(n)” instead of “3n+5is O(3n)” Examples
Comparing Growth Rates
f(n)=2n+12 O(n)
f(n)=25 O(1)
f(n)=n2+4n+6 O(n2)
• 
Ideally, we would like data structure operations to run
in times proportional to the constant or logarithm
function, and we would like our algorithms to run in
linear or n-log-n time. • 
Algorithms with quadratic or cubic running times are
less practical, but algorithms with exponential running
times are infeasible for all but the smallest sized
inputs. f(n)=n3+100n+99 O(n3)
f(n)=5n2+3nlogn+2n+5 O(n2)
f(n)=3logn+2 O(logn)
4
Growth Rates
O(1)– Constant time
public boolean isFirstZero(int arr[]){
if (array[0] == 0)
return true;
else
return false;
}
• 
O(logn)– Logarithmic Time • 
Any algorithm which cuts the problem in half each time
is O(logn). • 
O(logn)operations run in logarithmic time - the
operation will take longer as the input size increases,
but once the input gets fairly large it won’t change
enough to worry about. • 
If you double n, you have to spend an extra amount of
time t to complete the task. If n doubles again, t won’t
double, but will increase by a constant amount. • 
An example of an O(logn)operation is a binary search
Binary Search
public static boolean binarySearch
(int[] data, int target, int low, int high) {
if (low > high)
return false;
else{
int mid = (low + high) / 2; // Determine the middle element
// Split the result in half and search again recursively until found
if (target == data[mid])
return true;
else if (target < data[mid])
return binarySearch(data, target, low, mid-1);
else
return binarySearch(data, target, mid+1, high);
}
}
Will take the same time to execute, no matter how
big arr is
Binary Search
O(n)– Linear time
• 
O(n)means that for every element, you are doing a
constant number of operations, such as comparing
each element to a known value. • 
run in linear time - the larger the input, the longer it
takes, in an even tradeoff. • 
• 
E.g. Every time you double n, the operation will take
twice as long. An example of an O(n)operation is finding the
largest value in an array 5
O(n)– Linear time
• 
The linear function also represents the best running
time we can hope to achieve for any algorithm that
processes each of n objects that are not already in
the computer’s memory, because reading in the n
objects already requires n operations. int x = A[0];
for (int i = 1; i < n; i++){
if (A[i] >= x) {
x = A[i];
}
}
O(nlogn)–Loglinear Time
• 
O(nlogn)means that you’re performing an O(logn)
operation for each item in your input. • 
Most (efficient) sort algorithms are an example of
this. • 
O(nlogn)operations run in loglinear time increasing the input size hurts, but may still be
manageable. Every time you double n, you spend
twice as much time plus a little more. • 
Examples of O(nlogn)operations are quicksort (in
the average and best case) and merge sort. O(n2)– Quadratic Time
• 
O(n2)means that for every element, you do something
with every other element, such as comparing them. • 
O(n2)operations run in quadratic time - the operation is
only really practical up to a certain input size. Every time
n doubles, the operation takes four times as long. • 
Examples of O(n2)operations are quicksort (in the worst
case) and bubble sort. • 
The following function is an example of an O(n2)
operation, where every element in an array is compared
to every other element : public boolean containsDup (int array[]){
int i = 0;
int j;
while (i < array.length){
j = i + 1;
while (j < array.length){
if (array[i] == array[j])
return true;
j++;
}
i++;
}
return false;
}
Asymptotic Analysis
• 
Suppose two algorithms solving the same problem
are available: an algorithm A, which has a running
time of O(n), and an algorithm B, which has a
running time of O(n2). • 
Which algorithm is better? • 
We know that n is O(n2), which implies that
algorithm A is asymptotically better than algorithm
B, although for a small value of n, B may have a
lower running time than A. 6