Download Tutorial 1 C++ Programming

Tutorial 7
Sorting
&
Complexity
Analysis
(Almost) always asked in exam! e.g.
a. Derive algorithm for this problem!
b. What is the time complexity of your algorithm?
Asymptotic Algorithm Analysis
• What to analyze:
– Given input of size n, what is the time (or space) required by the algorithm?
• f(n) = O(g(n)) // Big O
– n is the input size!
– f(n) is the actual algorithm complexity, can be “hard to compute”…
– g(n) is a simple function that upper bounds f(n)!
• Drop lower order terms and constants in g(n) (hence the name: asymptotic)
– Big O is usually for worst case analysis
– Formal definition: 0 ≤ f(n) ≤ c*g(n) for all n ≥ n0
• This is like saying: “My algorithm’s growth rate cannot be WORSE than g(n)!”
• Purpose of Big O analysis:
– Simply to see whether the required time (and/or) space of our current algorithm
suits the given budget or not.
– If no, a better algorithm is needed!
• We will practice some algorithm analysis skills today!
How long to process n items?
• Assume that 1 item can be processed in 1 ms
Order of Growth
n
Time (ms)
Comment
O(1)
1000
1
Excellent, But almost impossible for most cases
O(log n)
1000
9.96
Very good, Example: Binary Search
O(n)
1000
1000
Normal, Linear time algorithm, 1000 operations == 1000 ms
O(n log n)
1000
9960
Average, This is usually found in sorting algorithm such as Quick Sort
O(n2)
1000
1000000
Slow, Typical in programs that have two nested loops.
O(n3)
1000
109
Slow, btw, All Pairs Shortest Paths algorithm: Floyd Warshall, is O(n3)
O(2n)
1000
21000
Poor, exponential growth... avoid this…
O(n!)
1000
Uncountable
Typical NP-Complete problems...
Unless the problem size is very small, there is no hope...
What can you do in 1 m?
• Assume that 1 item can be processed in 1 ms
Order of Growth
Time (ms)
Max Possible n
Comment
O(1)
60.000 ms
…
Can be virtually infinite…
O(log n)
60.000 ms
260000
A (very) very large number
O(n)
60.000 ms
60000
Baseline, still a very big number…
O(n log n)
60.000 ms
~ 5000
Moderate, average size for many real life problems
O(n2)
60.000 ms
244
Small
O(n3)
60.000 ms
39
Very small
O(2n)
60.000 ms
16
Try to avoid this if you can…
O(n!)
60.000 ms
8
Extremely small...
• Order of Growth Matters…
– O(1) < O(log n) < O(n) < O(n log n) < O(n2) < O(n3) < O(2n) < O(n!)
• We will use this O notation from now towards the end of semester!
Sorting: Motivation
• When a list is sorted, many operations become easier!
• Imagine if your dictionaries, phone books, etc are NOT sorted…
Comparison-Based Sorting
• From simple (slow) to complex (faster)
–
–
–
–
–
Bubble Sort
Insertion Sort
Selection Sort
Merge Sort
Quick Sort
O(n2)
O(n2)
O(n2)
O(n log n)  Discussed next week
O(n2), but on average O(n log n)  Discussed next2 week
• Lower bound for comparison-based sorting algorithms
– (n log n)  Proven, we cannot do better than this!
Special Purpose Sorting
•
•
•
•
Radix Sort
O(d(n+k))
Counting Sort
O(n+k)
Bucket Sort
O(n)
Hey, you said that sorting is (n log n) ?
– These sorting algorithms are for special purposes only.
– They do not do “comparison”…
• There are more sorting algorithms in this world…
– http://www.cs.ubc.ca/~harrison/Java/sorting-demo.html
– http://thomas.baudel.name/Visualisation/VisuTri/index.html
• But in real life,
– We just use generic built-in algorithms like Java “Collections.Sort”. 
– We are going to learn this in question 2 today.
Sort: Fundamental CS Problem
• “Give rise” or “used to explain” algorithmic concepts:
– Divide and Conquer
– Recursion
– That efficiency matters
• Sorting is mostly used as preliminary step for many other algorithms.
– When in doubt, sort!... Try this in CS1102 exams!
• Sorting problem is “well solved”.
– The best algorithm to-date is (n log n)
– It matches the lower bound of sorting (n log n)
• Also see:
– http://www.comp.nus.edu.sg/~stevenha/myteaching/notes/5_sorting.html
Student Presentation
• Gr3
1.
2.
3.
Du Xin and (David Seo or Tanvir Islam)
Leow Wei Jie or Hema Kumar or Jacob Pang
Lim Wei Hong and (Nicholas Koh or Chia Jie Shen)
• Gr4
1.
2.
3.
Wong Suet Teng, Melissa and Liew Hui Sun
Wang Kang or Ng Xue Lin Sherilyn or Tan Yan Hao (Gr5)
Goh Khoon Hiang and Chua Kien Chuan
• Gr5
1.
2.
3.
Joyeeta Biswas (all q1)
Ong Kian An
Wu Shujun and Zheng Yang
• Gr6
1.
2.
3.
Koh Jye Yiing and (Zhang Chao or Siddhartha)
Nguyen Duy Hoang
Kwong Gary and Wang Shuling
Overview of the questions:
1. Scenarios and Simple Problems
(2 students, a-b and then c-d)
2. F1, Comparator, Collections.sort
(1 student, a-c)
3. Complexity Analysis
(2 students, a-b-c, and then d)
9
Q1: Scenarios & Simple Probs
10 Sorting Scenarios:
1.
2.
3.
4.
5.
Finding word in dictionary
Finding duplicates in data
Finding median in data
Finding pairs A and B, where A+B=V
etc…
• Finding Duplicates
– Sort the numbers: O(n log n)
– Scan the numbers if two adjacent
numbers are the same: O(n)
• Finding Median
– Sort the numbers: O(n log n)
– Return number[size/2]; O(1)
– Fastest method: O(n), find k-index
(see note 5, slide 34 or this link)
• Finding Pair A and B, A+B=V
– Sort the numbers: O(n log n)
– Scan the numbers: O(n) * O(log n)
•
•
If the first number is A, then the other
number must be B=V-A!
Use binary search to find B: O(log n)
– If no pair found, report “no pair”.
Q2: F1 Again, but Sorted
• This is what you need to remember beyond this CS1102.
– Something to tell Java which object should be in front of which object!
• Java Interface: Comparator
– Or alternatively, Interface: Comparable
• Implements compare(o1, o2)
– Or alternatively, implements: compareTo(o)
• By default, generic data type (integer, float, double, string, etc) are comparable!
– Java generic sorting algorithm
•
•
•
•
•
Collections.sort(array)  works for generic data type
Collections.sort(array, comparator)  specify comparator for ADTs
O(n log n) – efficient implementation of comparison based sorting algorithm
Bug free
Easy to use
– See source codes for details!
Q3: Complexity Analysis (1)
• Case 1
• Case 2
j=1;
while (j<=n) {
for (i=n*n; i>=1; i=i-3)
x=x+1;
j=j*2;
}
•
O(n2
log n)
– The innermost loop is
– The while loop is log n
n2
• j=1
• j=j*2
• j will reach n in log2 n steps!!
for (i=1; i<=100; i++)
for (j=1; j<i*n; j++)
x=x+1;
• O(n)
– 100*(100*n) ~ 10000*n is still O(n)
• Case 3
for (i=1; i<=100; i++)
for (k=1; k<=i; k++)
for (j=1; k<=k; j++)
x=x+1;
• O(∞)
– The innermost loop
will never terminate…
Please look at my website for more practice related to algorithm analysis!
Q3: Complexity Analysis (2)
• Suppose we have:
int f(int n) {
if (n > 1) {
g(n);
return f(n/2) + f(n/2);
}
}
• What is the time complexity of f(n), if g(n) is:
To answer this, we must draw the recursive execution tree…
a) g(n) = O(1)
O(n), a sum of geometric series of 1+2+4+…+2log2 n = 1+2+4+…+n = c*n
b) g(n) = O(n)
O(n log n), a sum of (n+n+n+…+n) log2 n times, so, n log n
c) g(n) = O(n2)
O(n2), a sum of geometric series of n2+n2/2+n2/4+…+n = c*n2 = n2
Please look at my website for more practice related to algorithm analysis!
Food For Thought
•
•
•
•
•
What is the time complexity of recursive program that we built last week?
void CS1102_Life(int week_ID) {
do_tutorial_and_lab(week_ID);
if (week_ID != end of semester)
CS1102_Life(week_ID+1);
}
O(13 teaching weeks) ~ O(1) :p
One semester is just “constant time”
The “torture” is not that long …