Download Amortized Algorithm Analysis

Amortized Algorithm Analysis
COP3503
July 25, 2007
Andy Schwartz
Contents
•
•
•
•
•
Indirect Solutions
Amortized Analysis Introduction
Extendable Array Example
Binomial Queue
Binomial Queue Example
Indirect Solution
10 yards / minute
100 yards / minute
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Indirect Solution
10 yards / minute
100 yards / minute
Geometric Series?
The easy solution is indirect. It takes a kitten 5 minutes to
go 50 yards, how far can the mother go in 5 minutes?....
500 yards!
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Amortized Analysis Introduction
• The worst-case running time is not always the
same as the worst possible average running
time.
• Example:
– Worst case-time is O(n)
– Amortized worst-case is O(1)
– This could be from a series of table inserts and clears
(Source: Arup Guha. CS2 Notes – Summer 2007.
Amortization Techniques
• Two Techniques:
– Potential Function
– Accounting Method
• Start with $X dollars for n operations.
• Each simple operation costs $1, but larger operations are more
expensive.
• If $X dollars is enough for n operations, then the amortized worstcase is $x / n.
• Example: School Semester costs (worst-case most expensive in
Fall, but amortized worst-case including spring and summer is
cheaper).
• Compute the amortized worst-case running time of n
consecutive variable-popping stack operations. (details
on board)
(Source: Arup Guha. CS2 Notes – Summer 2007.
Extendable Array Example
• ArrayList (or Vector) in Java
• Can we maintain an extendable array and
keep the amortized worst-case running
time down to O(1)?
– Yes! (or this would be a bad example)
– How?....
– Simple case: add an element and space is left
over.
– Worst case: ?
(Source: Arup Guha. CS2 Notes – Summer 2007.
Extendable Array Example
• Simple Case: Add an element and space is left over.
O(1)
• Worst Case: Add an element, and there is no more
space after  we need to make space  allocate new
O(k)
array and copy elements
– The time complexity of this worst-case depends on n, the size of
the original array. How big should we make the new array?
– There were n-1 simple operations in order to fill the array to this
point, so let’s double it. That way a series of n-1 simple
insertions, followed by one worst case will result in about n-1 + n
= ~2n total operations.
– 2n operations / n add operations = 2 = O(2) = O(1)
*There were really more than n-1 operations if this was not the first
extension, but we know a series of those operations were O(1) and O(1)
+ O(2) = O(1).
(Source: Arup Guha. CS2 Notes – Summer 2007.
Binomial Queue
• Binomial Trees:
B0
B1
B2
B3
B4
Each tree “doubles”
the previous.
Binomial Queue
• Binomial Queue
• A queue of binomial trees. A “Forest”.
• Each tree is essentially a heap constrained to
the format of a binary tree.
• Example: B0, B2, B3
• Insertion: Create a B0 and merge
• Deletion: remove minimum( the root) from tree
Bk. This leaves a queue of trees:
B0, B1, …, Bk-1. Merge this queue with the rest of
the original queue.
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Binomial Queue Example
• The Most important step is Merge
• Merge Rules: (for two binomial queues)
– 0 or 1 Bk trees  just leave merged
– 2 Bk trees  meld into 1 Bk+1 tree.
– 3 Bk trees  meld two into 1 Bk+1, and leave the third.
• Insertion Runtime:
– M+1 steps, where M is smallest tree not present.
Worst-case is k+2, when Bk+1 is the smallest tree not
present. How does k relate to the total number of
nodes in the tree?
k = lg n, thus (nonamortized) worst-case time is O(lg n).
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Binomial Queue Example
• MakeBinQ Problem: Build a binomial queue of N
elements. (Like makeHeap).
How long should this take?
• Insertion worst-case runtime:
– Worst-case O(lg n) for 1 insert O(n lg n) n inserts,
but we want O(n) – like makeHeap
• Try amortized analysis directly:
– Considering each linking step of the merge. The 1st, 3rd, 5th,
ettc… odd steps require no linking step because there will be no
B0. So ½ of all insertions require no linking, similarly ¼ require 1
linking steps.
– We could continue down this path, but the it’ll be come
especially difficult for deletion (we should learn an indirect
analysis).
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Binomial Queue Example
• Indirect Analysis (time = M + 1)
– If no B0  cost is 1 (M is 0)
• Results in 1 B0 tree added to the forest
– If B0 but no B1  cost is 2 (M is 1)
• Results in same # of trees (new B1 but B0 is gone)
– When cost is 3 (M is 2)
• Results in 1 less tree (new B2 but remove B0 and B1)
….etc…
– When cost is c (M is c – 1)
• Results in increase of 2 – c trees
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Binomial Queue Example
• increase of 2 – c trees
• How can we use this?
Ti = # of trees after ith iteration
T0 = 0 = # of trees initially
Ci = Cost of ith iteration
Then, Ti = Ti-1 + (2 – Ci)

Ci + (Ti – Ti-1) = 2
• This is only the ith iteration
(Source: Mark Allen Weiss. Data Structures and Algorithm Analysis in Java.
Binomial Queue Example
Ci + (Ti – Ti-1) = 2
• To get all iterations:
C1 + (T1 – T0) = 2
C2 + (T2 – T1) = 2
…
Cn-1 + (Tn-1 – Tn-2) = 2
Cn + (Tn – Tn-1) = 2
n
Σ Ci + (Tn – T0) = 2n
i=1
(T1 .. Tn-1 cancel out)
Binomial Queue Example
n
Σ Ci + (Tn – T0) = 2n
i=1
• T0 = 0 and Tn is definitely not negative, so
Tn – T0 is not negative.
n
Σ Ci <= 2n
i=1
Thus, the total cost < 2n 
makeBinQ = O(n)
Since, makeBinQ consists of O(n)
inserts, then the amortized worstcase of each insert is O(1).