Download Lecture7PL

Priority Queues and
Binary Heaps
15-211
Fundamental Data Structures and
Algorithms
Peter Lee
February 4, 2003
Announcements
• HW3 is available
 due Monday, Feb.10, 11:59pm
 start today!
• Quiz #1 available Thursday
 an online quiz
 requires up to one hour of
uninterrupted time with a web browser
 must be completed by Friday, 11:59pm
• HW4 in teams of two
 go to recitation for details
Priority Queues
Priority queues
• Today we consider a useful abstract
data type called the priority queue.
Priority queue ADT
• Assume we have objects x that can
be compared using the < and ==
operators.
 These are Comparable objects.
• insert(k, o)
 Inserts comparable object k into the
priority queue with object o.
• deleteMin()
 Returns and removes the minimum
element from the priority queue.
The priority queue interface
• create
• insert
void insert (Comparable key, Object r)
• findMin
Comparable findMin ()
• deleteMin
Comparable deleteMin ()
Priority Queues: Applications
• Event simulations (key is time)
• Shortest paths in a graph
• Huffman codes
• Sorting
Possible priority queue implementations
• Linked list
 deleteMin
 insert
O(1)
O(N)
or
O(N)
O(1)
• Search trees
 All operations
• Heaps
 deleteMin
 insert
O(log N)
avg (assume random) worst
O(log N)
2.6
O(log N)
O(log N)
O(N)
O(N)
special case:
 buildheap
i.e., insert*N
Possible priority queue implementations
• Linked list
 deleteMin
 insert
O(1)
O(N)
or
O(N)
O(1)
• Search trees
 All operations
• Heaps
 deleteMin
 insert
O(log N)
avg (assume random) worst
O(log N)
2.6
O(log N)
O(log N)
O(N)
O(N)
special case:
 buildheap
i.e., insert*N
Possible priority queue implementations
• Linked list
 deleteMin
 insert
O(1)
O(N)
or
O(N)
O(1)
• Search trees
 All operations
• Heaps
 deleteMin
 insert
O(log N)
avg (assume random) worst
O(log N)
2.6
O(log N)
O(log N)
O(N)
O(N)
special case:
 buildheap
i.e., insert*N
A Digression:
Perfect and Complete
Binary Trees
Perfect binary trees
13
21
16
24
65
31
26
32
19
26
65
68
26
65
26
Perfect binary trees
13
21
16
24
31
65
65
26
26
65
32
26 65
19
26
26
65
65
26 65
26
68
26
65
65
26 65
26
26
65
26
Perfect binary trees
13
21
16
24
31
65
65
26
26
65
32
26 65
19
26
26
65
65
26 65
26
68
26
65
65
26 65
26
26
65
26
6526 6526 6526 6526 6526 6526 6526 6526 6526 6526 6526 65266526 6526 6526 6526
Perfect binary trees
• How many nodes?
13
21
24
65
16
31
19
68
26 32 26 65 26 65
26
Perfect binary trees
• How many nodes?
 N = 24 - 1 = 15
13
21
24
65
16
31
19
68
26 32 26 65 26 65
26
h=3
N=15
Perfect binary trees
• How many nodes?
 N = 24 - 1 = 15

 Most of the nodes are ______
13
21
24
65
16
31
19
68
26 32 26 65 26 65
26
h=3
N=15
Perfect binary trees
• How many nodes?
 N = 24 - 1 = 15

 Most of the nodes are leaves
13
21
24
65
How
many
edges?
16
31
19
68
26 32 26 65 26 65
26
h=3
N=15
Perfect binary trees
• How many nodes?
 N = 24 - 1 = 15
 In general: N =
 Most of the nodes are leaves
13
21
24
65
16
31
19
h=3
N=15
68
26 32 26 65 26 65
26
Perfect binary trees
• How many nodes?
 N = 24 - 1 = 15
 In general: N =
= 2h+1 - 1
 Most of the nodes are leaves
13
21
24
65
16
31
19
h=3
N=15
68
26 32 26 65 26 65
26
Quiz Break
Red-green quiz
• In a perfect binary tree,
what is the sum of the
heights of the nodes?
 Give a mathematical
characterization.
 Give a tight upper bound
(in big-O terms)
Perfect binary trees
• What is the sum of the heights?
S=
< N = O(N)
13
21
24
65
16
31
19
68
26 32 26 65 26 65
26
h=3
N=15
Complete binary trees
13
21
24
65
26
16
31
32
19
68
Complete binary trees
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Linked structures?
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Linked structures? No!
• Instead, use arrays!
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Arrays
 Parent at position i in the array
 Children at positions 2i and 2i+1
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Arrays (1-based)
 Parent at position i
 Children at 2i and 2i+1.
1 2 3
4 5 6 7
8 9 10
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Arrays (1-based)
 Parent at position i
 Children at 2i and 2i+1.
1 2 3
4 5 6 7
8 9 10
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Arrays (1-based)
 Parent at position i
 Children at 2i (and 2i+1).
1 2 3
4 5 6 7
8 9 10
1
2
3
4
8
5
9
10
6
7
Representing complete binary trees
• Arrays (1-based)
 Parent at position i
 Children at 2i and 2i+1.
public class BinaryHeap {
private Comparable[] heap;
private int size;
public BinaryHeap(int capacity) {
size=0;
heap = new Comparable[capacity+1];
}
. . .
Representing complete binary trees
• Arrays
 Parent at position i
 Children at 2i and 2i+1.
• Example: find the leftmost child
int left=1;
for(; left<size; left*=2);
return heap[left/2];
• Example: find the rightmost child
int right=1;
for(; right<size; right=right*2+1);
return heap[(right-1)/2];
Back to Our Main Topic:
Implementing Priority
Queues with Binary Heaps
Binary heaps: the invariant
• Representation invariants
1. Structure property
• Complete binary tree (i.e. the elements of
the heap are stored in positions 1…size of
the array)
2. Heap order property
• Parent keys less than children keys
Heaps
• Representation invariant
1. Structure property
• Complete binary tree
• Hence: efficient compact representation
2. Heap order property
• Parent keys less than children keys
• Hence: rapid insert, findMin, and deleteMin
• O(log(N)) for insert and deleteMin
• O(1) for findMin
The heap order property
• Each parent is less than each of its
children.
• Hence: Root is less than every other
node.
 (obvious proof by induction)
13
21
24
65
26
16
31
32
19
68
Operating with heaps
Representation invariant:
• All methods must:
1. Produce complete binary trees
2. Guarantee the heap order property
• All methods may assume
1. The tree is initially complete binary
2. The heap order property holds
Constructor method
• All methods must:
1. Produce complete binary trees
• Trivially true
2. Guarantee the heap order property
• Also trivially true
• This is the base case
findMin ()
• The code
public boolean isEmpty() {
return size == 0;
}
public Comparable findMin() {
if(isEmpty()) return null;
return heap[1];
}
• Does not change the tree
 Trivially preserves the invariant
insert (Comparable x)
• Process
1. Create a “hole” at the next tree cell for x.
heap[size+1]
This preserves the completeness of the tree.
2. Percolate the hole up the tree until the
heap order property is satisfied.
This assures the heap order property is satisfied.
insert (Comparable x)
• Process
1. Create a “hole” at the next tree cell for x.
heap[size+1]
This preserves the completeness of the tree
assuming it was complete to begin with.
2. Percolate the hole up the tree until the
heap order property is satisfied.
This assures the heap order property is
satisfied assuming it held at the outset.
Percolation up
public void insert(Comparable x)
throws Overflow
{
if(isFull()) throw new Overflow();
int hole = ++size;
for(;
hole>1 && x.compareTo(heap[hole/2])<0;
hole/=2)
heap[hole] = heap[hole/2];
heap[hole] = x;
}
Percolation up
• Bubble the hole up the tree until the
heap order property is satisfied.
hole = 11
HOP false
13
21
24
65
26
16
31
32
14
19
68
Not really there...
Percolation up
• Bubble the hole up the tree until the heap
order property is satisfied.
hole = 11
HOP false
hole = 5
HOP false
13
21
13
16
24
31
65 26
32 14
19 68
21
16
24
14
65 26
32 31
19 68
Percolation up
• Bubble the hole up the tree until the heap
order property is satisfied.
hole = 5
HOP false
hole = 2
HOP true
13
21
13
16
24
14
65 26
32 31
done
19 68
14
16
24
21
65 26
32 31
19 68
Percolation up
public void insert(Comparable x)
throws Overflow
{
if(isFull()) throw new Overflow();
int hole = ++size;
for(;
hole>1 && x.compareTo(heap[hole/2])<0;
hole/=2)
Integer division
heap[hole] = heap[hole/2];
heap[hole] = x;
}
deleteMin()
/**
* Remove the smallest item from the priority queue.
* @return the smallest item, or null, if empty.
*/
public Comparable deleteMin( )
{
if(isEmpty()) return null;
Comparable min = heap[1];
heap[1] = heap[size--];
percolateDown(1);
return min;
}
Grab min
element
Temporarily
place last
element at top
!!!
Percolation down
• Bubble the transplanted leaf value down
the tree until the heap order property is
satisfied.
1
2
-14
16
24
21
65 26
32 31
19 68
31
14
24
65 26
16
21
32
19 68
Percolation down
• Bubble the transplanted leaf value down
the tree until the heap order property is
satisfied.
2
3
31
14
24
65 26
16
21
32
19 68
14
31
24
65 26
16
21
32
19 68
Percolation down
• Bubble the transplanted leaf value down
the tree until the heap order property is
satisfied.
3
4
14
31
24
65 26
16
21
32
19 68
14
21
24
65 26
done
16
31
32
19 68
percolateDown(int hole)
private void percolateDown( int hole )
{
int child = hole;
Initially 1
Comparable tmp = heap[hole];
for( ; hole*2 <= size; hole=child )
Start at left child
{
child = hole * 2;
Is there a right child?
if(child!=size &&
heap[child+1].compareTo(heap[child]) < 0)
child++;
Select smaller child
if(array[child].compareTo(tmp) < 0)
heap[hole] = heap[child];
else break;
}
}
heap[hole] = tmp;
Bubble up if smaller than tmp
Exit loop if not bubbling
Finally, place the orphan
deleteMin ()
• Observe that both components of
the representation invariant are
preserved by deleteMin.
1. Completeness
• The last cell ( heap[size] ) is vacated,
providing the value to percolate down.
• This assures that the tree remains
complete.
2. Heap order property
deleteMin ()
• Observe that both components of
the representation invariant are
preserved by deleteMin.
1. Completeness
• The last cell ( heap[size] ) is vacated,
providing the value to percolate down.
• This assures that the tree remains
complete.
2. Heap order property
deleteMin ()
• Observe that both components of
the representation invariant are
preserved by deleteMin.
1. Completeness
• The last cell ( heap[size] ) is vacated,
providing the value to percolate down.
• This assures that the tree remains
complete.
2. Heap order property
• The percolation algorithm assures that the
orphaned value is relocated to a suitable
position.
buildHeap()
• Start with complete (unordered) tree
• Starting from bottom, repeatedly call
percolateDown()
Void buildHeap () {
for (int i = size/2; i>0; i--)
percolateDown(i);
}
buildHeap() performance
• At each iteration, have to do work
proportional to the height of the
current node
• Therefore, total running time is
bounded by the sum of the heights
of all of the nodes
Another Digression:
Invariants
Representation Invariants
• Necessary for a clear understand an
algorithm or piece of code.
• The most useful kind of comment
you can make in a piece of code!
(Invariants)
Plus ça change, plus c’est la même chose
• The role of an induction hypothesis
(Invariants and induction)
• Induction hypothesis
 What you are allowed to assume
• At the start
• About the result values of a recursive call
• About the object state when a method is
called
 What you must deliver
• At the end
• Of the result values when the recursive call
returns
• About the object state when the method
returns
(Invariants and induction)
• Induction hypothesis
 What you are allowed to assume
• At the start of a loop iteration
• About the result values of a recursive call
• About the object state when a method is
called
 What you must deliver
• At the end of the loop iteration
• Of the result values when the recursive call
returns
• About the object state when the method
returns
(Invariants and induction)
• Induction hypothesis
 What you are allowed to assume
•
• About the result values of a recursive call
• About the object state when a method is
called
 What you must deliver
•
• Of the result values when the recursive call
returns
• About the object state when the method
returns
(Invariants and induction)
• Induction hypothesis
 What you are allowed to assume
• At the start of a loop iteration
• About the result values of a recursive call
• About the object state when a method is
called
 What you must deliver
• At the end of the loop iteration
• Of the result values when the recursive call
returns
• About the object state when the method
returns
(Invariants and induction)
• Induction hypothesis
 What you are allowed to assume
• At the start of a loop iteration
• About the result values of a recursive call
• About the object state when a method is
called
 What you must deliver
• At the end of the loop iteration
• Of the result values when the recursive call
returns
• About the object state when the method
returns
(Invariants)
Plus ça change, plus c’est la même chose
• The role of an induction hypothesis
• Invariants in programs
 Loop invariants
 Recursion invariants
 Representation invariants
(Representation invariant)
• What must always be true of the data
structure when an operation completes.
• Theorem:
 Suppose each constructor assures the representation
invariant is initially correct.
 Suppose each method preserves the representation
invariant, assuming it is true initially.
 Then the representation invariant will always be true at
the completion of each method.
(Representation invariant)
• What must always be true of the data
structure when an operation completes.
• Theorem:
 Suppose each constructor assures the representation
invariant is initially correct.
 Suppose each method preserves the representation
invariant, assuming it is true initially.
 Then the representation invariant will always be true at
the completion of each method.
(Representation invariants)
• What must always be true of the data
structure when an operation completes.
• Theorem:
 Suppose each constructor assures the representation
invariant is initially correct.
 Suppose each method preserves the representation
invariant, assuming it is true initially.
 Then the representation invariant will always be true at
the completion of each method.
 Assuming the code is not concurrent!
Heapsort
• Obviously we can use a priority
queue to sort . . . just insert all
the keys then do repeated
deleteMins
• However we can take advantage
of the fact that the heap always
uses the first part of the array to
do this with no extra space.
• This is called heapsort.
Heapsort
• Reverse the sense of the heap order (largest at the root)
• Start from position 0 in the array (children are 2i+1 and
2i+2)
• Call it percDown(a, i, len)
Public static void heapsort(Comparable[] a) {
for(int i = a.length/2; i>=0; i--)
percDown(a, i, a.length);
for (int j = a.length-1; j>0; j--) {
swapReferences(a, 0, j);
percDown(a, 0, j);
}
}
Heapsort invariant
• At the start of the loop over j:
 a[j]…a[a.length-1] are sorted
 a[0]…a[j-1] are a heap