Download Algorithms and Data Structures

Priority Queues and Heaps
Page 1
Berner Fachhochschule - Technik und Informatik
Algorithms and Data Structures
Priority Queues and Heaps
Dr. Rolf Haenni
Fall 2008
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Page 2
Outline
Priority Queues
Heaps
Heap-Based Priority Queues
Bottom-Up Heap Construction
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 3
Outline
Priority Queues
Heaps
Heap-Based Priority Queues
Bottom-Up Heap Construction
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 4
Priority Queue ADT
I
I
I
A priority queue stores a collection of items
An item is a pair (key , element)
Characteristic operations:
Ý insertItem(k,e): inserts an item with key k and element e
Ý removeMin(): removes the item with the smallest key and
returns its element
Ý minKey(): returns the smallest key of an item (no removal)
Ý minElement(): returns the element of an item with smallest
key (no removal)
I
General operations:
Ý size(): returns the number of items
Ý isEmpty(): indicates whether the priority queue is empty
I
Two distinct items in a priority queue can have the same key
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 5
Total Order
I
I
Keys in a priority queue can be arbitrary objects on which a
total order is defined
Mathematically, a total order is a binary relation defined on
a set K which satisfies three properties:
Ý Totality: x y or y x, for all x, y ∈ K
Ý Antisymmetry: x y and y x implies x = y , for all x, y ∈ K
Ý Transitivity: x y and y z implies x z, for all x, y , z ∈ K
I
Totality implies Reflexivity: x x, for all x ∈ K
I
Examples: ≤ or ≥ for R, alphabetical or reverse alphabetical
order for {A, . . . , Z }, lexicographical order for {A, . . . , Z }∗ ,
etc.
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 6
Comparator ADT
I
A comparator encapsulates the action of comparing two keys
according to a given total order relation
I
A generic priority queue uses an auxiliary comparator (passed
as a parameter to the constructor)
I
When the priority queue needs to compare two keys, it uses its
comparator
Operations (all with Boolean return type):
I
Ý
Ý
Ý
Ý
Ý
Ý
isLessThan(x,y)
isLessThanOrEqualTo(x,y)
isEqualTo(x,y)
isGreaterThan(x,y)
isGreaterThanOrEqualTo(x,y)
isComparable(x)
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 7
Sequence-Based Priority Queue
I
I
There are two ways to implement a priority queue with a
sequence (list or vector)
Using an unsorted sequence
Ý insertItem(k,e) runs in O(1) time, since we can insert the
item at the beginning of the sequence
Ý removeMin(), minKey(), minElement() run in O(n) time
since we have to traverse the entire sequence to find the
smallest key
I
Using a sorted sequence
Ý insertItem(k,e) runs in O(n) time, since we have to find
the place where to insert the item
Ý removeMin(), minKey(), minElement() run in O(1) time
since the smallest key is at the beginning or end of the
sequence
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues
Priority Queues and Heaps
Page 8
UML Diagram: Composition
<<interface>>
<<interface>>
Comparator
BasicSequence
isLessThan(x,y)
isEqualTo(x,y)
etc.
isEmpty()
size()
<<interface>>
<<interface>>
PriorityQueue
List
insertItem(k,e)
removeMin()
minKey()
minElement()
SortedPQ
list: SinglyLinkedList
C: Comparator
etc.
UnsortedPQ
list: SinglyLinkedList
C: Comparator
Berner Fachhochschule
Technik und Informatik
SinglyLinkedList
first: ListNode
n: Integer
<<interface>>
Position
etc.
ListNode
element: Object
next: ListNode
Rolf Haenni
Algorithms and Data Structures
Priority Queues
Priority Queues and Heaps
Page 9
UML Diagram: Inheritance
<<interface>>
<<interface>>
Comparator
BasicSequence
isEmpty()
size()
isLessThan(x,y)
isEqualTo(x,y)
etc.
<<interface>>
<<interface>>
PriorityQueue
insertItem(k,e)
removeMin()
minKey()
minElement()
SortedPQ
Berner Fachhochschule
Technik und Informatik
Position
etc.
SinglyLinkedList
first: ListNode
n: Integer
C: Comparator
<<interface>>
List
etc.
ListNode
element: Object
next: ListNode
UnsortedPQ
C: Comparator
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 10
Sorting with a Priority Queue
I
We can use a priority queue to sort a collection of comparable
elements
Ý Insert the elements one by one with a series of
insertItem(e,e) operations
Ý Remove the elements in sorted order with a series of
removeMin() operations
I
Depending on how the priority queue is implemented (sorted
or unsorted), this leads to two well-known sorting algorithms
Ý Selection-Sort
Ý Insertion-Sort
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 11
PQ-Sort Algorithm
Solution in pseudo-code:
Algorithm PQ−Sort(S,C )
P ← new priority queue with comparator C
while not S.isEmpty() do
e ← S.removeElement(S.first())
P.insertItem(e,e)
while not P.isEmpty() do
e ← P.removeMin()
S.insertLast(e)
return S
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 12
Selection-Sort
I
Selection-sort is the variation of PQ-sort where the priority
queue is implemented with an unsorted sequence
I
The main task is to select the elements from the unsorted
sequence in the right order
Running time analysis:
I
Ý Inserting the elements into the priority queue with n
insertItem(e,e) operations runs in O(n) time
Ý Removing the elements in sorted order from the priority queue
with n removeMin() operations takes time proportional to
n + . . . + 2 + 1, and thus runs in O(n2 ) time
I
Selection-sort runs in O(n2 ) time
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 13
Insertion-Sort
I
Selection-sort is the variation of PQ-sort where the priority
queue is implemented with an sorted sequence
I
The main task is to insert the elements at the right place
Running time analysis:
I
Ý Inserting the elements into the priority queue with n
insertItem(e,e) operations takes time proportional to
1 + 2 + . . . + n, and thus runs in O(n2 ) time
Ý Removing the elements in sorted order from the priority queue
with n removeMin() operations runs in O(n) time
I
In general (worst case), insertion-sort runs in O(n2 ) time
I
If the input sequence is in reverse order (best case), it runs in
O(n) time
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Priority Queues
Page 14
In-Place Sorting
I
I
I
Instead of using external data
structures, we can implement
insertion- and selection-sort
in-place
A portion of the input sequence
itself serves as priority queue
For in-place insertion-sort we
Ý keep the initial portion of the
sequence sorted
Ý use swapElements(p,q) instead
of modifying the sequence
Berner Fachhochschule
Technik und Informatik
5
4
2
3
1
5
4
2
3
1
4
5
2
3
1
2
4
5
3
1
2
3
4
5
1
1
2
3
4
5
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 15
Outline
Priority Queues
Heaps
Heap-Based Priority Queues
Bottom-Up Heap Construction
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 16
The Heap ADT
I
A heap is a specialized binary tree storing keys at its nodes
(positions) and satisfying the following properties:
Ý Heap-Order: key (p) key (parent(p)), for every node p other
than the root
Ý Completeness: let h be the height of the tree
1) there are 2i nodes of depth i for i = 0, . . . , h − 1
2) at depth h, nodes are filled up from the left
I
I
The last node of a heap is the rightmost node of depth h
Heap operations:
Ý insertKey(k): inserts a key k into the heap
Ý removeMin(): removes smallest key and returns it
Ý minKey(): returns the smallest key (no removal)
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Heaps
Priority Queues and Heaps
Page 17
Heap Example
Depth
0
2
5
7
9
13
10
17 15 12
I
1
6
9
11
8
12
2
8
14
9
Last node
11
3
4
A heap storing n keys has height O(log n)
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 18
Insertion to a Heap
2
I
The insertion of a key k
consists of 3 steps:
5
9
Ý Add a new node q (the new
last node)
Ý Store k at q
Ý Restore the heap order
property (discussed next)
7
2
5
9
Berner Fachhochschule
Technik und Informatik
6
q
6
7
1
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 19
Upheap
I
Algorithm upheap restores the
heap-order property
Ý Swap k along an upward path
from the insertion node
Ý Stop when k reaches the root
or a node whose parent has a
key smaller than or equal to k
Ý Since the height of the heap
is O(log n), upheap runs in
O(log n) time
I
2
5
9
1
7
6
1
5
9
2
7
6
insertKey(k) runs in O(log n)
time
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 20
Removal from a Heap
2
I
The removal algorithm consists
of 4 steps:
Ý Return the key of the root
Ý Replace the root key with the
key k of the last node
Ý Delete the last node
Ý Restore the heap order
property (discussed next)
Berner Fachhochschule
Technik und Informatik
5
9
6
Last node
7
7
5
6
9
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heaps
Page 21
Downheap
I
Algorithm downheap restores the
heap-order property
Ý Swap k along an downward
path from the root (always with
the smaller key of its children)
Ý Stop when k reaches a leaf or a
node whose children have keys
greater than or equal to k
Ý Since the height of the heap is
O(log n), downheap runs in
O(log n) time
I
7
5
6
9
5
7
6
9
removeMin() runs in O(log n)
time
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Heaps
Priority Queues and Heaps
Page 22
Finding the Insertion Node
I
Starting from the last node, the insertion node can be found
by traversing a path of O(log n) nodes
Ý
Ý
Ý
Ý
If the last node is a left child, return the parent node
While the current node is a right child, go to the parent node
If the current node is a left child, go to the right child
While the current node is internal, go to the left child
2
5
6
9
13
7
10
9
Berner Fachhochschule
Technik und Informatik
11
8
Last node
8
Insertion node
Rolf Haenni
Algorithms and Data Structures
Heaps
Priority Queues and Heaps
Page 23
Array-Based Heap Implementation
I
We can represent a heap with n keys directly by means of an
array of length N > n
Ý By-pass the binary tree ADT
Ý No explicit position ADT needed
I
Idea similar to array-based implementation of binary trees
Ý
Ý
Ý
Ý
I
The
The
The
The
array cell at index 0 is unused
root of the heap is at index 1
left child of the node at index i is at index 2i
right child of the node at index i is at index 2i + 1
The insertion operations corresponds to inserting at index
n + 1 and thus runs in O(1) time
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Heaps
Priority Queues and Heaps
Page 24
Array-Based Heap: Example
2
5
6
9
13
7
10
11
9
8
Last node
8
2 5 6 9 7 11 8 13 10 9 8
0
1
2
3
Berner Fachhochschule
Technik und Informatik
4
5
6
7
8
9
10 11 12 13 14
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heap-Based Priority Queues
Page 25
Outline
Priority Queues
Heaps
Heap-Based Priority Queues
Bottom-Up Heap Construction
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heap-Based Priority Queues
Page 26
Heap-Based Priority Queues
I
We can use a heap to implement a priority queue by storing a
pair item = (key , element) at each node of the heap
I
Running times for different implementations
Operation
size, isEmpty
minElement, minKey
insertItem
removeMin
PQ-Sort
I
Unsorted
Sequence
1
n
1
n
n2
Sorted
Sequence
1
1
n
1
n2
Heap
1
1
log n
log n
n· log n
In the long run, the heap-based implementation beats any
sequence-based implementation
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Heap-Based Priority Queues
Priority Queues and Heaps
Page 27
UML Diagram: Composition
<<interface>>
<<interface>>
<<interface>>
Tree
Comparator
BasicSequence
isLessThan(x,y)
isEqualTo(x,y)
etc.
etc.
isEmpty()
size()
<<interface>>
<<interface>>
<<interface>>
BinaryTree
Heap
PriorityQueue
etc.
ArrayBinaryTree
T: Array
n: Integer
insertItem(k,e)
removeMin()
minKey()
minElement()
insertKey(k)
removeMin()
minKey()
BinaryTreeHeap
T: ArrayBinaryTree
C: Comparator
Berner Fachhochschule
Technik und Informatik
ArrayHeap
A: Array
C: Comparator
HeapPQ
H: ArrayHeap
C: Comparator
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Heap-Based Priority Queues
Page 28
Heap-Sort
I
I
Heap-sort is the variation of PQ-sort where the priority queue
is implemented with a heap
Running time analysis:
Ý Inserting the elements into the priority queue with n
insertItem(e,e) operations runs in O(n log n) time
Ý Removing the elements in sorted order from the priority queue
with n removeMin() operations runs in O(n log n) time
I
In general (worst case), heap-sort runs in O(n log n) time
I
For large n, heap-sort is much faster than quadratic sorting
algorithms such as insertion-sort or selection-sort
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Heap-Based Priority Queues
Priority Queues and Heaps
Page 29
In-Place Heap-Sort
I
To implement Heap-Sort in-place, use the front of the original
array to implement the heap
Ý Step 1: Use the reverse comparator (e.g. instead of ) to
build up the heap (by swapping elements)
Ý Step 2: Iteratively remove the maximum element from the
heap and insert it in front of the sequence
Phase 1:
4 7 2 5 3
4 7 2 5 3
7 4 2 5 3
7 4 2 5 3
7 5 2 4 3
4
7
7
7
4
4
7
5
4
Phase 2:
2
2
7 5 2 4 3
4
4
2
3
5 4 2 3 7
Berner Fachhochschule
Technik und Informatik
7
3
5
2
4
5
3
5
7 5 2 4 3
4
2
3
3
2
4 3 2 5 7
2
3 2 4 5 7
2
2 3 4 5 7
2 3 4 5 7
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Bottom-Up Heap Construction
Page 30
Outline
Priority Queues
Heaps
Heap-Based Priority Queues
Bottom-Up Heap Construction
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Bottom-Up Heap Construction
Priority Queues and Heaps
Page 31
Bottom-up Heap Construction
I
We can construct a heap storing n = 2h − 1 keys using a
recursive bottom-up construction with h − 1 phases
I
In phase i, pairs of heaps with 2i − 1 keys are merged into
heaps with 2i+1 − 1 keys
Merging two heaps (and a new key k):
I
Ý Create a new heap with the root node storing k and with the
two heaps as subtrees
Ý Perform downheap to restore the heap-order property
2i+1–1
i
2 –1
i
2 –1
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Bottom-Up Heap Construction
Page 32
Recursive Algorithm in Pseudo-Code
Algorithm bottomUpHeap(S) // sequence of keys of length 2ˆh−1
if S. isEmpty() then
return new heap
else
k ← S. elemAtRank(0)
S. removeAtRank(0)
H1 ←bottomUpHeap(firstHalf(S)) // 1st recursive call
H2 ←bottomUpHeap(secondHalf(S)) // 2nd recursive call
return mergeHeaps(k, H1 , H2 ) // merge the results
In the general case, i.e. if the sequence is not of size n = 2h − 1,
we can adapt the splitting of the sequence accordingly
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures
Bottom-Up Heap Construction
Priority Queues and Heaps
Page 33
Example: Phase 1
16
15
4
25
16
6
5
15
4
15
16
12
Berner Fachhochschule
Technik und Informatik
5
23
11
12
6
4
25
9
27
9
23
6
12
11
20
20
20
9
23
27
Rolf Haenni
Algorithms and Data Structures
Bottom-Up Heap Construction
Priority Queues and Heaps
Page 34
Example: Phase 2
15
16
4
25
5
6
12
11
20
9
7
4
25
5
6
12
11
20
9
4
5
25
Berner Fachhochschule
Technik und Informatik
23
27
6
15
16
27
8
15
16
23
7
8
12
11
20
9
23
27
Rolf Haenni
Algorithms and Data Structures
Bottom-Up Heap Construction
Priority Queues and Heaps
Page 35
Example: Phase 3
4
6
15
16
5
25
7
8
12
11
20
9
23
27
10
4
6
15
16
5
25
7
8
12
11
20
9
23
27
4
5
6
15
16
7
25
Berner Fachhochschule
Technik und Informatik
10
8
12
11
20
9
23
27
Rolf Haenni
Algorithms and Data Structures
Priority Queues and Heaps
Bottom-Up Heap Construction
Page 36
Complexity Analysis
I
The recursive BottomUpHeap algorithm decomposes a
problem of size n into two problems of size n−1
2
Ý Rule D2 with q = r = 2 (see Topic 2, Page 31)
I
The running time f (n) of each recursive step is O(log n)
Ý with an array-based sequence implementation, firstHalf()
and secondHalf() run in O(1) time
Ý the necessary downheap in mergeHeaps runs in O(log n) time
I
Apply the second column of D2, i.e. bottom-up heap
construction runs in O(n)
I
Speeds up the heap construction (Phase 1) of the Heap-Sort
algorithm, but not Phase 2
Berner Fachhochschule
Technik und Informatik
Rolf Haenni
Algorithms and Data Structures