Download ・ P Q RIORITY

Priority queue data type
A min-oriented priority queue supports the following core operations:
P RIORITY Q UEUES
・MAKE-HEAP(): create an empty heap.
・INSERT(H, x): insert an element x into the heap.
・EXTRACT-MIN(H): remove and return an element with the smallest key.
・DECREASE-KEY(H, x, k): decrease the key of element x to k.
‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
The following operations are also useful:
・IS-EMPTY(H): is the heap empty?
・FIND-MIN(H): return an element with smallest key.
・DELETE(H, x): delete element x from the heap.
・UNION(H1, H2): replace heaps H1 and H2 with their union.
‣ Fibonacci heaps
Lecture slides by Kevin Wayne
Copyright © 2005 Pearson-Addison Wesley
Note. Each element contains a key (duplicate keys are permitted)
http://www.cs.princeton.edu/~wayne/kleinberg-tardos
from a totally-ordered universe.
Last updated on Apr 10, 2013 5:50 AM
2
Priority queue applications
Applications.
P RIORITY Q UEUES
・A* search.
・Heapsort.
・Online median.
・Huffman encoding.
・Prim's MST algorithm.
・Discrete event-driven simulation.
・Network bandwidth management.
・Dijkstra's shortest-paths algorithm.
・...
‣ binary heaps
‣ d-ary heaps
‣ binomial heaps
Algorithms
F O U R T H
E D I T I O N
R OBERT S EDGEWICK | K EVIN W AYNE
SECTION 2.4
http://younginc.site11.com/source/5895/fos0092.html
3
‣ Fibonacci heaps
Complete binary tree
A complete binary tree in nature
Binary tree. Empty or node with links to two disjoint binary trees.
Complete tree. Perfectly balanced, except for bottom level.
complete tree with n = 16 nodes (height = 4)
Property. Height of complete binary tree with n nodes is ⎣log2 n⎦.
Pf. Height increases (by 1) only when n is a power of 2. ▪
5
6
Binary heap
Explicit binary heap
Binary heap. Heap-ordered complete binary tree.
Pointer representation. Each node has a pointer to parent and two children.
・Maintain number of elements n.
・Maintain pointer to root node.
・Can find pointer to last node or next node in O(log n) time.
Heap-ordered. For each child, the key in child ≤ key in parent.
root
6
6
parent
10
child
21
8
12
18
17
child
11
10
25
8
12
19
21
18
17
25
19
last
7
11
next
8
Implicit binary heap
Binary heap demo
Array representation. Indices start at 1.
・Take nodes in level order.
・Parent of node at k is at ⎣k / 2⎦.
・Children of node at k are at 2k and 2k + 1.
heap ordered
1
6
6
2
3
10
8
10
4
5
6
7
12
18
11
25
8
9
10
21
17
19
12
18
21
1
2
3
4
5
6
7
8
9
10
6
10
8
12
18
11
25
21
17
19
11
12
13
14
15
8
17
11
25
19
16
9
10
Binary heap: insert
Binary heap: extract the minimum
Insert. Add element in new node at end; repeatedly exchange new element
Extract min. Exchange element in root node with last node; repeatedly
with element in its parent until heap order is restored.
exchange element in root with its smaller child until heap order is restored.
6
10
8
12
21
18
17
19
element to
remove
6
11
7
25
12
add key to heap
(violates heap order)
7
8
21
10
17
11
18
19
25
exchange
with root
sink down
6
violates
heap order
18
7
8
7
12
10
11
17
19
18
8
25
12
21
7
10
8
swim up
21
11
10
17
19
11
6
25
remove
from heap
12
21
18
17
19
11
25
6
12
Binary heap: decrease key
Binary heap: analysis
Decrease key. Given a handle to node, repeatedly exchange element with
Theorem. In an implicit binary heap, any sequence of m INSERT, EXTRACT-MIN,
its parent until heap order is restored.
and DECREASE-KEY operations with n INSERT operations takes O(m log n) time.
Pf.
・Each heap op touches nodes only on a path from the root to a leaf;
the height of the tree is at most log2 n.
decrease key of node x to 11
・The total cost of expanding and contracting the arrays is O(n).
6
10
8
▪
Theorem. In an explicit binary heap with n nodes, the operations INSERT,
DECREASE-KEY, and EXTRACT-MIN take O(log n) time in the worst case.
12
21
18
17
11
25
19
x
13
14
Binary heap: find-min
Binary heap: delete
Find the minimum. Return element in the root node.
Delete. Given a handle to a node, exchange element in node with last node;
either swim down or sink up the node until heap order is restored.
delete node x or y
root
6
6
7
x
10
12
8
11
25
7
10
12
8
11
25
y
21
17
9
21
17
9
last
15
16
Binary heap: union
Binary heap: heapify
Union. Given two binary heaps H1 and H2, merge into a single binary heap.
Heapify. Given n elements, construct a binary heap containing them.
Observation. Can do in O(n log n) time by inserting each element.
Observation. No easy solution: Ω(n) time apparently required.
Bottom-up method. For i = n to 1, repeatedly exchange the element in node i
with its smaller child until subtree rooted at i is heap-ordered.
H1
H2
7
10
1
2
12
8
11
17
3
12
9
25
4
21
8
5
7
6
22
7
3
26
9
8
14
9
10
11
11 22
15
8
12
9
7
22
3
26
14
11
15
22
1
2
3
4
5
6
7
8
9
10
11
17
Binary heap: heapify
18
Priority queues performance cost summary
Theorem. Given n elements, can construct a binary heap containing those n
elements in O(n) time.
Pf.
・There are at most ⎡n / 2h+1⎤ nodes of height h.
・The amount of work to sink a node is proportional to its height h.
work is bounded by:
・Thus, thelogtotal
n
log n
2
h=0
linked list
binary heap
MAKE-HEAP
O(1)
O(1)
ISEMPTY
O(1)
O(1)
INSERT
O(1)
O(log n)
EXTRACT-MIN
O(n)
O(log n)
DECREASE-KEY
O(1)
O(log n)
DELETE
O(1)
O(log n)
UNION
O(1)
O(n)
FIND-MIN
O(n)
O(1)
2
n / 2h+1 h
log
h=0
2n
2
operation
n / 2h+1 h
n h / 2h
log
h=0
2n
2
n h / 2h
nh=0 h / 2h
h=1
= n2n ▪h / 2h
h=1
k
i=1
i
= 2
2i
k
2k
1
2k
1
2
= 2n
Corollary. Given two binary heaps H1 and H2 containing n elements in total,
can implement UNION in O(n) time.
19
20
Priority queues performance cost summary
Q. Reanalyze so that EXTRACT-MIN and DELETE take O(1) amortized time?
operation
linked list
binary heap
binary heap
MAKE-HEAP
O(1)
O(1)
O(1)
ISEMPTY
O(1)
O(1)
O(1)
INSERT
O(1)
O(log n)
O(log n)
EXTRACT-MIN
O(n)
O(log n)
O(1) †
DECREASE-KEY
O(1)
O(log n)
O(log n)
DELETE
O(1)
O(log n)
UNION
O(1)
O(n)
O(n)
FIND-MIN
O(n)
O(1)
O(1)
O(1)
P RIORITY Q UEUES
‣ binary heaps
†
‣ d-ary heaps
‣ binomial heaps
Algorithms
F O U R T H
‣ Fibonacci heaps
E D I T I O N
R OBERT S EDGEWICK | K EVIN W AYNE
SECTION 2.4
†
† amortized
21
Complete d-ary tree
Multiway heap: insert
Binary tree. Empty or node with links to d disjoint d-ary trees.
Insert. Add node at end; repeatedly exchange element in child with element
in parent until heap order is restored.
Complete tree. Perfectly balanced, except for bottom level.
Running time. Proportional to height = O(logd n).
4
30
80
32
20
10
34
55
22
34
46
20
40
Fact. The height of a complete d-ary tree with n nodes is ≤ ⎡logd n⎤.
82
23
90
24
Multiway heap: extract the minimum
Multiway heap: decrease key
Extract min. Exchange root node with last node; repeatedly exchange
Decrease key. Given a handle to an element x, repeatedly exchange it with
element in parent with element in largest child until heap order is restored.
its parent until heap order is restored.
Running time. Proportional to d ⨉ height = O(d logd n).
Running time. Proportional to height = O(logd n).
4
30
80
82
32
4
20
10
34
55
22
46
34
20
30
40
80
90
82
32
20
10
34
55
22
34
46
20
90
25
26
Priority queues performance cost summary
P RIORITY Q UEUES
operation
linked list
binary heap
d-ary heap
MAKE-HEAP
O(1)
O(1)
O(1)
‣ d-ary heaps
ISEMPTY
O(1)
O(1)
O(1)
‣ binomial heaps
INSERT
O(1)
O(log n)
O(logd n)
‣ Fibonacci heaps
EXTRACT-MIN
O(n)
O(log n)
O(d logd n)
DECREASE-KEY
O(1)
O(log n)
O(logd n)
DELETE
O(1)
O(log n)
O(d logd n)
UNION
O(1)
O(n)
O(n)
FIND-MIN
O(n)
O(1)
O(1)
‣ binary heaps
CHAPTER 6 (2ND EDITION)
27
40
Priority queues performance cost summary
operation
linked list
binary heap
Binomial heaps
d-ary heap
Programming
Techniques
S.L. Graham, R.L. Rivest
Editors
MAKE-HEAP
O(1)
O(1)
O(1)
ISEMPTY
O(1)
O(1)
O(1)
INSERT
O(1)
O(log n)
O(logd n)
EXTRACT-MIN
O(n)
O(log n)
O(d logd n)
DECREASE-KEY
O(1)
O(log n)
O(logd n)
DELETE
O(1)
O(log n)
O(d logd n)
UNION
O(1)
O(n)
O(n)
A data structure is described which can be used for
representing a collection of priority queues. The
primitive operations are insertion, deletion, union,
update, and search for an item of earliest priority.
Key Words and Phrases: data structures,
implementation of set operations, priority queues,
mergeable heaps, binary trees
CR Categories: 4.34, 5.24, 5.25, 5.32, 8.1
FIND-MIN
O(n)
O(1)
O(1)
I. Introduction
A Data Structure for
Manipulating Priority
Queues
Jean Vuillemin
Universit6 de Paris-Sud
In order to design correct and efficient algorithms for
solving a specific problem, it is often helpful to describe
our first approach to a solution in a language close to
that in which the problem is formulated. One such
language is that of set theory, augmented by primitive
set manipulation operations. Once the algorithm is outlined in terms of these set operations, one can then look
for data structures most suitable for representing each of
the sets involved. This choice depends only upon the
collection of primitive operations required for each set.
It is thus important to establish a good catalogue of such
data structures. A summary of the state of the art on this
question can be found in [2]. In this paper, we add to
this catalogue a data structure which allows efficient
manipulation of priority queues.
Goal. O(log n) INSERT, DECREASE-KEY, EXTRACT-MIN, and UNION.
mergeable heap
29
Binomial tree
Binomial tree properties
Def. A binomial tree of order k is defined recursively:
Properties. Given anGeneral
order
k binomial
tree
Bk, of all
permission
to make fair use in teaching
or research
・Order 0:
・Order k:
or part of this material is granted to individual readers and to nonprofit
libraries acting for them providedthat ACM's copyrightnotice is given
and that referenceis made to the publication, to its date of issue, and
to the fact that reprinting privilegeswere granted by permission of the
Association for Computing Machinery. To otherwise reprint a figure,
table, other substantial excerpt, or the entire work requires specific
permission as does republication,or systematicor multiple reproduction.
Author's address: Laboratoire de Recherche en Informatique,
Brit. 490, Universit6 de Paris-Sud, Centre d'Orsay 91405--Orsay,
France.
© 1978ACM 0001-0782/78/0400-0309$00.75
・Its height is k.
・It has !2k "nodes.
・It has ki nodes at depth i.
・The degree of its root is k.
・Deleting its root yields k binomial trees Bk–1, …, B0.
single node.
one binomial tree of order k – 1 linked to another of order k – 1.
B0
A priority queue is a set; each element of such a set
has a name, which is used to uniquely identify the
element, and a label or priority drawn from a totally
ordered set. Elements of the priority queue can be
thought of as awaiting service, where the item with the ~
smallest label is always to be served next. Ordinary
stacks and queues are special cases of priority queues.
A variety of applications directly require using priority queues: job scheduling, discrete simulation languages
where labels represent the time at which events are to
occur, as well as various sorting problems. These are
discussed, for example, in [2, 3, 5, 11, 15, 17, 24]. Priority
queues also play a central role in several good algorithms,
such as optimal code constructions, Chartre's prime
number generator, and Brown's power series multiplication (see [16] and [17]); applications have also been
found in numerical analysis algorithms [10, 17, 19] and
in graph algorithms for such problems as finding shortest
paths [2, 13] and minimum cost spanning tree [2, 4, 25].
Typical applications require primitive operations
among the following five: INSERT, DELETE, MIN, UPDATE, and UNION. The operation INSERT (name, label,
Q) adds an element to queue Q, while DELETE (name)
removes the element having that name. Operation MIN
(Q) returns the name of the element in Q having the least
label, and UPDATE (name, label) changes the label of the
element named. Finally, UNION (Q1, Q2, Q3) merges into
Qa all elements of Q1 and Q2; the sets Q1 and Q2 become
empty. In what follows, we assume that names are
handled in a separate dictionary [2, 17] such as a hashtable or a balanced tree. If deletions are restricted to
elements extracted by MIN, such an auxiliary symbol
table is not needed. 1
The heap, a truly elegant data structure discovered
by J. W. Williams and R. W. Floyd, handles a sequence
of n primitives INSERT, DELETE, and MIN, and runs in
O(nlogn) elementary operations using absolutely mini-30
mal storage [17]. For applications in which UNION is
necessary, more sophisticated data structures have been
devised, such as 2-3 trees [2, 17], leftist trees [5, 17], and
binary heaps [9].
The data structure we present here handles an arbitrary sequence of n primitives, each drawn from the five
described above, in O(nlogn) machine operations and
O(n) memory cells. It also allows for an efficient treatment of a large number of updates, which is crucial in
connection with spanning tree algorithms: Our data
structure provides an implementation (described in
[25]) of the Cheriton-Tarjan-Yao [3] minimum cost spanning tree algorithm which is much more straightforward
than the original one.
The proposed data structure uses less storage than
leftist, AVL, or 2-3 trees; in addition, when the primitive
operations are carefully machine coded from the programs given in Section 4, they yield worst case running
times which compare favorably with those o f their corn-
Bk
309
We-g'gsumehere that indexing through the symbol table is done
in constant time.
Communications
of
the ACM
Bk-1
April 1978
Volume 21
Number 4
Pf. [by induction on k]
Bk-1
Bk+1
B2
B1
B0
Bk
B0
B1
B2
B3
B4
B4
31
32
Binomial heap
Binomial heap representation
Def. A binomial heap is a sequence of binomial trees such that:
Binomial trees. Represent trees using left-child, right-sibling pointers.
・
・There is either 0 or 1 binomial tree of order k.
Each tree is min-heap ordered.
Roots of trees. Connect with singly-linked list, with degrees decreasing
from left to right.
root
6
10
31
17
44
37
29
10
31
17
6
3
44
37
18
23
22
48
48
45
32
24
3
48
10
50
50
55
B1
B4
B0
50
binomial heap
31
17
44
leftist power-of-2 heap representation
33
34
Binomial heap properties
Binomial heap: union
Properties. Given a binomial heap with n nodes:
Union operation. Given two binomial heaps H1 and H2, (destructively)
・The node containing the min element is a root of B0, B1, …, or Bk.
・It contains the binomial tree Bi iff bi = 1, where bk⋅ b2 b1 b0 is binary
replace with a binomial heap H that is the union of the two.
representation of n.
Warmup. Easy if H1 and H2 are both binomial trees of order k.
・Connect roots of H1 and H2.
・Choose node with smaller key to be root of H.
・It has ≤ ⎣log2 n⎦ + 1 binomial trees.
・Its height ≤ ⎣log2 n⎦.
8
45
55
30
23
32
24
18
37
29
le
30
6
nt
re
pa
ht
rig
29
18
ft
8
3
22
29
48
31
10
6
3
44
37
6
18
8
n = 19
# trees = 3
height = 4
binary = 10011
17
45
50
B4
B1
55
B0
35
30
23
32
24
22
48
29
10
31
17
44
50
H1
H2
36
12
18
8
30
23
22
48
29
10
31
17
6
3
44
37
15
45
+
32
24
7
18
8
45
28
33
23
32
24
22
48
10
31
17
3
44
37
12
50
55
30
29
6
25
+
15
7
33
25
12
50
28
55
41
41
37
7
18
3
12
37
18
38
3
28
25
15
7
33
25
37
7
3
12
37
18
25
41
8
30
23
22
48
29
10
31
17
6
3
44
37
15
45
+
55
32
24
7
18
8
45
33
23
32
24
22
48
10
31
17
3
44
37
12
50
28
30
29
6
25
+
41
55
15
7
33
25
18
12
50
28
41
12
12
18
18
39
40
3
28
15
7
33
25
7
37
3
12
37
18
3
28
25
41
8
30
23
22
48
29
10
31
17
+
32
24
7
33
25
7
37
3
44
37
7
18
8
37
18
25
45
28
55
33
30
23
32
24
22
48
29
10
31
17
6
3
44
37
12
50
25
+
15
7
33
25
15
7
33
25
18
12
50
28
55
41
28
12
41
6
15
45
15
3
41
3
12
37
18
6
8
45
41
41
30
23
32
24
22
48
29
10
31
17
44
28
50
15
7
33
25
3
12
37
18
41
42
55
Binomial heap: union
Union operation. Given two binomial heaps H1 and H2, (destructively)
replace with a binomial heap H that is the union of the two.
8
29
10
31
17
6
3
44
37
18
Solution. Analogous to binary addition.
Running time. O(log n).
45
+
30
23
32
24
22
48
15
7
33
25
12
Pf. Proportional to number of trees in root lists ≤ 2 ( ⎣log2 n⎦ + 1). ▪
50
28
55
41
19 + 7 = 26
+
1
1
1
1
0
0
1
1
0
0
1
1
1
1
1
0
1
0
19 + 7 = 26
43
+
1
1
1
1
0
0
1
1
0
0
1
1
1
1
1
0
1
0
44
Binomial heap: extract the minimum
Binomial heap: extract the minimum
Extract-min. Delete the node with minimum key in binomial heap H.
Extract-min. Delete the node with minimum key in binomial heap H.
・
・Find root x with min key in root list of H, and delete.
・H' ← broken binomial trees.
・H ← UNION(H', H).
Find root x with min key in root list of H, and delete.
Running time. O(log n).
8
45
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
6
8
H
45
50
30
23
32
24
22
48
29
10
31
17
44
18
37
H
50
H'
55
55
45
46
Binomial heap: decrease key
Binomial heap: delete
Decrease key. Given a handle to an element x in H, decrease its key to k.
Delete. Given a handle to an element x in a binomial heap, delete it.
・DECREASE-KEY(H, x, -∞).
・DELETE-MIN(H).
・Suppose x is in binomial tree Bk.
・Repeatedly exchange x with its parent until heap order is restored.
Running time. O(log n).
Running time. O(log n).
8
x
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
8
H
45
50
55
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
H
50
55
47
48
Binomial heap: insert
Binomial heap: sequence of insertions
Insert. Given a binomial heap H, insert an element x.
Insert. How much work to insert a new node x ?
・If n =
・If n =
・If n =
・If n =
・
・H' ← INSERT(H', x).
・H ← UNION(H', H).
H' ← MAKE-HEAP( ).
.......0, then only 1 credit.
6
44
37
x
.......01, then only 2 credits.
.......011, then only 3 credits.
.......0111, then only 4 credits.
48
29
10
31
17
50
Running time. O(log n).
8
3
29
10
3
6
44
37
18
x
Observation. Inserting one element can take Ω(log n) time.
H'
if n = 11...111
Theorem. Starting from an empty binomial heap, a sequence of n
45
30
23
32
24
22
48
31
17
consecutive INSERT operations takes O(n) time.
H
Pf. (n / 2) (1) + (n / 4)(2) + (n / 8)(3) + … ≤ 2 n. ▪
50
k
i=1
55
i
= 2
2i
k
2k
1
2k
1
2
49
Binomial heap: amortized analysis
Binomial heap: amortized analysis
Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).
Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
50
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
Case 1. [INSERT]
Case 2. [ DECREASE-KEY ]
・Actual cost ci = number of trees merged + 1.
・∆Φ = Φ(Hi) – Φ(Hi–1) = number of trees merged – 1.
・Amortized cost = ĉi = ci + Φ(Hi) – Φ(Hi–1) = 2.
・Actual cost ci = O(log n).
・∆Φ = Φ(Hi) – Φ(Hi–1) = 0.
・Amortized cost = ĉi = ci =
51
O(log n).
52
Binomial heap: amortized analysis
Priority queues performance cost summary
Theorem. In a binomial heap, the amortized cost of INSERT is O(1) and the
worst-case cost of EXTRACT-MIN and DECREASE-KEY is O(log n).
Pf. Define potential function Φ(Hi) = trees(Hi) = # trees in binomial heap Hi.
・Φ(H0) = 0.
・Φ(Hi) ≥ 0 for each binomial heap Hi.
Case 3. [ EXTRACT-MIN or DELETE ]
・Actual cost ci = O(log n).
・∆Φ = Φ(Hi) – Φ(Hi–1) ≤ ⎣log2 n⎦.
・Amortized cost = ĉi = ci + Φ(Hi)
– Φ(Hi–1) = O(log n). ▪
operation
linked list
binary heap
binomial heap
binomial heap
MAKE-HEAP
O(1)
O(1)
O(1)
O(1)
ISEMPTY
O(1)
O(1)
O(1)
O(1)
INSERT
O(1)
O(log n)
O(log n)
O(1) †
EXTRACT-MIN
O(n)
O(log n)
O(log n)
O(log n)
DECREASE-KEY
O(1)
O(log n)
O(log n)
O(log n)
DELETE
O(1)
O(log n)
O(log n)
O(log n)
UNION
O(1)
O(n)
O(log n)
FIND-MIN
O(n)
O(1)
O(log n)
O(1)
homework
†
O(1)
† amortized
Hopeless challenge. O(1) INSERT, DECREASE-KEY and EXTRACT-MIN. Why?
Challenge. O(1) INSERT and DECREASE-KEY, O(log n) EXTRACT-MIN.
53
54