Download Heaps and Priority Queues

Priority Queues
T. M. Murali
August 31, 2009
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Motivation: Sort a List of Numbers
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
T. M. Murali
n
such that
i
August 31, 2009
CS 4104: Priority Queues
Motivation: Sort a List of Numbers
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
n
such that
i
I Possible algorithm:
I
I
T. M. Murali
Store all the numbers in a data structure D .
Repeatedly nd the smallest number in D , output it, and remove it.
August 31, 2009
CS 4104: Priority Queues
Motivation: Sort a List of Numbers
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
n
such that
i
I Possible algorithm:
I
I
Store all the numbers in a data structure D .
Repeatedly nd the smallest number in D , output it, and remove it.
O (n log n) running time, each nd minimum step must take
O (log n) time.
I To get
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Candidate Data Structures for Sorting
Data structure must support insertion, nding minimum, and
deleting minimum.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Candidate Data Structures for Sorting
Data structure must support insertion, nding minimum, and
deleting minimum.
List
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Candidate Data Structures for Sorting
Data structure must support insertion, nding minimum, and
deleting minimum.
List Insertion and deletion take
scanning the list and takes
T. M. Murali
O (1) time but nding minimum requires
Ω(n) time.
August 31, 2009
CS 4104: Priority Queues
Candidate Data Structures for Sorting
Data structure must support insertion, nding minimum, and
deleting minimum.
List Insertion and deletion take
scanning the list and takes
O (1) time but nding minimum requires
Ω(n) time.
Sorted array
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Candidate Data Structures for Sorting
Data structure must support insertion, nding minimum, and
deleting minimum.
List Insertion and deletion take
scanning the list and takes
Sorted array Finding minimum takes
take
T. M. Murali
Ω(n)
O (1) time but nding minimum requires
Ω(n) time.
O (1) time but insertion and deletion can
time in the worst case.
August 31, 2009
CS 4104: Priority Queues
Priority Queue
I Store a set
S
of elements, where each element
I Smaller key values
≡
v
has a priority value
key(v ).
higher priorities.
I Operations supported: nd the element with smallest key, remove the
smallest element, update the key of an element, insert an element, delete an
element.
I Key update and element deletion require knowledge of the position of the
element in the priority queue.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Heaps
I Combine benets of both lists and sorted arrays.
I Conceptually, a heap is a balanced binary tree.
I
Heap order:
satises
T. M. Murali
For every element
key(w ) ≤ key(v ).
v
at a node
August 31, 2009
i , the element w
at
i 's parent
CS 4104: Priority Queues
Heaps
I Combine benets of both lists and sorted arrays.
I Conceptually, a heap is a balanced binary tree.
I
Heap order:
satises
For every element
key(w ) ≤ key(v ).
v
at a node
i , the element w
at
i 's parent
I We can implement a heap in a pointer-based data structure.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Heaps
I Combine benets of both lists and sorted arrays.
I Conceptually, a heap is a balanced binary tree.
I
Heap order:
satises
For every element
key(w ) ≤ key(v ).
v
at a node
i , the element w
at
i 's parent
I We can implement a heap in a pointer-based data structure.
I Alternatively, assume maximum number
N of elements is known in advance.
I Store nodes of the heap in an array.
I
I
I
T. M. Murali
Node at index i has children at indices 2i and 2i + 1 and parent at index bi /2c.
Index 1 is the root.
How do you know that a node at index i is a leaf?
August 31, 2009
CS 4104: Priority Queues
Heaps
I Combine benets of both lists and sorted arrays.
I Conceptually, a heap is a balanced binary tree.
I
Heap order:
satises
For every element
key(w ) ≤ key(v ).
v
at a node
i , the element w
at
i 's parent
I We can implement a heap in a pointer-based data structure.
I Alternatively, assume maximum number
N of elements is known in advance.
I Store nodes of the heap in an array.
I
I
I
T. M. Murali
Node at index i has children at indices 2i and 2i + 1 and parent at index bi /2c.
Index 1 is the root.
How do you know that a node at index i is a leaf? If 2i > n, the number of
elements in the heap.
August 31, 2009
CS 4104: Priority Queues
Example of a Heap
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Inserting an Element:
1. Insert new element at index
2. Fix heap order using
T. M. Murali
Heapify-up
n + 1.
Heapify-up(H , n + 1).
August 31, 2009
CS 4104: Priority Queues
Example of
T. M. Murali
Heapify-up
August 31, 2009
CS 4104: Priority Queues
Correctness of
T. M. Murali
August 31, 2009
Heapify-up
CS 4104: Priority Queues
Correctness of
I
I
Heapify-up
H is almost a heap with key of H [i ] too small if there is a value
α ≥ key(H [i ]) such that increasing key(H [i ]) to α makes H a heap.
Prove by induction on i .
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
I
I
Heapify-up
H is almost a heap with key of H [i ] too small if there is a value
α ≥ key(H [i ]) such that increasing key(H [i ]) to α makes H a heap.
Prove by induction on i .
Proof base case: i = 1.
Proof inductive step: If H is almost a heap with key of H [i ] too small, after
Heapify-up(H , i ), H is a heap or almost a heap with the key of H [j ] too
small.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
I
I
Heapify-up
H is almost a heap with key of H [i ] too small if there is a value
α ≥ key(H [i ]) such that increasing key(H [i ]) to α makes H a heap.
Prove by induction on i .
Proof base case: i = 1.
Proof inductive step: If H is almost a heap with key of H [i ] too small, after
Heapify-up(H , i ), H is a heap or almost a heap with the key of H [j ] too
small.
I Running time is
T. M. Murali
O (log i ).
August 31, 2009
CS 4104: Priority Queues
Deleting an Element:
Heapify-down
H has n + 1 elements.
Delete element at H [i ] by moving element at H [n + 1] to H [i ].
If element at H [i ] is too small, x heap order using Heapify-up(H , i ).
If element at H [i ] is too large, x heap order using Heapify-down(H , i ).
I Suppose
1.
2.
3.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Example of
T. M. Murali
Heapify-down
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
Heapify-down
H is almost a heap with key of H [i ] too big if there is a value α ≤ key(H [i ])
such that decreasing key(H [i ]) to α makes H a heap.
Proof by reverse induction on i .
I Proof base case:
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
I
Heapify-down
H is almost a heap with key of H [i ] too big if there is a value α ≤ key(H [i ])
such that decreasing key(H [i ]) to α makes H a heap.
Proof by reverse induction on i .
Proof base case: 2i > n.
I Proof inductive step:
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
I
I
Heapify-down
H is almost a heap with key of H [i ] too big if there is a value α ≤ key(H [i ])
such that decreasing key(H [i ]) to α makes H a heap.
Proof by reverse induction on i .
Proof base case: 2i > n.
Proof inductive step: after Heapify-down(H , i ), H is a heap or almost a
heap with the key of H [j ] too big.
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Correctness of
I
I
I
I
I
Heapify-down
H is almost a heap with key of H [i ] too big if there is a value α ≤ key(H [i ])
such that decreasing key(H [i ]) to α makes H a heap.
Proof by reverse induction on i .
Proof base case: 2i > n.
Proof inductive step: after Heapify-down(H , i ), H is a heap or almost a
heap with the key of H [j ] too big.
Running time of Heapify-down(H , i ) is O (log n).
T. M. Murali
August 31, 2009
CS 4104: Priority Queues
Sorting Numbers with the Priority Queue
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
T. M. Murali
n
such that
i
August 31, 2009
CS 4104: Priority Queues
Sorting Numbers with the Priority Queue
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
n
such that
i
I Final algorithm:
I
I
T. M. Murali
Insert each number in a priority queue H .
Repeatedly nd the smallest number in H , output it, and delete it from
August 31, 2009
H.
CS 4104: Priority Queues
Sorting Numbers with the Priority Queue
Sort
x1 , x2 , . . . , x of integers.
SOLUTION: A permutation y1 , y2 , . . . , y of x1 , x2 , . . . , x
y ≤ y +1 , for all 1 ≤ i < n.
INSTANCE: Nonempty list
n
n
i
n
such that
i
I Final algorithm:
I
I
Insert each number in a priority queue H .
Repeatedly nd the smallest number in H , output it, and delete it from
I Each insertion and deletion takes
O (n log n).
T. M. Murali
H.
O (log n) time for a total running time of
August 31, 2009
CS 4104: Priority Queues