Download Heaps

HEAPS & PRIORITY QUEUES
Array and Tree
implementations
1
Priority queue
A stack is first in, last out
A queue is first in, first out
A priority queue is least-first-out
The “smallest” element is the first one removed
2
The definition of “smallest” is up to the programmer
(for example,
you might define it by implementing Comparator
or Comparable)
3
Priority queue
If there are several “smallest” elements,
the implementer must decide which to remove first
Remove any “smallest” element (don’t care
which)
Remove the first one added
4
A priority queue is an:
ADT where items are removed based on their priority
(highest-to-lowest), regardless of the order they ARE
inserted in the structure
5
A priority queue ADT
PriorityQueue(): Methods
void add(Comparable o): inserts o into the priority queue
Comparable removeLeast(): removes and returns the least
element
Comparable getLeast(): returns (but does not remove) the
least element
boolean isEmpty(): returns true iff empty
int size(): returns the number of elements
void clear(): discards all elements
6
Evaluating implementations
When we choose a data structure,
it is important to look at usage patterns
If we load an array once and do thousands of searches on it,
we want to make searching fast—
so we would probably sort the array
7
When we choose a data structure,
it is important to look at usage patterns
If we load a huge array and expect to do only a few searches,
we probably don’t want to spend time sorting the array
8
Evaluating implementations
For almost all uses of a queue (including a
priority queue),
we eventually remove everything that we add
9
TIMING
Hence,
when we analyze a priority queue,
neither “add” nor “remove” is more important—
we need to look at the timing for (“add + remove”)
10
Array implementations
A priority queue could be implemented as an unsorted array
(with a count of elements)
Adding an element would take O(1) time (why?)
Removing an element would take O(n) time (why?)
Hence, (adding and removing) an element takes O(n) time
This is an inefficient representation – a BigO(N)
11
Array implementations
A priority queue could be implemented as a
array (again, with a counter of elements)
sorted
Adding an element would take O(n) time (why?)
Removing an element would take O(1) time (why?)
So adding and removing) an element takes
O(n) time
Again, this is very inefficient
12
Linked list implementations
A priority queue could be implemented as an
–
unsorted linked list
Adding an element would take O(1) time (why?)
Removing an element would take (N) time (why?)
Again, inefficient!
13
SORTED LINKED LIST
A priority queue could be implemented as a
sorted linked list
Adding an element would take O(n) time (why?)
Removing an element would take O(1) time (why?)
Again, an inefficient algorithm
14
Binary tree implementations
A priority queue could be represented as a
– balanced
binary search tree
Insertion and removal could destroy the balance
We would need an algorithm to rebalance the binary tree
Good rebalancing algorithms require O(log n) time, but are
complicated
15
Heap Implementaton
The concepts of heaps is used to refer to memory in a
computer,
E.g. the part of the memory that provides dynamic
memory allocation ( get it as you need it)
“Heap” is also used as a name for a special kind of
binary tree
16
Heap is a complete binary tree
A heap is a complete binary tree
with values stored in its nodes such that
no child has a value bigger than the value of its
parent –
(though it can be equal)
17
Therefore we would like to find a better
algorithm. We want to:
Insert and Remove with a Big O of one
18
HEAPS & Priority Queues
A binary tree represented as a heap facilitates the operation
to find the maximum element:
“it is always in the root of the tree “
Because finding the node with the maximum value in a heap is a
BigO(I)
A HEAP is the most efficient structure for implementing
priority queues
19
Heaps
A heap is a certain
kind of complete
binary tree.
Root
When a complete
binary tree is built,
its first node must be
the root.
20
Heaps
Complete binary
tree.
Left child
Right child
The Second and third
Nodes are always the
left and right child
of the root.
21
Heaps
Complete binary
tree.
The next nodes
always fill the next
level from left-to-right.
22
Heaps
Complete binary tree.
23
Heaps
45
A heap is a
certain kind of
complete binary
tree.
35
27
19
23
21
22
4
Each node in a heap
contains a key that
can be compared to
other nodes' keys.
24
To implement a heap as a priority queue
A node has the heap property if it is
Equal to or greater than as its children
OR
if it is smaller than or equal to its children (since smaller
numbers represent higher priorities)
3
12
8
3
Heap: Yellow node
has the heap property
8
12
Priority queue: Yellow node has the
heap property - less than its children
25
Heaps
A heap is a
certain kind of
complete binary
tree.
45
35
This example is a
max heap.
27
19
23
21
22
4
The "heap property"
requires that each
node's key is >= to the
keys of its children
26
Priority Queues
A heap can easily implement a priority queue
but what happens with we remove the element of
highest priority?
How do we re-arrange the tree?
27
Removing from a heap
There are two things we need to consider when
rearranging:
We want to end up with a heap which means that
The tree has to be complete –
ALL HEAPS ARE COMPLETE TREES!
28
To remove an element at top of tree:
Remove the element at location 0
Move the element in the lastIndex to location 0,
decrement lastIndex
(lastIndex is at the end of the heap)
Reheap the new root node (the one now at location 0)
This is called down-heap bubbling or
percolating down
Down-heap bubbling requires O(log n) time
to remove an element
29
Removing the Top of a Heap
(max heap)
Move the last node
onto the root.
45
42
35
19
23
21
22
4
27
30
Removing the Top of a Heap
(max heap)
Move the value in
the last node into
the root.
27
42
35
23
21
22
4
19
31
Removing the Top of a Heap
27
 Move the last node onto the
root.
42
Push the out-of-place
node downward,
35
swapping with its
larger child
until the new node
reaches an
acceptable location.
23
21
22
4
19
32
Removing the Top of a Heap
42
 Move the last node onto the
root.
Push the out-of-place
node downward,
swapping with its
larger child until the
new node reaches an
acceptable location.
27
35
23
21
22
4
19
33
Removing the Top of a Heap

Move the last node onto the root.

Push the out-of-place node downward,
swapping with its larger child until the new
node reaches an acceptable location.
42
 Note that we discard half
of the tree with each
move downward
35
27
23
21
22
4
19
34
reheapification downward.
42
 We stop when:
The children all have
keys <= to the out-ofplace node, or
35
The node reaches the
27
leaf.
The process of pushing
23
21
22
4
19
the node downward is
called:
reheapification
downward.
35
Priority Queues
Another Example :
How to delete a node from a Priority Queue
Since it is queue - first in first out,
we remove the first one
The Root
36
An Example of a MAX Heap
The root has largest value
23
12
16
7
1
9
3
13
5
2
37
Deleting a node from the heap
We use a heap to implement a priority queue
the "next" element ( the root ) in the queue was removed
(pop operation).
We end up with a root with no value
12
16
7
9
13
5
Root has no value
1
3
2
38
Deleting a node from the heap
The obvious question
is: which node can we
use to replace this
one?
12
16
7
1
9
3
13
5
2
39
Deleting a node from the heap
12
16
7
1
9
3
2
13
5
If we want this tree to stay
complete
the rightmost element in the
bottommost level is the
obvious choice
40
Deleting a node from the heap
We now have a
problem: We have a complete tree but
this is not a heap!
WHY???
This can be solved by applying
systematic swap of nodes.
2
12
7
1
16
9
13
5
3
41
Deleting a node from the heap
2
12
7
1
16
9
3
13
5
We systematically swap
the root with the larger
of the children nodes
until no more swaps can
be done
42
Deleting a node from the heap
16
12
7
1
2
9
13
5
3
43
Deleting a node from the heap
16
12
7
1
13
9
2
5
3
44
Deleting a node from the heap
16
12
7
1
13
9
2
5
3
The tree has restored its
heap property!
45
Insert operation
Another example - the heap below
Recall that while delete causes pairwise swaps from the root to the
bottom,
insert causes pairwise swaps from the bottom to the top.
Suppose that we want to insert an element with value 20
22
17
12
11
6
3
4
13
5
1
46
insert (20)
22
12
17
11
6
3
4
1
13
5
20
47
insert (20)
22
12
17
11
6
20
4
1
13
5
3
48
insert (20)
22
17
20
11
6
12
4
1
13
5
3
49
insert (20)
22
20
17
11
6
12
4
1
13
5
3
50
Implementing a Heap
We will store the
data from the
nodes in a
partially-filled
array.
42
35
27
23
21
An array of data
51
Implementing a Heap
Data from the root
goes in the
first
location
of the
array.
42
35
27
23
21
42
An array of data
52
Implementing a Heap
Data from the
next row goes in
the next two
array locations.
42
35
27
42
35
23
21
23
An array of data
53
Implementing a Heap
Data from the
next row goes in
the next two
array locations.
42
35
27
42
35
23
27
23
21
21
An array of data
54
Implementing a Heap
Data from the
next row goes in
the next two
array locations.
42
35
27
42
35
23
27
23
21
21
An array of data
55
Points about the Implementation
The links between the tree's nodes
are not actually stored as pointers,
or in any other way.
The only way we "know" that "the
array is a tree" is from the way we
manipulate the data.
42
35
23
27
42
35
27
23
21
21
An array of data
56
Array representation of a heap
3
12
18
0
6
14
1
2
lastIndex = location 5
8
3
4
5
6
7
8
9
10
11 12
3 12 6 18 14 8
Left child of node i is 2*i + 1, right child is 2*i + 2
Unless the value is larger than lastIndex -therefore no such child
Parent of node i is (i – 1)/2
unless i == 0
57
Adding to heap - - Implementation
algorithm for the array implementation
Increase lastIndex and put the new value where lastIndex now
references -that is add to the end of the array
Reheap the newly added node
This is called up-heap bubbling or percolating up
Up-heap bubbling requires O(log n) time
58
Points about the Implementation
42
If you know the index of a
node, then it is easy to figure
out the
indexes of that node's
parent and children.
35
27
23
21
42
35
23
27
21
[0]
[1]
[2]
[3]
[4]
59
Points about the Implementation
42
If we add 45 to the tree
and to the array,
45’s parent is at what
index? i = ?
(i-1)/2 = 2
35
27
23
21
42
35
23
27
21
45
[0]
[1]
[2]
[3]
[4]
[5]
45
60
Points about the Implementation
If we add 45 to the tree and the array, 45’s
parent is at what index?
42
(i-1)/2 which is 2.
35
We do a comparison of 45 with its
parent and
27
swap if it is greater than the
parent - it is greater than it
23
21
42
35
23
27
21
[0]
[1]
[2]
[3]
[4]
45
45
[5]
61
Points about the Implementation
If we add 45 to the tree and to the tree and
the array,
42
45’s parent is at what index?
(i-1)/2 which is 2.
35
45
We do a comparison and swap.
27
It is still out of place.
21
42
35
45
27
21
[0]
[1]
[2]
[3]
[4]
23
23
[5]
62
Points about the Implementation
45
45 is still out of place,
so we do another comparison
35
and swap with 45’s parent which
is 42.
So what is the terminating
condition)(s)?
27
42
21
45
35
42
27
21
[0]
[1]
[2]
[3]
[4]
23
23
[5]
63
Summary
A heap is a complete binary tree, where
the entry at each node is greater than or equal
to the entries in its children.
OR
the entry at each node is less than or equal to
the entries in its children.
64
Summary Adding/Deleting
To add an entry to a heap,
place the new entry at the next available spot,
and perform a reheapification upward.
To remove the biggest entry,
move the value in the last node onto the root, and
perform a reheapification downward.
65
Reheapifying a Tree
Fine,
we know how to delete one element and
restore the heap,
but there are several other operations involving
heaps
66
If we find a systematic way of heapifying a tree,
we could perform any operation
because we can execute the procedure heapify
67
Reheapifying a Tree
we can heapify a tree using a quite simple idea.
Given a binary tree T (not necessarily a heap) with root R
and left subtree Tleft and Tright,
if these two subtrees are heaps,
we can make T a heap using the process described in the
previous slide (of applying systematic swaps)
We start the process of heapifying a tree from
the lower levels
68
Algorithm for Reheapifying Tree
The algorithm aims at heapifying a tree by always
comparing three nodes at a time.
To start from the lower levels
we choose to look at the internal nodes in
reverse level order
External nodes have no children so they are (individually)
heaps
69
HEAPIFY
void heapify (Heap h) {
NodeType n;
for (n = the internal nodes of h in reverse levelorder) {
Reheapify the heap h starting at node n
}
}
70
Let change this tree into a heap - A MAX HEAP
6
4
17
12
11
1
22
13
5
3
71
An example
6
4
17
12
11
1
22
13
5
3
Internal Nodes in Reverse Level-Order are:
1, 12, 17, 4, 6 ,
start with the leftmost leaf : compare 3 To 1
72
6
4
17
12
11
3
22
13
5
1
3 and 1 have swapped
Internal Nodes in Reverse Level-Order: 3, 12, 17, 4, 6
73
An example
6
4
17
12
11
3
22
13
5
1
Compare 12 with its children and swap 12 and 22 :
Internal Nodes in Reverse Level-Order: 3, 22, 17, 4, 6
74
An example
6
4
17
22
11
3
12
13
5
1
Compare 17 with its children and no changes necessary:
3, 22, 17, 4, 6
--- NEXT COMPARE 4 AND 22
75
6
22
17
4
11
3
12
13
5
1
Compare 22 with 4 and swap :
4 IS NOW OUT OF PLACE
76
An example
6
22
17
12
11
3
4
13
5
1
Compare 12 and 4 and swap, NEXT compare 22 to 6 :
3, 12, 17, 22, 6
77
22
6
17
12
11
3
4
13
5
1
Heapify 6 downwards until it finds its proper location :
3, 12, 17, 6
78
22
12
17
6
11
3
4
13
5
1
Reheap down 6 : Compare 6 and 11
79
An example
6 is in place and the tree is reheaped:
22
12
17
11
6
3
4
13
5
1
80
An example
22
12
17
11
6
3
4
13
5
1
81
heapsort
Heapsort is another algorithm for sorting lists
The idea behind heap sort is that
to sort a list
we can insert the elements of a list into a heap using
heapify
Then remove all the elements using remove.
82
The idea behind heap sort is that
to sort a list we can insert the elements of a list into a heap
using heapify
Then remove all the elements using remove.
It is guaranteed that the largest element is removed first,
So by systematically removing the elements
we get the list ordered ( in reverse order)
83