Download COMP 620 Algorithm Analysis

COMP 620 Algorithm Analysis
Franklin University
Module 3: Advanced Data
Structures
Trees, Heap, Binomial Heap,
Fibonacci Heap
1
COMP 620 Algorithm Analysis
Franklin University
Trees
•
Definition: A tree (which is one type of a data structure)
is a finite set of one or more nodes such that:
There is a specially designated node called root.
The remaining nodes are divided into n>=0 disjoint sets
T1, T2, ..... Tn where each of these sets is a tree. T1, T2,
.... Tn are called the subtrees of the root.
•
•
•
Node - represents an item of information stored in the tree.
Branches - represent the links between the nodes.
Root of the tree or root node - node at the top of the tree, the start of
the tree.
Degree of a node - the number of subtrees of the node (i.e., the
number of children from any one node).
Degree of a tree - the maximum degree of any of the nodes in the
tree.
Leaf or terminal nodes - nodes with degree 0.
•
•
•
2
COMP 620 Algorithm Analysis
Franklin University
Tree Terminology (contd.)
7.
8.
9.
10.
11.
12.
Child/children - the roots of the subtrees of the parent node.
Siblings - children of the same parent.
Parent - a node that has subtrees (i.e., a node that has
children).
Level of a node - defined by root = 1, children of root = 2,
grandchildren of root = 3, etc. (In some textbooks, the root is
defined to be at level 0, children of the root at level 1, etc.)
Depth or height of a tree - the maximum level of any node in
the tree.
Binary trees are trees with no more than two-way branching from
each node in the tree. A binary tree is either empty or consists of a
root node and two disjoint binary trees called left and right subtrees.
3
COMP 620 Algorithm Analysis
Franklin University
Tree - Example
A
C
B
E
K
F
D
G
I
H
J
L
M
N
Degree of the tree =
O
Height (depth) of the tree =
Terminal nodes of the tree are:
4
COMP 620 Algorithm Analysis
Franklin University
Binary Trees: Properties
• The maximum number of nodes on level i of a binary
tree is 2 (i-1) , where i >= 1.
• The maximum number of nodes in a binary tree of
depth k is 2k –1,where k >= 1.
• For any non-empty binary tree, T, if n0 represents the
number of leaf nodes and n2 represents the number
of nodes with degree 2, then n0 = n2 + 1.
• A full binary tree of depth k is a binary tree of depth k
having 2k - 1 nodes, where k >= 1.
• A binary tree with n nodes and depth k is complete if
and only if its nodes correspond to the nodes
numbered from 1 to n in the full binary tree of depth
k, filling in from left to right on each level.
5
COMP 620 Algorithm Analysis
Franklin University
Full and Complete Binary Trees
1
A
C
B
D
2
F
E
G
4
3
5
Complete Binary Tree
Full Binary Tree
Array Representing Above Tree
A B C D E F G
0
1
2
3
4
5
6
7
•Leftchild(i) is at 2i if 2i >n no left child
•Rightchild(i) is at 2i + 1 if 2i +1 > n no right child
6
COMP 620 Algorithm Analysis
Franklin University
Neither Full nor Complete Trees
A
B
D
C
F
E
G
Array Representation of the Above Tree
A B
0
1
2
C D
3
4
5
E F
..
10
11
G
..
23
The array representation of the tree that is neither full nor complete is very
wasteful of memory
7
COMP 620 Algorithm Analysis
Franklin University
Linked Representation
template <class BaseData>
Class BtNode
{
public:
BaseData info;
BtNode *leftChild, *rightChild;
};
8
COMP 620 Algorithm Analysis
Franklin University
•
Traversing Trees
Inorder traversal
1.
Traverse the left subtree
2.
Visit the root
3.
Traverse the right subtree
Void BinTree::inord( BtNode *rt)
{
if (rt != NULL)
{
inord(rt->leftChild);
processNode(rt->info);
inord(rt->rightChild);
}
}
9
COMP 620 Algorithm Analysis
Franklin University
Traversing Trees
• Preorder traversal
1. Visit root
2. Traverse the left subtree
3. Traverse the right subtree
Void BinTree::preord( BtNode *rt)
{
if (rt != NULL)
{
processNode(rt->info);
preord(rt->leftChild);
preord(rt->rightChild);
}
}
10
COMP 620 Algorithm Analysis
Franklin University
Traversing Trees
•
Postorder traversal
1.
2.
3.
Traverse the left subtree
Traverse the right subtree
Visit the root
•
Levelorder traversal
Every node is visited in turn from left to right on every level,
starting at level 1, then level 2, etc., until level n, where n
represents the height (depth) of the tree.
11
COMP 620 Algorithm Analysis
Franklin University
Binary Search Trees
• Binary search tree is of significant importance in
Computer Science.
• Has better performance than many other data structures
especially for operations like insertion, deletion, and
searching.
• Definition:A binary search tree is a binary tree. It may
be empty. If not empty, it satisfies the following
properties:
Every element has a key, and traditionally no two
elements have the same key.
Keys in a non-empty left subtree must be smaller
than the key in the root of the subtree.
Keys in a non-empty right subtree must be larger
than the key in the root of the subtree.
Left and right subtrees are also binary search trees.
12
COMP 620 Algorithm Analysis
Franklin University
Binary Search Tree
G
F
A
O
H
E
I
C
B
J
P
M
K
D
N
L
13
COMP 620 Algorithm Analysis
Franklin University
•
•
Binary Search Trees: Operations
Searching a binary search tree:
Begin at the root. If the key of the element to be
searched = root key, then the search is successful.
If the key of the element to be searched < root key,
then search the left subtree.
If the key of the element to be searched > root key,
then search the right subtree.
Inserting into a binary search tree:
- Search for the key
- If the key is not present, locate the parent node
- Insert the new node as the left/right child of the
parent node
All insertions take place at leaf nodes.
14
COMP 620 Algorithm Analysis
Franklin University
Binary Search Trees: Operations
•
Deleting from a binary search tree:
The deletion process begins with a search to find the node to
be deleted from the binary search tree.
Three possible scenarios exist in the deletion process:
1. Delete a leaf node.
- Make the appropriate pointer in x’s parent a null pointer
2. Delete a node with 1 child.
- Set the appropriate pointer in x’s parent to point to this child
3. Delete a node with 2 children.
- Replace the value stored in the node x by its inorder
successor (predecessor) and then delete the successor
(predecessor)
15
COMP 620 Algorithm Analysis
Franklin University
Binary Search Trees: Variations
• AVL (or height-balanced) trees
A binary search tree in which the balance factor of
each node is 0, 1, or –1, where the balance factor of
a node x is defined as the height of the left subtree of
x minus the height of x’s right subtree.
• 2-3-4 trees:
A tree with the following properties:
Each node stores at most 3 data values.
Each internal node is a 2-node, 3-node, or a 4node.
All the leaves are on the same level.
16
COMP 620 Algorithm Analysis
Franklin University
Motivation for AVL Tree
2
1
2
1
4
3
3
5
4
What is the value of balance of each node?
5
The height of AVL tree is approximately Log2 n
17
COMP 620 Algorithm Analysis
Franklin University
2-3-4 Tree
53
27
16
25
38
33
60
36
41 46
48
55
59
65
70
68
73 75
79
18
COMP 620 Algorithm Analysis
Franklin University
Red Black Trees
• Red Black trees
A binary search tree with two kinds of nodes, red and
black, which satisfy the following properties:
• Every node is either red or black. Root is black.
• Every leaf (NIL) is black.
• If a node is red, both its children are black.
• Every simple path from node to descendant leaf contains
the same number of black nodes.
• A red-black tree with n internal nodes has height at most
2log(n+1).
19
COMP 620 Algorithm Analysis
Franklin University
11
14
2
1
15
7
5
8
RED-BLACK TREE
20
COMP 620 Algorithm Analysis
Franklin University
B-trees
• B-trees of minimum degree t
A tree with the following properties:
- Each node stores at most 2t -1 data values.
- Every node other than the root must have at least t1 data values.
- n[x] keys in each node x stored in nondecreasing
order, so that key1[x] <= key2[x] .. <= key n[x] [x].
- Each internal node contains n[x] + 1 children. X
contains n +1 pointers c1[x], c2[x] … c n[x]+1[x].
21
COMP 620 Algorithm Analysis
Franklin University
B-trees
- The keys keyi [x] separates the ranges of keys
stored in each subtree: if ki is any key stored in the
subtree with root ci[x] then, then
k1 <= key1[x] <= key2[x] <= ….<= key n[x] [x] <= k n[x]+1
- All the leaves are on the same level.
- Balanced and designed to work efficiently on disks and
other direct access secondary storage devices. Many
database systems use B-tree.
- the maximum height of a n-key B tree is logt ((n+1)/2)
- insert, delete, and search time on a B-Tree is (log n)
22
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
23
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
24
COMP 620 Algorithm Analysis
Franklin University
B-tree of minimum degree t = 2
Is 2-3-4 tree a B-tree?
M
Q T
D H
B C
F G
J K
L
N P
X
R
S
V
W
Y Z
25
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
26
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
27
COMP 620 Algorithm Analysis
Franklin University
B-Tree
• Insert
- locate the node where the key should be inserted
- split any full nodes encountered while descending the tree
• Deletion
Deletion is similar to Insertion. Make sure that the tree is still
a B-Tree after deletion.
Detailed discussion of deleting from a B-tree, refer to Section
18.3, pages 450-453, of Cormen, Leiserson, and Rivest.
28
COMP 620 Algorithm Analysis
Franklin University
B-Tree Insertion - Inserts only in leaf node
G M P X
A C D E
J K
N O
R S TU V
Y Z
(a) Initial Tree
G M P X
A B C D E
J K
N O
R S TU V
Y Z
(b) B Inserted
29
COMP 620 Algorithm Analysis
Franklin University
B-Tree Insertion - Inserts only in leaf node
G M P T X
A B C D E
J K
N O
Q R S
U V
Y Z
(C ) Q Inserted
P
G M
A B C D E
J K L
T X
N O
Q R S
U V
Y Z
(d ) L Inserted
30
COMP 620 Algorithm Analysis
Franklin University
B-Tree Insertion - Inserts only in leaf node
P
C G M
A B
D E F
J K L
T X
N O
Q R S
U V
Y Z
(e ) F Inserted
31
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
32
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
33
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
34
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
35
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
36
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
37
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
38
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
COMP 620 Algorithm Analysis
Franklin University
39
COMP 620 Algorithm Analysis
Franklin University
Heaps
• Heaps data structures are mostly used to support the
following operations (not efficient to use for search):
Insert element x
Return min element
Delete minimum element
Union of two heaps
• Application:
Dijkstra’s Shortest Path
Prim’s MST
Huffman Encoding
Heap Sort
40
COMP 620 Algorithm Analysis
Franklin University
Binary Heap
A heap is a complete binary tree such that the value
of the key in the root is greater than the value of the
key in each of its children, and that both subtrees are
also heaps (a recursive definition).
http://homepages.ius.edu/rwisman/C455/html/notes/Chapte
r6/HeapSort.htm
Heapsort:
(1) Make a heap of the elements to be sorted.
(2) Convert the heap into a sorted list.
10
6
9
3
2
5
41
COMP 620 Algorithm Analysis
Franklin University
Heap Sort
HeapSort(A)
1
Build_Max_Heap(A)
2
for i ← length[A] downto 2 do
3
exchange A[1] ↔ A[i]
4
heap-size[A] ← heap-size[A] – 1
5
Max_Heapify(A, 1)
Build_Max_Heap(A)
1
heap-size[A] ← length[A]
2
for i ← floor(length[A]/2) downto 1 do
3
Max_Heapify (A, i)
42
COMP 620 Algorithm Analysis
Franklin University
HEAP SORT
Heapify(A, i)
1
l ← LEFT(i)
2
r ← RIGHT(i)
3
if l ≤ heap-size[A] and A[l] > A[i]
4
then largest ← l
5
else largest ← i
6
if r ≤ heap-size[A] and A[r] > A[largest]
7
then largest ← r
8 if largest ≠ i
9
then exchange A[i] ↔ A[largest]
10
Max_Heapify (A, largest)
43
COMP 620 Algorithm Analysis
Franklin University
HEAPS
Operation
Linked
List
Binary
Heap
Binomial Fibonacci
Heap
Heap
make-heap
1
1
1
1
insert
1
Log N
Log N
1
find-min
N
1
Log N
1
delete-min
N
Log N
Log N
Log N
decrease-key
1
Log N
Log N
1
delete
N
Log N
Log N
Log N
union
1
N
Log N
1
44
COMP 620 Algorithm Analysis
Franklin University
BINOMIAL TREE
• Recursive Definition : Bk consists of 2 binomial trees B k-1
that are linked together; the root of the one is the leftmost
Child of the root of the other.
B0
B1
B2
B3
45
COMP 620 Algorithm Analysis
Franklin University
Binomial Tree Properties
•
•
•
•
Number of nodes = 2 k
Height of the tree = k
k
There are nodes  i  nodes at level i
 
The root has degree k and it children are B k-1, B k-2,.. B 0
from left to right.
46
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
• A binomial heap is a set of binomial trees
• Each binomial tree in H is heap-ordered key (x) >= key
(parent(x)).
• There never exist 2 or more trees with same degree in the
heap.
• Linked list of roots in order of increasing degree
• Binomial tree is stored in a left child-right sibling
representation.
47
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
Roots of the trees connected with singly linked list
Head points to the first node in the linked list
Head
7
5
3
6
B0
Parent
10
4
Key
degree
B1
17
B2
Sibling
Child
V
i
e
View the operations on binomial heap at the class web site
48
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
MAKE_BINOMIAL_HEAP()
allocate(H)
head[H] = NIL
Insert(H, x)
H’ = MAKE-BINOMIAL-HEAP()
set x’s field appropriately
head[H’] = x
n[H’] = 1
H = BINOMIAL-HEAP-UNION(H, H’]
49
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
Binomial-Heap-Minimum(H)
y = NIL
x = head[H]
min = inf
while x <> NIL
do if key[x] < min
then min = key[x]
y=x
x = sibling[x]
return y;
50
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
BINOMIAL-HEAP-UNION (H1, H2)
Merge the root lists of binomial heaps H1 and H2 into a single linked
linked list H that is sorted by degree into monotonically increasing
order.
Links roots of equal degree until at most one root remains of each
degree.
Binomial-Link (y,z)
p[y] = z
sibling[y] = child[z]
child[z] = y
degree [z] = degree[z] + 1
51
COMP 620 Algorithm Analysis
Franklin University
Binomial Heap
BINOMIAL-HEAP-EXTRACT-MIN(H)
find the root x with the minimum key in the root list of H,
and remove x from the root list of H
H’ = MAKE-BINOMIAL-HEAP()
reverse the order of the linked list of x’s children and set
head[H’] to point to the head of the resulting list
H = BINOMIAL-HEAP-UNION (H, H’)
return (x)
52
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
• A set of min-heap-ordered trees
• Roots of trees are connected with circular doubly linked
list. Children of a node are connected with circular doubly
linked list.
• Pointer to root of tree with minimum element.
Parent
Key
Mark: Newly created nodes are
unmarked. This field becomes
true if the node has lost a child
Since the node became a child
Of another node.
Degree
Left
Right
Child
53
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
View the operations on Fibonacci heap at the class web site
23
Min[H]
3
7
18
39
52
24
17
38
30
41
26
46
35
54
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
Make_heap()
allocate(H)
min[H] = NIL
n[H]= 0
Insert(H, x)
set x’s field appropriately
add x to root list of H
reset min[H] if needed
n[H] = n[H] + 1
55
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
Union(H1, H2)
H = new heap whose root list contains roots from H1 and H2
Min[H] = min[H1]
If ( min[H1] == NIL) or (min [H2] <> NIL and min[H2] < min[H1])
then min [H] = min[H2]
N[H] = n[H1] + n[H2]
free the objects H1 and H2
return H
56
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
Extract-min (H)
z = min[H]
Add z’s children to root list
Remove z from root list
If root list <> {}
then Consolidate H
else min[H] = NIL
n[H] = n[H] - 1
57
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap
Consolidate (H)
While 2 trees in H (T1, T2) have the same degree:
if root(T1) < root (T2)
then make T1 child of T2
else make T2 child of T1
for i = 0 to D(n[H] ) // D is max possible trees
if tree T of degree i has root-key < min[H]
then min[H] = T
58
COMP 620 Algorithm Analysis
Franklin University
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Fibonacci Heap Decrease Key
FIB-HEAP-DECREASE-KEY(H, x, k)
1 if k > key[x]
2 then error "new key is greater than current key"
3 key[x] ← k
4 y ← p[x]
5 if y ≠ NIL and key[x] < key[y]
6 then CUT(H, x, y)
7
CASCADING-CUT(H, y)
8 if key[x] < key[min[H]]
9 then min[H] ← x
CUT(H, x, y)
1 remove x from the child list of y, decrementing degree[y]
2 add x to the root list of H
3 p[x] ← NIL
4 mark[x] ← FALSE
CASCADING-CUT(H, y)
1 z ← p[y]
2 if z ≠ NIL
3 then if mark[y] = FALSE
4
then mark[y] ← TRUE
5
else CUT(H, y, z)
6
CASCADING-CUT(H, z)
59
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap Decrease Key
60
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap Decrease Key
a.
b.
The initial Fibonacci heap
The node with key 46 has its key decreased to 15. The node
becomes a root, and its parent ( with key 24) which had
previously been unmarked, becomes marked.
c-e The node with key 35 has its key decreased to 5. In part (c) ,
the node now with key 5, becomes a root. Its parent, with
key 26, is cut from its parent and made an unmarked root in
(d). Another cascading cut occurs, since the node with key
24 is marked as well. This node is cut from its parent and
made an unmarked root in part (e). The cascading cuts stop
at this point, since the node with key 7 is a root.
61
COMP 620 Algorithm Analysis
Franklin University
Fibonacci Heap Delete Node
• FIB-HEAP-DELETE(H, x)
• 1 FIB-HEAP-DECREASE-KEY(H, x, -∞)
• 2 FIB-HEAP-EXTRACT-MIN(H)
62