Download Balanced Search Trees

Balanced Search Trees
CS 302 - Data Structures
Mehmet H Gunes
Modified from authors’ slides
Contents
•
•
•
•
•
Balanced Search Trees
2-3 Trees
2-3-4 Trees
Red-Black Trees
AVL Trees
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Balanced Search Trees
• Height of a binary search tree sensitive to
order of insertions and removals
– Minimum = log2 (n + 1)
– Maximum = n
• Various search trees can retain balance
despite insertions and removals
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Balanced Search Trees
• (a) A binary search tree of maximum height;
(b) a binary search tree of minimum height
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3 Trees
• A 2-3 tree is not a binary tree
• A 2-3 tree never taller than a minimumheight binary tree
• A 2-3 tree of height 3
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3 Trees
• Placing data items in nodes of a 2-3 tree
– A 2-node (has two children) must contain single
data item greater than left child’s item(s) and less
than right child’s item(s)
– A 3-node (has three children) must contain two
data items, S and L , such that
• S is greater than left child’s item(s) and less than middle
child’s item(s);
• L is greater than middle child’s item(s) and less than
right child’s item(s).
– Leaf may contain either one or two data items.
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3 Trees
• Nodes in a 2-3 tree: (a) a 2-node; (b) a 3-node
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013`
2-3 Trees
View Header file for a class of nodes for a
2-3 tree, Listing 19-1
• A 2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Traversing a 2-3 Tree
• Traverse 2-3 tree
in sorted order
by performing
analogue of
inorder traversal
on binary tree:
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
• Retrieval operation for 2-3 tree similar to
retrieval operation for binary search tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
• Possible to search 2-3 tree and shortest binary
search tree with approximately same
efficiency, because:
– Binary search tree with n nodes cannot be shorter
than log2 (n + 1)
– 2-3 tree with n nodes cannot be taller than
log2 (n + 1)
– Node in a 2-3 tree has at most two items
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
• A balanced binary search tree;
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
• a 2-3 tree with the same entries
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Searching a 2-3 Tree
• (a) The binary
search tree of
Figure 19-5a after
inserting the
sequence of values
32 through 39
• (b) the 2-3 tree of
Figure 19-5 b after
the same insertions
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• After inserting 39 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• The steps for inserting 38 into the tree:
(a) The located node has no room;
(b) the node splits; (c) the resulting tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• After inserting 37 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• (a), (b), (c) The steps for inserting 36 into
the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• (d) the resulting tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• The tree after the insertion of 35, 34, and
33 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• Splitting a leaf in a 2-3 tree when the leaf is
a (a) left child; (b) right child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• Splitting an internal node in a 2-3 tree when
the node is a (a) left child; (b) right child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• Splitting the root of a 2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• Summary of insertion strategy
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Inserting Data into a 2-3 Tree
• Summary of insertion strategy
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• A 2-3 tree; (b), (c), (d), (e) the steps for
removing 70;
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• (f) the resulting tree;
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• (a), (b), (c) The steps for removing 100 from
the tree; (d) the resulting tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• The steps for removing 80 from the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• The steps for removing 80 from the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• Results of removing 70, 100, and 80 from
(a) the 2-3 tree and (b) the binary search tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• Algorithm for removing data from a 2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• Algorithm for removing data from a 2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• Algorithm for removing data from a 2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• (a) Redistributing values;
(b) merging a leaf;
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• (c) redistributing values and children;
• (d) merging internal nodes
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Removing Data from a 2-3 Tree
• (e) deleting the root
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• A 2-3-4 tree with the same data items as the
2-3 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Rules for placing data items in the nodes of a
2-3-4 tree
– 2-node (two children), must contain a single data
item that satisfies relationships
– 3-node (three children), must contain two data
items that satisfies relationships
– ...
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
– 4-node (four children) must contain three data
items S , M , and L that satisfy:
• S is greater than left child’s item(s) and less than
middle-left child’s item(s)
• M is greater than middle-left child’s item(s) and less
than middle-right child’s item(s);
• L is greater than middle-right child’s item(s) and less
than right child’s item(s).
– A leaf may contain either one, two, or three data
items
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• A 4-node in a 2-3-4 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Has more efficient insertion and removal
operations than a 2-3 tree
• Has greater storage requirements due to the
additional data members in its 4-nodes
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Searching and Traversing a 2-3-4 Tree
– Simple extensions of the corresponding
algorithms for a 2-3 tree
• Inserting Data into a 2-3-4 Tree
– Insertion algorithm splits a node by moving one of
its items up to its parent node
– Splits 4-nodes as soon as it encounters them on
the way down the tree from the root to a leaf
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Inserting 20 into a one-node 2-3-4 tree
(a) the original tree; (b) after splitting the node;
(c) after inserting 20
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• After inserting 50 and 40 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• The steps for inserting 70 into the tree :
(a) after splitting the 4-node;
(b) after inserting 70
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• After inserting 80 and 15 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• The steps for inserting 90 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• The steps for inserting 100 into the tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Splitting a 4-node root during insertion into
a 2-3-4 tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Splitting a 4-node whose parent is a 2-node
during insertion into a 2-3-4 tree, when the
4-node is a (a) left child; (b) right child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Splitting a 4-node whose parent is a 3-node
during insertion into a 2-3-4 tree, when the
4-node is a (a) left child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Splitting a 4-node whose parent is a 3-node
during insertion into a 2-3-4 tree, when the
4-node is a (b) middle child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Splitting a 4-node whose parent is a 3-node
during insertion into a 2-3-4 tree, when the
4-node is a (c) right child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
2-3-4 Trees
• Removing Data from a 2-3-4 Tree
– Removal algorithm has same beginning as removal
algorithm for a 2-3 tree
– Locate the node n that contains the item I you
want to remove
– Find I ’s inorder successor and swap it with I so
that the removal will always be at a leaf
– If leaf is either a 3-node or a 4-node, remove I .
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Modified from Dr George Bebis and Dr Monica Nicolescu
Red-Black Trees
• Use a special binary search tree—a red-black
tree —to represent a 2-3-4 tree
• Retains advantages of a 2-3-4 tree without
storage overhead
• The idea is to represent each 3-node and 4node as an equivalent binary search tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Red-black representation of (a) a 4-node;
(b) a 3-node
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• A red-black tree that represents the 2-3-4
tree
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Searching and traversing
– Red-black tree is a binary search tree, search and
traverse it by using algorithms for binary search
tree
• Inserting, removing with a red-black tree
– Adjust the 2-3-4 insertion algorithms to
accommodate the red-black representation
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Splitting a red-black representation of a
4-node that is the root
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Splitting a red-black representation of a
4-node whose parent is a 2-node, when
the 4-node is a (a) left child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Splitting a red-black representation of a
4-node whose parent is a 2-node, when
the 4-node is a (b) right child
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black
Trees
• Splitting a red-black representation of a
4-node whose parent is a 3-node
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black
Trees
• Splitting a red-black representation of a
4-node whose parent is a 3-node
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black Trees
• Splitting a redblack
representation
of a 4-node
whose parent is
a 3-node
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
Red-Black-Trees Properties
(**Binary search tree property is satisfied**)
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
• No two consecutive red nodes on a simple path
from the root to a leaf
5. For each node, all paths from that node to a leaf
contain the same number of black nodes
68
Example: RED-BLACK-TREE
26
17
NIL
41
NIL
30
NIL
47
38
NIL
NIL
NIL
50
NIL
NIL
• For convenience, we add NIL nodes and refer to them as the
leaves of the tree.
– Color[NIL] = BLACK
69
Definitions
26
h=1
bh = 1
NIL
h=4
bh = 2
17
41
NIL
NIL
h=2
30 bh = 1
h=3
bh = 2
h=1
bh = 1
38
NIL
NIL
47
NIL
h=2
bh = 1
50
NIL
h=1
bh = 1
NIL
• Height of a node: the number of edges in the longest path
to a leaf
• Black-height bh(x) of a node x:
the number of black
nodes (including NIL) on the path from x to a leaf, not counting x
70
Height of Red-Black-Trees
A red-black tree with n internal nodes
has height at most 2log(N+1)
71
Insert Item
26
17
41
30
What color to make the new node?
47
38
• Red?
– Let’s insert 35!
• Property 4 is violated: if a node is red, then both children
are black
• Black?
– Let’s insert 14!
• Property 5 is violated: all paths from a node to its leaves
contain the same number of black nodes
50
Delete Item
26
17
41
30
What color was the node that was removed? Red?
OK!
1. Every node is either red or black
OK!
2. The root is black
3. Every leaf (NIL) is black
OK!
4. If a node is red, then both its children are black
47
38
50
OK!
5. For each node, all paths from the node to descendant leaves
OK!
contain the same number of black nodes
73
Delete Item
26
17
41
30
47
What color was the node that was removed? Black? 38
50
OK!
1. Every node is either red or black
Not OK! If removing the root and
2. The root is black
the child that replaces it is red
3. Every leaf (NIL) is black OK!
4. If a node is red, then both its children are black
Not OK! Could change the
black heights of some nodes
Not OK! Could create
two red nodes in a row
5. For each node, all paths from the node to descendant leaves
contain the same number of black nodes
74
Rotations
• Operations for re-structuring the tree after
insert and delete operations
– Together with some node re-coloring, they help
restore the red-black-tree property
– Change some of the pointer structure
– Preserve the binary-search tree property
• Two types of rotations:
– Left & right rotations
75
Left Rotations
• Assumptions for a left rotation on a node x:
– The right child y of x is not NIL
• Idea:
–
–
–
–
Pivots around the link from x to y
Makes y the new root of the subtree
x becomes y’s left child
y’s left child becomes x’s right child
76
Example: LEFT-ROTATE
77
LEFT-ROTATE(T, x)
1. y ← right[x]
►Set y
2. right[x] ← left[y] ► y’s left subtree becomes x’s right subtree
3. if left[y]  NIL
4. then p[left[y]] ← x ► Set the parent relation from left[y] to x
5. p[y] ← p[x]
► The parent of x becomes the parent of y
6. if p[x] = NIL
7. then root[T] ← y
8. else if x = left[p[x]]
9.
then left[p[x]] ← y
10.
else right[p[x]] ← y
11. left[y] ← x
► Put x on y’s left
12. p[x] ← y
► y becomes x’s parent
Right Rotations
• Assumptions for a right rotation on a node x:
– The left child x of y is not NIL
• Idea:
–
–
–
–
Pivots around the link from y to x
Makes x the new root of the subtree
y becomes x’s right child
x’s right child becomes y’s left child
79
Insert Item
• Goal:
– Insert a new node z into a red-black tree
• Idea:
– Insert node z into the tree as for an ordinary
binary search tree
– Color the node red
– Restore the red-black tree properties
80
RB-INSERT(T, z)
1. y ← NIL
2. x ← root[T]
• Initialize nodes x and y
• Throughout the algorithm
y points to the parent of x
3. while x  NIL
4.
5.
do y ← x
if key[z] < key[x]
6.
7.
8. p[z] ← y
then x ← left[x]
else x ← right[x]
• Sets the parent
of z to be y
• Go down the tree until
reaching a leaf
• At that point y is the
parent of the node to be
inserted
26
17
41
30
47
38
50
RB-INSERT(T, z)
9. if y = NIL
The tree was empty:
set the new node to be the root
10. then root[T] ← z
11. else if key[z] < key[y]
12.
then left[y] ← z
13.
else right[y] ← z
Otherwise, set z to be the left or
right child of y, depending on
whether the inserted node is
smaller or larger than y’s key
14. left[z] ← NIL
15. right[z] ← NIL
Set the fields of the newly added node
16. color[z] ← RED
17. RB-INSERT-FIXUP(T, z)
Fix any inconsistencies that could have been
introduced by adding this new red node
RB Properties Affected by Insert
OK!
1. Every node is either red or black
2. The root is black
If z is the root  not OK
3. Every leaf (NIL) is black
OK!
4. If a node is red, then both its children are black
If p(z) is red  not OK
z and p(z) are both red
OK!
5. For each node, all paths
from the node to descendant
leaves contain the same number
of black nodes
26
17
41
38
47
50
RB-INSERT-FIXUP
Case 1: z’s “uncle” (y) is red
(z could be either left or right child)
Idea:
• p[p[z]] (z’s grandparent) must be black
• color p[z]  black
• color y  black
• color p[p[z]]  red
• z = p[p[z]]
– Push the “red” violation up the tree
84
RB-INSERT-FIXUP
Idea:
Case 2:
• color p[z]  black
• z’s “uncle” (y) is black
• color p[p[z]]  red
• z is a left child
• RIGHT-ROTATE(T, p[p[z]])
• No longer have 2 reds in a row
• p[z] is now black
Case 2
85
RB-INSERT-FIXUP
Case 3:
• z’s “uncle” (y) is black
• z is a right child
Idea:
• z p[z]
• LEFT-ROTATE(T, z)
 now z is a left child, and both z and p[z] are red  case 2
Case 3
Case 2
86
Example
Insert 4
Case 1
11
2
14
15
7
1
8 y
z and p[z] are both red
z’s uncle y is red
5
z 4
14 y
2
7
1
Case 3
11
5
4
11
z
15
8
z and p[z] are both red
z’s uncle y is black
z is a right child
7
14 y
7
z
8
2
z
2
Case 2
15
1
5
1
4
z and p[z] are red
z’s uncle y is black
z is a left child
11
5
4
8
14
15
87
RB-INSERT-FIXUP(T, z)
1. while color[p[z]] = RED
2.
3.
4.
5.
6.
if p[z] = left[p[p[z]]]
The while loop repeats only when
case1 is executed: O(logN) times
Set the value of x’s “uncle”
then y ← right[p[p[z]]]
if color[y] = RED
then Case1
else if z = right[p[z]]
7.
then Case3
8.
Case2
9.
else (same as then clause with “right” and “left”
exchanged for lines 3-4)
10. color[root[T]] ← BLACK
We just inserted the root, or
The red violation reached the root
Analysis of InsertItem
• Inserting the new element into the tree
O(logN)
• RB-INSERT-FIXUP
– The while loop repeats only if CASE 1 is executed
– The number of times the while loop can be
executed is O(logN)
• Total running time of Insert Item: O(logN)
89
Delete Item
• Delete as usually, then re-color/rotate
• A bit more complicated though …
• Demo
– http://gauss.ececs.uc.edu/RedBlack/redblack.html
90
Problems
91
Problems
• What red-black tree property is violated in the tree below? How
would you restore the red-black tree property in this case?
– Property violated: if a node is red, both its children are black
– Fixup: color 7 black, 11 red, then right-rotate around 11
7
z
2
1
11
5
4
8
14
15
92
Problems
• Let a, b, c be arbitrary nodes in subtrees , ,  in the tree
below.
• How do the depths of a, b, c change when a left rotation is
performed on node x?
– a: increases by 1
– b: stays the same
– c: decreases by 1
93
Problems
• When we insert a node into a red-black tree,
we initially set the color of the new node to
red.
Why didn’t we choose to set the color to
black?
• Would inserting a new node to a red-black
tree and then immediately deleting it, change
the tree?
94
AVL Trees
• Named for inventors
– Adel’son-Vel’skii and Landis
• A balanced binary search tree
– Maintains height close to the minimum
– After insertion or deletion, check the tree is still
AVL tree
• determine whether any node in tree has left and right
subtrees whose heights differ by more than 1
• Can search AVL tree almost as efficiently as
minimum-height binary search tree.
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
AVL Trees
• (a) An unbalanced binary search tree;
(b) a balanced tree after rotation;
(c) a balanced tree after insertion
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
AVL Trees
• (a) Before;
• (b) and after a single left rotation that
decreases the tree’s height;
• (c) the rotation in general
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
AVL Trees
• (a) Before;
• (b) and after a single left rotation that does
not affect the tree’s height;
• (c) the rotation in general
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013
AVL Trees
• (d) the double rotation in general
Data Structures and Problem Solving with C++: Walls and Mirrors, Carrano and Henry, © 2013