Download s05.2 - UBC ECE

The complexity and correctness of algorithms
(with binary trees as an example)
An algorithm is an
unambiguous sequence of
simple, mechanically
executable instructions.
3
Trees
• A tree is a data structure used to represent
different kinds of data and help solve a
number of algorithmic problems
• Game trees (i.e., chess ), UNIX directory
trees, sorting trees etc.
• We will study at least two kinds of trees:
Binary Search Trees and Red-Black Trees.
4
Trees
• Trees have nodes. They also have edges that
connect the nodes. Between two nodes there is
always only one path.
Tree nodes
Tree edges
5
Trees
• Trees are rooted. Once the root is defined (by the user) all
nodes have a specific level.
• Trees have internal nodes and leaves. Every node (except
the root) has a parent and it also has zero or more children.
root
level 0
level 1
internal nodes
level 2
level 3
leaves
parent
and
child
6
Binary trees
• A binary tree is a tree such that each node
has at most 2 children.
7
Binary trees
Recursive definition (important):
1. An empty tree is a binary tree
2. A node with two child sub-trees is a binary tree
3. [often implicit] Only what you get from 1 by a
finite number of applications of 2 is a binary
tree.
Values stored at tree nodes are called keys.
56
26
200
28
18
12
24
190
213
27
8
Binary search trees
• A binary search tree (BST) is a binary tree
with the following properties:
– The key of a node is always greater than the keys
of the nodes in its left subtree.
– The key of a node is always smaller than the keys
of the nodes in its right subtree.
9
Binary search trees
• Stored keys must satisfy
the binary search tree
property.
56
–  y in left subtree of x,
then key[y]  key[x].
–  y in right subtree of x,
then key[y]  key[x].
26
28
18
12
200
24
190
213
27
10
Binary search trees: examples
root
root
14
C
A
10
D
16
root
8
11
15
18
14
10
8
16
11
15
11
Binary search trees
• Data structures that can support dynamic set
operations.
– Search, minimum, maximum, predecessor,
successor, insert, and delete.
• Can be used to build
– Dictionaries.
– Priority Queues.
12
BST – Representation
• Represented by a linked data structure of
nodes.
• root(T) points to the root of tree T.
• Each node contains fields:
–
–
–
–
key
left – pointer to left child: root of left subtree.
right – pointer to right child : root of right subtree.
p – pointer to parent; p[root[T]] = NIL (optional).
13
Inorder traversal of a BST
The binary search tree property allows the keys of a binary search tree to
be printed, in (monotonically increasing) order, recursively.
Inorder-Tree-Walk (p)
if p  NIL then
56
Inorder-Tree-Walk(left[p])
print key[x]
Inorder-Tree-Walk(right[p])
26
28
18
12
200
24
190
213
27
• Can you prove the correctness on in-order traversal?
• How long does an in-order walk take?
14
Correctness of inorder traversal
• Must prove that it prints all elements, in order,
and that it terminates.
• By induction on size of tree. Size=0: Easy.
• Size >1:
– Prints left subtree in order by induction.
– Prints root, which comes after all elements in left
subtree (still in order).
– Prints right subtree in order (all elements come
after root, so still in order).
15
Notice how we used the recursive
definition of a tree in our inductive proof.
We exploit the recursive structure of a
tree, and this approach - which is general
to all recursive definitions, and not
restricted to trees - is called structural
induction.
16
Searching in a BST
Tree-Search(x, k)
if x = NIL or k = key[x] then
return x
if k < key[x]
then return Tree-Search(left[x], k)
else return Tree-Search(right[x], k)
26
24
27
Height 3
200
28
18
12
Height 4
56
190
213
Height 2
Height 1
Time complexity is proportional
with h is the height of the tree.
17
Inserting an element in a BST
• Change the dynamic set
represented by a BST.
• Ensure the binary search
tree property holds after
change.
• Insertion is easier than
deletion.
26
12
24
y  x
if key[z] < key[x]
then x  left[x]
else x  right[x]
then root[t]  z
else if key[z] < key[y]
200
28
y  NIL
x  root[T]
while x  NIL do
p[z]  y
if y = NIL
56
18
Tree-Insert(Tree T, int z)
190
213
then left[y]  z
else right[y]  z
27
18
Analysis of Insertion
• Initialization: constant time
• While loop in lines 3-7
searches for place to insert
z, maintaining parent y.
This takes time proportional
with h
• Lines 8-13 insert the value:
constant time
 TOTAL: C1 + C2 + C3*h
Tree-Insert(T, z)
y  NIL
x  root[T]
while x  NIL do
y  x
if key[z] < key[x]
then x  left[x]
else x  right[x]
p[z]  y
if y = NIL
then root[t]  z
else if key[z] < key[y]
then left[y]  z
else right[y]  z
19
Deletion
• Goal: Delete a given node z from a binary search
tree.
• We need to consider several cases.
– Case 1: z has no children.
• Delete z by making the parent of z point to NIL, instead of to z.
15
15
5
16
3
5
12
10
13
18
delete
6
7
3
20
z
16
12
23
10
20
18
23
6
7
20
Deletion
• Case 2: z has one child.
– Delete z by making the parent of z point to z’s
child, instead of to z.
– Update the parent of z’s child to be z’s parent.
15
5
3
z
16
5
12
10
6
18
20
3
20
13
15
delete
12
23
18
23
10
6
7
7
21
Deletion
• Case 3: z has two
children.
delete
15
z
5
16
3
– z’s successor (y) is the
minimum node in z’s right
subtree.
– y has either no children or
one right child (but no left
child).
12
10
20
13
18
23
y 6
7
15
• Why?
– Delete y from the tree (via
Case 1 or 2).
– Replace z’s key and
satellite data with y’s.
6
6
16
3
12
10
20
13
18
23
7
22
Successor node
• The successor of node x is the node y such that key[y]
is the smallest key greater than key[x].
• The successor of the largest key is NIL.
• Search for a successor consists of two cases.
– If node x has a non-empty right subtree, then x’s successor is
the minimum in the right subtree of x.
– If node x has an empty right subtree, then:
• As long as we move to the left up the tree (move up through right
children), we are visiting smaller keys.
• x’s successor y is the node that x is the predecessor of (x is the
maximum in y’s left subtree).
• In other words, x’s successor y, is the lowest ancestor of x whose
left child is also an ancestor of x.
• We can define the predecessor node similarly.
23
Exercise: Sorting using BSTs
Sort (A)
for i  1 to n
do tree-insert(A[i])
inorder-tree-walk(root)
– What are the worst case and best case
running times?
– In practice, how would this compare to other
sorting algorithms?
24
Wrap-up
• Determining the complexity of algorithms is
usually the first step in obtaining better
algorithms.
– Or in realizing we cannot do any better.
• What should you know?
– Inductive proofs on trees.
– Binary search trees and operations on BSTs.
– Height of trees and how it influences efficiency.
25