Download Lecture 18 - UMass CS !EdLab

CMPSCI 187
Introduction to Programming
with Data Structures
Lecture 18
Binary Search Trees and
Priority Queues
1
CMPSCI 187
Binary Search Trees

A binary search tree is a special binary tree with the
following properties that hold for every node N:
 Every data element in N’s left subtree is less than (or
less than or equal to) the element in N.
 Every data element in N’s right subtree is greater than
the element in node N.


Implies an ordering on the data elements of the tree.
Note that if the nodes of a BST are printed using an
inorder traversal, the values will be in sorted order
(smallest to largest).
2
CMPSCI 187
Binary Search Trees: Keys and Data





Usually think of a data item stored in a node with an
associated search key.
The search keys are the elements compared.
When looking for an item, we are given a search key
and we search the tree for a matching key. The
entire data item is returned.
When inserting an item in a tree, we use the key to
determine where the item belongs and then insert
both the key and the data item at this location.
The node for a binary tree is something like [key,
data, left subtree, right subtree]:
Key
Binary Tree Node:
Data
Left Right
3
CMPSCI 187
Comparing Keys

If keys are integers, or strings, etc. they can be compared using Java
built-in methods (we’ll do this for our first examples).
 For example, use the compareTo(String str) method for strings

If keys are general objects, how can we compare them?

Use the comparable interface.
interface Comparable
{ // if k1 and k2 are Comparable keys, then
//
k1.compareTo(k2) returns
//
0 if k1==k2
//
+1 if k1>k2
//
-1 if k1<k2
int compareTo(Comparable key);
}
4
CMPSCI 187
Overview of Binary Search Tree
Binary search tree definition:
T is a binary search tree if either of these is
true
T is empty; or
Root has two subtrees:
 Each is a binary search tree
 Value in root > all values of the left subtree
 Value in root < all values in the right subtree
5
CMPSCI 187
Binary Search Tree Example
6
CMPSCI 187
Searching a Binary Tree
7
CMPSCI 187
Searching a Binary Search Tree: Algorithm
1.
if root is null
2.
item not in tree: return null
3.
compare target and root.data
4.
if they are equal
5.
6.
7.
8.
9.
target is found, return root.data
else if target < root.data
return search(left subtree)
else
return search(right subtree)
8
CMPSCI 187
Operations on Binary Search Trees

traverse a tree in pre-, post-, and inorder patterns
 find smallest value in the tree
 find largest value in the tree
 find a specific key (and retrieve item)
These operations allow us
 insert an item into the tree
to construct and
manipulate binary search
 delete an item from the tree
trees.

We can:
 Create a new binary search tree class
 Use the adaptor pattern to implement a binary search tree on
top of our binary tree
 Add binary search tree methods to our existing tree class
9
CMPSCI 187
Binary Search Tree Interface
public interface SearchTree {
// add/remove say whether tree changed
public boolean add (Object e);
public boolean remove (Object e);
// contains tests whether e is in tree
public boolean contains (Object e);
// find and delete return e if present,
// or null if it is not present
public Object find (Object e);
public Object delete (Object e);
}
10
CMPSCI 187
TreeInterface
(same as before)
public interface TreeInterface
{
public Object getRootData();
public int getHeight();
public int getNumberOfNodes();
public boolean isEmpty();
public void clear();
}
11
CMPSCI 187
A Generic Binary Tree
root
57
18
184
5
-8
40
14
28
63
56
229
387
Do we allow duplicate elements in the tree? No (book doesn’t do it)
12
CMPSCI 187
BinarySearchTree:
Implementation
public class BinarySearchTree
extends BinaryTree
implements SearchTree {
// Data Fields
/** Return value from the public add method. */
protected boolean addReturn;
/** Return value from the public delete method. */
protected Object deleteReturn;
//Constructors
public BinarySearchTree()
{super();}
//creates a null tree
public BinarySearchTree(Object e)
{super(e);}
//creates a tree with one node
//containing the data element e
13
CMPSCI 187
Methods to Implement
boolean add (Object e);
boolean remove (Object e);
boolean contains (Object e);
Object find (Object e);
Object delete (Object e);
14
CMPSCI 187
Finding a specific value in the tree
root
57
18
5
-8



184
40
14 28
63
56
229
387
Suppose I gave you a key, like 28.
Return the data item in the node containing 28 or null if there
is no 28 in the tree.
How would you do it?
15
CMPSCI 187
Searching for a node with a value
root
 Search
key = 28
 Compare against root
compare
and choose
57
 If <, choose left
18
 If ==, return root
184
 If >, choose right
 Recurse
on left or right
subtree until either
5
40
63
229
 find element
 run off a leaf node
(failure)
-8
14
28
56
387
16
CMPSCI 187
The method findKey
public BinaryTree findKey(int key) throws BinaryTreeException
{
if (isEmpty()) return null;
// we have an empty tree
Integer dataElement = (Integer) getRootElement();
if (key == dataElement.intValue()) return new BinaryTree(getRootElement());
else
if (key <= dataElement.intValue()) //look for it in the left subtree
return (getLeftTree().findKey(key));
else
//look for it in the right subtree
return (getRightTree().findKey(key));
}
17
CMPSCI 187
Inserting a Key into a Binary Search Tree
root
57
18
184
5
-8


40
14
28
63
56
229
387
Suppose I want to insert the key 60 to this tree.
How would I do it and maintain the structure of the binary search
tree?
18
CMPSCI 187
Inserting a Key
root
57
18
184
5
-8
40
14
28
63
56

Search for the key, as we did
earlier.
 If we don’t find it, the place
where we ‘fall off the tree’ is
where a new node containing
the key should be inserted.
 If we do find it, we have to
229
have a convention about
where to put a duplicate item
(arbitrarily choose the left
387
subtree).
compare
and choose
Element
should be
here if it’s in
the tree!
In this case, we need to create a
new node as the left subtree of the
node containing 63
19
CMPSCI 187
The method insertKey
public void insertKey(int key) throws BinaryTreeException
{ insertKeyNode(root, key); }
private BinaryNode insertKeyNode(BinaryNode t, int key)
throws BinaryTreeException
{
if (t==null)
t = new BinaryNode(new Integer(key));
// we have a null tree, so create a node at the root and return it
else
{ Integer dataElement = (Integer) t.element();
if (key <= dataElement.intValue())
{ t.setLeft(insertKeyNode(t.getLeft(), key)); }
else
{ t.setRight(insertKeyNode(t.getRight(), key));}
}
return t;
}
20
CMPSCI 187



Removing a Key from a Binary
Search Tree
Suppose I wanted to remove a key (and associated data item) from a tree.
How would I do it?
Several cases to consider:
root
57
18
184
5
-8
40
14
28
63
56
229
387
21
CMPSCI 187
Removing a Key

To remove a key R from the binary search tree first
requires that we find R.
 (we know how to do this)



If R is (in) a leaf node, then R’s parent has its
corresponding child set to null.
If R is in a node that has one child, then R’s parent
has its corresponding pointer set to R’s child.
If R is in a node that has two children, the solution is
more difficult.
22
CMPSCI 187
Removing a node with two children

root

57

18

184

5
40
63
229


-8
14
28
56
387

Don’t remove the NODE
containing the key 18.
Instead, consider replacing its
key using a key from the left or
right subtree.
Then remove THAT node from
the tree.
We need to maintain binary tree
properties.
So what values could we use.
What about 14 or 28?
Max value from left subtree or
minimum value from right
subtree!
Why are these the ones to use?
23
CMPSCI 187
Removing a node with two children
root
root
57
57
14
5
-8
184
40
14 28
63
56
28
229
387
5
-8
184
40
14 28
63
56
229
387
Notice that in both cases, the binary tree property is preserved.
24
CMPSCI 187
Removing a node with two children
When no duplicates are allowed, doesn’t matter which node we choose.
 If duplicates are allowed, and if they are stored in the left subtree, then we must
choose the replacement from the left subtree.
 Why?

31
20
5
42
20
42
25 42
Original Tree
Remove Key 31
5
25
20
42
25 42
Selecting from the
Right Subtree
5
42
25 42
Selecting from the
Left Subtree
Which of these two alternatives preserve
the binary search tree property???
25
CMPSCI 187
The method removeKey
public BinaryNode removeKey(int key)
{ return removeKeyNode(root, key); }
private BinaryNode removeKeyNode(BinaryNode root, int key)
{
if (root == null) return null; //should never have an empty tree, but...
Integer dataElement = (Integer) root.element();
if (key<dataElement.intValue()) //if true, remove the node from the left subtree
root.setLeft(removeKeyNode(root.getLeft(),key));
else
if (key>dataElement.intValue()) //if true, remove the node from the right subtree
root.setRight(removeKeyNode(root.getRight(),key));
Assumed methods in
else //found the node with the key
the BinaryTree class;
{
if (root.getRight() == null) root=root.getLeft();
you write them.
else if (root.getLeft() == null) root=root.getRight();
else //two children
{ Integer temp = (Integer) getRightMostData(root.getLeft());
root.setElement(temp);
root.setLeft(removeRightMost(root.getLeft()));}
}
return root;
}
26
CMPSCI 187
BinarySearchTree Test Program

We’ll build the tree from before:
root
57
18
184
5
-8

40
14
28
63
56
229
387
And then play with the entries
27
CMPSCI 187
BinarySearchTree Test Program
Construct tree
public static void main(String[] args)
{
BinarySearchTree myTree = new BinarySearchTree(new Integer(57));
System.out.println("Element at root of tree = " + myTree.getRootData());
System.out.println("Initial tree is: ");
myTree.inorderTraverse();
System.out.println();
••••••••••••••••continued next page
Element at root of tree = 57
Initial tree is:
( 57 )
28
CMPSCI 187
BinarySearchTree Test Program
Add remaining nodes
int[] nodes = {18, 184, 5,40, -8, 14, 28, 56, 63, 229, 387};
for (int i=0; i<nodes.length; i++)
{
System.out.println(("Inserting: "+ nodes[i]));
Inserting: 18
myTree.add(new Integer(nodes[i]));
Tree is now:
(( 18 ) 57 )
System.out.println("Tree is now: ");
Inserting: 184
Tree is now:
myTree.inorderTraverse();
(( 18 ) 57 ( 184 ))
System.out.println();
Inserting: 5
Tree is now:
}
((( 5 ) 18 ) 57 ( 184 ))
……………..
Inserting: 63
Tree is now:
(((( -8 ) 5 ( 14 )) 18 (( 28 ) 40 ( 56 ))) 57 (( 63 ) 184 ))
Inserting: 229
Tree is now:
(((( -8 ) 5 ( 14 )) 18 (( 28 ) 40 ( 56 ))) 57 (( 63 ) 184 ( 229 )))
Inserting: 387
Tree is now:
(((( -8 ) 5 ( 14 )) 18 (( 28 ) 40 ( 56 ))) 57 (( 63 ) 184 ( 229 ( 387 ))))
29
CMPSCI 187
BinarySearchTree Test Program
Look for keys and remove them if found

Look for the keys -8, 22, and 40.
System.out.println("=================================");
Comparable entry = new Integer(-8);
if (myTree.contains(entry))
{
System.out.println("Entry found; now remove it.");
System.out.println("Removing element: "+ entry);
myTree.remove(entry);
System.out.println("Tree is now: ");
myTree.inorderTraverse();
System.out.println();
}
else System.out.println("Entry " + entry + "not found");
====================================================================
Entry found; now remove it.
Removing element: -8
Tree is now:
((( 5 ( 14 )) 18 (( 28 ) 40 ( 56 ))) 57 (( 63 ) 184 ( 229 ( 387 ))))
====================================================================
30
CMPSCI 187
BinarySearchTree Test Program
Look for keys and remove them if found

Look for the keys -8, 22, and 40.
System.out.println("=================================");
Comparable entry = new Integer(22);
if (myTree.contains(entry))
{
System.out.println("Entry found; now remove it.");
System.out.println("Removing element: "+ entry);
myTree.remove(entry);
System.out.println("Tree is now: ");
myTree.inorderTraverse();
System.out.println();
}
else System.out.println("Entry " + entry + "not found");
============================================================
Entry 22 not found
============================================================
31
CMPSCI 187
BinarySearchTree Test Program
Look for keys and remove them if found

Look for the keys -8, 22, and 40.
System.out.println("=================================");
Comparable entry = new Integer(40);
if (myTree.contains(entry))
{
System.out.println("Entry found; now remove it.");
System.out.println("Removing element: "+ entry);
myTree.remove(entry);
System.out.println("Tree is now: ");
myTree.inorderTraverse();
System.out.println();
}
else System.out.println("Entry " + entry + "not found");
==============================================================
Removing element: 40
Tree is now:
((( 5 ( 14 )) 18 ( 28 ( 56 ))) 57 (( 63 ) 184 ( 229 ( 387 ))))
==============================================================
32
CMPSCI 187
BinarySearchTree Test Program
Restore tree and remove node 18
System.out.println("===================================");
System.out.println("Now put the two elements back in the tree.");
myTree.add(new Integer(-8));
myTree.add(new Integer(40));
System.out.println("Tree is now: ");
myTree.inorderTraverse();
System.out.println();
==========================================================================
Now put the two elements back in the tree.
Tree is now:
(((( -8 ) 5 ( 14 )) 18 ( 28 (( 40 ) 56 ))) 57 (( 63 ) 184 ( 229 ( 387 ))))
==========================================================================
33
CMPSCI 187
BinarySearchTree Test Program
Restore tree and remove node 18
System.out.println("==========================");
entry = new Integer(18);
if (myTree.contains(entry))
{
System.out.println("Removing element: "+ entry);
myTree.remove(entry);
System.out.println("Tree is now: ");
myTree.inorderTraverse();
System.out.println();
}
else System.out.println("Entry " + entry + " not found");
System.out.println("================================");
====================================================================
Removing element: 18
Tree is now:
(((( -8 ) 5 ) 14 ( 28 (( 40 ) 56 ))) 57 (( 63 ) 184 ( 229 ( 387 ))))
====================================================================
34
CMPSCI 187
A Test Program, cont’d
System.out.println("The tree has " + t.treeSize() + " nodes.");
System.out.println("Print tree using inorder traversal:");
t.inOrderPrint();
System.out.println();
System.out.println("Print tree using preorder traversal:");
t.preOrderPrint();
System.out.println();
System.out.println("Print tree using postorder traversal:");
t.postOrderPrint();
The tree has 8 nodes.
Print tree using inorder traversal:
2 8 9 15 18 24 28 31
Print tree using preorder traversal:
15 8 2 9 31 24 18 28
Print tree using postorder traversal:
2 9 8 18 28 24 31 15
35
CMPSCI 187
A Test Program, con’td
int key = 24;
BinaryTree temp = t.findKey(key);
System.out.println();
System.out.println("Search key = " + key +"; Value found = "+temp.getRootElement());
Search key = 24; Value found = 24
Search key = 13; Key was not found
key = 13;
Search key = 15; Value found = 15
temp = t.findKey(key);
System.out.println();
if (temp==null) System.out.println("Search key = " + key +"; Key was not found");
else System.out.println("Search key = " + key +"; Value found = "+temp.getRootElement());
key = 15;
temp = t.findKey(key);
System.out.println();
if (temp==null) System.out.println("Search key = " + key +"; Key was not found");
else System.out.println("Search key = " + key +"; Value found = "+temp.getRootElement ());
36
CMPSCI 187
A Test Program, con’td
key=29;
System.out.println("Inserting key = "+key+" into tree.");
t.insertKey(key);
System.out.println("Print tree using inorder traversal:");
t.inOrderPrint();
System.out.println();
key=40;
System.out.println("Inserting key = "+key+" into tree.");
t.insertKey(key);
………….
key=1;
System.out.println("Inserting key = "+key+" into tree.");
t.insertKey(key);
…………..
key=57;
System.out.println("Inserting key = "+key+" into tree.");
t.insertKey(key);
…………..
key=29;
System.out.println("Removing node with key = "+key);
t.removeKey(key);
………….
}
}
Inserting key = 29 into tree.
In insertKeyNode, going right, value at node = 15
In insertKeyNode, going left, value at node = 31
In insertKeyNode, going right, value at node = 24
In insertKeyNode, going right, value at node = 28
Print tree using inorder traversal:
2 8 9 15 18 24 28 29 31
Inserting key = 40 into tree.
In insertKeyNode, going right, value at node = 15
In insertKeyNode, going right, value at node = 31
Print tree using inorder traversal:
2 8 9 15 18 24 28 29 31 40
Inserting key = 1 into tree.
In insertKeyNode, going left, value at node = 15
In insertKeyNode, going left, value at node = 8
In insertKeyNode, going left, value at node = 2
Print tree using inorder traversal:
1 2 8 9 15 18 24 28 29 31 40
Inserting key = 57 into tree.
In insertKeyNode, going right, value at node = 15
In insertKeyNode, going right, value at node = 31
In insertKeyNode, going right, value at node = 40
Print tree using inorder traversal:
1 2 8 9 15 18 24 28 29 31 40 57
Removing node with key = 29
Print tree using inorder traversal:
1 2 8 9 15 18 24 28 31 40 57
37
CMPSCI 187
Priority Queues: Stock Trading


Consider trading of a single security, say Akamai
Technologies, founded in 1998 by CS professors and
students at MIT.
Investors place orders consisting of three items (action,
price, size), where
 action is either buy or sell,
 price is the worst price you are willing to pay for the purchase or
get from your sale, and
 size is the number of shares

At equilibrium, all the buy orders (bids) have prices lower
than all the sell orders (asks)
38
CMPSCI 187
Stock Trading, cont'd.




A level 1 quote gives the highest bid and lowest ask
(as provided by popular financial sites, and e-brokers
for the naive public)
A level 2 quote gives all the bids and asks for
several price steps (Island ECN on the Web and
quote subscriptions for professional traders)
A trade occurs whenever a new order can be
matched with one or more existing orders, which
results in a series of removal transactions
Orders may be canceled at any time
39
CMPSCI 187
A Data Structure for Trading


For each stock, keep two structures, one for the buy
orders (bids), and the other for the sell orders (asks)
Operations that need to be supported
Action
Ask Structure
place an order
insert(price, size)
get level 1 quote min()
trade
removeMin()
cancel
remove(order)


Bid Structure
insert(price, size)
max()
removeMax()
remove(order)
These data structures are called priority queues.
The NASDAQ priority queues support an average daily
trading volume of 1B shares ($50B)
40
CMPSCI 187
Keys and Total Order Relations

A Priority Queue ranks its elements by key with a total
order relation.

Keys:
 - Every element has its own key
 - Keys are not necessarily unique

Total Order Relation
 Denoted by 
 Reflexive: k  k
 Antisymetric: if k1  k2 and k2  k 1 , then k1  k2
 Transitive: if k1  k2 and k2  k3 , then k1  k3
41
CMPSCI 187
Priority Queue Operations

A Priority Queue supports these fundamental
methods on key-element pairs:
 min()
 insertItem(k, e)
 removeMin()

where k is a key and e is the element associated with
the key
42
CMPSCI 187
The Priority Queue ADT

A priority queue P supports the following methods:
 size(): //Return the number of elements in P
 isEmpty(): //Test whether P is empty
 insertItem(k,e): //Insert a new element e with key k into P
 minElement(): //Return (but don’t remove) an element of P
with smallest key; an error occurs if P is empty.
 minKey(): //Return the smallest key in P; an error occurs if P
is empty
 removeMin(): //Remove from P and return an element with the
smallest key; an error condition occurs if P is empty.
43
CMPSCI 187
Sorting with a Priority Queue

A Priority Queue P can be used for sorting a sequence
S by:
 inserting the elements of S into P with a series of
insertItem(e, e) operations
 removing the elements from P in increasing order and
putting them back into S with a series of removeMin()
operations

Such a sort is a template for a family of sorting
algorithms which differ only in how the priority queue is
implemented.
 To be discussed:
 Selection sort,
 insertion sort,
 heap sort
LATER!
But I left the notes
pages in anyway.
44
CMPSCI 187
Pseudo-Code for Sorting Template
Algorithm PriorityQueueSort(S, P):
Input: A sequence S storing n elements, on which a total order
relation is defined, and a Priority Queue P that compares keys
with the same relation
Output: The Sequence S sorted by the total order relation
while !S.isEmpty() do
e  S.removeFirst()
P.insertItem(e, e)
while P is not empty do
e  P.removeMin()
S.insertLast(e)
Phase 1: insert all the elements into the queue
Phase 2: remove items and return to sequence
45
CMPSCI 187
Implementation with an Unsorted Sequence




Try to implement a priority queue with an unsorted
sequence S.
The elements of S are a composition of two
elements, k, the key, and e, the element.
We can implement insertItem() by using insertLast()
on the sequence. This takes O(1) time.
However, because we always insert at the end,
irrespective of the key value, our sequence is not
ordered.
46
CMPSCI 187



Implementation with an Unsorted Sequence
continued
Thus, for methods such as minElement(), minKey(),
and removeMin(), we need to look at all the
elements of S.
The worst case time complexity for these methods is
O(n).
Performance summary
insertItem
minKey, minElement
removeMin
O(1)
O(n)
O(n)
47
CMPSCI 187
Selection Sort

Implement P with an unsorted sequence
 Phase 1: O(n)
 Phase 2:
O(n + (n-1) + …..+ 2 + 1) = O (
n
S
i=1
i ) = O(
n(n+1)
2
)
= O (n2)
Overall complexity of O(n2)
 Sort amounts to selecting the smallest element from
the queue and placing it into the sorted sequence
 Called selection sort.

48
CMPSCI 187
Implementation with a Sorted Sequence


Another implementation uses a sequence S, sorted by
increasing keys
minElement(), minKey(), and removeMin() take O(1) time

However, to implement insertItem(), we must now scan
through the entire sequence in the worst case.
 Thus, insertItem() runs in O(n) time
 Performance summary
insertItem
minKey, minElement
removeMin
O(n)
O(1)
O(1)
49
CMPSCI 187
Insertion Sort

Implement P with a sorted sequence
 Phase 1:
O(n + (n-1) + …..+ 2 + 1) = O (
n
S
i=1
i ) = O( n(n+1)
)
2
= O (n2)
 Phase 2: O(n)



Overall complexity of O(n2)
Sort amounts to inserting the next element in the
sequence into a sorted queue and then copying the
queue back to the sequence.
Called insertion sort.
50