1. 1
Binary Search Trees
basic implementations
randomized BSTs
deletion in BSTs
References:
Algorithms in Java, Chapter 12
Intro to Programming, Section 4.4
http://www.cs.princeton.edu/introalgsds/43bst

2. 2
Elementary implementations: summary
Challenge:
Efficient implementations of get() and put() and ordered iteration.
implementation
worst case average case ordered
iteration?
operations
on keys
search insert search insert
unordered array N N N/2 N/2 no equals()
ordered array lg N N lg N N/2 yes compareTo()
unordered list N N N/2 N no equals()
ordered list N N N/2 N/2 yes compareTo()

3. 3
basic implementations
randomized BSTs
deletion in BSTs

4. 4
Binary Search Trees (BSTs)
Def. A BINARY SEARCH TREE is a binary tree in symmetric order.
A binary tree is either:
• empty
• a key-value pair and two binary trees
[neither of which contain that key]
Symmetric order means that:
• every node has a key
• every node’s key is
larger than all keys in its left subtree
smaller than all keys in its right subtree
smaller larger
x
node
subtrees
the
was
it
of times
best
equal keys ruled out to facilitate
associative array implementations

5. 5
BST representation
A BST is a reference to a Node.
A Node is comprised of four fields:
• A key and a value.
• A reference to the left and right subtree.
Key and Value are generic types;
Key is Comparable
root
it 2
was 2
the 1
best 1
of 1 times 1
private class Node
{
Key key;
Value val;
Node left, right;
}
smaller keys larger keys

6. public class BST<Key extends Comparable<Key>, Value>
implements Iterable<Key>
{
private Node root;
private class Node
{
Key key;
Value val;
Node left, right;
Node(Key key, Value val)
{
this.key = key;
this.val = val;
}
}
public void put(Key key, Value val)
// see next slides
public Val get(Key key)
// see next slides
}
6
BST implementation (skeleton)
instance variable
inner class

7. 7
BST implementation (search)
public Value get(Key key)
{
Node x = root;
while (x != null)
{
int cmp = key.compareTo(x.key);
if (cmp == 0) return x.val;
else if (cmp < 0) x = x.left;
else if (cmp > 0) x = x.right;
}
return null;
}
get(“the”)
returns 1
get(“worst”)
returns null
root
it 2
was 2
the 1
best 1
of 1 times 1

8. 8
BST implementation (insert)
public void put(Key key, Value val)
{ root = put(root, key, val); }
root
it 2
was 2
the 1
best 1
of 1 times 1
put(“the”, 2)
overwrites the 1
put(“worst”, 1)
adds a new entry
worst 1
private Node put(Node x, Key key, Value val)
{
if (x == null) return new Node(key, val);
int cmp = key.compareTo(x.key);
if (cmp == 0) x.val = val;
else if (cmp < 0) x.left = put(x.left, key, val);
else if (cmp > 0) x.right = put(x.right, key, val);
return x;
}
Caution: tricky recursive code.
Read carefully!

9. 9
BST: Construction
Insert the following keys into BST. A S E R C H I N G X M P L

10. Tree shape.
• Many BSTs correspond to same input data.
• Cost of search/insert is proportional to depth of node.
Tree shape depends on order of insertion
10
Tree Shape
E
S
A
C
H
H
A E I S
C R
R
I
H
A
E
I
S
C
R
typical best
worst

11. BST implementation: iterator?
11
public Iterator<Key> iterator()
{ return new BSTIterator(); }
private class BSTIterator
implements Iterator<Key>
{
BSTIterator()
{ }
public boolean hasNext()
{ }
public Key next()
{ }
}
E
S
A
C
H R
I
N

12. BST implementation: iterator?
12
public void visit(Node x)
{
if (x == null) return;
visit(x.left)
StdOut.println(x.key);
visit(x.right);
}
E
S
A
C
H R
I
N
Approach: mimic recursive inorder traversal
visit(E)
visit(A)
print A
visit(C)
print C
print E
visit(S)
visit(I)
visit(H)
print H
print I
visit(R)
visit(N)
print N
print R
print S
A
C
E
H
I
N
R
S
E
A E
E
C E
E
S
I S
H I S
I S
S
R S
N R S
R S
S
Stack contents
To process a node
• follow left links until empty
(pushing onto stack)
• pop and process
• process node at right link

13. 13
BST implementation: iterator
public Iterator<Key> iterator()
{ return new BSTIterator(); }
private class BSTIterator
implements Iterator<Key>
{
private Stack<Node>
stack = new Stack<Node>();
private void pushLeft(Node x)
{
while (x != null)
{ stack.push(x); x = x.left; }
}
BSTIterator()
{ pushLeft(root); }
public boolean hasNext()
{ return !stack.isEmpty(); }
public Key next()
{
Node x = stack.pop();
pushLeft(x.right);
return x.key;
}
}
E
S
A
C
H R
I
A E
A C E
C E
E H I S
H I S
I N R S
N R S
R S
S
N

14. 1-1 correspondence between BSTs and Quicksort partitioning
14
A
C
E
I
K
L
M
O
P
Q
R
S
T
U
XE
no
equal
keys

15. 15
BSTs: analysis
Theorem. If keys are inserted in random order, the expected number
of comparisons for a search/insert is about 2 ln N.
Proof: 1-1 correspondence with quicksort partitioning
Theorem. If keys are inserted in random order, height of tree
is proportional to lg N, except with exponentially small probability.
But… Worst-case for search/insert/height is N.
e.g., keys inserted in ascending order
mean 6.22 lg N, variance = O(1)
1.38 lg N, variance = O(1)

16. Searching challenge 3 (revisited):
Problem: Frequency counts in “Tale of Two Cities”
Assumptions: book has 135,000+ words
about 10,000 distinct words
Which searching method to use?
1) unordered array
2) unordered linked list
3) ordered array with binary search
4) need better method, all too slow
5) doesn’t matter much, all fast enough
6) BSTs
16
insertion cost < 10000 * 1.38 * lg 10000 < .2 million
lookup cost < 135000 * 1.38 * lg 10000 < 2.5 million

17. 17
Elementary implementations: summary
Next challenge:
Guaranteed efficiency for get() and put() and ordered iteration.
implementation
guarantee average case ordered
iteration?
operations
on keys
search insert search insert
unordered array N N N/2 N/2 no equals()
ordered array lg N N lg N N/2 yes compareTo()
unordered list N N N/2 N no equals()
ordered list N N N/2 N/2 yes compareTo()
BST N N 1.38 lg N 1.38 lg N yes compareTo()

18. 18
basic implementations
randomized BSTs
deletion in BSTs

19. Two fundamental operations to rearrange nodes in a tree.
• maintain symmetric order.
• local transformations (change just 3 pointers).
• basis for advanced BST algorithms
Strategy: use rotations on insert to adjust tree shape to be more balanced
Key point: no change in search code (!)
19
Rotation in BSTs
h = rotL(u)
h = rotR(v)
A B
C
CB
A
u
h
h
v
u
v

20. 20
Rotation
Fundamental operation to rearrange nodes in a tree.
• easier done than said
• raise some nodes, lowers some others
private Node rotL(Node h)
{
Node v = h.r;
h.r = v.l;
v.l = h;
return v;
}
private Node rotR(Node h)
{
Node u = h.l;
h.l = u.r;
u.r = h;
return u;
}
root = rotL(A) A.left = rotR(S)

21. 21
Recursive BST Root Insertion
Root insertion: insert a node and make it the new root.
• Insert as in standard BST.
• Rotate inserted node to the root.
• Easy recursive implementation
insert G
private Node putRoot(Node x, Key key, Val val)
{
if (x == null) return new Node(key, val);
int cmp = key.compareTo(x.key);
if (cmp == 0) x.val = val;
else if (cmp < 0)
{ x.left = putRoot(x.left, key, val); x = rotR(x); }
else if (cmp > 0)
{ x.right = putRoot(x.right, key, val); x = rotL(x); }
return x;
}
Caution: very tricky recursive
code.
Read very carefully!

22. 22
Constructing a BST with root insertion
Ex. A S E R C H I N G X M P L
Why bother?
• Recently inserted keys are near the top (better for some clients).
• Basis for advanced algorithms.

23. Randomized BSTs (Roura, 1996)
Intuition. If tree is random, height is logarithmic.
Fact. Each node in a random tree is equally likely to be the root.
Idea. Since new node should be the root with probability 1/(N+1),
make it the root (via root insertion) with probability 1/(N+1).
23
private Node put(Node x, Key key, Value val)
{
if (x == null) return new Node(key, val);
int cmp = key.compareTo(x.key);
if (cmp == 0) { x.val = val; return x; }
if (StdRandom.bernoulli(1.0 / (x.N + 1.0))
return putRoot(h, key, val);
if (cmp < 0) x.left = put(x.left, key, val);
else if (cmp > 0) x.right = put(x.right, key, val);
x.N++;
return x;
}
need to maintain count of
nodes in tree rooted at x

24. 24
Constructing a randomized BST
Ex: Insert distinct keys in ascending order.
Surprising fact:
Tree has same shape as if keys were
inserted in random order.
Random trees result from any insert order
Note: to maintain associative array abstraction
need to check whether key is in table and replace
value without rotations if that is the case.

25. 25
Randomized BST
Property. Randomized BSTs have the same distribution as BSTs under
random insertion order, no matter in what order keys are inserted.
• Expected height is ~6.22 lg N
• Average search cost is ~1.38 lg N.
• Exponentially small chance of bad balance.
Implementation cost. Need to maintain subtree size in each node.

26. Summary of symbol-table implementations
Randomized BSTs provide the desired guarantee
Bonus (next): Randomized BSTs also support delete (!) 26
implementation
guarantee average case ordered
iteration?
operations
on keys
search insert search insert
unordered array N N N/2 N/2 no equals()
ordered array lg N N lg N N/2 yes compareTo()
unordered list N N N/2 N no equals()
ordered list N N N/2 N/2 yes compareTo()
BST N N 1.38 lg N 1.38 lg N yes compareTo()
randomized BST 7 lg N 7 lg N 1.38 lg N 1.38 lg N yes compareTo()
probabilistic, with
exponentially small
chance of quadratic time

27. 27
basic implementations
randomized BSTs
deletion in BSTs

28. 28
BST delete: lazy approach
To remove a node with a given key
• set its value to null
• leave key in tree to guide searches
[but do not consider it equal to any search key]
Cost. O(log N') per insert, search, and delete, where N' is the number
of elements ever inserted in the BST.
Unsatisfactory solution: Can get overloaded with tombstones.
E
S
A
C
H R
I
N
E
S
A
C
H R
I
N
remove I a “tombstone”

29. 29
BST delete: first approach
To remove a node from a BST. [Hibbard, 1960s]
• Zero children: just remove it.
• One child: pass the child up.
• Two children: find the next largest node using right-left*
swap with next largest
remove as above.
Unsatisfactory solution. Not symmetric, code is clumsy.
Surprising consequence. Trees not random (!) sqrt(N) per op.
Longstanding open problem: simple and efficient delete for BSTs
zero children one child two children

30. Deletion in randomized BSTs
To delete a node containing a given key
• remove the node
• join the two remaining subtrees to make a tree
Ex. Delete S in
30
E
S
A
C
H R
I
N
X

31. Deletion in randomized BSTs
To delete a node containing a given key
• remove the node
• join its two subtrees
Ex. Delete S in
31
E
A
C
H R
I
N
X
join these
two subtrees
private Node remove(Node x, Key key)
{
if (x == null)
return new Node(key, val);
int cmp = key.compareTo(x.key);
if (cmp == 0)
return join(x.left, x.right);
else if (cmp < 0)
x.left = remove(x.left, key);
else if (cmp > 0)
x.right = remove(x.right, key);
return x;
}

32. Join in randomized BSTs
To join two subtrees with all keys in one less than all keys in the other
• maintain counts of nodes in subtrees (L and R)
• with probability L/(L+R)
make the root of the left the root
make its left subtree the left subtree of the root
join its right subtree to R to make the right subtree of the root
• with probability L/(L+R) do the symmetric moves on the right
32
H R
I
N
X
to join these
two subtrees
H
R
N
X
make I the root
with probability 4/5
I
need to join these
two subtrees

33. Join in randomized BSTs
To join two subtrees with all keys in one less than all keys in the other
• maintain counts of nodes in subtrees (L and R)
• with probability L/(L+R)
make the root of the left the root
make its left subtree the left subtree of the root
join its right subtree to R to make the right subtree of the root
• with probability L/(L+R) do the symmetric moves on the right
33X
to join these
two subtrees
R
N
X
make R the root
with probability 2/3
R
N
private Node join(Node a, Node b)
{
if (a == null) return a;
if (b == null) return b;
int cmp = key.compareTo(x.key);
if (StdRandom.bernoulli((double)*a.N / (a.N + b.N))
{ a.right = join(a.right, b); return a; }
else
{ b.left = join(a, b.left ); return b; }
}

34. Deletion in randomized BSTs
To delete a node containing a given key
• remove the node
• join its two subtrees
Ex. Delete S in
Theorem. Tree still random after delete (!)
Bottom line. Logarithmic guarantee for search/insert/delete 34
E
S
A
C
H R
I
N
X
E
X
A
C H
R
I
N

35. Summary of symbol-table implementations
Randomized BSTs provide the desired guarantees
Next lecture: Can we do better? 35
implementation
guarantee average case ordered
iteration?
search insert delete search insert delete
unordered array N N N N/2 N/2 N/2 no
ordered array lg N N N lg N N/2 N/2 yes
unordered list N N N N/2 N N/2 no
ordered list N N N N/2 N/2 N/2 yes
BST N N N 1.38 lg N 1.38 lg N ? yes
randomized BST 7 lg N 7 lg N 7 lg N 1.38 lg N 1.38 lg N 1.38 lg N yes
probabilistic, with
exponentially small
chance of error