2. Application Examples
Useful for locating items in a list
Used in Huffman coding algorithm
Study games like checkers and chess to
determine winning strategies
Weighted trees used to model various
categories of problems
3. Trees
Definition: A connected undirected
graph with no simple circuits
Characteristics
- No multiple edges
- No loops
5. Which of the following graphs are trees?
a
c
b
d
f
e
b
a
e
f
c
d
f
a
b c
d
e
a
b
d
c
e
f
A B C D
A & B : Trees
C: Not a Tree (Cycle abdcfa)
D: Not a Tree (Not connected). However it is a forest i.e. unconnected
graphs
6. Tree Theorem
Theorem 1 : An undirected Graph is a tree if and only
if there is a unique simple path between any two of
its vertices
Proof (=>) The graph is a Tree and it does not have cycles. If
there were cycles then there will be more than one simple path
between two vertices. Hence, there is a unique simple path
between every pair of vertices
Proof (<=) There exists a unique simple path between any pair
of vertices. Therefore the graph is connected and there are no
cycles in the graph. Hence, the graph is a tree
7. Tree terminology
Rooted tree: one vertex designated as
root and every edge directed away
Parent: u is the parent of v
iff (if and only if) there is an
edge directed from u to v
Child: v is called the child of u
Every non-root node has a unique parent (?)
8. Siblings: vertices with same parent
Ancestors: all vertices on path from the root to this
vertex, excluding the vertex
Descendants: Descendants of vertex v are all vertices
that have v as an ancestor
Internal Nodes: Nodes that have children
External or Leaf Nodes: Nodes that have no children
Tree terminology
9. A
B C
D E F
G H I J K
Root Node = A
Internal Nodes = B, C, E
and F
External Nodes = D,G, H ,I, J and K
Given : Tree rooted at A
Find: Descendants (B), Ancestor (A), Siblings (G) ?
10. Definition - Level
The level of vertex v in a rooted tree is
the length of the unique path from the
root to v
11. What is the level of Ted?
Hal
Lou
Ken
Joe Ted
Sue
Ed
Max
12. Definition - Height
The height of a rooted tree is the
maximum of the levels of its vertices
13. What is the height?
Hal
Lou
Ken
Joe Ted
Sue
Ed
Max
14. Definition: m-ary trees
Rooted tree where every vertex has no
more than ‘m’ children
Full m-ary if every internal vertex has
exactly ‘m’ children (i.e., except
leaf/external vertices).
m=2 gives a binary tree
16. Definition: Binary Tree
Every internal vertex has a maximum of 2
children
An ordered rooted tree is a rooted tree where
the children of each internal vertex are
ordered.
In an ordered binary tree, the two possible
children of a vertex are called the left child
and the right child, if they exist.
18. Definition: Balanced
A rooted binary tree of height h is called
balanced if all its leaves are at levels h
or h-1
19. Is this tree balanced?
Hal
Lou
Ken
Joe Ted
Sue
Ed
Max
20. Tree Property 1
Theorem 2 : A tree with n vertices has n-1 edges
Proof: By induction
Basis Step: for n = 1 there are (1 – 1) 0 edges
Inductive Step: We assume that n vertices has n – 1 edges . We
need to prove that n + 1 vertices have n edges.
A tree with n + 1 vertices can be achieved by adding an edge
from a vertex of the tree to the new vertex (why 1 edge ?). The
total number of edges is (n – 1 + 1 = n).
21. Tree Property 2
Theorem 3: A full m-ary tree with i internal vertices
contains n = mi +1 vertices
Proof: i internal vertices have m children. Therefore, we have mi
vertices. Since the root is not a child we have n = mi + 1
vertices.
22. Tree Property 3
Theorem 4: A full m-ary tree with
1) n vertices has i = (n-1)/m internal vertices and
l = [(m-1)n+1]/m leaves
2) i internal vertices has n = mi + 1 vertices and
l = (m-1)i+1 leaves
3) l leaves has n = (ml-1)/(m-1) vertices and
i = (l-1)/(m-1) internal vertices
23. Tree Property 4
Theorem 5 – Proof for 1)
l: Number of leaves
i: Number of internal vertices
n: total number of vertices.
By theorem 3, n = mi + 1, and we have n = l + i (why ?)
n = mi + 1 => i = (n-1)/m
By subsitution in n = l + i
l = n – i
l = n – (n-1)/m
[(m-1)n+1]/m
24. Tree Property 4
Theorem 5 – Proof for 2)
l: Number of leaves
i: Number of internal vertices
n: total number of vertices.
By theorem 3, n = mi + 1, and we have n = l + i
By subsitution in n = l + i
l + i = mi + 1
l = (m-1) i + 1
25. Tree Property 4
Theorem 5 – Proof for 3)
l: Number of leaves
i: Number of internal vertices
n: total number of vertices.
By theorem 3, n = mi + 1, and we have n = l + i (i = n – l)
By subsitution of i in n = mi + 1
n = m (n –l) + 1
n (m -1) = (ml -1)
n = (ml -1)/ (m -1)
By subsitution of n in i = n – l we have i = (l – 1)/(m – 1)
26. Tree Property 5
Theorem 6: There are at most mh leaves in a m-ary
tree of height h
Proof: By Induction
Basic Step: m-ary tree of height 1 => leaves = m
which is true since the root has m-ary children
27. Tree Property 5
Inductive Step:
Assume that the result is true for tree of height less than h
Let Tree T be m-ary of height h
To build T we can add m such sub-tree of height ≤ h -1 such
that the root node acts as the child to the root of T
T has ≤ m.mh -1 ≤ mh
30. Binary Search Trees (BST)
Applications
A Binary Search Tree can be used to
store items in its vertices
It enables efficient searches
31. Definition: BST
A special kind of binary tree in which:
Each vertex contains a distinct key value,
The key values in the tree can be compared
using “greater than” and “less than”, and
The key value of each vertex in the tree is
less than every key value in its right subtree,
and greater than every key value in its left
subtree.
32. Shape of a binary search tree
Depends on its key values and their order of insertion.
Example: Insert the elements 6, 3, 4, 2, 10 in that
order . The first value to be inserted is put into the
root
6
33. Shape of a binary search tree
Inserting 3 into the BST
6
3
34. Shape of a binary search tree
Inserting 4 into the BST
6
3
4
35. Shape of a binary search tree
Inserting 4 into the BST
6
3
4
2
36. Shape of a binary search tree
Inserting 10 into the BST
6
3
4
2
10
12
37. Binary search tree algorithm
procedure insertion (T:binary search tree, X: item)
v:= root of T
{a vertex not present in T has the value null}
while v != null and label(v) != x
begin
if x < label(v) then
if left child of v != null then v:=left child of v
else add new vertex as left child of v and set v:= null
else
if right child of v != null then v:=right child of v
else add new vertex as right child of v to T and set v := null
end
if root of T = null then add a vertex v to the tree and label it with x
else if v is null or label(v) != x then label new vertex with x and let v be this
new vertex
{v = location of x}
38. Codes
Text Encoding
Our next goal is to develop a code that represents a given text as compactly
as possible.
A standard encoding is ASCII, which represents every character using 7 bits:
“An English sentence” = 133 bits ≈ 17 bytes
1000001 (A) 1101110 (n) 0100000 ( ) 1000101 (E) 1101110 (n) 1100111 (g)1101100
(l) 1101001 (i) 1110011 (s) 1101000 (h)
0100000 ( ) 1110011 (s) 1100101 (e) 1101110 (n) 1110100 (t) 1100101 (e) 1101110
(n) 1100011 (c)
1100101 (e)
39. Codes
Text Encoding
Of course, this is wasteful because we can encode 12 characters in 4 bits:
‹space› = 0000 A = 0001 E = 0010 c = 0011 e = 0100 g = 0101 h =
0110
i = 0111 l = 1000 n = 1001 s = 1010 t = 1011
Then we encode the phrase as
0001 (A) 1001 (n) 0000 ( ) 0010 (E) 1001 (n) 0101 (g) 1000 (l) 0111 (i)
1010 (s) 0110 (h) 0000 ( ) 1010 (s) 0100 (e) 1001 (n) 1011 (t) 0100
(e)
1001 (n) 0011 (c) 0100 (e)
This requires 76 bits ≈ 10 bytes
40. Codes
Text Encoding
An even better code is given by the following encoding:
‹space› = 000 A = 0010 E = 0011 s = 010 c = 0110 g = 0111
h = 1000 i = 1001 l = 1010 t = 1011 e = 110 n = 111
Then we encode the phrase as
0010 (A) 111 (n) 000 ( ) 0011 (E) 111 (n) 0111 (g) 1010 (l)
1001 (i) 010 (s) 1000 (h) 000 ( ) 010 (s) 110 (e) 111 (n) 1011
(t) 110 (e) 111 (n) 0110 (c) 110 (e)
This requires 65 bits ≈ 9 bytes
41. Codes
Codes That Can Be Decoded
Fixed-length codes:
Every character is encoded using the same number of bits.
To determine the boundaries between characters, we form
groups of w bits, where w is the length of a character.
Examples:
■ASCII
■Our first improved code
Prefix codes:
No character is the prefix of another character.
Examples:
■Fixed-length codes
■Huffman codes
42. Why Prefix Codes ?
Consider a code that is not a prefix code:
a = 01 m = 10 n = 111 o = 0 r = 11 s = 1 t = 0011
Now you send a fan-letter to your favorite movie star. One of
the sentences is “You are a star.”
You encode “star” as “1 0011 01 11”.
Your idol receives the letter and decodes the text using your
coding table:
100110111 = 10 0 11 0 111 = “moron”
43. Our example: ‹space› = 000 A = 0010 E = 0011 s = 010 c = 0110 g =
0111 h = 1000 i = 1001 l = 1010 t = 1011 e = 110 n = 111
Representing a Prefix-Code Dictionary
‹spc›
A E
s
c g h i l t
e n
0 1
0
0
0
0
0
0
0
0
0 0
1
1
1
1 1 1 1
1 1
1
44. Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
Huffman’s Algorithm
f:5 b:13
c:12 d:16
e:9 a:45
45. Huffman’s Algorithm
f:5 b:13
c:12 d:16
e:9 a:45
14
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
48. Huffman’s Algorithm
f:5 b:13
c:12
d:16
e:9
a:45
14 25
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
49. Huffman’s Algorithm
f:5 b:13
c:12
d:16
e:9
a:45
14 25
30
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
50. Huffman’s Algorithm
f:5
b:13
c:12 d:16
e:9
a:45
14
25 30
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
51. Huffman’s Algorithm
f:5
b:13
c:12 d:16
e:9
a:45
14
25 30
55
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
52. Huffman’s Algorithm
f:5
b:13
c:12 d:16
e:9
a:45
14
25 30
55
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
53. Huffman’s Algorithm
f:5
b:13
c:12 d:16
e:9
a:45
14
25 30
55
100
0 1
0
0 0
0
1
1 1
1
0
100 101
1100 1101
111
Huffman(C)
1 n ← |C|
2 Q ← C
3 for i = 1..n – 1
4 do allocate a new node z
5 left[z] ← x ← Delete-Min(Q)
6 right[z] ← y ← Delete-Min(Q)
7 f [z] ← f [x] + f [y]
8 Insert(Q, z)
9 return Delete-Min(Q)
54. Game Trees
Writing down all the possible moves by both players down onto a tree
Root node represents the current status
Internal nodes at even levels correspond to positions in which the first player
is to move (MAX nodes)
Internal nodes at odd levels correspond to positions in which the second
player is to move (Min nodes)
Leaves correspond to positions at which the game has ended. One player or
the other has won, or perhaps the game is drawn.
55. Game Trees: Partial Game Tree for Tic-Tac-Toe
Here's a little piece of the game tree for Tic-Tac-Toe, starting from an empty board.
Note that even for this trivial game, the search tree is quite big.
56. Game Trees
In case of TIC TAC TOE, there are 9 choices for the first player
Next, the opponent have 8 choices, …, and so on
Number of nodes = 1 + 9 + 9*8 + 9*8*7 + … +9*8*7*6*5*4*3*2*1 = ???
Exponential order of nodes
If the depth of the tree is m and there are b moves at each node,
the time complexity of evaluating all the nodes are O(bm)
Even in case of TICTACTOE, there are so many nodes
57. Game Trees: Min-Max Procedure
So instead of expanding the whole tree, we use a min-max
strategy
Starting from the current game position as the node, expand the
tree to some depth
Apply the evaluation function at each of the leaf nodes
“Back up” values for each of the non-leaf nodes until a value is
computed for the root node
At MIN nodes, the backed-up value is the minimum of the values
associated with its children.
At MAX nodes, the backed-up value is the maximum of the values
associated with its children.
Pick the edge at the root associated with the child node whose
backed-up value determined the value at the root
58. Game Trees: Minimax Search
2 7 1 8
MAX
MIN
2 7 1 8
2 1
2 7 1 8
2 1
2
2 7 1 8
2 1
2
This is the move
selected by minimax
Static evaluator
value
59. Tree Traversal
Information is stored in Ordered Rooted tree
Methods are needed for visiting each vertex of the
tree in order to access data
Three most commonly used traversal algorithms are:
1. Preorder Traversal
2. Inorder Traversal
3. Postorder Traversal
60. Preorder Traversal
Let T be an ordered binary tree with root r
If T has only r, then r is the preorder traversal. Otherwise,
suppose T1, T2 are the left and right subtrees at r. The
preorder traversal begins by visiting r. Then traverses T1
in preorder, then traverses T2 in preorder.
62. Preorder Traversal
Procedure preorder (T: ordered rooted tree)
r := root of T
List r;
for each child c of r from left to right
T(c) := subtree with c as its root
preorder(T(c));
End.
63. Inorder Traversal
Let T be an ordered binary tree with root r.
If T has only r, then r is the inorder traversal. Otherwise, suppose
T1, T2 are the left and right subtrees at r. The inorder traversal
begins by traversing T1 in inorder. Then visits r, then traverses T2
in inorder.
65. Inorder Traversal
Procedure inorder (T: ordered rooted tree)
r := root of T
if r is a leaf then list r
else
begin
l := first child of r from left to right;
T(l) := subtree with l as its root
inorder (T(l));
list r;
for each child c of r except for l from left to right
T(c) := subtree with c as its root
inorder(T(c));
End.
66. Postorder Traversal
Let T be an ordered binary tree with root r.
If T has only r, then r is the postorder traversal. Otherwise,
suppose T1, T2 are the left and right subtrees at r. The postorder
traversal begins by traversing T1 in postorder. Then traverses T2
in postorder, then ends by visiting r.
68. Postorder Traversal
Procedure postorder (T: ordered rooted tree)
r := root of T
for each child c of r from left to right
T(c) := subtree with c as its root
postorder(T(c));
End.
List r.
69. Binary Expression Tree
Ordered rooted tree representing complicated expression such
compound propositions, combinations of sets, and arithmetic
expression are called Binary Expression Tree.
In this special kind of tree,
1. Each leaf node contains a single operand,
2. Each nonleaf node contains a single binary operator, and
3. The left and right subtrees of an operator node represent
subexpressions that must be evaluated before applying the
operator at the root of the subtree.
70. Infix, Prefix, and Postfix Expressions
‘*’
‘-’
‘8’ ‘5’
‘/’
‘+’
‘4’
‘3’
‘2’
What infix, prefix, postfix expressions does it represent?
72. Spanning Tree
A spanning tree of a (connected) graph
G is a connected, acyclic subgraph of G
that contains all of the vertices of G
73. Spanning Tree Example
Below is an example of a spanning tree T, where the
edges in T are drawn as solid lines and the edges in G
but not in T are drawn as dotted lines.
A
B
C
D
E
J
I
H
G
F
74. Algorithms to find spanning
tree in a graph
Depth-First-Search (DFS)
Breadth-First-Search (BFS)
75. Depth First Search (DFS)
• The algorithm starts from a node, selects a node connected to it,
then selects a node connected to this new node and so on, till no
new nodes remain.
• Then, it backtracks to the node next to the last one and discovers
any new nodes connected to it.
• Move back 2 vertices in the path and try again.
• Repeat this move back operation until no more edges to be
added.
• A different ordering of the vertices can lead to a different DFS
spanning tree
• Stacks are used to implement DFS algorithms
82. Breadth First Search (BFS)
Compared to DFS breadth first search is a simple
algorithm
Timidly tries one edge. Totally exhaust neighbors of a
vertex then goes to next neighbors. It radiates in
waves in balanced manner.
Implemented using queues
Whatever is in queues tells u what to explore next.
Once the queue is empty algorithm comes to an end
89. Given: Graph G=(V, E), |V|=n
Cost function c: E R .
Goal: Find a minimum-cost spanning tree for V i.e.,
find a subset of arcs E* E which connects any two
nodes of V with minimum possible cost
Example:
2
3
3
4
4
5
7
8
e
b
c
d
a
2
3
3
4
4
5
7
8
e
b
c
d
a
G=(V,E) Min. span. tree: G*=(V,E*)
Red bold arcs are in E*
Minimum Spanning Tree (Min SPT)
91. Prims Algorithm
Procedure Prims (G: weighted connected undirected graph with n
vertices)
T := a minimum-weight edge
For i := 1 to n-2
Begin
e:= an edge of minimum weight incident to a vertex in T and
not forming a simple circuit in T if added to T
T := T with e added
End {T is a minimum spanning tree of G}
97. Kruskal’s algorithm
Procedure Kruskal (G: weighted connected undirected graph
with n vertices)
T := empty graph
For i := 1 to n-1
Begin
e := any edge in G with smallest weight that does not form
simple circuit when added to T
T := T with e added
End { T is the minimum spanning tree of G}
103. Prim’s and Kruskal’s
Prim’s
Chosen edges: minimum weight & incident
to a vertex already in the tree.
Kruskal’s
Chosen edge: minimum weight & not
necessarily incident to a vertex in the tree.