The document discusses graph search algorithms. It begins by defining the problem of finding all nodes reachable from a single source node s in a graph G. It then outlines the basic steps of a graph search: (1) start with the known reachable nodes as just s; (2) repeatedly find neighbors of known reachable nodes and add them as new reachable nodes; (3) terminate when no more progress can be made. It then describes breadth-first search (BFS) in detail, showing how BFS finds nodes in order of distance from s by maintaining a queue of found nodes not yet handled. At each step, BFS handles the next node in the queue, finds its neighbors, and adds them to the found

1. Searching in a Graph
Jeff Edmonds
York University COSC 3101LectureLecture 55

2. Generic Search
Breadth First Search
Dijkstra's Shortest Paths Algorithm
Depth First Search
Linear Order
Network Flow
Linear Programming

3. Graph
a
c
b Node ~ city or computer
Edge ~ road or network
Undirected or Directed
A surprisingly large number of problems
in computer science can be expressed
as a graph theory problem.

4. Graph Search
Specification: Reachability-from-single-source s
•Preconditions:
The input is a graph G
(either directed or undirected)
and a source node s.
•Postconditions:
Output all the nodes u that are
reachable by a path in G from s.

5. Graph Search
Basic Steps:
s
u
Suppose you know that
u is reachable from s
v
& there is an edge
from u to v
You know that
v is reachable from s
Build up a set of reachable nodes.

6. Graph Search
s
reachable
f
u
k reachable
d
v
w reachable
e
h
d reachable
b
t
reachable
d
v
w reachable
e
h
d reachable
b
t
•How do we keep track of
all of this information?
•How do we avoid cycling?

7. s
a
c
h
k
f
i
l
m
j
e
b
g
d
Graph Search

8. s
a
c
h
k
f
i
l
m
j
e
b
g
d
Graph Search

9. s
a
c
h
k
f
i
l
m
j
e
b
g
d
Graph Search

10. s
a
c
h
k
f
i
m
j
e
b
g
d
We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
l

11. s
a
c
h
k
f
i
m
j
e
b
g
d
We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
l
Handle some foundNotHandled node
i.e. find its neighbors
Don’t refind a node.

12. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
Handle some foundNotHandled node
i.e. find its neighbors
s
a
c
h
k
f
i
m
j
e
b
g
d
l

13. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
# of found nodes.
s
a
c
h
k
f
i
m
j
e
b
g
d
l
measure
progress
# of handled nodes.
Might not increase.

14. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
Node s is foundNotHandled
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Other nodes NotFound

15. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
All nodes found.
Exit
a
c
h
k
f
i
m
j
e
b
g
d
l
When can’t make any more progress.
No. Might not find all.
When all found nodes are have been handled.
Handle some foundNotHandled node
s

17. Graph Search
We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
a
c
h
k
f
i
m
j
e
b
g
d
l
Exit All found nodes are handled.
Exit
•Postconditions:
Output all the nodes u that are
reachable by a path in G from s.
Output Found nodes

18. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
Exit
a
c
h
k
f
i
m
j
e
b
g
d
l
All found nodes are handled.
Exit Found nodes are reachable from s.
Reachable nodes have been Found.
•Postconditions:

19. We know found nodes are
reachable from s because we
have traced out a path.
If a node has been handled,
then all of its neighbors
have been found.
Graph Search
Exit
a
c
h
k
f
i
m
j
e
b
g
d
l
All found nodes are handled.
Exit Reachable nodes have been Found.
•Postconditions:
found = handled
handled
Notfound
[A ⇒ B] = [¬B ⇒ ¬A]
NotFound nodes not reachable.

20. Graph Search
Specification of Reachability-from-single-source s
•Preconditions:
The input is a graph G
(either directed or undirected)
and a source node s.
•Postconditions:
Output all the nodes u that are
reachable by a path in G from s.
EndingInitial ConditionsMake Progress
Maintain Loop InvDefine Exit ConditionDefine Step
Define Measure of
Progress
Define Loop
Invariants
Define Problem
km∞
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit

21. Graph Search
# of handled nodes.
Handle some foundNotHandled node
Time = O(n)
f
u
k
O(n) iterations,
but iteration takes more than O(1)
O(n) neighbors= O(n2
)
Could be fewer?
Each edge visited, times.2
= O(E)
Size = O(E)
Linear time.

22. Graph Search
Handle some foundNotHandled node
•Queue: Handle in order found.
•Breadth-First Search
•Stack: Handle most recently found
•Depth-First Search
•Priority Queue:
Handle node that seems to be closest to s.
•Shortest (Weighted) Paths:

23. Breadth First Search
Pre & Post Conditions
Algorithm

24. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue

25. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
s

26. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
a
b
g
d

27. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
b
g
d
c
f

28. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
b
g
c
f
m
e

29. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
b
c
f
m
e
j

30. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
c
f
m
e
j

31. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
f
m
e
j
h
i

32. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
m
e
j
h
i

33. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
e
j
h
i
l

34. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
j
h
i
l

35. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
h
i
l

36. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
i
l
k

37. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
l
k

38. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
k

39. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue

40. BFS
What order are the nodes found?
i.e. Breadth First Search
Algorithm
So far, the nodes have been
found in order of length from s.

41. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue

42. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
s
d=0
d=0

43. BFS Found
Not Handled
Queue
d=0
a
b
g
d
d=1
s
a
c
h
k
f
i
l
m
j
e
b
g
d
d=0
d=1

44. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
a
b
g
d
d=0
d=1
d=1

45. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
b
g
d
c
f
d=0
d=1
d=2
d=1
d=2

46. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
b
g
c
f
m
e
d=0
d=1
d=2
d=1
d=2

47. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
d=0
d=1
d=2
b
j
c
f
m
e
d=1
d=2

48. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
d=0
d=1
d=2
j
c
f
m
e
d=1
d=2

49. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
c
f
m
e
j
d=0
d=1
d=2
d=2

50. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
f
m
e
j
h
i
d=0
d=1
d=2
d=3
d=2
d=3

51. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
m
e
j
h
i
d=0
d=1
d=2
d=3
d=2
d=3

52. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
e
j
h
i
l
d=0
d=1
d=2
d=3
d=2
d=3

53. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
j
h
i
l
d=0
d=1
d=2
d=3
d=2
d=3

54. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
h
i
l
d=0
d=1
d=2
d=3
d=2
d=3

55. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
h
d=0
d=1
d=2
d=3
i
l
d=3

56. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
i
l
k
d=0
d=1
d=2
d=3
d=4
d=3
d=4

57. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
l
k
d=0
d=1
d=2
d=3
d=4
d=3
d=4

58. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
k
d=0
d=1
d=2
d=3
d=4
d=3
d=4

59. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
k
d=0
d=1
d=2
d=3
d=4
d=4

60. BFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Queue
d=0
d=1
d=2
d=3
d=4
d=4
d=5

61. BFS
What order are the nodes found?
So far, the nodes have been
found in order of length from s.
Post Conditions: Finds a shortest path from s
to each node v and its length.
To prove path is shortest:
Prove there is a path of this length.
Prove there are no shorter paths.
Give a path
(witness)

62. Basic Steps:
s
u
The shortest path to u
has length d
v
& there is an edge
from u to v
There is a path to v with length d+1.
BFS

63. BFS
What order are the nodes found?
So far, the nodes have been
found in order of length from s.
Post Conditions: Finds a shortest path from s
to each node v and its length.
When we find v, we know
there isn't a shorter path to it because ?
Prove there are no shorter paths.
Otherwise, we would have found it already.

64. BFS
Data structure for storing tree:
For each node v, store π(v) to be parent of v.

65. Basic Steps:
s
u
Path to u
v
& there is an edge
from u to v
Parent of v is
π(v)
π(v) = u.
BFS

67. Graph Search
Handle some foundNotHandled node
•Queue: Handle in order found.
•Breadth-First Search
•Stack: Handle most recently found
•Depth-First Search
•Priority Queue:
Handle node that seems to be closest to s.
•Shortest (Weighted) Paths:

68. Dijkstra's Shortest-Weighted Paths
Specification: Dijkstra's Shortest-Weighted Paths
Reachability-from-single-source s
•Preconditions:
The input is a graph G
(either directed or undirected)
with positive edge weights
and a source node s.
•Postconditions:
Finds a shortest weighted path from s
to each node v and its length.

69. Dijkstra's Shortest-Weighted Paths
s
u
v
100
1
1
1
1
w
r
Length of shortest
path from s to u?
4

70. So far, the nodes have been
found in order of length from s.
BFS
s
u
v
100
1
1
1
1
w
r
Which node is found first?
Is the same true for Dijkstra's Alg?

71. So far, the nodes have been
found in order of length from s.
s
u
v
100
1
1
1
1
w
r
It has the longest path from s.
Which node is found first?
Is the same true for Dijkstra's Alg?
BFS

72. So far, the nodes have been
found in order of length from s.
s
u
v
100
1
1
1
1
w
r
In what order do we handle the FoundNotHandled nodes?
handled
Dijkstra's
Handle node that seems to be closest to s.

73. To prove path is shortest:
Prove there is a path of this length.
Prove there are no shorter paths.
Give a path
(witness)
Dijkstra's
So far, the nodes have been
handled in order of length from s.
Post Conditions: Finds a shortest weighted path
from s to each node v and its length.

74. To prove path is shortest:
Prove there is a path of this length.
Prove there are no shorter paths.
Dijkstra's
When we handle v, we know
there isn't a shorter path to it because ?
Otherwise, we would have handled it already.
Post Conditions: Finds a shortest weighted path
from s to each node v and its length.
So far, the nodes have been
handled in order of length from s.

75. Basic Steps:
u
The shortest of handled paths
to u has length d(u)
Dijkstra's
& there is an edge
from u to v
w<u,v>
vs
The shortest of handled paths
to v has length d(v)
The shortest known path to v has length
min( d(v), d(u)+w<u,v> ).
Handle node that seems to be closest to s.
Need to keep approximate shortest distances.

76. …

77. Dijkstra's
The shortest of handled paths
to v has length d(v)
u
vs

78. Dijkstra's
u
vs
Handled nodes Found nodes
Handled Edges?
The shortest of handled paths
to v has length d(v)

79. Dijkstra's
u
vs
Handled Nodes Found Nodes
Handled Edges
The shortest of handled paths
to v has length d(v)
Handled Paths?

80. Dijkstra's
u
vs
Handled Nodes Found Nodes
Handled Edges
The shortest of handled paths
to v has length d(v)
Handled Paths

81. Dijkstra's
u
vs
Handled Nodes
The shortest of handled paths
to v has length d(v)
d(v) is the length of the
shortest handled paths to v.
For handled u,
d(u) is the length of the
shortest paths to u.
NotFound
d(v’)= ∞

82. Exit

83. Dijkstra's
For handled w,
d(w) is the length of the
shortest paths to w.
Handle node with smallest d(u).
For handled u, d(u) is the length
of the shortest paths to u.
d(v) is the length
of the shortest handled
path to v.

84. Dijkstra's
For handled w,
d(w) is the length of the
shortest paths to w.
Handle node with smallest d(u).
For handled u, d(u) is the length
of the shortest paths to u.
d(u) is the length
of the shortest handled
path to u.
d(u)

85. Dijkstra's
For handled w,
d(w) is the length of the
shortest paths to w.
Handle node with smallest d(u).
For handled u, d(u) is the length
of the shortest paths to u.
d(u)
d(u) is the length
of the shortest handled
path to u.
First not handled
node in path.

86. Dijkstra's
For handled w,
d(w) is the length of the
shortest paths to w.
Handle node with smallest d(u).
For handled u, d(u) is the length
of the shortest paths to u.
d(u)
First not handled
node in path.d(u’)
d(u) & d(u’) are the length
of the shortest handled
paths to u and u’.

87. Dijkstra's
d(v) is the length
of the shortest handled
path to v.
For handled w,
d(w) is the length of the
shortest paths to w.
Handle node with smallest d(u).
d(v) is the length of the shortest
handled path to v.
d(u)
d(v)
w<u,v>
d(v) = min( d(v), d(u)+w<u,v> ).

88. Time = O(E)
Each edge visited, times.2
?
Dijkstra's
This number of times following an edge.
Must update Priority Queue.
Takes time O(log(n))
Time = O(E log(n))
For each update,
d(v) = min( d(v), d(u)+w<u,v> ).
Heap

89. Dijkstra's Shortest-Weighted Paths
Specification: Dijkstra's Shortest-Weighted Paths
Reachability-from-single-source s
•Preconditions:
The input is a graph G
(either directed or undirected)
with positive edge weights
and a source node s.
•Postconditions:
Finds a shortest weighted path from s
to each node v and its length.
EndingInitial ConditionsMake Progress
Maintain Loop InvDefine Exit ConditionDefine Step
Define Measure of
Progress
Define Loop
Invariants
Define Problem
km∞
79 km
to school
Exit
Exit
79 km 75 km
Exit
Exit
0 km Exit

90. Dijkstra's

91. Dijkstra's

92. Graph Search
Handle some foundNotHandled node
•Queue: Handle in order found.
•Breadth-First Search
•Stack: Handle most recently found
•Depth-First Search
•Priority Queue:
Handle node that seems to be closest to s.
•Shortest (Weighted) Paths:

93. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack

94. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
s

95. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
a
b
g
d

96. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
a
m
g
d
e

97. DFS

98. DFS
The nodes in the stack form a path starting at s.

99. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
<node,# edges>

100. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
<node,# edges>
s,0

101. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,0

102. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,0

103. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,1
h,0

104. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,1
h,1
k,0

105. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,1
h,1
Tree Edge
Path on Stack

106. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,1

107. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,0

108. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,1
Cross Edge
to handled node

109. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,2

110. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,3
l,0

111. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,3
l,1

112. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,3

113. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,0

114. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1
j,0

115. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1
j,1
Back Edge
to node on Stack
Forms a cycle

116. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1
j,2
m,0

117. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1
j,2
m,1

118. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1
j,2

119. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4
g,1

120. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,4

121. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,5
f,0

122. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,5
f,1

123. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2
i,5

124. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,2

125. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1
c,3

126. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,1

127. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>
a,2

128. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,1
Found
Not Handled
Stack
<node,# edges>

129. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>
d,1

130. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>
d,2

131. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>
d,3
e,0

132. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>
d,3
e,1

133. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>
d,3

134. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,2
Found
Not Handled
Stack
<node,# edges>

135. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,3
Found
Not Handled
Stack
<node,# edges>

136. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,4
Found
Not Handled
Stack
<node,# edges>
b,0

137. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,4
Found
Not Handled
Stack
<node,# edges>
b,1

138. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,4
Found
Not Handled
Stack
<node,# edges>
b,2

139. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,4
Found
Not Handled
Stack
<node,# edges>
b,3

140. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
s,4
Found
Not Handled
Stack
<node,# edges>

141. DFS
s
a
c
h
k
f
i
l
m
j
e
b
g
d
Found
Not Handled
Stack
<node,# edges>

144. Linear Order

146. Ingredients:
•Instances: The possible inputs to the problem.
•Solutions for Instance: Each instance has an
exponentially large set of solutions.
•Cost of Solution: Each solution has an easy to
compute cost or value.
Specification
•Preconditions: The input is one instance.
•Postconditions: An valid solution with optimal
cost. (minimum or maximum)
Optimization Problems

147. Specification: Network Flow
•Preconditions:
The input is a network G
directed edges with capacities,
a source node s, and a sink node t.
•Postconditions:
Finds a Maximum Flow F.
Network Flow

148. Network Flow
No leaks, not extra flow
Value of flow
= flow out of s
= flow into t
= flow across any cut

149. Network Flow

150. Network Flow
•Given Flow F
•Construct Augmenting Graph GF
•Find path P using BFS, DFS,
or generic search algorithm
•Let w be the max amount flow
can increase along path P.
•Increase flow along path P by w.
i.e newF = oldF + w × P

151. Network Flow
Stop

152. Hill Climbing
We have a valid solution.
(not necessarily optimal)
Take a step that goes up.
measure
progress
Value of our solution.
Problems:
Exit
Cant take a step that goes up.
Running time?
Initially have the “zero” solution
Local Max
Global Max
Can our Network Flow
Algorithm get stuck
in a local maximum?
Make small local changes to your solution to
construct a slightly better solution.

153. Network Flow
Previous Input
Previous Output
Same Input
Yes! Our Network Flow
Algorithm can get stuck.
Worse Output
Need only one example.

154. Network Flow

155. Network Flow

156. Network Flow

157. Network Flow

158. Network Flow
Previous Input
Previous Output
Same Input
Yes! Our Network Flow
Algorithm can get stuck.
Worse Output
Need only one example.

159. Hill Climbing
Avoiding getting stuck
in a local maximum
Good ExecutionBad Execution
•Made better choices of direction
•Hard
•Back up an retry
•Exponential time
•Define a bigger step

160. Hill Climbing
Different
Solutions
Current
Solution
Alg defines
to where alg can step
i.e. what small local
changes can be made to
current solution
This defines
the topography

161. Hill Climbing
Different
Solutions
Current
Solution
Define a slightly
bigger step
This defines
the topography
Perhaps removes some local maxima

162. Network Flow
Previous Input
Previous Output
Same Input
Yes! Our Network Flow
Algorithm can get stuck.
Worse Output
Need only one example.
Mistake?
Putting 2 through
this edge

163. Network Flow
u v
0/75
0/10
Flow Graph
u v
75
10
Augmentation Graph
75-21=54
21+10=31
u v
21/75
10
Flow Graph
u v
Augmentation Graph
2

164. Network Flow

165. Network Flow

166. Network Flow

167. Network Flow

168. Network Flow

169. Network Flow

171. Hill Climbing

173. Network Flow

176. Linear Programming

177. A Hotdog
A combination of pork,A combination of pork,
grain, and sawdust, …grain, and sawdust, …
Constraints:Constraints:
•Amount of moistureAmount of moisture
•Amount of protean,Amount of protean,
•……

178. The Hotdog Problem
Given today’s prices,
what is a fast algorithm
to find the cheapest
hotdog?

179. There are deep ideas within the simplicity.
=
AbstractionAbstraction
Goal: Understand and think about complex
things in simple ways.
There are deep ideas within the simplicity.
Rudich www.discretemath.com

180. Abstract Out Essential Details
Cost: 29, 8, 1, 2
Amount to add: x1, x2, x3, x4
pork
grain
water
sawdust
3x1 + 4x2 – 7x3 + 8x4 ≥ 12
2x1 - 8x2 + 4x3 - 3x4 ≥ 24
-8x1 + 2x2 – 3x3 - 9x4 ≥ 8
x1 + 2x2 + 9x3 - 3x4 ≥ 31
Constraints:Constraints:
•moisturemoisture
•protean,protean,
•……
29x1 + 8x2 + 1x3 + 2x4Cost of Hotdog:Cost of Hotdog:

181. 3x1 + 4x2 – 7x3 + 8x4 ≥ 12
2x1 - 8x2 + 4x3 - 3x4 ≥ 24
-8x1 + 2x2 – 3x3 - 9x4 ≥ 8
x1 + 2x2 + 9x3 - 3x4 ≥ 31
29x1 + 8x2 + 1x3 + 2x4
Subject to:Subject to:
Minimize:Minimize:
Abstract Out Essential Details

182. A Fast Algorithm
For decades people thought
that there was no fast
algorithm.
Then one was found!
Theoretical Computer Science
finds new algorithms every day.
3x1 + 4x2 – 7x3 + 8x4 ≥ 12
2x1 - 8x2 + 4x3 - 3x4 ≥ 24
-8x1 + 2x2 – 3x3 - 9x4 ≥ 8
x1 + 2x2 + 9x3 - 3x4 ≥ 31
29x1 + 8x2 + 1x3 + 2x4
Subject to:Subject to:
Minimize:Minimize:
≈

184. End