Lecture13
Upcoming SlideShare
Loading in...5
×
 

Lecture13

on

  • 201 views

dsuc

dsuc

Statistics

Views

Total Views
201
Views on SlideShare
201
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Lecture13 Lecture13 Document Transcript

  • Reading CSE 326: Data Structures Chapter 9.1, 9.2, 9.3 Graph Algorithms Graph Search Lecture 13 1 Graph Algorithms, Graph Search - Lecture 13 Graphs In Practice Graph ADT Graphs are a formalism for representing relationships between objects • a graph G is represented as G = (V, E) – V is a set of vertices – E is a set of edges • operations include: – – – – q Web graph Han • Vertices are web pages • Edge from u to v is a link to v appears on u Luke q Call graph of a computer program Leia V = {Han, Leia, Luke} E = {(Luke, Leia), (Han, Leia), (Leia, Han)} iterating over vertices iterating over edges iterating over vertices adjacent to a specific vertex asking whether an edge exists connected two vertices • Vertices are functions • Edge from u to v is u calls v q Task graph for a work flow • Vertices are tasks • Edge from u to v if u must be completed before v begins 3 Graph Algorithms, Graph Search - Lecture 13 Graph Representation 2: Adjacency List A |V| x |V| array in which an element (u, v) is true if and only if there is an edge from u to v Luke Han Luke 0 1 Han 0 0 0 A |V|-ary list (array) in which each entry stores a list (linked list) of all adjacent vertices Leia 2 Han Luke 1 Leia Leia Luke 1 0 0 1 Runtime: iterate over vertices O(|v|) Leia 2 1 0 0 iterate ever edges O(|v|2) iterate edges adj. to vertex O(|v|) edge exists? O(1) Space required: O(|v|2) Graph Algorithms, Graph Search - Lecture 13 4 Graph Algorithms, Graph Search - Lecture 13 Graph Representation 1: Adjacency Matrix Han 2 Graph Algorithms, Graph Search - Lecture 13 Han 0 2 Luke 1 2 Runtime: Leia 2 0 iterate over vertices O(v) iterate ever edges O(|e|) iterate edges adj. to vertex O(d) (d is number of adj. vertices) Space required: O(|v| +|e|) edge exists? O(d) 5 Graph Algorithms, Graph Search - Lecture 13 6 1
  • Terminology Weighted Graphs Each edge has an associated weight or cost. In directed graphs, edges have a specific direction In undirected graphs, edges are two-way Vertices u and v are adjacent if (u, v) ∈ E A sparse graph has O(|V|) edges (upper bound) A dense graph has Ω(|V|2) edges (lower bound) A complete graph has an edge between every pair of vertices q An undirected graph is connected if there is a path between any two vertices q q q q q q 20 Clinton Mukilteo Kingston 30 Bainbridge 35 Edmonds Seattle 60 Bremerton 7 Graph Algorithms, Graph Search - Lecture 13 8 Graph Algorithms, Graph Search - Lecture 13 Paths and Cycles Path Length and Cost A path is a list of vertices {v1, v2, …, vn} such that (vi, vi+1) ∈ E for all 0 ≤ i < n. A cycle is a path that begins and ends at the same node. Path length: the number of edges in the path Path cost: the sum of the costs of each edge Chicago Chicago 3.5 Seattle Seattle 2 Salt Lake City 2 2 Salt Lake City 2.5 2.5 2.5 3 San Francisco San Francisco Dallas Dallas p = {Seattle, Salt Lake City, Chicago, Dallas, San Francisco, Seattle} 9 Graph Algorithms, Graph Search - Lecture 13 Trees as Graphs Every tree is a graph with some restrictions: • the tree is directed • there are no cycles (directed or undirected) • there is a directed path from the root to every node B Graph Algorithms, Graph Search - Lecture 13 cost(p) = 11.5 10 Graph Algorithms, Graph Search - Lecture 13 Directed Acyclic Graphs (DAGs) C E program call graph DAGs are directed graphs with no cycles. A D length(p) = 5 main() mult() F add() G H access() read() Trees ⊂ DAGs ⊂ Graphs 11 Graph Algorithms, Graph Search - Lecture 13 12 2
  • Topological Sort Topological Sort Given a directed graph, G = (V, E), output all the vertices in V such that no vertex is output before any other vertex with an edge to it. reserve flight Label each vertex’s in-degree Initialize a queue to contain all in-degree zero vertices While there are vertices remaining in the queue Remove a vertex v with in-degree of zero and output it Reduce the in-degree of all vertices adjacent to v Put any of these with new in-degree zero on the queue check in airport call taxi take flight taxi to airport locate gate pack bags Runtime: O(|v| + |e|) 13 Graph Algorithms, Graph Search - Lecture 13 Example 2 1 check in airport 4 call taxi take flight taxi to airport pack bags locate gate 6 5 Example In-degree 3 reserve flight 7 Queue = 1 2 5 Output = 1 2 3 4 5 6 7 2 1 0 6 0 3 2 4 check in airport take flight taxi to airport pack bags locate gate 6 5 0 6 2 3 1 7 4 7 Queue = 1 2 Output = 5 15 check in airport 4 call taxi take flight taxi to airport pack bags 5 6 Queue = 1 Output = 5 2 locate gate 7 1 2 3 4 5 6 7 0 6 0 3 2 4 2 0 6 1 3 1 4 7 16 Graph Algorithms, Graph Search - Lecture 13 Example In-degree 3 reserve flight 4 call taxi Example 2 In-degree 3 reserve flight 2 Graph Algorithms, Graph Search - Lecture 13 1 14 Graph Algorithms, Graph Search - Lecture 13 1 2 3 4 5 6 7 2 1 0 6 0 3 1 4 3 1 pack bags 4 Graph Algorithms, Graph Search - Lecture 13 4 take flight taxi to airport 6 1 check in airport call taxi 5 6 2 0 In-degree 3 reserve flight 7 17 Queue = 6 Output = 5 2 1 locate gate 7 1 2 3 4 5 6 7 0 6 0 3 1 4 2 0 6 0 3 1 4 Graph Algorithms, Graph Search - Lecture 13 7 18 3
  • Example 2 1 In-degree 3 reserve flight check in airport 4 call taxi take flight taxi to airport pack bags locate gate 6 5 Example 7 Queue = 3 7 Output = 5 2 1 6 1 2 3 4 5 6 7 0 6 0 3 0 4 0 3 0 pack bags check in airport 4 call taxi take flight taxi to airport pack bags locate gate 6 5 7 Queue = 4 Output = 5 2 1 6 7 3 5 locate gate 6 7 4 7 19 Queue = 3 Output = 5 2 1 6 7 1 2 3 4 5 6 7 0 6 0 3 0 4 1 0 6 0 3 0 4 Graph Algorithms, Graph Search - Lecture 13 7 20 Exercise In-degree 3 reserve flight 4 take flight taxi to airport Example 2 check in airport call taxi 6 0 In-degree 3 reserve flight 2 Graph Algorithms, Graph Search - Lecture 13 1 2 1 1 2 3 4 5 6 7 0 3 0 Design the algorithm to initialize the in-degree array. Assume the adjacency list representation. 6 0 4 0 0 6 0 3 0 4 Graph Algorithms, Graph Search - Lecture 13 7 21 Graph Search Graph Algorithms, Graph Search - Lecture 13 22 General Graph Search Algorithm Open – some data structure (e.g., stack, queue, heap) Many problems in computer science correspond to searching for a path in a graph, given a start node and goal criteria • • • • • • Criteria – some method for removing an element from Open Route planning – Mapquest Packet-switching VLSI layout 6-degrees of Kevin Bacon Program synthesis Speech recognition – We’ll discuss these last two later… Graph Algorithms, Graph Search - Lecture 13 23 Search( Start, Goal_test, Criteria) insert(Start, Open); repeat if (empty(Open)) then return fail; select Node from Open using Criteria; Mark Node as visited; 24 Graph Algorithms, Graph Search - Lecture if (Goal_test(Node)) then 13 return Node; 4
  • Depth-First Graph Search Breadth-First Graph Search Open – Stack Open – Queue Criteria – Pop Criteria – Dequeue (FIFO) DFS( Start, Goal_test) push(Start, Open); repeat if (empty(Open)) then return fail; Node := pop(Open); Mark Node as visited; 25 Graph Algorithms, Graph Search - Lecture if (Goal_test(Node)) then 13 return Node; BFS( Start, Goal_test) enqueue(Start, Open); repeat if (empty(Open)) then return fail; Node := dequeue(Open); Mark Node as visited; if (Goal_test(Node)) then return Node; for each Child of node do if (Child not already visited) then enqueue(Child, Open); end BFS - Example 2 26 Graph Algorithms, Graph Search - Lecture 13 DFS - Example 2 3 1 3 1 4 6 4 6 5 5 7 7 QUEUE = 1 STACK = 1 Graph Algorithms, Graph Search - Lecture 13 27 Two Models Graph Algorithms, Graph Search - Lecture 13 28 Planning Example A huge graph may be implicitly specified by rules for generating it on-the-fly Blocks world: 1. Standard Model: Graph given explicitly with n vertices and e edges. • vertex = relative positions of all blocks • edge = robot arm stacks one block q Search is O(n + e) time in adjacency list representation stack(blue,table) stack(green,blue) 2. AI Model: Graph generated on the fly. stack(blue,red) q Time for search need not visit every vertex. stack(green,red) stack(green,blue) Graph Algorithms, Graph Search - Lecture 13 29 Graph Algorithms, Graph Search - Lecture 13 30 5
  • DFS Space Requirements AI Comparison: DFS versus BFS Depth-first search Assume: • Does not always find shortest paths • Must be careful to mark visited vertices, or you could go into an infinite loop if there is a cycle • Longest path in graph is length d • Highest number of out-edges is k DFS stack grows at most to size dk Breadth-first search • Always finds shortest paths – optimal solutions • Marking visited nodes can improve efficiency, but even without doing so search is guaranteed to terminate • For k=10, d=15, size is 150 Is BFS always preferable? Graph Algorithms, Graph Search - Lecture 13 31 32 Conclusion BFS Space Requirements Assume In the AI Model, DFS is hugely more memory efficient, if we can limit the maximum path length to some fixed d. • Distance from start to a goal is d • Highest number of out edges is k BFS Queue could grow to size kd • If we knew the distance from the start to the goal in advance, we can just not add any children to stack after level d • But what if we don’t know d in advance? • For k=10, d=15, size is 1,000,000,000,000,000 Graph Algorithms, Graph Search - Lecture 13 Graph Algorithms, Graph Search - Lecture 13 33 Recursive Depth-First Search Graph Algorithms, Graph Search - Lecture 13 34 Finding Connected Components DFS(v: vertex) mark v; for each vertex w adjacent to v do if w is unmarked then DFS(w) For each vertex v do mark[v]= 0; C := 1. For each vertex v do if mark[v] = 0 then dfs(v); C := C+1; Note: the recursion has the same effect as a stack dfs(v: vertex) mark[v] := C; for each vertex w adjacent to v do if mark[w] = 0 then dfs(w) All those vertices with the same mark are in the same connected component Graph Algorithms, Graph Search - Lecture 13 35 Graph Algorithms, Graph Search - Lecture 13 36 6
  • Example Saving the Path mark 2 1 3 4 6 5 7 For undirected graphs, each edge appears twice on the adjacency lists. 1 2 3 4 5 6 7 2 6 5 6 1 4 3 2 7 2 7 6 Our pseudocode returns the goal node found, but not the path to it How can we remember the path? • Add a field to each node, that points to the previous node along the path • Follow pointers from goal back to start to recover path 5 Graph Algorithms, Graph Search - Lecture 13 1 37 Example Graph Algorithms, Graph Search - Lecture 13 38 Example (Unweighted Graph) Seattle Seattle Salt Lake City Salt Lake City San Francisco San Francisco Dallas Graph Algorithms, Graph Search - Lecture 13 Dallas 39 Example (Unweighted Graph) Graph Algorithms, Graph Search - Lecture 13 40 Graph Search, Saving Path Search( Start, Goal_test, Criteria) insert(Start, Open); repeat if (empty(Open)) then return fail; select Node from Open using Criteria; Mark Node as visited; if (Goal_test(Node)) then return Node; for each Child of node do if (Child not already visited) then Seattle Salt Lake City Child.previous := Node; San Francisco Insert( Child, Open ); Dallas Graph Algorithms, Graph Search - Lecture 13 end 41 Graph Algorithms, Graph Search - Lecture 13 42 7
  • Shortest Path for Weighted Graphs Edsger Wybe Dijkstra (1930-2002) Given a graph G = (V, E) with edge costs c(e), and a vertex s ∈ V, find the shortest (lowest cost) path from s to every vertex in V q Invented concepts of structured programming, synchronization, weakest precondition, and "semaphores" for controlling computer processes. The Oxford English Dictionary cites his use of the words "vector" and "stack" in a computing context. q Believed programming should be taught without computers q 1972 Turing Award q “In their capacity as a tool, computers will be but a ripple on the surface of our culture. In their capacity as intellectual challenge, they are without precedent in the cultural history of mankind.” Assume: only positive edge costs Graph Algorithms, Graph Search - Lecture 13 43 Dijkstra’s Algorithm for Single Source Shortest Path Pseudocode for Dijkstra Initialize the cost of each node to ∞ s.cost := 0 insert(s,0,heap); While (! empty(heap)) n := deleteMin(heap); For each edge e=(n,a) do if (n.cost + e.cost < a.cost) then a.cost = n.cost + e.cost; a.previous = n; if (a is in the heap) then decreaseKey(a, a.cost, heap) else insert(a, a.cost, heap) end end Similar to breadth-first search, but uses a heap instead of a queue: • Always select (expand) the vertex that has a lowest-cost path to the start vertex Correctly handles the case where the lowest-cost (shortest) path to a vertex is not the one with fewest edges Graph Algorithms, Graph Search - Lecture 13 45 Important Features 46 Graph Algorithms, Graph Search - Lecture 13 Dijkstra’s Algorithm in Action 0 ∞ 2 1 Once a vertex is removed from the heap, the cost of the shortest path to that node is known While a vertex is still in the heap, another shorter path to it might still be found The shortest path itself can found by following the backward pointers stored in node.previous 4 ∞ D 1 2 47 1 G 3 ∞ 11 E visited H 2 ∞ 7 vertex A B C D E F G H ∞ 3 F 10 9 C ∞ 2 B A Graph Algorithms, Graph Search - Lecture 13 44 Graph Algorithms, Graph Search - Lecture 13 ∞ 1 cost 0 ∞ ∞ ∞ ∞ ∞ ∞ ∞ Graph Algorithms, Graph Search - Lecture 13 48 8
  • Dijkstra’s Algorithm in Action 0 2 2 1 4 4 1 D 2 1 4 4 E ∞ 1 49 2 2 4 4 1 D 2 12 cost 0 visited x x x 1 4 1 2 2 4 4 D 1 C 2 51 3 7 0 E x 1 4 8 x 1 10 1 G 3 ∞ 11 E 12 cost 0 1 2 1 4 12 4 ∞ x 7 2 2 D 1 52 2 H 2 3 1 1 G 8 11 E 7 53 7 3 F 10 9 C 4 2 B vertex A B C D E F G H 2 1 4 12 4 8 7 Graph Algorithms, Graph Search - Lecture 13 H 2 1 visited x x x x A 4 12 cost 0 7 3 F Dijkstra’s Algorithm in Action 11 visited x x x x 2 1 G 50 Graph Algorithms, Graph Search - Lecture 13 H 3 2 1 7 vertex A B C D E F G H 4 1 C 4 2 B vertex A B C D E F G H F 10 9 D 2 1 4 12 4 ∞ ∞ 2 B 1 2 2 7 Dijkstra’s Algorithm in Action A 1 2 1 4 12 ∞ ∞ ∞ x 9 ∞ Graph Algorithms, Graph Search - Lecture 13 0 12 cost 0 visited x A 4 E vertex A B C D E F G H 0 11 7 ∞ 11 E 1 G 3 1 G 3 1 1 Dijkstra’s Algorithm in Action H 2 H 2 Graph Algorithms, Graph Search - Lecture 13 ∞ 3 F 10 9 C 4 2 B ∞ 3 F 10 7 vertex A B C D E F G H Dijkstra’s Algorithm in Action 1 D 2 1 4 ∞ ∞ ∞ ∞ A 1 C ∞ 2 B 2 cost 0 Graph Algorithms, Graph Search - Lecture 13 0 2 9 ∞ 11 visited x 2 A 1 G 3 0 H 2 1 7 vertex A B C D E F G H ∞ 3 F 10 9 C ∞ 2 B A Dijkstra’s Algorithm in Action visited x x x x x x x 11 cost 0 1 2 1 4 11 4 8 7 Graph Algorithms, Graph Search - Lecture 13 54 9
  • Data Structures for Dijkstra’s Algorithm Dijkstra’s Algorithm in Action 0 2 2 1 4 4 1 C 2 H 2 3 1 |V| times: Select the unknown node with the lowest cost 1 G 8 11 E 11 cost 0 7 vertex A B C D E F G H 7 3 F 10 9 D 4 2 B A visited x x x x x x x x 1 findMin/deleteMin O(log |V|) |E| times: a’s cost = min(a’s old cost, …) 2 1 4 11 4 8 7 decreaseKey or insert runtime: O(|E| log |V|) 55 Graph Algorithms, Graph Search - Lecture 13 O(log |V|) 56 Graph Algorithms, Graph Search - Lecture 13 Problem: Large Graphs Example q It is expensive to find optimal paths in large graphs, using BFS or Dijkstra’s algorithm (for weighted graphs) q How can we search large graphs efficiently by using “commonsense” about which direction looks most promising? 53nd St 52nd St G 51st St S 50th St 2nd Ave 3rd Ave 4th Ave 5th Ave 6th Ave 7th Ave 8th Ave 9th Ave 10th Ave Plan a route from 9th & 50th to 3rd & 51st 57 Graph Algorithms, Graph Search - Lecture 13 Graph Algorithms, Graph Search - Lecture 13 58 Best-First Search Example The Manhattan distance (∆ x+ ∆ y) is an estimate of the distance to the goal 53nd St • It is a search heuristic 52nd St q Best-First Search G 51st St • Order nodes in priority to minimize estimated distance to the goal S 50th St 2nd Ave 3rd Ave 4th Ave 5th Ave 6th Ave 7th Ave 8th Ave 9th Ave 10th Ave q Compare: BFS / Dijkstra • Order nodes in priority to minimize distance from the start Plan a route from 9th & 50th to 3rd & 51st Graph Algorithms, Graph Search - Lecture 13 59 Graph Algorithms, Graph Search - Lecture 13 60 10
  • Best-First Search Obstacles Open – Heap (priority queue) Criteria – Smallest key (highest priority) h(n) – heuristic estimate of distance from n to closest goal 50th St 2nd Ave 3rd Ave 4th Ave 5th Ave 6th Ave 7th Ave 8th Ave 62 q Best-first is often tremendously faster than BFS/Dijkstra, but might stop with a non-optimal solution q How can it be modified to be (almost) as fast, but guaranteed to find optimal solutions? q A* - Hart, Nilsson, Raphael 1968 52nd St G 50th St 2nd Ave 3rd Ave 4th Ave 5th Ave 6th Ave 7th Ave 8th Ave 9th Ave 10th Ave • One of the first significant algorithms developed in AI • Widely used in many applications Shortest Path 63 A* Graph Algorithms, Graph Search - Lecture 13 64 Optimality of A* Exactly like Best-first search, but using a different criteria for the priority queue: minimize (distance from start) + (estimated distance to goal) priority f(n) = g(n) + h(n) Suppose the estimated distance is always less than or equal to the true distance to the goal • heuristic is a lower bound Then: when the goal is removed from the priority queue, we are guaranteed to have found a shortest path! f(n) = priority of a node g(n) = true distance from start h(n) = heuristic distance to goal Graph Algorithms, Graph Search - Lecture 13 Graph Algorithms, Graph Search - Lecture 13 Improving Best-First Path found by Best-first Graph Algorithms, Graph Search - Lecture 13 9th Ave 61 53nd St S G 51st St Non-Optimality of Best-First 51st St S 52nd St 10th Ave Best_First_Search( Start, Goal_test) insert(Start, h(Start), heap); repeat if (empty(heap)) then return fail; Node := deleteMin(heap); Graph Algorithms, Graph Search - Lecture 13 Mark Node as visited; Best-FS eventually will expand vertex to get back on the right track 65 Graph Algorithms, Graph Search - Lecture 13 66 11
  • A* in Action Applications of A*: Planning A huge graph may be implicitly specified by rules for generating it on-the-fly Blocks world: h=7+3 h=6+2 53nd St 52nd St S 51st St • vertex = relative positions of all blocks • edge = robot arm stacks one block G stack(blue,table) stack(green,blue) 50th St stack(blue,red) 2nd Ave 3rd Ave 4th Ave 5th Ave 6th Ave 7th Ave 8th Ave 9th Ave 10th Ave H=1+7 stack(green,red) stack(green,blue) Graph Algorithms, Graph Search - Lecture 13 67 68 Graph Algorithms, Graph Search - Lecture 13 Application of A*: Speech Recognition Blocks World (Simplified) Problem: Blocks world: • distance = number of stacks to perform • heuristic lower bound = number of blocks out of place • System hears a sequence of 3 words • It is unsure about what it heard – For each word, it has a set of possible “guesses” – E.g.: Word 1 is one of { “hi”, “high”, “I” } • What is the most likely sentence it heard? # out of place = 2, true distance to goal = 3 Graph Algorithms, Graph Search - Lecture 13 69 Speech Recognition as Shortest Path 70 Graph Algorithms, Graph Search - Lecture 13 W11 W12 W13 Convert to a shortest-path problem: • Utterance is a “layered” DAG • Begins with a special dummy “start” node • Next: A layer of nodes for each word position, one node for each word choice • Edges between every node in layer i to every node in layer i+1 W21 W23 W22 W311 W1 W33 W41 W43 – Cost of an edge is smaller if the pair of words frequently occur together in real speech + Technically: - log probability of co-occurrence • Finally: a dummy “end” node • Find shortest path from start to end node Graph Algorithms, Graph Search - Lecture 13 71 Graph Algorithms, Graph Search - Lecture 13 72 12
  • Summary: Graph Search Depth First • Little memory required • Might find non-optimal path Breadth First • Much memory required • Always finds optimal path Dijskstra’s Short Path Algorithm • Like BFS for weighted graphs Best First • Can visit fewer nodes • Might find non-optimal path A* • Can visit fewer nodes than BFS or Dijkstra • Optimal if heuristic estimate is a lower-bound Graph Algorithms, Graph Search - Lecture 13 73 13