Your SlideShare is downloading. ×
0
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
PowerPoint - K-State Laboratory for Knowledge Discovery in ...
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

PowerPoint - K-State Laboratory for Knowledge Discovery in ...

513

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
513
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lecture 4 of 42<br />State Spaces, Graphs, Uninformed (Blind) Search:<br />ID-DFS, Bidirectional, UCS/B&amp;B<br />Discussion: Term Projects 4 of 5<br />William H. Hsu<br />Department of Computing and Information Sciences, KSU<br />KSOL course page: http://snipurl.com/v9v3<br />Course web site: http://www.kddresearch.org/Courses/CIS730<br />Instructor home page: http://www.cis.ksu.edu/~bhsu<br />Reading for Next Class:<br />Sections 3.5 – 3.7, p. 81 – 88; 4.1 – 4.2, p. 94 - 109, Russell &amp; Norvig 2nd ed.<br />Instructions for writing project plans, submitting homework<br />
  • 2. Lecture Outline<br /><ul><li>Reading for Next Class: Sections 3.5 – 3.7, 4.1 – 4.2, R&amp;N 2e
  • 3. Past Week: Intelligent Agents (Ch. 2), Blind Search (Ch. 3)
  • 4. Basic search frameworks: discrete and continuous
  • 5. Tree search intro: nodes, edges, paths, depth
  • 6. Depth-first search (DFS) vs. breadth-first search (BFS)
  • 7. Completeness and depth-limited search (DLS)
  • 8. Coping with Time and Space Limitations of Uninformed Search
  • 9. Depth-limited and resource-bounded search (anytime, anyspace)
  • 10. Iterative deepening (ID-DFS) and bidirectional search
  • 11. Project Topic 4 of 5: Natural Lang. Proc. (NLP) &amp; Info. Extraction
  • 12. Preview: Intro to Heuristic Search (Section 4.1)
  • 13. What is a heuristic?
  • 14. Relationship to optimization, static evaluation, bias in learning
  • 15. Desired properties and applications of heuristics</li></li></ul><li>Problem-Solving Agents<br />© 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 16. State Space Graph:<br />Vacuum World<br />Based on slide © 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 17. State Space Example:<br />Vacuum World<br />Sensorless<br />Conditional<br />Based on slide © 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 18. Graph Search Example:<br />Route Planning<br />Edge weights<br />based on actual driving distance<br />Based on slides © 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 19. Single-State Problem Formulation<br />© 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 20. Selecting A State Space<br />© 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 21. General Search Algorithm:Review<br /><ul><li>functionGeneral-Search (problem, strategy) returns a solution or failure
  • 22. initialize search tree using initial state of problem
  • 23. loop do
  • 24. if there are no candidates for expansion thenreturn failure
  • 25. choose leaf node for expansion according to strategy
  • 26. If node contains a goal state thenreturn corresponding solution
  • 27. else expand node and add resulting nodes to search tree
  • 28. end
  • 29. Note: Downward Function Argument (Funarg) strategy
  • 30. Implementation of General-Search
  • 31. Rest of Chapter 3, Chapter 4, R&amp;N
  • 32. See also:
  • 33. Ginsberg (handout in CIS library today)
  • 34. Rich and Knight
  • 35. Nilsson: Principles of Artificial Intelligence</li></li></ul><li>Depth-First Search:<br />Review<br />© 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 36. Iterative Deepening Search (ID-DFS):<br />Example<br />Based on slides © 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 37. Iterative Deepening Search (ID-DFS):<br />Properties<br />© 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 38. Comparison of Search Strategies<br />for Algorithms Thus Far<br />Based on slides © 2003 S. Russell &amp; P. Norvig. Reused with permission.<br />
  • 39. Bidirectional Search:Review<br /><ul><li>Intuitive Idea
  • 40. Search “from both ends”
  • 41. Caveat: what does it mean to “search backwards from solution”?
  • 42. Analysis
  • 43. Solution depth (in levels from root, i.e., edge depth): d
  • 44. Analysis
  • 45. bi nodes generated at level i
  • 46. At least this many nodes to test
  • 47. Total: I bi = 1 + b + b2 + … + bd/2= (bd/2)
  • 48. Worst-Case Space Complexity:(bd/2)
  • 49. Properties
  • 50. Convergence: suppose b, l finite and l  d
  • 51. Complete: guaranteed to find a solution
  • 52. Optimal: guaranteed to find minimum-depth solution
  • 53. Worst-case time complexity is square root of that of BFS</li></li></ul><li>Uniform-Cost Search<br />(aka Branch &amp; Bound)And Heuristics<br /><ul><li>Previously: Uninformed (Blind) Search
  • 54. No heuristics: only g(n) used
  • 55. Breadth-first search (BFS) and variants: uniform-cost, bidirectional
  • 56. Depth-first search (DFS) and variants: depth-limited, iterative deepening
  • 57. Heuristic Search
  • 58. Based on h(n) – estimated cost of path to goal (“remaining path cost”)
  • 59. h – heuristic function
  • 60. g: node  R; h: node  R; f: node  R
  • 61. Using h
  • 62. h only: greedy (akamyopic) informed search
  • 63. f = g + h: (some) hill-climbing, A/A*
  • 64. Uniform-Cost (Branch and Bound) Search
  • 65. Originates from operations research (OR)
  • 66. Special case of heuristic search: treat as h(n) = 0
  • 67. Sort candidates by g(n)</li></li></ul><li>Heuristic Search [1]:Terminology<br /><ul><li>Heuristic Function
  • 68. Definition: h(n) = estimated cost of cheapest path from state at node n to a goal state
  • 69. Requirements for h
  • 70. In general, any magnitude (ordered measure, admits comparison)
  • 71. h(n) = 0 iffn is goal
  • 72. For A/A*, iterative improvement: want
  • 73. h to have same type as g
  • 74. Return type to admit addition
  • 75. Problem-specific (domain-specific)
  • 76. Typical Heuristics
  • 77. Graph search in Euclidean space</li></ul> hSLD(n) = straight-line distance to goal<br /><ul><li>Discussion (important): Why is this good?</li></li></ul><li>Heuristic Search [2]:Background<br /><ul><li>Origins of Term
  • 78. Heuriskein – to find (to discover)
  • 79. Heureka (“I have found it”) – attributed to Archimedes
  • 80. Usage of Term
  • 81. Mathematical logic in problem solving
  • 82. Polyà [1957]
  • 83. Methods for discovering, inventing problem-solving techniques
  • 84. Mathematical proof derivation techniques
  • 85. Psychology: “rules of thumb” used by humans in problem-solving
  • 86. Pervasive through history of AI
  • 87. e.g., Stanford Heuristic Programming Project
  • 88. One origin of rule-based (expert) systems
  • 89. General Concept of Heuristic (A Modern View)
  • 90. Standard (rule, quantitative measure) used to reduce search
  • 91. “As opposed to exhaustive blind search”
  • 92. Compare (later): inductive bias in machine learning</li></li></ul><li>Best-First Search [1]:Evaluation Function<br /><ul><li>Recall: General-Search
  • 93. Applying Knowledge
  • 94. In problem representation (state space specification)
  • 95. At Insert(), akaQueueing-Fn()
  • 96. Determines node to expand next
  • 97. Knowledge representation (KR)
  • 98. Expressing knowledge symbolically/numerically
  • 99. Objective
  • 100. Initial state
  • 101. State space (operators, successor function)
  • 102. Goal test: h(n) – part of (heuristic) evaluation function</li></li></ul><li>Best-First Search [2]:Characterization of Algorithm Family<br /><ul><li>Best-First: Family of Algorithms
  • 103. Justification: using only g doesn’t direct search toward goal
  • 104. Nodes ordered
  • 105. Node with best evaluation function (e.g., h) expanded first
  • 106. Best-first: any algorithm with this property (NB: not just using h alone)
  • 107. Note on “Best”
  • 108. Refers to “apparent best node”
  • 109. based on eval function
  • 110. applied to current frontier
  • 111. Discussion: when is best-first not really best?</li></li></ul><li>Best-First Search [3]:Implementation<br /><ul><li>functionBest-First-Search (problem, Eval-Fn) returns solution sequence
  • 112. inputs: problem, specification of problem (structure or class) Eval-Fn, an evaluation function
  • 113. Queueing-Fn  function that orders nodes by Eval-Fn
  • 114. Compare: Sort with comparator function &lt;
  • 115. Functional abstraction
  • 116. returnGeneral-Search (problem, Queueing-Fn)
  • 117. Implementation
  • 118. Recall: priority queue specification
  • 119. Eval-Fn: node  R
  • 120. Queueing-Fn Sort-By: node list  node list
  • 121. Rest of design follows General-Search
  • 122. Issues
  • 123. General family of greedy (akamyopic, i.e., nearsighted) algorithms
  • 124. Discussion: What guarantees do we want on h(n)? What preferences?</li></li></ul><li>Next Topic:More on Informed Search<br /><ul><li>Branch-and-Bound Search
  • 125. Heuristics for General-Search Function of Problem-Solving-Agent
  • 126. Informed (heuristic) search: heuristic definition, development process
  • 127. Best-First Search
  • 128. Greedy
  • 129. A/A*
  • 130. Admissibility property
  • 131. Developing good heuristics
  • 132. Humans
  • 133. Intelligent systems (automatic derivation): case studies and principles
  • 134. Constraint Satisfaction Heuristics
  • 135. This Week: More Search Basics
  • 136. Memory bounded, iterative improvement (gradient, Monte Carlo search)
  • 137. Introduction to game tree search</li></li></ul><li>Project Topic 4 of 5:<br />NLP and Information Extraction (IE)<br />Machine Translation<br />© 2009 Google, Inc. <br />Input<br />Named<br />Entity<br />Recognition<br />Output<br />Conversational Agent<br />© 1997 InstitutJozef<br />Stefan (IJS), Slovenia<br />
  • 138. Terminology<br /><ul><li>State Space Search
  • 139. Search Types: Uninformed (“Blind”) vs. Informed (“Heuristic”)
  • 140. Basic Search Algorithms
  • 141. British Museum (depth-firstakaDFS)
  • 142. Breadth-Firstaka BFS
  • 143. Depth-Limited Search (DLS)
  • 144. Refinements
  • 145. Iterative-deepening DFS (ID-DFS)
  • 146. Bidirectional (as adaptation of BFS or ID-DFS)
  • 147. Cost, c(n1, n2) and Cumulative Path Cost, g(n)
  • 148. Online (Path) Cost, g(goal) vs. Offline (Search) Cost
  • 149. Heuristic: Estimate of Remaining Path Cost, h(n)
  • 150. Uniform-Cost (akaBranch-and-Bound): g(n) only, h(n) = 0</li></li></ul><li>Summary Points<br /><ul><li>Reading for Next Class: Sections 3.5 – 3.7, 4.1 – 4.2, R&amp;N 2e
  • 151. This Week: Search, Chapters 3 - 4
  • 152. State spaces
  • 153. Graph search examples
  • 154. Basic search frameworks: discrete and continuous
  • 155. Uninformed (“Blind”) vs. Informed (“Heuristic”) Search
  • 156. h(n) and g(n) defined: no h in blind search; online cost = g(goal)
  • 157. Properties: completeness, time and space complexity, offline cost
  • 158. Uniform-cost search (B&amp;B) as generalization of BFS: g(n) only
  • 159. Relation to Intelligent Systems Concepts
  • 160. Knowledge representation: evaluation functions, macros
  • 161. Planning, reasoning, learning
  • 162. Coming Week: Heuristic Search, Chapter 4
  • 163. Later: Goal-Directed Reasoning, Planning (Chapter 11)</li>

×