NLP (Fall 2013): Spatial Semantic Hierarchy & Narrative Maps


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

NLP (Fall 2013): Spatial Semantic Hierarchy & Narrative Maps

  1. 1. Natural Language Processing Spatial Semantic Hierarchy & Narrative Maps Vladimir Kulyukin
  2. 2. Outline Spatial Semantic Hierarchy (SSH)  SSH Levels  Axioms & Inferences  Deduction, Induction, Abduction  Narrative Maps  Extraction of SSHs from Narrative Maps 
  3. 3. Background
  4. 4. Background Spatial Semantic Hierarchy (SSH) was discovered and developed by Benjamin Kuipers and his students  SSH is a model of knowledge of environments that consists of multiple interacting representations  These representations are quantitative and qualitative 
  5. 5. Types of Spatial Knowledge Large-scale space – space that exceeds the agent’s sensory horizon  Visual space – immediate environment that the agent can explore by gaze  Graphical space – spatial layouts and relations among symbols expressed graphically (e.g., on paper or tablet)  The term cognitive map refers to human knowledge of large-scale space 
  6. 6. Why Study Spatial Knowledge? Spatial knowledge is fundamental to commonsense knowledge  We use spatial knowledge daily to navigate  We use spatial knowledge represent concepts graphically  We use spatial knowledge of the real world to organize virtual worlds (e.g., the concept of computer desktop) 
  7. 7. SSH Levels
  8. 8. SSH Levels SSH consists of five levels: 1) sensory; 2) control; 3) causal; 4) topological; and 5) metrical  Kuipers organizes these levels in a lattice of nodes where each node corresponds to a representation with its own ontology (aka conceptualization in the terminology of this course), axioms, and inference rules 
  9. 9. Relations among SSH Levels Level X is dependent on level Y when X “presupposes, or is defined in terms of, or is inferred from, knowledge in the representation at” Y  Level X receives information from level Y when knowledge stored and/or computed at Y is accessible to X 
  10. 10. Organization of SSH Lattice Nodes SSH lattice nodes are organized along two dimensions: qualitative vs. quantitative (horizontal) and ontological (vertical)  The horizontal level indicates that spatial knowledge can be either qualitative or quantitative  The vertical level organizes nodes in terms of ontological dependencies 
  11. 11. Sensory Level Sensory level is the interface to the agent’s sensorimotor system  Sensors can be continuous (camera, laser, sonar, etc.) or discrete (digital compass, odometer, RFID reader, Wi-Fi receiver, etc.)  Distinction continuous vs. discrete is arbitrary because a continuous sensor’s output can be made discrete 
  12. 12. Control Level Control level is a set of control laws that bind the agent and its environment in a dynamic system within some uniform segment of that environment  Each control law has conditions for its appropriateness and termination  There are two broad classes of control laws: hill-climbing & trajectory following 
  13. 13. Causal Level Causal level discretizes the continuous world and the agent’s actions in terms of sensory views, actions, and relations among views and actions  This is similar to the STRIPS action semantics of the PDDL operators that we have investigated before  Plans constructed at the causal level are executed on the control level 
  14. 14. Quick Review of PDDL
  15. 15. PDDL Problem Specification Knowledge engineers must do two things to do problem solving describe in PDDL: 1) describe a domain and 2) describe a problem  The description of the domain is placed into a domain file  The description of the problem is placed into a problem file 
  16. 16. Domain Definition (define (domain <DOMAIN NAME>) <REQUIREMENT>* <PREDICATE>* <ACTION>* ) <REQUIREMENT>* is a statement that specifies requirements (e.g., :typing or :equal) <PREDICATE>* is a sequence of predication specifications <ACTION>* is a sequence of action specifications
  17. 17. Problem Definition (define (problem <PROBLEM NAME>) <DOMAIN NAME> <OBJECT STATEMENT> <INITIAL STATE DESCRIPTION> <GOAL DESCRIPTION> ) <DOMAIN NAME> is a statement that references the domain in which the problem must be solved <OBJECT STATEMENT> is a sequence of object constants <INITIAL STATE DESCRIPTION> is a sequence of predicates that describe the initial state of the world <GOAL DESCRIPTION> is a sequence of predicates that describe the goal state of the world
  18. 18. STRIPS Semantics of PDDL Actions STRIPS (Stanford Research Institute Planning System) is an AI Planner developed by Richard Fikes and Nils Nilson in 1971  A STRIPS operator has preconditions (a set of predicates that must be true in the current state of the world for the operator to be considered application) and postconditions (a set of predicates that will be true in the state of the world that results from the operator’s application)  A PDDL action also has preconditions and postconditions called effects 
  20. 20. Topological Level Topological level describes the world in terms of places, paths, regions, and their connectivity, order, and containment  SSH makes a claim that a topological network map is more effective for planning that the flat causal model  SSH makes another claim that “the ability to plan and act is not dependent on the availability of quantitative spatial knowledge”  The latter claim is very interesting, because, if it is true, planning can be done without direct contact with the world 
  21. 21. Metrical Level Metrical level is a global geometric map of the environment with a single frame of reference  Kuipers does say that quantitative geometric information is also present at each SSH level: local analog maps at control level; action magnitudes at causal level; headings and distances at topological level  Smaller local frames can be linked into a global frame of reference 
  22. 22. Deeper Dive into SSH Levels
  23. 23. Control Level
  24. 24. Uniform Segments Environment is divided into uniform segments each of which has its own control law (e.g., follow middle line, follow right wall, follow left wall, etc.)  The agent is assumed to use only sensory input to execute control laws  The agent receives a continuous time series of sensory values and outputs a continuous time series of motor signals  A control law is a relation b/w sensory inputs and motor outputs 
  25. 25. Control Laws as Differential Equations The agent, the environment, and a given control law is a dynamic system  This system can be modeled by a differential equation  The system’s behavior is described by a solution to the equation 
  26. 26. Hill-Climbing vs. Trajectory Following A hill-climbing control law brings the agent into a locally distinctive state  A hill-climbing control law terminates when a distinctiveness measure (e.g., distance) reaches a local maximum  A trajectory-following control law brings the agent from one distinctive state to the neighborhood of the next 
  27. 27. Low-Level Details of Control Laws In Sections 2.1, 2.2., 2.3, 2.4, and 2.5 of his article “The Spatial Semantic Hierarchy,” Kuipers discusses low-level details of control laws  These are fascinating and insightful but are peripheral to the NLP problem of automating SSH acquisition from narrative maps that we are investigating here  Hence, we will skip them in this presentation 
  28. 28. Guarantees of Control Level Guarantees of control level are more interesting to us, because they specify what we can assume about the agent’s physical abilities  There are two broad guarantees: 1) After a hill-climbing law terminates at a distinctive state, at least one trajectory following law is applicable (no dead ends) 2) After a trajectory-following law terminates at least one hill-climbing law is applicable 
  29. 29. Causal Level
  30. 30. Action Abstraction Schema A sequence of control laws can be abstracted into an action that starts at a given sensory view V and ends at another sensory view V’  This abstraction is called the schema <V, A, V’>  Situation calculus is a suitable formalism of the causal level  Causal level, unlike control level, consists of discrete states  The agent performs a sequence of discrete actions that result in state transitions 
  31. 31. View A view is a description of the sensory input vector s(t) = [s1(t), …, sn(t)]  My guess is that this definition is deliberately vague to give the knowledge engineer a lot of elbow space to play with various representations  For example, a description can specify a Wi-Fi cluster or the color histogram of an image take by the robot’s camera 
  32. 32. Actions, Schemas, Routines An action is a sequence of one or more control laws  An action is initiated at a locally distinctive state specified by one view description and terminates at another locally distinctive state specified by another view description  Actions are specified by schemas <V, A, V’>  A routine is a set of schemas indexed by initial view 
  33. 33. Declarative & Procedural Schema Interpretations ℎ𝑜𝑙𝑑𝑠 𝑉, 𝑠0 && ℎ𝑜𝑙𝑑𝑠 𝑉 ′ , 𝑟𝑒𝑠𝑢𝑙𝑡 𝐴, 𝑠0 ℎ𝑜𝑙𝑑𝑠 𝑉, 𝑛𝑜𝑤 ⇒ 𝑑𝑜 𝐴, 𝑛𝑜𝑤 The first interpretation (declarative) means that a view V is observed in situation s0 and the view V’ holds in the result of executing action A in situation s0  The second interpretation (procedural) means that if a view V is observed, then execute action A right away (now) 
  34. 34. Turns & Travels 𝑇𝑢𝑟𝑛 α , 𝑤ℎ𝑒𝑟𝑒 α 𝑖𝑠 𝑎𝑛 𝑟𝑜𝑡𝑎𝑡𝑖𝑜𝑛 𝑎𝑛𝑔𝑙𝑒 𝑇𝑟𝑎𝑣𝑒𝑙 δ, ΔΘ , 𝑤ℎ𝑒𝑟𝑒 δ, ΔΘ 𝑎𝑟𝑒 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑎𝑛𝑑 𝑜𝑟𝑖𝑒𝑛𝑡𝑎𝑡𝑖𝑜𝑛 𝑐ℎ𝑎𝑛𝑔𝑒, 𝑟𝑒𝑝𝑠𝑒𝑐𝑡𝑖𝑣𝑒𝑙𝑦 At the causal level, all actions are classified into two categories: turn and travel  This categorization may be too restrictive: SSH article states that one can construct environments for which these categories break down  A claim is made, however, that these actions are sufficient for office spaces and street networks  A turn is an action that leaves the agent in the same place; a travel is an action that takes the agent from one place to another 
  35. 35. Routines A routine is a sequence of schemas  A routine is indexed by its initial view, i.e., the view of the 1st schema  A routine represents a behavior that moves the agent from one distinctive state to another distinctive state  Figure 8 in “The Spatial Semantic Hierarchy” seems to imply that distinctive states are described by views  A routine can be viewed as a description of a behavior or as a procedure for executing that behavior in the world 
  36. 36. Complete & Partial Schemas in Routines A schema of the form <V, A, V’> is complete  A schema of the form <V, A, nil> is partial  If a routine is defined in terms of complete schemas, the agent can both execute and describe it  If a routine is defined in terms of partial schemas, the agent can only execute it in the world but not describe it 
  37. 37. Complete & Adequate Routines Suppose 𝑉0 𝐴0 𝑉1 𝐴1 𝑉2 𝐴2 , … , 𝑉 𝑛−1 𝐴 𝑛−1 , 𝑉𝑛 is an alternating sequence of views and actions  A routine R is complete from 𝑉 to 𝑉𝑛 if R has a 0 complete schema < 𝑉𝑖 , 𝐴 𝑖 , 𝑉𝑖+1 > for each 0< 𝑖< 𝑛  A routine R is adequate from 𝑉0 to 𝑉𝑛 if it contains either a complete schema < 𝑉𝑖 , 𝐴 𝑖 , 𝑉𝑖+1 > or a partial schema < 𝑉𝑖 , 𝐴 𝑖 , 𝑉𝑖+1 > for each 0 < 𝑖 < 𝑛 
  38. 38. Complete & Adequate Routines Adequate routines support situated action, i.e., behavior that can be executed only when the agent is placed (“situated”) in the world  Complete routines support both situated action and cognitive manipulation, i.e., the agent can both execute and describe the behavior  Example: if the agent can only navigate a route, the navigation routine is adequate; if the agent can both navigate and describe it, the navigation routine is complete 
  39. 39. Deduction, Induction, & Abduction
  40. 40. Deduction Given the implication A  B and the truth of A, infer B  Example: Implication: if the agent’s Wi-Fi classifier classifies the input signal at cluster C, the agent is at location L Truth of antecedent: the agent’s Wi-Fi classifier classifies the input signal at cluster C Inference: the agent is at cluster location L 
  41. 41. Induction Infer the consequent B from the antecedent A if B has so far always followed A  Example: you observe a swan and note that its color is white; you observe another swan and note that its color is white; you observe another n swans and they are all white; you conclude that if a bird is a swan (A), then its color is white (B)  Inductive inferences are always congruent with agents’ experience but are not always be true: there are, in fact, black swans 
  42. 42. Abduction Infer A as a possible antecedent of the observed consequent B  In the literature on abduction, A is sometimes referred to as a possible explanation of B  Example: you observe that your neighbor’s lot is wet on a dry day (B) and infer that your neighbor has watered the lawn (A); chances are your inference is true but there is a chance that it is wrong: there may be a leaking sprinkler 
  43. 43. Topological Level
  44. 44. Elements of Topological Level A place is a zero-dimensional part of the environment  A path is a one-dimensional subspace (e.g., a street in a city)  There are two directions along a path: dir = +1 and dir =-1  These directions may be loosely interpreted and forward and backward  A travel action moves the agent from one place on a path to another place on a path  A turn action keeps the agent in the same place  A region is a 2D subset of the environment  A region may be abstracted into a place 
  45. 45. Abduction Given a sequence of views and actions, the agent infers places, paths, and regions by abduction  The agent postulates a minimal set of places, paths, and regions consistent with the views and actions  Abduced elements may or may not be sufficient to explain the sequence of observed views and actions 
  46. 46. Topological Relations Given a sequence of views and actions, the agent infers places, paths, and regions by abduction  The agent postulates a minimal set of places, paths, and regions consistent with the views and actions  Abduced elements may or may not be sufficient to explain the sequence of observed views and actions 
  47. 47. Topological Relations 𝑎𝑡 𝑣𝑖𝑒𝑤, 𝑝𝑙𝑎𝑐𝑒 − 𝑣𝑖𝑒𝑤 is seen at 𝑝𝑙𝑎𝑐𝑒 𝑎𝑙𝑜𝑛𝑔 𝑣𝑖𝑒𝑤, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 − 𝑣𝑖𝑒𝑤 is seen along 𝑝𝑎𝑡ℎ in direction 𝑑𝑖𝑟 𝑜𝑛 𝑝𝑙𝑎𝑐𝑒, 𝑝𝑎𝑡ℎ − 𝑝𝑙𝑎𝑐𝑒 is on 𝑝𝑎𝑡ℎ 𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝑝𝑙𝑎𝑐𝑒1, 𝑝𝑙𝑎𝑐𝑒2, 𝑑𝑖𝑟 − the order on path from 𝑝𝑙𝑎𝑐𝑒1 to 𝑝𝑙𝑎𝑐𝑒2 is 𝑑𝑖𝑟 𝑟𝑖𝑔ℎ𝑡𝑂𝑓 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝𝑎𝑡ℎ, facing direction 𝑑𝑖𝑟, has 𝑟𝑒𝑔𝑖𝑜𝑛 on its right 𝑙𝑒𝑓𝑡𝑂𝑓 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝𝑎𝑡ℎ, facing direction 𝑑𝑖𝑟, has 𝑟𝑒𝑔𝑖𝑜𝑛 on its left 𝑖𝑛 𝑝𝑙𝑎𝑐𝑒, 𝑟𝑒𝑔𝑖𝑜𝑛 − 𝑝lace is in 𝑟𝑒𝑔𝑖𝑜𝑛
  48. 48. Topological Axioms 𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, 𝑑𝑖𝑟 → 𝑜𝑛 𝐴, 𝑝𝑎𝑡ℎ & 𝑜𝑛(𝐵, 𝑝𝑎𝑡ℎ) ┐𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐴, 𝑑𝑖𝑟 𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, +1 ↔ 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝐵, 𝐴, −1) 𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐴, 𝐵, 𝑑𝑖𝑟 &𝑜𝑟𝑑𝑒𝑟 𝑝𝑎𝑡ℎ, 𝐵, 𝐶, 𝑑𝑖𝑟 → 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝐴, 𝐶, 𝑑𝑖𝑟) ∃𝛼 𝑎𝑙𝑜𝑛𝑔 𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 & 𝑉, 𝑡𝑢𝑟𝑛, 𝛼 , 𝑉 ′ & 𝑎𝑙𝑜𝑛𝑔 𝑉 ′ , 𝑝𝑎𝑡ℎ, −𝑑𝑖𝑟
  49. 49. Abducing Places & Paths from Views & Actions Every view is observed at some place. ∀𝑣𝑖𝑒𝑤 ∃𝑝𝑙𝑎𝑐𝑒 𝑎𝑡 𝑣𝑖𝑒𝑤, 𝑝𝑙𝑎𝑐𝑒
  50. 50. Abducing Places & Paths from Views & Actions If the agent turns, it does not change its place: 𝑉, 𝑡𝑢𝑟𝑛 𝛼 , 𝑉′ → ∃𝑝𝑙𝑎𝑐𝑒 𝑎𝑡 𝑉, 𝑝𝑙𝑎𝑐𝑒 &𝑎𝑡 𝑉 ′ , 𝑝𝑙𝑎𝑐𝑒
  51. 51. Abducing Places & Paths from Views & Actions If the agent travels a non-zero distance, then the first and second view exist at two distinct places. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 𝛼 , 𝑉′ & 𝛿 ≠ 0 → ∃𝑝𝑙𝑎𝑐𝑒1 , 𝑝𝑙𝑎𝑐𝑒2 𝑝𝑙𝑎𝑐𝑒1 ≠ 𝑝𝑙𝑎𝑐𝑒2 & 𝑎𝑡 𝑉, 𝑝𝑙𝑎𝑐𝑒1 & 𝑎𝑡 𝑉 ′ , 𝑝𝑙𝑎𝑐𝑒2
  52. 52. Abducing Places & Paths from Views & Actions If the agent travels, then there are a path and direction such that the 1st view V exists on that path in that direction and the 2nd view V’ exists on that path in the same direction. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 𝛼 , 𝑉′ → ∃𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 𝑎𝑙𝑜𝑛𝑔(𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟) & 𝑎𝑙𝑜𝑛𝑔 𝑉 ′ , 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟
  53. 53. Abducing Places & Paths from Views & Actions If the agent travels, then there are a path and a direction with two places such that the first place has the first view, the second place has the second view, both views exist along the path and can be ordered along the same direction. 𝑉, 𝑡𝑟𝑎𝑣𝑒𝑙 δ , 𝑉′ & 𝛿 ≠ 0 → ∃𝑝𝑙𝑎𝑐𝑒1 , 𝑝𝑙𝑎𝑐𝑒2 , 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 𝑎𝑡(𝑉, 𝑝𝑙𝑎𝑐𝑒1 ) & 𝑎𝑡 𝑉 ′ , 𝑝𝑙𝑎𝑐𝑒2 & 𝑎𝑙𝑜𝑛𝑔(𝑉, 𝑝𝑎𝑡ℎ, 𝑑𝑖𝑟 & 𝑜𝑟𝑑𝑒𝑟(𝑝𝑎𝑡ℎ, 𝑝𝑙𝑎𝑐𝑒1 , 𝑝𝑙𝑎𝑐𝑒2 , 𝑑𝑖𝑟 )
  54. 54. Regions, Boundaries, Abstractions   Regions are sets of places Places are grouped into regions because 1) they are located on one side of a specific boundary; 2) they share a 2D metrical frame; 3) they are abstracted to the same place in a higher-level topological map
  55. 55. Regions, Boundaries, Abstractions SSH supports upward and downward mapping  Upward mapping: multiple places at a lower level map to a single place/region at a higher level  Downward mapping: a single place at a higher level map to multiple places at a lower level  An abstraction region is the set of places in a more detailed map abstracted to a particular place  Example: a corridor can be abstracted to a single place in a higher-level map 
  56. 56. Topological Level Uses Topological level of representation supports various problem-solving methods  It can be searched as a graph (DFS, BFS)  Distance measures, when and if they are available, support A* and Dijkstra  Topological level can support goals and subgoals 
  57. 57. Metrical Level
  58. 58. Global Metrical Mapping An agent may have a global single frame of reference (2D or 3D)  Many useful state of knowledge cannot be expressed numerically in terms of real numbers (orientation error)  Storage of large global frame of references may present problems  Global frame of reference can be split into a patchwork of local frame of references 
  59. 59. Narrative Maps
  60. 60. Narrative Map Narrative map consists of two verbal descriptions: verbal route descriptions and verbal surveys of a given environment  The origins of narrative maps vary from blogs, forums, books, O&M instructors  Question: To what extent can narrative maps be mined/parsed to extract various levels of SSH? 
  61. 61. Route Description 01 Exit Northbound M 4 or M 104 bus on Broadway between 120th and 121st streets. Walk straight to the inside guideline of the sidewalk and turn right. Trail along the left and in 50 feet you will reach the corner of 120th and Broadway. source:
  62. 62. Route Description 02 Turn left and continue trailing. In 50 feet, you will reach the first left side opening. Pass this opening and continue straight. In 200 feet, you will reach the second left side opening, a driveway that leads to Thorndike. Take this left turn. source:
  63. 63. Route Description 03 Pass through the second set of double doors and you will be in the main lobby of the Thorndike building. At 1:00, 40 feet away is the disability services main office. source:
  64. 64. Route Description 04 As you continue along this bypass lane, you will notice a rough gravel driveway near its endpoint. Five feet after crossing this driveway, there is a mailbox that sticks out into the walking path. Continue following along the left side edge, and in 800 to 1000 feet, you will reach a second bypass lane. source:
  65. 65. Route Description 05 At any point, turn and cross to the South side of route 83 and then turn East with the grass line on your right. It will be easiest for both you and Pixie to walk along the dirt or grass to the right of the pavement. The first intersection you reach will be 173rd street. Cross and turn right to proceed towards your home. source:
  66. 66. Thoughts on Automated SSH Extraction from Narrative Maps
  67. 67. Tentative Suggestions Represent landmarks in terms of SSH: place, path, region  Represent actions that the agent can execute on route (CD primitive acts are a possibility)  Develop a parser that parses narrative maps into landmarks and actions  Develop an SSH compiler that takes graphs produced by the parser and creates various levels of SSH 
  68. 68. CA: CD Parsing Conceptual Analyzer CD Graphs (aka CDs) Natural Language Input Inference Engine Modified and/or New CDs LTM
  69. 69. PTRANS PTRANS – transfer of the physical location of an object Examples: 1) GO is an PTRANS of oneself to a place 2) PUT is an PTRANS of an object to a place The robot went to the lab. The robot put the block on the table.
  70. 70. PROPEL PROPEL – application of a physical force to an object; this primitive applies whenever any force is applied Examples: PUSH, PULL, KICK, THROW have the PROPEL primitive The robot pushed the chair to the wall. This is an instance of PROPEL by the robot to the chair that caused a PTRANS of the chair from its current location to the wall.
  71. 71. MOVE MOVE – the movement of a body part of an agent/animal by that agent/animal Examples: KICK, HAND have the MOVE primitive The boy kicked the ball. This is an instance of MOVE by the boy of his foot to the ball that causes a PTRANS of the ball from its current location to some unknown location.
  72. 72. GRASP GRASP – the grasping of an object by an actor Examples: HOLD, GRAB have the GRASP primitive The robot picked up the ball from the floor. This is an instance of GRASP by the robot of the ball to the ball that causes a PTRANS of the ball from the floor into the robot’s gripper. This is also an instance of MOVE by the robot of its gripper to the ball.
  73. 73. INGEST INGEST – the taking of an object by an animal/agent to the inside of that animal agent Examples: EAT, DRINK, SMOKE, BREATHE have the INGEST primitive The robot charged. John ate an apple. These are instances of INGEST. The first sentence is an INGEST by the robot of electricity inside the robot’s batter. The second sentence is an instance of INGEST by John of the apple to John’s stomach.
  74. 74. MTRANS MTRANS – the transfer of mental information within one animal/agent or between/among animals/agents. CD Theory partitions the agent’s memory into two components: CP (conscious processor where current mental manipulation occurs) and LTM (long-term memory where things are stored) Examples: TELL, INFORM, SEE, FORGET have the MTRANS primitive Mary told the robot how to get to the lab. The robot told Mary which rooms it had cleaned. Both sentences are instances of MTRANS. Mary does an MTRANS of a route from some location to the lab. The robot does an MTRANS of the rooms it had cleaned to Mary.
  75. 75. SPEAK SPEAK – the production of sounds by an animal/agent. Examples: SHOUT, PURR, BEEP have the SPEAK primitive The robotic car beeped twice. Mary yelled at John.
  76. 76. ATTEND ATTEND – the focusing of a sense organ by an animal/agent toward a stimulus. Examples: ATTEND(EAR) – LISTEN ATTEND(EYE) – SEE ATTEND(NOSE) – SMELL ATTEND(SKIN) – TOUCH The robot detected a door. John saw an exit.
  77. 77. A Field Study of the SSH Topological Level in Blind Navigation of Modern Supermarket
  78. 78. Background We investigated the utility of verbal route instructions in a longitudinal study of blind shopping in supermarkets  In our system, called ShopTalk, verbal route directions were generated from a manually constructed topological map (inspired by the topological level of the SSH) of the supermarket’s locomotor space 
  79. 79. ShopTalk 1.0
  80. 80. Field Study Results Ten visually impaired participants were able to detect environmental cues needed to make sense of the generated verbal instructions (provided at beginning of experiment)  The participants used their O&M skills to localize and orient themselves in the store, without any wearable or environment-embedded sensors 
  81. 81. Field Study Results A key finding was that verbal route directions were sufficient for our sample of independent travelers to navigate this supermarket reliably  The more they used the system, the less they requested verbal route directions  This finding suggests that the stores (and other dynamic and complex environments) may not need to be instrumented with any external sensors, such as RFID tags, Wi-Fi routers, IR transmitters, etc. 
  82. 82. References & Reading Suggestions B. Kuipers. (2000). “The Spatial Semantic Hierarchy.” Artificial Intelligence 119, pp. 191-233.  R. Schank, C. Riesbeck W. A. (1981) Inside Computer Understanding. Lawrence Erlbaum & Associates.  J. Nicholson, V. Kulyukin, D. Coster. (2009). “ShopTalk: Independent Blind Shopping Through Verbal Route Directions and Barcode Scans.” The Open Rehabilitation Journal, ISSN: 1874-9437 Volume 2, 2009, DOI 10.2174/1874943700902010011. 