Your SlideShare is downloading. ×
NLP (Fall 2013): Conceptual Dependency Theory
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

NLP (Fall 2013): Conceptual Dependency Theory

523
views

Published on

Published in: Technology, Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
523
On Slideshare
0
From Embeds
0
Number of Embeds
35
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Natural Language Processing Conceptual Dependency Theory Vladimir Kulyukin www.vkedco.blogspot.com
  • 2. Outline Conceptual Dependency (CD) Theory Background  Conceptual Dependency Primitives  Conceptual Analysis of Natural Language 
  • 3. Background
  • 4. Background Language is a medium whose primary purpose is communication  Primary focus of linguistics is to determine what kinds of things can be communicated and how  Primary emphasis is on content, not form (this is in opposition to formal language theory) 
  • 5. Is Pro Content Anti-Syntax? Primary focus on content does not mean that the focus is anti-syntax  Syntax is very important but its role should be secondary to the study of knowledge and meaning  There should be no independent syntactic pass over the input: syntactic & semantic processing should go hand in hand 
  • 6. Basic Axioms of CD Theory For any two sentences that are identical in meaning, regardless of language, there should be only one representation  Any information in a sentence that is implicit must be made explicit in the representation of the meaning of that sentence 
  • 7. Basic Definitions of CD Theory CD proposes a meaning representation formalism  The meaning propositions underlying language are called conceptualizations  A conceptualization can be active or stative  An active conceptualization has the form: <ACTOR, ACTION, OBJECT, DIRECTION, INSTRUMENT>  A stative conceptualization has the form: IS_IN(OBJECT, STATE, VALUE) – OBJECT is in STATE whose value is equal to VALUE 
  • 8. Event Representation Every EVENT has: ACTOR ACTION performed by ACTOR OBJECT that ACTION is performed on DIRECTION in which ACTION is oriented INSTRUMENT with which ACTOR does ACTION
  • 9. Note on ACTOR & OBJECT ACTOR is a concrete object (aka PICTURE PRODUCER or PP)  ACTOR can decide to apply ACTION to another PP called OBJECT  A rock is a PP but cannot be an ACTOR because it cannot decide to apply ACTION to any other object  Honesty, justice, truth and other mass nouns are not PPs  ACTION represent a physical action or a mental action 
  • 10. Semantic Action & State Primitives CD constructs the meaning of natural language input from a finite set of semantic primitives  Semantic primitives can be considered as an interlingual vocabulary in terms of which one can, in principle, represent the meaning of every word in every language  To date, there is no universally accepted set of semantic action & state primitives but the search for this set continues 
  • 11. CD Primitive Acts
  • 12. Verb Representation in CD A verb is represented as a particular combination of primitive actions (acts) and states none of which are unique to that verb but whose combination is entirely unique R. Schank, R. Abelson. “Scripts, Plans, Goals, & Understanding: An Inquiry into Human Knowledge Structures”
  • 13. ATRANS ATRANS – transfer of an abstract relationship (e.g., possession, ownership, control) Examples: 1) GIVE is an ATRANS of something to someone else 2) TAKE is an ATRANS of something to oneself 3) BUY is an ATRANS of something to oneself and another ATRANS of money from oneself to the owner of something The robot gave John a cup of coffee. The robot took a cup of coffee from the coffee machine. John bought a new car.
  • 14. PTRANS PTRANS – transfer of the physical location of an object Examples: 1) GO is an PTRANS of oneself to a place 2) PUT is an PTRANS of an object to a place The robot went to the lab. The robot put the block on the table.
  • 15. PROPEL PROPEL – application of a physical force to an object; this primitive applies whenever any force is applied Examples: PUSH, PULL, KICK, THROW have the PROPEL primitive The robot pushed the chair to the wall. This is an instance of PROPEL by the robot to the chair that caused a PTRANS of the chair from its current location to the wall.
  • 16. MOVE MOVE – the movement of a body part of an agent/animal by that agent/animal Examples: KICK, HAND have the MOVE primitive The boy kicked the ball. This is an instance of MOVE by the boy of his foot to the ball that causes a PTRANS of the ball from its current location to some unknown location.
  • 17. GRASP GRASP – the grasping of an object by an actor Examples: HOLD, GRAB have the GRASP primitive The robot picked up the ball from the floor. This is an instance of GRASP by the robot of the ball to the ball that causes a PTRANS of the ball from the floor into the robot’s gripper. This is also an instance of MOVE by the robot of its gripper to the ball.
  • 18. INGEST INGEST – the taking of an object by an animal/agent to the inside of that animal agent Examples: EAT, DRINK, SMOKE, BREATHE have the INGEST primitive The robot charged. John ate an apple. These are instances of INGEST. The first sentence is an INGEST by the robot of electricity inside the robot’s batter. The second sentence is an instance of INGEST by John of the apple to John’s stomach.
  • 19. EXPEL EXPEL – the expulsion of an object from the body of an animal/agent to the outside of the body Examples: SWEAT, CRY have the EXPEL primitive Mary cried. John spat on the floor. Both sentences are instances of EXPEL. The object of the first instance of EXPEL is tears. The object of the second instance of EXPEL is saliva.
  • 20. MTRANS MTRANS – the transfer of mental information within one animal/agent or between/among animals/agents. CD Theory partitions the agent’s memory into two components: CP (conscious processor where current mental manipulation occurs) and LTM (long-term memory where things are stored) Examples: TELL, INFORM, SEE, FORGET have the MTRANS primitive Mary told the robot how to get to the lab. The robot told Mary which rooms it had cleaned. Both sentences are instances of MTRANS. Mary does an MTRANS of a route from some location to the lab. The robot does an MTRANS of the rooms it had cleaned to Mary.
  • 21. MBUILD MBUILD – the construction of an agent/animal of new information from old information. Examples: DECIDE, CONCLUDE, REMEMBER have the MBUILD primitive The robot concluded that it is lost. John remembered that he had promised Mary to take her to the movies.
  • 22. SPEAK SPEAK – the production of sounds by an animal/agent. Examples: SHOUT, PURR, BEEP have the SPEAK primitive The robotic car beeped twice. Mary yelled at John.
  • 23. ATTEND ATTEND – the focusing of a sense organ by an animal/agent toward a stimulus. Examples: ATTEND(EAR) – LISTEN ATTEND(EYE) – SEE ATTEND(NOSE) – SMELL ATTEND(SKIN) – TOUCH The robot detected a door. John saw an exit.
  • 24. Categorization of Primitive ACTs Physical ACTs: 1) PROPEL, 2) MOVE, 3) INGEST, 4) EXPEL, 5) GRASP  ACTs that cause state changes: 1) PTRANS, 2) ATRANS  Instrumental ACTs: 1) SPEAK, 2) ATTEND  Mental ACTs: 1) MTRANS, 2) MBUILD 
  • 25. CD Representation of States States are presented as attribute-value pairs  The values come from arbitrary ranges constructed by the knowledge engineer  For example, an agent’s health can be represented on a scale from -10 to +10  CD has never formulated a coherent set of state primitives comparable to its primitive acts and adhered to by all its proponents 
  • 26. Conceptual Analysis (CA) of Natural Language
  • 27. CA: CD Parsing Conceptual Analyzer CD Graphs (aka CDs) Natural Language Input Inference Engine Modified and/or New CDs LTM
  • 28. CD Representation Rules Rule 1: PP ACT Picture Producer PP can perform some act ACT
  • 29. CD Representation Rules Rule 2: ACT o PP Some act ACT has some PP as its object
  • 30. CD Representation Rules PP1 Rule 3: ACT D PP2 Some act ACT is directed from PP2 to PP1
  • 31. CD Representation Rules PP1 Rule 4: ACT R PP2 Some act ACT receives something from PP2 and gives it to PP1
  • 32. CD Representation Rules ACT2 Rule 5: ACT1 I PP2 Some act ACT1 is accomplished (instrumented) by another act ACT2 done by some picture producer PP2
  • 33. CD Graph Examples
  • 34. CD Graph Example 01 The robot went to the kitchen.
  • 35. CD Graph Example 01 Robot PTRANS O Robot D Unknown Kitchen
  • 36. CD Graph Example 02 The robot went from the lab to the kitchen.
  • 37. CD Graph Example 02 Robot PTRANS O Robot D Lab Kitchen
  • 38. CD Graph Example 03 John ate an apple.
  • 39. CD Graph Example 03 Unknown John Mouth D MOVE I John INGEST O O Hand Apple D Unknown Mouth
  • 40. CD Graph Example 04 The robot saw a door.
  • 41. CD Graph Example 04 Unknown Robot Door D ATTEND I Robot MTRANS O O Camera Door D Camera CP
  • 42. CD Graph Example 04 John saw an exit.
  • 43. CD Graph Example 05 Unknown John Exit D ATTEND I John MTRANS O O Eyes Exit D Eyes CP
  • 44. CD Graph Example 06 The robot read a street sign.
  • 45. CD Graph Example 06 Unknown Robot Sign D ATTEND I Robot MTRANS O O Camera Sign D Camera CP
  • 46. CD Graph Example 07 John promised to give Mary a book.
  • 47. CD Graph Example 07 Mary John John D ATRANS O Book O John MTRANS D LTM Unknown
  • 48. Mapping NL Input to CD Graphs
  • 49. Syntactic Parsing vs. Conceptual Analysis The objective of syntactic parsing is to construct a parse tree (or multiple parse trees) of the input  The objective of conceptual analysis (CA) is to construct a conceptual dependency graph representing the meaning of the input 
  • 50. Expectations Conceptual analysis (CA) is an expectation-driven process  Syntactic parsers (e.g., Early parser) also use expectations  In a syntactic parser, expectations come from a grammar  In a conceptual analyzer, expectations come from a library of CD structures 
  • 51. CD Database Suppose that we have compiled a database of PP and CD structures  For example, our database may have CD structures like <INGEST :ACTOR NULL :OBJECT NULL :FROM NULL :TO NULL :INSTRUMENT NULL :TIME NULL> <PP :CLASS APPLE :REF NULL NUMBER: NULL> <PP :CLASS BOOK :REF NULL NUMBER: NULL> <PP :CLASS HUMAN :NAME JOHN :GENDER MALE> 
  • 52. CD Database CD structures have slots and fillers  Consider this CD:  <INGEST :ACTOR <PP :CLASS HUMAN :NAME JOHN :GENDER MALE> :OBJECT NULL :FROM NULL :TO NULL :INSTRUMENT NULL :TIME NULL> In this CD, the slot :ACTOR has the filler <PP :CLASS HUMAN :NAME JOHN :GENER MALE> while the slot :OBJECT has the filler 
  • 53. Requests Expectations are implemented as requests  A Request consists of a Boolean test and a sequence of actions that can be run if the test is true  More on test and action types below  <REQUEST TEST: a Boolean test ACTION 1, ACTION 2, … ACTION N>
  • 54. Control & Data Structures CA uses two lists: CONCEPT LIST (C-LIST) and REQUEST LIST (R-LIST)  C-LIST models the short-term memory (STM)  R-LIST models an expectation management mechanism 
  • 55. Basic CA Algorithm 0) C-LIST = [] // empty list 1) R-LIST = [] // empty list 3) While ( True ) { 4) NextItem = Read the next item from the input 5) If ( NextItem == NULL ) Then RETURN 6) Requests = RetrieveRequests(NextItem, Dictionary) 7) Add Requests to R-LIST 8) Run all active requests in R-LIST 9) }
  • 56. Test Types Type 01: Test for the occurrence of a particular word or phrase in the input  Type 02: Test for the occurrence or ordering properties of concepts on C-LIST 
  • 57. Action Types Type 01: Add a CD to C-LIST  Type 02: Fill a slot of a CD with another CD  Type 03: Activate another request, i.e., add it to R-LIST  Type 04: De-activate another request, i.e., remove it from R-LIST 
  • 58. CA Example
  • 59. CD Dictionary Entry for “JOHN” “JOHN”  <REQUEST TEST: TRUE ACTIONS: 1. Add <PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”> to C-LIST >
  • 60. CD Dictionary Entry for “ATE” “ATE”  <REQUEST TEST: TRUE ACTIONS: 1. Add <INGEST :ACTOR NULL :OBJECT NULL :FROM NULL :TO NULL :INSTRUMENT NULL :TIME PAST> to C-LIST 2. Add <REQUEST :TEST Can you find a PP Human on C-LIST before INGEST? :ACTIONS 1. Put the found PP human to the ACTOR slot of the INGEST CD> to R-LIST 3. Add <REQUEST :TEST Can you find a PP Edible on CLIST after INGEST? :ACTIONS 1. Put the found PP Edible into the OBJECT slot of the INGEST CD> to R-LIST >
  • 61. CD Dictionary Entry for “AN” “AN”  <REQUEST TEST: TRUE ACTIONS: 1. Add <REQUEST :TEST TRUE :ACTIONS 1. Add a dummy CD with a unique ID to C-LIST> to R-LIST 2. Add <REQUEST :TEST Can you find a PP of CATEGORY COUNTABLE after the dummy CD? :ACTIONS 1. Modify the REF slot of the found PP to “INDEF”> to R-LIST >
  • 62. CD Dictionary Entry for “APPLE” “APPLE”  <REQUEST :TEST TRUE :ACTIONS 1. Add <PP :CLASS :CATEGORY :TYPE :REF to C-LIST > APPLE COUNTABLE EDIBLE NULL>
  • 63. Sample Input John ate an apple.
  • 64. Sample Input John ate an apple. 1. Read “John” 2. Retrieve this request and add it to R-LIST <REQUEST TEST: TRUE ACTIONS: 1. Add <PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”> to C-LIST> 3. Run all requests: there is only one active request whose test is true so you run its actions and take off R-LIST 4. The action is to add <PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”> to C-LIST 5. So, C-LIST = [<PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”>]
  • 65. Sample Input John ate an apple. 1. Read “ate” 2. Retrieve this request associated with “ate” and add it to R-LIST 3. Run all requests on R-LIST So, R-LIST = [ <REQUEST :TEST Can you find a PP Human on C-LIST before INGEST? :ACTIONS 1. Put the found PP human to the ACTOR slot of the INGEST CD> <REQUEST :TEST Can you find a PP Edible on CLIST after INGEST? :ACTIONS 1. Put the found PP Edible into the OBJECT slot of the INGEST CD> ] 4. C-LIST = [<PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”>, <INGEST :ACTOR NULL :OBJECT NULL :FROM NULL :TO NULL :INSTRUMENT NULL :TIME PAST>]
  • 66. Sample Input John ate an apple. 1. Read “an” 2. Retrieve the request associated with “an” and add it to R-LIST 3. Run all requests on R-LIST So, R-LIST = [<REQUEST :TEST Can you find a PP Edible on CLIST after INGEST? :ACTIONS 1. Put the found PP Edible into the ACTOR slot of the INGEST CD>, <REQUEST :TEST Can you find a PP of CATEGORY COUNTABLE after the dummy CD? :ACTIONS 1. Modify the REF slot of the found PP to “INDEF”> ] C-LIST = [<INGEST :ACTOR <PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”>, :OBJECT NULL :FROM NULL :TO NULL :INSTRUMENT NULL :TIME PAST>, <DUMMY CD>]
  • 67. Sample Input John ate an apple. 1. Read “apple” 2. Retrieve the request associated with “apple” and add it to R-LIST 3. Run all requests on R-LIST R-LIST = [] C-LIST = [<INGEST :ACTOR <PP :CLASS “HUMAN” :NAME “John” :GENDER “MALE”>, :OBJECT <PP :CLASS APPLE :CATEGORY COUNTABLE :TYPE EDIBLE :REF “INDEF”> :FROM NULL :TO NULL :INSTRUMENT NULL :TIME PAST> ]
  • 68. Critique of CD Theory
  • 69. Sapir-Whorf Hypothesis: Alternative to Semantic Primitives Edward Sapir, 1884 - 1939 Benjamin Lee Whorf, 1897 - 1941
  • 70. Linguistic Relativity Benjamin Lee Whorf (1897 – 1941) was an American linguist & anthropologist whose mentor was Edward Sapir (1884 – 1939), another American linguist & anthropologist  Whorf formulated the Principle of Linguistic Relativity (aka the Sapir-Whorf Hypothesis) that states that speakers of different languages conceptualize and experience the world differently due to linguistic differences in grammar and usage  Linguistic Relativity Principle states that it is impossible to find the universal set of semantic action and state primitives 
  • 71. Charles Dunlop’s Critique The reduction of all human actions and all physical objects to a small set of primitives has little promise of succeeding  Primitive ACTs cannot capture intentional states 
  • 72. References & Reading Suggestions R. Schank, C. Riesbeck W. A. (1981) Inside Computer Understanding. Lawrence Erlbaum & Associates.  C. Dunplop. (1990). Conceptual Dependency as the Language of Thought. Synthese 82:275-296. Kluwer. 