Your SlideShare is downloading. ×
0
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Computational
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Computational

1,111

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,111
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Johan Bos University of Rome " La Sapienza “ Computational Semantics and Knowledge Engineering
  • 2. A text understanding system <ul><li>Mary has a dog. </li></ul><ul><li>Does Mary have an animal? </li></ul>User: System:
  • 3. A text understanding system <ul><li>Mary has a dog. </li></ul><ul><li>Does Mary have an animal? </li></ul>Yes! (dogs are animals) User: System:
  • 4. A text understanding system <ul><li>Vincent is not married. </li></ul><ul><li>Vincent is married. </li></ul>User: System:
  • 5. A text understanding system <ul><li>Vincent is not married. </li></ul><ul><li>Vincent is married. </li></ul>Hey! That’s inconsistent. User: System:
  • 6. A text understanding system <ul><li>Florence is the cultural capital of Italy. </li></ul><ul><li>Is Florence the capital of Italy? </li></ul>User: System:
  • 7. A text understanding system <ul><li>Florence is the cultural capital of Italy. </li></ul><ul><li>Is Florence the capital of Italy? </li></ul>Yes! (a cultural capital is a capital) User: System:
  • 8. A text understanding system <ul><li>Mia’s husband is a gangster. </li></ul><ul><li>Is Mia married? </li></ul>User: System:
  • 9. A text understanding system <ul><li>Mia’s husband is a gangster. </li></ul><ul><li>Is Mia married? </li></ul>Uhh, don’t know… User: System:
  • 10. Knowledge acquisition <ul><li>You need knowledge? </li></ul><ul><li>You want a machine to get it? </li></ul><ul><li>Why don’t you get it from … text ? </li></ul><ul><ul><li>Wikipedia </li></ul></ul><ul><ul><li>Definitions </li></ul></ul><ul><ul><li>Dictionary glosses </li></ul></ul>
  • 11. Catch 22 <ul><li>To build an intelligent NLP system we need background knowledge </li></ul><ul><li>To acquire background knowledge automatically we need NLP </li></ul>
  • 12. This talk <ul><li>Background: computational semantics </li></ul><ul><li>Building a text understanding system </li></ul><ul><ul><li>Syntax-semantics interface </li></ul></ul><ul><ul><li>Semantic representation and inference </li></ul></ul><ul><ul><li>Knowledge </li></ul></ul><ul><li>Open-domain question answering </li></ul><ul><ul><li>Why you need semantics </li></ul></ul><ul><ul><li>Why you need knowledge </li></ul></ul><ul><li>A case study </li></ul>
  • 13. Computational Semantics <ul><li>Semantics The branch of logic and linguistics concerned with meaning </li></ul><ul><li>Computational Semantics Using a computer to build meaning representations, and reason with the result (inference) </li></ul>
  • 14. Applications <ul><li>Spoken Dialogue Systems </li></ul><ul><li>Question-Answering Systems </li></ul><ul><li>Textual Inference Systems </li></ul>
  • 15. Reasoning and Natural Language <ul><li>To reason you need </li></ul><ul><ul><li>A logical formalism </li></ul></ul><ul><ul><li>A reasoning engine for your logic </li></ul></ul><ul><ul><li>Something that maps language into logic </li></ul></ul><ul><ul><li>Supporting background knowledge </li></ul></ul>
  • 16. Reasoning and Natural Language <ul><li>To map language into logic you need </li></ul><ul><ul><li>A parser that produces syntactic structure </li></ul></ul><ul><ul><li>A syntax-semantics interface </li></ul></ul><ul><ul><li>A semantic formalism </li></ul></ul>
  • 17. The bigger picture Syntactic Analysis Semantic Analysis Natural language Syntactic structure Semantic representation Inference Model Builder Theorem Prover Background Knowledge proofs models First-order logic axioms
  • 18. The bigger picture Syntactic Analysis Semantic Analysis Natural language Syntactic structure Semantic representation Inference Model Builder Theorem Prover Background Knowledge proofs models First-order logic axioms
  • 19. Syntactic analysis: C&C <ul><li>Creation of tree-banks </li></ul><ul><li>Stochastic parsers trained on such tree-banks </li></ul><ul><li>C&C parser (Clark & Curran) </li></ul><ul><ul><li>Combinatory Categorial Grammar </li></ul></ul><ul><ul><li>Efficient and robust </li></ul></ul>
  • 20. The bigger picture C&C parser Semantic Analysis Natural language Syntactic structure Semantic representation Inference Model Builder Theorem Prover Background Knowledge proofs models First-order logic axioms
  • 21. Semantic Analysis: Boxer <ul><li>The recent developments in parsing has lead to developing wide-coverage semantic analysis components </li></ul><ul><li>One of such systems is Boxer, developed over the last five years </li></ul>
  • 22. Boxer <ul><li>Follows the principles of Hans Kamp's Discourse Representation Theory </li></ul><ul><li>Produces formal semantic representations </li></ul><ul><li>Translation to first-order logic </li></ul><ul><li>Systematic syntax-semantic interface, using lambda calculus </li></ul><ul><li>Pronoun and presupposition resolution </li></ul>
  • 23. Discourse Representation Theory <ul><li>Box-like structures as semantic representation </li></ul><ul><li>Structure plays role in pronoun resolution </li></ul><ul><li>Neo-Davidsonian event semantics </li></ul>A spokesman lied. Every spokesman lied. spokesman(x) lie(e) agent(e,x) x e  spokesman(x) x lie(e) agent(e,x) e
  • 24. Syntax-semantics interface
  • 25. Breakthrough <ul><li>Why C&C and Boxer make a difference </li></ul><ul><ul><li>Broad-coverage </li></ul></ul><ul><ul><li>Reasonably efficient </li></ul></ul><ul><ul><li>Clean syntax-semantics interface </li></ul></ul><ul><ul><li>Interpretable structures </li></ul></ul>
  • 26. The bigger picture C&C parser Boxer Natural language Syntactic structure Semantic representation Inference Model Builder Theorem Prover Background Knowledge proofs models First-order logic axioms
  • 27. Inference: Nutcracker <ul><li>Selects appropriate background knowledge for inference </li></ul><ul><li>Acts as a mediator between Boxer and inference engines for first-order logic </li></ul><ul><ul><li>Theorem prover </li></ul></ul><ul><ul><li>(Finite) Model builders </li></ul></ul><ul><li>Reports results back to Boxer or front-end application </li></ul>
  • 28. The bigger picture C&C parser Boxer Natural language Syntactic structure Semantic representation Nutcracker Model Builder Theorem Prover Background Knowledge proofs models First-order logic axioms
  • 29. Inference engines <ul><li>Which inference engines? Off-the-shelf! </li></ul><ul><li>How do we know which are the best? </li></ul><ul><ul><li>CADE world cup automated deduction </li></ul></ul><ul><ul><li>Theorem proving: vampire </li></ul></ul><ul><ul><li>Model building: paradox </li></ul></ul>
  • 30. Some concerns <ul><li>Isn’t first-order logic undecidable? </li></ul><ul><li>Why do we need both a theorem prover and a model builder? </li></ul>
  • 31. Yin and Yang of Inference <ul><li>Theorem Proving and Model Building function as opposite forces </li></ul>
  • 32. The bigger picture C&C tools Boxer Natural language Syntactic structure Semantic representation Nutcracker Paradox Vampire Background Knowledge proofs models First-order logic axioms
  • 33. Background Knowledge <ul><li>Format: first-order logic </li></ul><ul><li>Amount: ideally as few axioms as possible </li></ul>
  • 34. Man or Machine? <ul><li>Manually constructed </li></ul><ul><ul><li>WordNet </li></ul></ul><ul><ul><li>NomLex </li></ul></ul><ul><ul><li>CIA Factbook </li></ul></ul><ul><ul><li>CYC/OpenCYC </li></ul></ul><ul><li>Automatically constructed </li></ul><ul><ul><li>Linguistic patterns [e.g. Hearst] </li></ul></ul><ul><ul><li>Paraphrases [e.g. Lin & Pantel] </li></ul></ul>
  • 35. WordNet <ul><li>Use hypernym relations from WordNet to build an isa-hierarchy </li></ul><ul><li>Create MiniWordNets for small texts </li></ul><ul><ul><li>Convert these into first-order axioms </li></ul></ul>
  • 36. MiniWordNet <ul><ul><li>There is no asbestos in our products now. </li></ul></ul><ul><ul><li>Neither Lorillard nor the researchers who studied the workers </li></ul></ul><ul><ul><li>were aware of any research on smokers of the Kent cigarettes. </li></ul></ul>
  • 37. MiniWordNet  x(asbestos(x)  entity(x))  x(cigarette(x)  entity(x))  x(asbestos(x)   cigarette(x))
  • 38. The bigger picture C&C tools Boxer Natural language Syntactic structure Semantic representation Nutcracker Paradox Vampire WordNet proofs models First-order logic axioms
  • 39. OK – does this work?
  • 40. It does work for small domains… <ul><li>Examples: </li></ul><ul><li>Talking robots </li></ul><ul><li>Smart houses </li></ul>
  • 41. But what about larger domains? <ul><li>Semantic interpretation of e.g. </li></ul><ul><ul><li>newspaper texts </li></ul></ul><ul><ul><li>web pages </li></ul></ul><ul><ul><li>wikipedia </li></ul></ul><ul><li>Let’s look at a real-world application in which we used C&C and Boxer: </li></ul><ul><ul><li>Open-domain question answering </li></ul></ul><ul><ul><li>NIST/TREC campaigns </li></ul></ul><ul><ul><li>Pronto QA system </li></ul></ul>
  • 42. What is Question Answering? <ul><li>Questions, no queries! </li></ul><ul><li>Answers, not documents! </li></ul>Where did Olof Palme die? Q: A:
  • 43. What is Question Answering? <ul><li>Questions, no queries! </li></ul><ul><li>Answers, not documents! </li></ul>Where did Olof Palme die? In Stockholm. Q: A:
  • 44. People ask questions <ul><li>Excite Search Engine Log with 2,477,283 queries </li></ul><ul><li>Collected at 20 December 1999 </li></ul><ul><li>Around 15% natural language questions </li></ul>
  • 45. Excite Query Log 090036 0E600BA4988CF3B4 40 www.prodreg.com/browning 090036 86BE2D578695A788 0 teen kim 090036 138E3A0E17E8462C 0 toys r us 090036 458707BFDA329E9A 0 iso AND 9000 AND training 090036 F54BC573FF3C94B1 0 Where can I see pictures of the female celebrity Michelle Pfeiffer? 090036 7E7D8993E53A1D43 0 www.ezone.com 090036 4AD556DDCA079EB8 0 Where can I buy a Tenesse Titians Jersey? 090036 593E6D1F438E4774 0 Seven Falls 090036 F36E9DDD57EC5BD1 90 Phipps Fine Jewelry.com 090036 B34CF6EA2B2231B7 0 Minnesota schools 090036 442BF0248CDF1D7D 0 palm pilot programmer 090036 18E21BC7D100C75A 0 where can i find info on the government of Turkey? 090036 9E32A306444FF93D 0 boobs tits 090036 C83E2204AE590930 0 san antonio 090036 A3D1A6107CED9F1A 0 local authorities christmas unitary councils 090036 EAD674E28277D799 0 chichen itza 090036 CEE0DBDC438233AD 0 music + free + downlaods 090036 D43552C9CAFB9CD8 0 &quot;marble jewelry box&quot; 090036 5C5BBD4BB2CEC87C 0 where can I play tetris online 090036 2DCCE5254C0AD49E 0 www.dragonballz.com 090036 82C5F15DC4124756 0 pussy 090036 E46C818CBE8F2DDE 0 PEANUT BUTTER COOKIES 090036 649BC4A73E402DCC 0 self esteem 090036 4CF71BBBA59B0210 0 Importers of Islamic Art 090037 AFF432E6129E8557 10 &quot;guitar microphones&quot; 090037 2F2653FB59E33842 0 new AND york AND jobs 090037 F54BC573FF3C94B1 0 athens 090037 8B00FD33C6474FA8 130 playstation tomb raider 4 090037 8B4ADE423F0C8B1C 10 bingo 090037 BB2EBD5ED48EE4B6 0 &quot;ball cower&quot; 090037 E150C7AB0BD87F00 0 J.P. Staticoy 090037 F386248EFE7B1D30 60 video,windows,mpeg,avi,quicktime 090037 5601E390E6D8ECAE 110 science museums 090037 EFB9E1F3C56A2E45 40 Cypress Bath Robes 090037 ED3387AD491107A9 0 Bartholomew Roberts 090037 62FEDCDC59C62662 0 korn addidas video videopics 090037 230B9E3ACD6588E6 0 Enter search query here. 090037 274EA149A81BB6B6 0 gifts for a golf fanatic 090037 F0F13D05734611BE 10 nyctinasty 090037 158C4B19C22E442B 10 +klinika 090037 A4ED32DCF420D9A8 0 elliptical & exercise & (study | comparison) 090037 39FE2F68D24F9037 0 dolphin photographs 090037 5C5BBD4BB2CEC87C 0 WHAT ARE THE WORDS TO GOOD FING WEN 090037 383F6A2FCD3FC025 0 greece art architecture 090037 4504D65E78E16DB3 0 nude celebreties 090037 A3BDF5662D31A9A1 0 PICO MACOM 090037 34A85FF35B505CF4 10 arrowsmith 090037 A2345DBA2067C914 10 Wrestlemania 2000 090037 E0879EB63392B1FA 0 WHat is the average life span of a human male? 090037 B5A2CCD65390DEA6 0 jc Whitney automotive 090037 99E59ED9100455A5 0 CANADA POST 090037 EE6268E5D773E6DE 0 CONSTITUTION 090037 940B48B101C7816B 20 hollywood celebrity residences 090037 FBEA3318F3DB8626 30 mbf 090037 105F1A2F7FCBF4AD 0 www.stmarymonastery.org/locatenvhtml 090037 8577FAAC4DFAE5C4 0 skirts 090037 629CD8D1EACCF218 0 Zelda how to get out of dodongo's cavern walkthrough 090037 590F4121ABC62C02 0 charles bosman sip of jeripego 090037 005E6360DE0BFB3D 0 All-American Sports Complex 090037 14D05008E05F104C 0 +franchise +restaurant +mcdonald's +capital +food 090037 F54BC573FF3C94B1 0 How do I find a poster? 090037 AE51E562208251D7 0 lyrics 090037 AC53E149F086BFCA 10 online rebates 090037 7C4895D89880FABD 0 gameboy color 090037 80E00221481F876F 0 SiteScope test 090037 9D9541F7837C5B13 0 deion sanders pictures 090037 FEF3B8BE27B32551 0 What does a koa tree look like? 090037 12905D8A160C8511 0 free e-mail 090037 D7C2C95E2100520D 20 pocketed mattress 090037 949341D683EF1A31 0 william bingham 090037 FD647F92E62C1999 0 Where can I find christmas classroom activities? 090037 43626A796894E0EE 90 cordwood 090037 E964D6A0DE533C46 0 corsets 090037 7DA9705171772D20 0 story + &quot;adult&quot; 090037 18E21BC7D100C75A 0 WHERE CAN I BUY A CHRISTMAS GIFT? 090037 4E47D669B072E110 0 [email_address] 090037 DA905D9DEC6800E2 0 CERT CIAC 090037 9F1AAFDF51958B76 30 SYMPTOMS OF MENOPAUSE 090037 10B371E09C0D35F0 0 españa 090037 4EDFE92AD6265CB4 100 massage 090037 5C70C43145B48D00 0 cellular phone 090037 6219027312AC2AE4 0 www.peeps.com 090037 21596CBA358337B2 80 disk operating system ms-dos 090037 12F3AC457EEA3D13 20 Ross Simmons 090037 F54BC573FF3C94B1 0 where can I find free games to download? 090037 754D23E3457AA30A 0 OFFROAD ATV 090037 01CC8FF9FD247836 0 Albanians, Macedonia 090037 9B2ACAA34F720788 0 &quot;young and the restless&quot; 090037 3088709788B12211 0 WWW.BMECU.ORG 090037 9E945FF875D60D96 0 stock market 090037 F54BC573FF3C94B1 0 all my children 090037 7BD52A47D59F3A8A 0 beavis and butthead wavs 090037 CA52ECD1524E737D 0 where can I find pages of nokia 090037 36F6E517810791CA 0 carol vorderman 090037 5E0C0F1E310C9240 0 potato soup recipes 090037 3666F406AC5DB584 0 historia del municipio de galeana en nuevo leon 090037 889BA47A5F4D6295 0 blue+book+bible 090037 91255EEF958730CE 0 possession is nine points of the law. 090037 61156CCDD77056A2 40 admin staff 090037 E81F3A8BC7B4D827 20 sailor+moon+nude 090037 80E00221481F876F 0 school 090037 F7B6638388EECFC9 10 free movies 090037 BF93868DEE790FFE 0 jamaica 090037 9CFBCA5CA918F815 10 McDonalds nutrition labes how nutritious the food is 090037 60C766139BBB9F86 0 where can i find embrodiery floss? 090037 4AD556DDCA079EB8 0 v 090037 9401A76DCCC15FF0 0 &quot;West Coast British&quot; +Rover 090037 FEF3B8BE27B32551 0 &quot;Where can I find pictures of Plastic Little?&quot; 090037 F05EA1123DD9A4DD 0 christmas songs 090037 FC00B873A0FD7CC5 0 milwaukee public library 090037 9D9541F7837C5B13 0 when was caesar born 090037 A1FE214ACBEC2D0C 0 &quot;zip code&quot; 090037 35B83536D6087739 0 hotdog carts 090037 F54BC573FF3C94B1 0 Leather key case 090037 37F9181F34759407 0 papiamento 090037 9AF56EA5CF84C306 90 horoscope 090037 514D8E8D1EDDCFA4 0 olympics 090037 554A8E3D803D4206 60 underneath 090037 9523F8882058998A 0 Dynatech 090037 CE11325DF6F7EF15 0 NFL 090037 C7893B5DAFC243E5 0 Ralph Lauren 090037 E5B196087B34F7A3 0 CHING 090038 95502391A3AA5D80 30 motorcycles 090038 65E62497C6F60716 0 fleetwood building block 090038 94ED8FB1905674D4 0 free nude pictures 090038 F54BC573FF3C94B1 0 who won the scopes trial 090038 3A2495EFB74F11DB 0 price waterhouse 090038 5B4E216DDFE062D8 10 Advances in Technology 090038 15AE8E6F90914FD2 0 bombay company 090038 78867524E53569AD 10 lyric search 090038 4605D6EB72FE3C11 0 &quot;BLAQUE&quot; 090038 97C1F70A5CF669C2 10 eletronics manufacturers suppliers 090038 E43DD6D82BFBD0B8 0 can You find the population of canada? 090038 BFD6526A5DA74676 10 porn 090038 80E00221481F876F 0 Pharmaceutical suppliers? 090038 18E21BC7D100C75A 0 Where can I find Asian XXX? 090038 F54BC573FF3C94B1 0 ncaa basketball polls 090038 292FF8ABE560CB61 0 newspapers 090038 B9BBCA5F01FBC052 370 muffy 090038 F07832CEDA8CA802 70 gout 090038 91255EEF958730CE 0 where can i get pictures of the rock from the wwf 090038 13EC4A85B298C559 0 links web 100 090038 F54BC573FF3C94B1 0 I am not sure what is the feeling with love? 090038 5BAE255480EC6569 0 Matt Damon and Ben Afflack 090038 C8267097505624FB 0 where can i find maps of gremany] 090038 CA52ECD1524E737D 0 Poop 090038 6D6221C9923D3D0E 0 puerto vallarta 090038 20FDC01F3CFF9B9E 20 amputee women 090038 DCFB38779D450561 50 partition magic serial number 090038 ABCD798116C85A24 0 einbaum 090038 90CC9EAF7E3EA7A3 50 redneck rampage cuss pack download 090038 162BF1167BF1AB8F 20 cross training shoes 090038 A9D200507BDEEED4 0 Bed & Breakfast UK 090038 D7515D49C25AEF66 0 white pages Hopewell, Va 090038 C8267097505624FB 0 reverse email search 090038 EC8992C4C7D4C067 0 tikes 090038 5290AF3692835BFD 0 ford engine recall 090038 1677D4C2AAE49285 0 crocket resort 090038 C4A6345C77DC97A4 0 Brunswick Bowling 090038 DD7E3B386701F491 30 protein expression pProEX 090038 2856BB471CF7E171 0 combo AND boat/pwc AND trailer 090038 200B8619B61472C7 20 talking clock software 090038 FE974DB2B743630F 0 Bass Brothers Fort Worth metroplex 090038 0C46406CEB75B16E 0 Island Duchess 090038 1EFA171DE74B1BF0 0 keith watts 090038 F54BC573FF3C94B1 0 digital camras 090038 E1FA3E641955BCDE 10 crate & barrell 090038 0272726847F58CFE 0 roofs house roof 090038 E43DD6D82BFBD0B8 0 how do I find out who belongs to a phone number? 090038 D9261D132C2A25FB 0 business shareware 090038 35FDCD84B03C82D1 0 nässlorna+blomma 090038 60B1E0B0F8F5289B 0 mail 090038 D2588D18868DCE60 0 harvard general hospital 090038 255DC03F259995EE 110 japanese preteen school girls 090038 400759108118569E 0 mgi picture software 090038 07B4940450470233 0 wcw 090038 6456584F5B316AAE 120 tiger electronics 090038 3BD556156A42EF7C 0 &quot;puerto rico&quot; 090038 B8BE65F36F5B0FC4 0 nudo 090038 DC38ABCBB52AFEF5 0 green bay gazzette 090038 7CA1487449619434 0 relatos de amor filial 090038 469105D1067B6E24 0 SUBLIME NUDES 090038 4F7762B41E9DF37B 10 &quot;shopping cart script&quot; 090038 9E945FF875D60D96 0 GREEN DAY 090038 70771A0A7143F9A3 0 Archive 090038 3C0384D86CD9A22E 0 Colonial New York nys 090038 F54BC573FF3C94B1 0 What is a multiárea 090038 54D5C4B884C232BA 0 naval doctrine publication 090038 F39AB84814D863D8 0 flowers.com 090038 3DF4E9B0AFF6B808 0 Christmas Traditions 090038 D8B39C6813A5B732 130 go carts 090038 CEF6A01CECEBAAED 10 New York State 090038 E4AD262B9121CA06 0 sexy 090038 B51E3758091FF471 80 HUNTING THE SANCTUARY 090038 F7E6B97DE1F69EBF 10 search recruiters casey mark associates 090038 43B0C06F6257A1BA 0 lesson plans 090038 1EDBC76CD39F353F 350 pinewood derby designs 090038 E683ABAAFF2B3FB7 20 +paul+revere jewel 090038 9F57839B10170414 0 Where can I find information on the US supreme court judges? 090038 245832BC3425778D 0 Software Forum 090038 77796C50C38B7754 30 Birthday party in Long Island 090038 3DC6D7915492DF12 40 computer 090038 AE51E562208251D7 0 electron 090038 714DBCB0C80FF699 0 lastminute and cottages and UK 090038 FAA504E9C64315F0 0 + computer notebook 090038 5C5BBD4BB2CEC87C 0 melody beattie 090039 6052086231BBBB90 10 alabama football 090039 B639E92CBBF0381E 0 ccd AND resolution and optic and vcr and colour and pixel and sony 090039 82F30745D2BF5C2C 0 golf 090039 9252B344DB1B7047 0 www.Odcgov/cia/Publication/Fatbook/index 090039 E6D3DCFA0A39BA3E 0 www.ufallfahrzeug.de 090039 AFCD0FD4A65D34C3 0 montogery ward 090039 F54BC573FF3C94B1 0 What is the website for the newspaper in Scranton, PA? 090039 3DF4E9B0AFF6B808 0 Universities in Wellington 090039 009CD46686D4A1FD 0 pantyfreek 090039 F26E5D31D804ED98 0 carman ministries 090039 D87CE5C149126B4B 0 Where can I find information about the television show Felicity? 090039 91255EEF958730CE 0 how can i convert liquid audio files to wav files? 090039 03D685B3F001A858 0 &quot;CD duplication software&quot; 090039 F54BC573FF3C94B1 0 What does the FDA say is its mandate? 090039 DAF1CD4E35BB6D98 0 ebony ayres+homepage 090039 BDF5F884F42DB05F 0 hacking VP Instant Message readers messanger 090039 1246BA311FD5AF65 0 thiazolidisediones 090039 9F0FF534C84D7873 0 chicago newspapers 090039 E00F9C2509761702 0 yahoo 090039 70B9F6D94EB08EB7 0 New Century Professions, Inc. 090039 5AE9D0A42F05FDE1 0 What presidents were born in texas 090039 6D3922A8D9E39775 0 dell 090039 FEF3B8BE27B32551 0 What can I find out about heavy menstral flow? 090039 801D47EB3B429B87 0 vadodara 090039 D7886E2745E6A42E 20 zak twins 090039 81C8AC0A3DA66AC4 0 what is ASC II format ? wordperfect 090039 BAB38DB2E0F20DBE 0 shemya --More-- 090039 82F30745D2BF5C2C golf 090039 9252B344DB1B7047 www.Odcgov/cia/Publication/Fatbook/index 090039 E6D3DCFA0A39BA3E www.ufallfahrzeug.de 090039 AFCD0FD4A65D34C3 montogery ward 090039 F54BC573FF3C94B1 What is the website for the newspaper in Scranton, PA? 090039 3DF4E9B0AFF6B808 Universities in Wellington 090039 009CD46686D4A1FD pantyfreek 090039 F26E5D31D804ED98 carman ministries 090039 D87CE5C149126B4B Where can I find information about the television show Felicity? 090039 91255EEF958730CE how can i convert liquid audio files to wav files? 090039 03D685B3F001A858 &quot;CD duplication software&quot; 090039 F54BC573FF3C94B1 What does the FDA say is its mandate? 090039 DAF1CD4E35BB6D98 ebony ayres+homepage 090039 BDF5F884F42DB05F hacking VP Instant Message readers messanger 090039 1246BA311FD5AF65 thiazolidisediones 090039 9F0FF534C84D7873 chicago newspapers 090039 E00F9C2509761702 yahoo 090039 70B9F6D94EB08EB7 New Century Professions, Inc. 090039 5AE9D0A42F05FDE1 What presidents were born in texas
  • 46. Question Types <ul><li>Wh-Questions: 342,075 </li></ul><ul><li>Yes/No-Questions: 23,882 </li></ul>
  • 47. Question Types How is the U.N. funded? Was Don lying about the shooting? Who invened the rangefinder? what exactly is a hydraulic system? When was Fisher Price started? Where can i find exams on the internet about database management systems course with solutions? Where can I find yesturday's weather? What is the probability of extraterrestrial life existing in the Universe? What vacation rentals are available in the pacific Northwest? Who was president Grant's private secretary? Where can I get a recipe for sloppy joes? What are the times of sunrises and sunsets in various cities in the US? pros and cons of professional managers in government? Where can I find information about FLSA? where can I find details of property in Scotland? Who sings the song&quot;Ice, Ice Baby&quot;? what percentage of schools have internet access? When were gingerbread houses invented? how do I find out the purchase price of a house that sold in California? Are cmputer chips made from sand? what is lupus? where can i find how to write a resume? WHERE CAN I FIND ON-LINE GREETING CARDS?
  • 48. Wh-Question Types
  • 49. Natural search <ul><li>Google gets about 200 million searches per day </li></ul><ul><li>That would mean about 30 million natural language searches </li></ul>
  • 50. Trying to guess What country is Berlin in? Q: Ties between Iran and Germany strained after a Berlin court verdict ruled on April 10 that Iran's top leaders were behind the assassination of four exiled Kurdish opposition leaders in a Berlin restaurant. A:
  • 51. Trying to guess What country is Berlin in? Q: Ties between Iran and Germany strained after a Berlin court verdict ruled on April 10 that Iran's top leaders were behind the assassination of four exiled Kurdish opposition leaders in a Berlin restaurant. A:
  • 52. Another guessing example Port Arthur Massacre. What was the killer's nationality? Q: A nation asks why, the portrait of a lone gunman, Martin Bryant, should have no reason to be a killer . The man arrested after the Port Arthur massacre should be a wealthy man, only four years ago he inherited more than 500,000 Australian dollars 375,000 US dollars from one of the heirs of George Adams's great Tattersalls fortune. A:
  • 53. Trying to google What is the state bird of Alaska? Q: Google search: &quot;The state bird of Alaska is the * &quot; A:
  • 54. Trying to google What is the state bird of Alaska? Q: Google results: The state bird of Alaska is the ptarmigan . The state bird of Alaska is the willow ptarmigan . A:
  • 55. Trying to google What is the state bird of Alaska? Q: Google results: The state bird of Alaska is the ptarmigan . The state bird of Alaska is the willow ptarmigan . The state bird of Alaska is the mosquito . The state bird of Alaska is the Mosquito . A:
  • 56. Trying GOFAI Where did Olof Palme die? Q: Stockholm. A:
  • 57. GOFAI, internally Where did Olof Palme die? Q: ……… . given clause #13: (wt=2) 69 [hyper,47,16] vehicle($c3). given clause #14: (wt=2) 71 [hyper,50,16] vehicle($c4). given clause #15: (wt=2) 74 [hyper,52,18] building($c5). given clause #16: (wt=3) 51 [] have(vincent,$c4). given clause #17: (wt=2) 76 [hyper,55,18] building($c6). given clause #18: (wt=2) 78 [hyper,61,16] vehicle($c6). given clause #19: (wt=2) 80 [hyper,65,10] organism($c1). given clause #20: (wt=2) 84 [hyper,67,10] organism($c2). given clause #21: (wt=3) 53 [] die(palme,$c5). given clause #22: (wt=2) 86 [hyper,69,15] instrument($c3). given clause #23: (wt=2) 88 [hyper,71,15] instrument($c4). given clause #24: (wt=2) 90 [hyper,74,7] artifact($c5). given clause #25: (wt=2) 94 [hyper,76,7] artifact($c6). given clause #26: (wt=3) 56 [] $c7=$c6. given clause #27: (wt=2) 96 [hyper,78,15] stockholm($c6). -----> EMPTY CLAUSE at 0.01 sec ----> 113 [hyper,96,24,76] $F. A:
  • 58. GOFAI, externally Where did Olof Palme die? Q: 1. You are looking for location. 2. I know that Stockholm is a city. 3. Every city is a location. 4. If x is shot to death then x died. 5. I found the following evidence in document APW20000227.0124: &quot;In 1986, Swedish Prime Minister Olof Palme was shot to death in central Stockholm. “ 6. Hence Stockholm is the answer. A:
  • 59. Predicate Argument Structure When was NATO established? Q: A: NATO launched its first attack against Yugoslavia on March 24 . WordNet: launch=establish
  • 60. Logical operators Where did Ricky Williams, American football player, grow up? Q: A: Texas running back Ricky Williams is from California. Those who follow the Longhorns don't like Williams any less because he didn't grow up in Texas .
  • 61. Word Sense Disambiguation Bing Crosby. What was his profession? Q: A: Crosby sang her first song, Starlight. She scribbled notes for the song on the back of a menu at a New York jazz bar in 1931. . WordNet: bar hyponym of profession
  • 62. Proper Name Reference Where is the Taj Mahal? Q: A: The Taj Mahal is a mausoleum located in Agra, India , that was built under Mughal Emperor Shah Jahan in memory of his favorite wife, Mumtaz Mahal.
  • 63. Proper Name Reference Where is the Taj Mahal? Q: A: The Taj Mahal is a mausoleum located in Agra, India , that was built under Mughal Emperor Shah Jahan in memory of his favorite wife, Mumtaz Mahal. A: The Taj Mahal first opened its door to a receptive but cautious public back in 1964. Being the first restaurant of its kind in Stevenage , our main problems were initially to get people to try foods that they had never tried before.
  • 64. Granularity Where did Franz Kafka die? In his bed Q: A:
  • 65. Granularity Where did Franz Kafka die? In his bed In a sanatorium Q: A: A:
  • 66. Granularity Where did Franz Kafka die? In his bed In a sanatorium In Kierling Q: A: A: A:
  • 67. Granularity Where did Franz Kafka die? In his bed In a sanatorium In Kierling Near Vienna Q: A: A: A: A:
  • 68. Granularity Where did Franz Kafka die? In his bed In a sanatorium In Kierling Near Vienna In Austria Q: A: A: A: A: A:
  • 69. Knowledge acquisition <ul><li>What kind of knowledge do we need for QA applications (apart from WordNet) </li></ul><ul><li>Can we produce this on the fly? </li></ul><ul><li>Let’s have a look at some examples </li></ul><ul><ul><li>Linguistic patterns </li></ul></ul><ul><ul><li>Paraphrase collections </li></ul></ul>
  • 70. Failing instances <ul><li>WordNet has no instances of airlines. </li></ul>TREC 20.2 (Concorde) What airlines have Concorde in their fleets?
  • 71. Find instances with patterns <ul><li>Search for linguistic patterns in corpora (Hearst 1992, Aguado de Cea et al. 2008) </li></ul><ul><li>Example pattern: “X such as Y and” </li></ul><ul><li>Text: … said that airlines such as Continental and United now fly… </li></ul>TREC 20.2 (Concorde) What airlines have Concorde in their fleets?
  • 72. Find instances with patterns <ul><li>Search for linguistic patterns in corpora (Hearst 1992, Aguado de Cea et al. 2008) </li></ul><ul><li>Example pattern: “ X such as Y and ” </li></ul><ul><li>Text: … said that airlines such as Continental and United now fly… </li></ul>TREC 20.2 (Concorde) What airlines have Concorde in their fleets?
  • 73. Pattern result <ul><li>Knowledge (Acquaint corpus): Air Asia, Air Canada, Air France, Air Mandalay, Air Zimbabwe, Alaska, Aloha, American Airlines, Angel Airlines, Ansett, Asiana, Bangkok Airways, Belgian Carrier Sabena, British Airways, Canadian, Cathay Pacific, China Eastern Airlines, China Xinhua Airlines, Continental, Garuda, Japan Airlines, Korean Air, Lai, Lao Aviation, Lufthansa, Malaysia Airlines, Maylasian Airlines, Midway, Northwest, Orient Thai Airlines, Qantas, Seage Air, Shanghai Airlines, Singapore Airlines, Skymark Airlines Co., South Africa, Swiss Air, US Airways, United, Virgin, Yangon Airways </li></ul>TREC 20.2 (Concorde) What airlines have Concorde in their fleets?
  • 74. Paraphrases <ul><li>DIRT database (Lin & Pantel): X was killed in Y == X died in Y </li></ul>TREC 4.2 (James Dean) When did James Dean die? ---- APW19990929.0165: In 1955 , actor James Dean was killed in a two-car collision near Cholame, Calif.
  • 75. Paraphrases <ul><li>In first-order logic:  x  t(  e(kill(e)&theme(e,x)&in(e,t))   e‘ ( die(e')&agent(e',x)&in(e',t))) </li></ul>TREC 4.2 (James Dean) When did James Dean die? ---- APW19990929.0165: In 1955 , actor James Dean was killed in a two-car collision near Cholame, Calif.
  • 76. General Knowledge <ul><li>Knowledge (manually coded?):  x(husband(x)  male(x))  x  y((son(x,y)&male(y))  father(y,x)) </li></ul>TREC 14.4 (Horus) Who was his father? ---- XIE19990713.004: It also hosted statues of Amon’s wife, Mut, the goddess Isis, her husband, Osiris , and their son Horus.
  • 77. Knowledge in QA <ul><li>We need knowledge for inference-based QA system </li></ul><ul><li>This is knowledge not explicitly expressed in the text or question </li></ul><ul><li>Current background knowledge resources are not sufficient </li></ul><ul><li>Automatically extracted knowledge </li></ul><ul><ul><li>Ideally word sense disambiguated </li></ul></ul><ul><ul><li>Deal with more complex relations </li></ul></ul>
  • 78. Automatically harvested knowledge <ul><li>Good at simple relations </li></ul><ul><ul><li>hyponyms, instances, synonyms </li></ul></ul><ul><ul><li>typically high precision and low recall </li></ul></ul><ul><li>Not so good at paraphrases </li></ul><ul><ul><li>DIRT paraphrases [Lin and Pantel] </li></ul></ul><ul><ul><li>no improvement on QA and RTE </li></ul></ul><ul><ul><li>typically low precision, high recall </li></ul></ul><ul><li>General knowledge is hard to get, but of course we can use Boxer itself! </li></ul>
  • 79. There is only one catch…
  • 80. A small case study <ul><li>Let’s try to derive knowledge from texts </li></ul><ul><li>Let’s use Boxer to analyse definitions </li></ul><ul><li>Pilot study: Find axioms for wife and husband </li></ul>
  • 81. Case study: wife and husband <ul><li>WordNet 3.0: wife (a married woman; a man’s partner in marriage) husband (a married man; a woman’s partner in marriage) </li></ul>
  • 82. Boxing WordNet glosses <ul><li>Original: wife (a married woman; a man’s partner in marriage) </li></ul><ul><li>Rephrased (1): Every wife is a married woman. </li></ul> wife(x) x married(y) woman(y) x=y y
  • 83. Boxing WordNet glosses <ul><li>Original: wife (a married woman; a man’s partner in marriage) </li></ul><ul><li>Rephrased (2) Every wife is a man’s partner in marriage. </li></ul> wife(x) x partner(v) of(v,u) man(u) in(v,y) marriage(y) x=v u v y
  • 84. Case study: wife and husband <ul><li>Wikipedia (Oct 1, 2008): A wife is a female spouse, or participant in a marriage, or civil union or civil partnership. A husband is a male spouse (participant) in a marriage, civil union or civil partnership. </li></ul>
  • 85. Boxing Wikipedia definitions <ul><li>Every wife is a female spouse, or participant in a marriage, or civil union or civil partnership. </li></ul>
  • 86. Boxing glosses and definitions <ul><li>Not straightforward to get good results </li></ul><ul><li>(Manual?) reformulation required </li></ul><ul><li>Several issues </li></ul><ul><ul><li>implication in one or two directions? </li></ul></ul><ul><ul><li>Word sense disambiguation </li></ul></ul><ul><ul><li>Modifier attachments </li></ul></ul><ul><ul><li>Scope of disjunction </li></ul></ul><ul><ul><li>Interpretation of disjunction </li></ul></ul><ul><ul><li>Strict or default rules? </li></ul></ul><ul><li>But: inference seems to play no major role </li></ul>
  • 87. Ideas for definition analysis <ul><li>Develop a controlled natural language for definition templates in sources like Wikipedia </li></ul><ul><li>Train new parsing models on annotated definitions </li></ul>
  • 88. Conclusions <ul><li>We can build semantic representations with high coverage and reasonable accuracy for open-domain natural texts </li></ul><ul><li>We also have reasonable inference engines at our disposals that work well enough for small texts </li></ul><ul><li>To use these in real-world applications inference tasks we need additional background knowledge </li></ul>
  • 89. More Conclusions <ul><li>Some of the background knowledge can be derived from existing resources and ontologies </li></ul><ul><li>The word sense problem makes symbol grounding in ontologies hard </li></ul><ul><li>Deriving general knowledge from natural language definitions sounds like a nice idea but is not without stumbles </li></ul>
  • 90. And finally… <ul><li>The C&C tools, Boxer and Nutcracker are freely available for research </li></ul><ul><ul><li>http://svn.ask.it.usyd.edu.au/trac/candc/wiki </li></ul></ul><ul><ul><li>There is also on online demo </li></ul></ul>

×