Cooking an ontology-based spoken dialogue system

1,342 views
1,246 views

Published on

The main goal of this thesis is to study how different levels of knowledge stored in ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning. We take McGuinness' ontologies spectrum - and related work - to support the division of a cooking ontology into increasingly complex knowledge levels.The hypothesis being tested is whether ontologies can be used to enrich a coaching spoken dialogue system and be used in it in such way that the system can abstract the source of domain-specific knowledge - related to the tasks being coached - focusing only on the dialogue phenomena. The integration of ontological knowledge should be done with few architecture adaptions to the dialogue system so that when adding a new domain - a new class of tasks - minor changes in special modules are sufficient. Software engineering techniques for systems integration, like extension-based modularization and design patterns, are to be considered and extensively used to minimize the number of architecture adaptions.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,342
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
48
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Cooking an ontology-based spoken dialogue system

  1. 1. Cooking an ontology-based coaching dialogue system — based on my PhD proposal — Joana Paulo Pardal Quality and Usability Colloquium December 7th, 2009 Berlin, GermanyMonday, December 7, 2009
  2. 2. Who am I?Monday, December 7, 2009
  3. 3. Researcher, PhD Student and Lecturer RESEARCHER: - Spoken Language Systems Laboratory (L2F) @ INESC-ID, Lisbon PHD STUDENT: - IST, Technical University of Lisbon - Ontology-based (spoken) Dialogue Systems LECTURER: - IST, Technical University of LisbonMonday, December 7, 2009
  4. 4. Monday, December 7, 2009
  5. 5. cooking a coaching ontology-based (spoken) dialogue systemsMonday, December 7, 2009
  6. 6. CookingMonday, December 7, 2009
  7. 7. House of the futureMonday, December 7, 2009
  8. 8. Monday, December 7, 2009
  9. 9. Part of our daily lifeMonday, December 7, 2009
  10. 10. With a lot of inspirationMonday, December 7, 2009
  11. 11. And interested peopleMonday, December 7, 2009
  12. 12. Eyes and hands busyMonday, December 7, 2009
  13. 13. Dialogue SystemsMonday, December 7, 2009
  14. 14. Typical Architecture User Speech Speech Synthesis Recognition Dialogue Management Natural Natural Language Language Generation Understanding back-end systemMonday, December 7, 2009
  15. 15. Typical Architecture User prosody acoustic models Speech Speech Synthesis Recognition coded speech grammar Dialogue Management lexicon lexicon Natural Natural Language Language grammar Generation Understanding grammar back-end systemMonday, December 7, 2009
  16. 16. DigA L2F INESC-ID Dialogue Manager Device Core IO Manager Core ASR Business Parser Service Manager JVoiceXML & JSpeechAPI 2.0 interface (Audimus) Interpretation Dialogue Manager Interface Microphone Manager Array Task Manager Device Core Behavioral Agent Data GUI base Generation Manager User TTS Surface Access (DIXI) Generation Manager Device Core Discourse Context Device QA@L2F Face Event Manager ManagerMonday, December 7, 2009
  17. 17. TRIPS Univ RochesterMonday, December 7, 2009
  18. 18. Olympus Carnegie Mellon #$%&()*+,-&./0$12%02,/% <%0=;9&:6;16% !"$%()& 0$1),"(#()" !"#$%&%$##()" !"#$%&(& )01&#.$(*$( !"#$%& )#*+,& -.#"&/0 !"#$%1#()"-.,% *(+),-.",$% 3405%67 1&234#5 )!#,,# ()*$%+,)- 23"#4$5(5 /$"$%#()" 2),,&#!$ (#.$//) 8)62$9&:6;16% !"#$%& !"Monday, December 7, 2009
  19. 19. Practical dialogueMonday, December 7, 2009
  20. 20. http://flickr.com/photos/carpathian/1656738696/in/photostream/ Practical Dialogue Hypothesis The conversational competence required for practical dialogues, while still complex, is significantly simpler to achieve than general human conversational competence.Monday, December 7, 2009
  21. 21. Domain-Independence Hypothesis Within the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management is independent of the task being performed.Monday, December 7, 2009
  22. 22. Types of dialogue systems: - Information providing - Appliances control - Planning & scheduling - Tutoring ...Monday, December 7, 2009
  23. 23. Tehran Times Sports Desk CoachingMonday, December 7, 2009
  24. 24. Simple but funcional dialogue systemsMonday, December 7, 2009
  25. 25. How different levels of knowledge stored in domain-specific ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning? http://www.flickr.com/photos/jhhwild/1781000505/Monday, December 7, 2009
  26. 26. short introduction to ONTOLOGIESMonday, December 7, 2009
  27. 27. Definition adapted from http://en.wiktionary.org/wiki/ontology on·tol·o·gy ORIGIN early 18th cent.: from modern Latin ontologia, from Greek ONT- ‘being’ -LOGY. (philosophy) the branch of metaphysics that addresses the nature or essential characteristics of being and of things that exist; the study of being in the capacity of being. (computer science) a structure of concepts or entities within a domain, organized by relationships; a system model.Monday, December 7, 2009
  28. 28. What is an Ontology? is a description of the existing concepts and relationships in a given domain “An ontology is a specification of a conceptualization.” [Gruber, 1993]Monday, December 7, 2009
  29. 29. Monday, December 7, 2009
  30. 30. http://www.w3.org/2000/Talks/1206-xml2k-tbl/slide10-0.html Tim Berners-Lee’s Semantic Web Machine processable (not natural language), for data, for the future.Monday, December 7, 2009
  31. 31. machine processable and interoperable systems http://flickr.com/photos/ul_marga/755378645/Monday, December 7, 2009
  32. 32. sharing knowledge http://flickr.com/photos/kgoldenchild/2223459035/Monday, December 7, 2009
  33. 33. easier communication http://flickr.com/photos/hchalkley/92120879/Monday, December 7, 2009
  34. 34. easier integration http://flickr.com/photos/stevehopson/1794895311Monday, December 7, 2009
  35. 35. Data organization when a large amount is available http://flickr.com/photos/shetha/1561376494/Monday, December 7, 2009
  36. 36. Ontologies in dialogue systemsMonday, December 7, 2009
  37. 37. Ontology in a dialog system User prosody acoustic models Speech Speech Synthesis Recognition coded speech grammar Dialogue Management lexicon lexicon Natural Natural Language Language grammar Generation Understanding grammar back-end system Domain OntologyMonday, December 7, 2009
  38. 38. TRIPS Univ Rochester !"#"$%& 01*$)22-."/3 @5""&D @5""&D ()#*+)*" 4)3".-# >"&-*#%E"$ @=#6D"3%3 ,-."/ &-$5+3.)6) C-$.3 C-$.3 !$)22)$ 8$-5"$ ("7%&-# 0-+#39 @+$F)&" :6&; !"#"$)G-# 8)$3"$ <#6-/-*= (B3 <#6-/-*= ,)55%#* (B3 >+/"3 J#6"$5$"6)G-# ,)#)*"$! H%3&-+$3" H-2)%#1@5"&%I& @6)6" ?-25-#"#63 ?8@A&63Monday, December 7, 2009
  39. 39. McGuinness’ spectrumMonday, December 7, 2009
  40. 40. My work so far...Monday, December 7, 2009
  41. 41. OntoChef a cooking ontology joint work with R. Ribeiro, F. Batista, N. Mamede and P. VazMonday, December 7, 2009
  42. 42. Main goal Recipe: Chocolate Cake Ingredients 4 oz of sweet dark chocolate 5 eggs ... Procedure separate egg whites and egg yolks with an electric mixer whisk the egg whites until forming stiff peaks put egg whites in a container ... video file ...Monday, December 7, 2009
  43. 43. Competency Questions Recipes: - How do I make recipe R? - What are the quantities to make recipe R for 4 persons? Actions: - How do I do A1? Times: - Which recipes that take less than 10 minutes to do? Food: - Which recipes have food item F1, but not F2? - Which recipes have as main ingredient food item F1? Utensils: - Which utensils are used in recipe R1? - Which recipes can be made using the microwave? Equivalencies: - How many liters is a cup?Monday, December 7, 2009
  44. 44. OntoChef cooking domain modelMonday, December 7, 2009
  45. 45. several modules Equivalences Actions Food Units & Recipes Utensils MeasuresMonday, December 7, 2009
  46. 46. Formalization overviewMonday, December 7, 2009
  47. 47. Resulting Ontology 4 main (reusable) modules: - Food - Utensils - Actions - Recipes 3 auxiliary (reusable) modules: - Measures - Equivalences - Types of dishes Metrics: - 1151 classes - 92 slots - 52 relations between classes - 311 instancesMonday, December 7, 2009
  48. 48. OntoChef building methodologyMonday, December 7, 2009
  49. 49. Main activities: - Knowledge acquisition - Conceptualization - Formalization - EvaluationMonday, December 7, 2009
  50. 50. Equivalences Actions Food Units & Recipes Utensils Measures separate teams for bulk workMonday, December 7, 2009
  51. 51. weekly brainstorm sessionsMonday, December 7, 2009
  52. 52. hardest: reach agreementMonday, December 7, 2009
  53. 53. OntoChef knowledge sourcesMonday, December 7, 2009
  54. 54. Monday, December 7, 2009
  55. 55. professional websites Web Site #Recipes #Photos www.vaqueiro.pt 3136 2790 www.continente.pt 467 467 sabores.sapo.pt 2760 1038 comezainas.clix.pt 1646 0 www.entrepratos.com 189 51 Total: 8198 4346 Recipes collected from professional websites.Monday, December 7, 2009
  56. 56. personal blogs Web Site #Recipes paladares.do.sapo.pt 329 receitasdeculinaria.blogs.sapo.pt 10 culinaria.no.sapo.pt 236 Total: 575 Recipes collected from personal blogs.Monday, December 7, 2009
  57. 57. CMU’s Recipe Database <recipe version="1.1"> <line> <originaltext>1 (18 ounce) package small chocolate chip cookies</originaltext> <annotation>create_ing(ing0, "1 (18 ounce) package small chocolate chip cookies")</annotation> </line> <line> <originaltext>1 cup hot fudge topping</originaltext> <annotation>create_ing(ing2, "1 cup hot fudge topping")</ annotation> </line> ... <line> <originaltext>Spread 3/4 cup fudge topping over crust. </ originaltext> <annotation>separate(ing2, ing12, "3/4 cup fudge", ing13, "1/4 cup fudge", "")</annotation> </line> ...Monday, December 7, 2009
  58. 58. USENET Cookbook .RH MOD.RECIPES-SOURCE CHOC-CAKE-1 D "20 May 86" 1986 .RZ "CHOCOLATE FANTASY CAKE" "A no-bake chocolate cake" This is a very simple, quick, no-bake cake. .IH "Serves 6-8" .SH! ! CAKE .IG "2(12 cups" "cream" "600 ml" .IG "3 cups" "milk" "750 ml" .IG "3 pkgs" "instant chocolate pudding" .PH .SK 1 Arrange a layer of biscuits at the bottom of a large baking pan. ... .SH RATING .I Difficulty: easy. .I Time: 10 minutes preparation, several hours chilling. .I Precision: measure the ingredients. .WR Mike Trachtman, Weizmann Institute, Rehovot, Israel,Monday, December 7, 2009
  59. 59. OntoChef conceptualizationMonday, December 7, 2009
  60. 60. OntoChef database population joint work with T. Machado and N. MamedeMonday, December 7, 2009
  61. 61. Manual inputMonday, December 7, 2009
  62. 62. Ontology-based semi-automatic input through natural language processing OntoChef Process (cooking ontology) Internet Recipes e-Books Database Cookbooks (in text) (ontology intances)Monday, December 7, 2009
  63. 63. OntoChef evaluationMonday, December 7, 2009
  64. 64. OntoChef future workMonday, December 7, 2009
  65. 65. additional languagesMonday, December 7, 2009
  66. 66. Knowledge sources Book’s Cover Book’s Bibliographic Reference English: “Ingredients”, by Loukie Werle, Ullmann Publishing, 2009, 384 pp. Portuguese: “Ingredientes”, by Loukie Werle, Könemann, 2009, 384 pp. English: “The Cooks Bible”, by Le Cordon Bleu, Carroll & Brown Publishers, 2005, 352 pp. Portuguese: “O Livro das Técnicas Culinárias”, by E. Treuille & J. Wright, Livros e Livros, 2006, 352 p. “Dicionário Visual Português/Inglês”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Espanhol”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Françês”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Italiano”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Alemão”, Civilização Editora, 2006, 360 pgs.Monday, December 7, 2009
  67. 67. Collected information Languages # pairs PT-EN Portuguese — English Ingredients 2112 Portuguese — English Actions 1658 5927 Portuguese — English 2157 Portuguese — Spanish 2157 Portuguese — French 2157 Portuguese — Italian 2157 Portuguese — German 2157 Total: 14555Monday, December 7, 2009
  68. 68. wine informationMonday, December 7, 2009
  69. 69. Cooking Coach an ontology-basedcoaching dialogue systemsMonday, December 7, 2009
  70. 70. Recipe descriptionMonday, December 7, 2009
  71. 71. Monday, December 7, 2009
  72. 72. Available ‘tasks’ - next - previous - repeat - how? (down) - continue (up)Monday, December 7, 2009
  73. 73. Cooking CorpusMonday, December 7, 2009
  74. 74. Monday, December 7, 2009
  75. 75. Corpus annotation Equivalences Actions Food Units & Recipes Utensils MeasuresMonday, December 7, 2009
  76. 76. Cooking Coach next steps...Monday, December 7, 2009
  77. 77. Explore semantic spectrum without automated Reasoning with automated Reasoning Complexity Catalog Terms / Term Formal Formal Frames Value Description General Thesaurus / ID Glossary Hierarchy Taxonomy Instance (props) Restrictions Logics LogicMonday, December 7, 2009
  78. 78. Reasoning ‘Caracóis’ (snails): broiled animal, a cake with snail shapeMonday, December 7, 2009
  79. 79. Tackle different usersMonday, December 7, 2009
  80. 80. Different coaching stylesMonday, December 7, 2009
  81. 81. System’s evaluation a lot Prior experience with the Dialog System some none none some a lot Knowledge of the Domain (Cooking prior experience)Monday, December 7, 2009
  82. 82. MIT’s CounterMonday, December 7, 2009
  83. 83. DFKI’s Smart KitchenMonday, December 7, 2009
  84. 84. Coaching another tasksMonday, December 7, 2009
  85. 85. Questions http://flickr.com/photos/jazz_dalek/1845705965/Monday, December 7, 2009
  86. 86. Thank you!http://flickr.com/photos/42304632@N00/232004616/Monday, December 7, 2009
  87. 87. Joana Paulo Pardal joana.paulo.pardal@gmail.com ladydoctorbug.blogspot.com www.l2f.inesc-id.pt ⁄!joanaMonday, December 7, 2009
  88. 88. Cooking an ontology-based coaching dialogue system Joana Paulo Pardal Quality and Usability Colloquium December 7th, 2009 Berlin, GermanyMonday, December 7, 2009

×