Cooking an ontology-based spoken dialogue system
Upcoming SlideShare
Loading in...5
×
 

Cooking an ontology-based spoken dialogue system

on

  • 1,080 views

The main goal of this thesis is to study how different levels of knowledge stored in ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning. We ...

The main goal of this thesis is to study how different levels of knowledge stored in ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning. We take McGuinness' ontologies spectrum - and related work - to support the division of a cooking ontology into increasingly complex knowledge levels.The hypothesis being tested is whether ontologies can be used to enrich a coaching spoken dialogue system and be used in it in such way that the system can abstract the source of domain-specific knowledge - related to the tasks being coached - focusing only on the dialogue phenomena. The integration of ontological knowledge should be done with few architecture adaptions to the dialogue system so that when adding a new domain - a new class of tasks - minor changes in special modules are sufficient. Software engineering techniques for systems integration, like extension-based modularization and design patterns, are to be considered and extensively used to minimize the number of architecture adaptions.

Statistics

Views

Total Views
1,080
Views on SlideShare
1,077
Embed Views
3

Actions

Likes
1
Downloads
17
Comments
0

1 Embed 3

http://www.slashdocs.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cooking an ontology-based spoken dialogue system Cooking an ontology-based spoken dialogue system Presentation Transcript

  • Cooking an ontology-based coaching dialogue system — based on my PhD proposal — Joana Paulo Pardal Quality and Usability Colloquium December 7th, 2009 Berlin, GermanyMonday, December 7, 2009
  • Who am I?Monday, December 7, 2009
  • Researcher, PhD Student and Lecturer RESEARCHER: - Spoken Language Systems Laboratory (L2F) @ INESC-ID, Lisbon PHD STUDENT: - IST, Technical University of Lisbon - Ontology-based (spoken) Dialogue Systems LECTURER: - IST, Technical University of LisbonMonday, December 7, 2009
  • Monday, December 7, 2009
  • cooking a coaching ontology-based (spoken) dialogue systemsMonday, December 7, 2009
  • CookingMonday, December 7, 2009
  • House of the futureMonday, December 7, 2009
  • Monday, December 7, 2009
  • Part of our daily lifeMonday, December 7, 2009
  • With a lot of inspirationMonday, December 7, 2009
  • And interested peopleMonday, December 7, 2009
  • Eyes and hands busyMonday, December 7, 2009
  • Dialogue SystemsMonday, December 7, 2009
  • Typical Architecture User Speech Speech Synthesis Recognition Dialogue Management Natural Natural Language Language Generation Understanding back-end systemMonday, December 7, 2009
  • Typical Architecture User prosody acoustic models Speech Speech Synthesis Recognition coded speech grammar Dialogue Management lexicon lexicon Natural Natural Language Language grammar Generation Understanding grammar back-end systemMonday, December 7, 2009
  • DigA L2F INESC-ID Dialogue Manager Device Core IO Manager Core ASR Business Parser Service Manager JVoiceXML & JSpeechAPI 2.0 interface (Audimus) Interpretation Dialogue Manager Interface Microphone Manager Array Task Manager Device Core Behavioral Agent Data GUI base Generation Manager User TTS Surface Access (DIXI) Generation Manager Device Core Discourse Context Device QA@L2F Face Event Manager ManagerMonday, December 7, 2009
  • TRIPS Univ RochesterMonday, December 7, 2009
  • Olympus Carnegie Mellon #$%&()*+,-&./0$12%02,/% <%0=;9&:6;16% !"$%()& 0$1),"(#()" !"#$%&%$##()" !"#$%&(& )01&#.$(*$( !"#$%& )#*+,& -.#"&/0 !"#$%1#()"-.,% *(+),-.",$% 3405%67 1&234#5 )!#,,# ()*$%+,)- 23"#4$5(5 /$"$%#()" 2),,&#!$ (#.$//) 8)62$9&:6;16% !"#$%& !"Monday, December 7, 2009
  • Practical dialogueMonday, December 7, 2009
  • http://flickr.com/photos/carpathian/1656738696/in/photostream/ Practical Dialogue Hypothesis The conversational competence required for practical dialogues, while still complex, is significantly simpler to achieve than general human conversational competence.Monday, December 7, 2009
  • Domain-Independence Hypothesis Within the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management is independent of the task being performed.Monday, December 7, 2009
  • Types of dialogue systems: - Information providing - Appliances control - Planning & scheduling - Tutoring ...Monday, December 7, 2009
  • Tehran Times Sports Desk CoachingMonday, December 7, 2009
  • Simple but funcional dialogue systemsMonday, December 7, 2009
  • How different levels of knowledge stored in domain-specific ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning? http://www.flickr.com/photos/jhhwild/1781000505/Monday, December 7, 2009
  • short introduction to ONTOLOGIESMonday, December 7, 2009
  • Definition adapted from http://en.wiktionary.org/wiki/ontology on·tol·o·gy ORIGIN early 18th cent.: from modern Latin ontologia, from Greek ONT- ‘being’ -LOGY. (philosophy) the branch of metaphysics that addresses the nature or essential characteristics of being and of things that exist; the study of being in the capacity of being. (computer science) a structure of concepts or entities within a domain, organized by relationships; a system model.Monday, December 7, 2009
  • What is an Ontology? is a description of the existing concepts and relationships in a given domain “An ontology is a specification of a conceptualization.” [Gruber, 1993]Monday, December 7, 2009
  • Monday, December 7, 2009
  • http://www.w3.org/2000/Talks/1206-xml2k-tbl/slide10-0.html Tim Berners-Lee’s Semantic Web Machine processable (not natural language), for data, for the future.Monday, December 7, 2009
  • machine processable and interoperable systems http://flickr.com/photos/ul_marga/755378645/Monday, December 7, 2009
  • sharing knowledge http://flickr.com/photos/kgoldenchild/2223459035/Monday, December 7, 2009
  • easier communication http://flickr.com/photos/hchalkley/92120879/Monday, December 7, 2009
  • easier integration http://flickr.com/photos/stevehopson/1794895311Monday, December 7, 2009
  • Data organization when a large amount is available http://flickr.com/photos/shetha/1561376494/Monday, December 7, 2009
  • Ontologies in dialogue systemsMonday, December 7, 2009
  • Ontology in a dialog system User prosody acoustic models Speech Speech Synthesis Recognition coded speech grammar Dialogue Management lexicon lexicon Natural Natural Language Language grammar Generation Understanding grammar back-end system Domain OntologyMonday, December 7, 2009
  • TRIPS Univ Rochester !"#"$%& 01*$)22-."/3 @5""&D @5""&D ()#*+)*" 4)3".-# >"&-*#%E"$ @=#6D"3%3 ,-."/ &-$5+3.)6) C-$.3 C-$.3 !$)22)$ 8$-5"$ ("7%&-# 0-+#39 @+$F)&" :6&; !"#"$)G-# 8)$3"$ <#6-/-*= (B3 <#6-/-*= ,)55%#* (B3 >+/"3 J#6"$5$"6)G-# ,)#)*"$! H%3&-+$3" H-2)%#1@5"&%I& @6)6" ?-25-#"#63 ?8@A&63Monday, December 7, 2009
  • McGuinness’ spectrumMonday, December 7, 2009
  • My work so far...Monday, December 7, 2009
  • OntoChef a cooking ontology joint work with R. Ribeiro, F. Batista, N. Mamede and P. VazMonday, December 7, 2009
  • Main goal Recipe: Chocolate Cake Ingredients 4 oz of sweet dark chocolate 5 eggs ... Procedure separate egg whites and egg yolks with an electric mixer whisk the egg whites until forming stiff peaks put egg whites in a container ... video file ...Monday, December 7, 2009
  • Competency Questions Recipes: - How do I make recipe R? - What are the quantities to make recipe R for 4 persons? Actions: - How do I do A1? Times: - Which recipes that take less than 10 minutes to do? Food: - Which recipes have food item F1, but not F2? - Which recipes have as main ingredient food item F1? Utensils: - Which utensils are used in recipe R1? - Which recipes can be made using the microwave? Equivalencies: - How many liters is a cup?Monday, December 7, 2009
  • OntoChef cooking domain modelMonday, December 7, 2009
  • several modules Equivalences Actions Food Units & Recipes Utensils MeasuresMonday, December 7, 2009
  • Formalization overviewMonday, December 7, 2009
  • Resulting Ontology 4 main (reusable) modules: - Food - Utensils - Actions - Recipes 3 auxiliary (reusable) modules: - Measures - Equivalences - Types of dishes Metrics: - 1151 classes - 92 slots - 52 relations between classes - 311 instancesMonday, December 7, 2009
  • OntoChef building methodologyMonday, December 7, 2009
  • Main activities: - Knowledge acquisition - Conceptualization - Formalization - EvaluationMonday, December 7, 2009
  • Equivalences Actions Food Units & Recipes Utensils Measures separate teams for bulk workMonday, December 7, 2009
  • weekly brainstorm sessionsMonday, December 7, 2009
  • hardest: reach agreementMonday, December 7, 2009
  • OntoChef knowledge sourcesMonday, December 7, 2009
  • Monday, December 7, 2009
  • professional websites Web Site #Recipes #Photos www.vaqueiro.pt 3136 2790 www.continente.pt 467 467 sabores.sapo.pt 2760 1038 comezainas.clix.pt 1646 0 www.entrepratos.com 189 51 Total: 8198 4346 Recipes collected from professional websites.Monday, December 7, 2009
  • personal blogs Web Site #Recipes paladares.do.sapo.pt 329 receitasdeculinaria.blogs.sapo.pt 10 culinaria.no.sapo.pt 236 Total: 575 Recipes collected from personal blogs.Monday, December 7, 2009
  • CMU’s Recipe Database <recipe version="1.1"> <line> <originaltext>1 (18 ounce) package small chocolate chip cookies</originaltext> <annotation>create_ing(ing0, "1 (18 ounce) package small chocolate chip cookies")</annotation> </line> <line> <originaltext>1 cup hot fudge topping</originaltext> <annotation>create_ing(ing2, "1 cup hot fudge topping")</ annotation> </line> ... <line> <originaltext>Spread 3/4 cup fudge topping over crust. </ originaltext> <annotation>separate(ing2, ing12, "3/4 cup fudge", ing13, "1/4 cup fudge", "")</annotation> </line> ...Monday, December 7, 2009
  • USENET Cookbook .RH MOD.RECIPES-SOURCE CHOC-CAKE-1 D "20 May 86" 1986 .RZ "CHOCOLATE FANTASY CAKE" "A no-bake chocolate cake" This is a very simple, quick, no-bake cake. .IH "Serves 6-8" .SH! ! CAKE .IG "2(12 cups" "cream" "600 ml" .IG "3 cups" "milk" "750 ml" .IG "3 pkgs" "instant chocolate pudding" .PH .SK 1 Arrange a layer of biscuits at the bottom of a large baking pan. ... .SH RATING .I Difficulty: easy. .I Time: 10 minutes preparation, several hours chilling. .I Precision: measure the ingredients. .WR Mike Trachtman, Weizmann Institute, Rehovot, Israel,Monday, December 7, 2009
  • OntoChef conceptualizationMonday, December 7, 2009
  • OntoChef database population joint work with T. Machado and N. MamedeMonday, December 7, 2009
  • Manual inputMonday, December 7, 2009
  • Ontology-based semi-automatic input through natural language processing OntoChef Process (cooking ontology) Internet Recipes e-Books Database Cookbooks (in text) (ontology intances)Monday, December 7, 2009
  • OntoChef evaluationMonday, December 7, 2009
  • OntoChef future workMonday, December 7, 2009
  • additional languagesMonday, December 7, 2009
  • Knowledge sources Book’s Cover Book’s Bibliographic Reference English: “Ingredients”, by Loukie Werle, Ullmann Publishing, 2009, 384 pp. Portuguese: “Ingredientes”, by Loukie Werle, Könemann, 2009, 384 pp. English: “The Cooks Bible”, by Le Cordon Bleu, Carroll & Brown Publishers, 2005, 352 pp. Portuguese: “O Livro das Técnicas Culinárias”, by E. Treuille & J. Wright, Livros e Livros, 2006, 352 p. “Dicionário Visual Português/Inglês”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Espanhol”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Françês”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Italiano”, Civilização Editora, 2006, 360 pgs. “Dicionário Visual Português/Alemão”, Civilização Editora, 2006, 360 pgs.Monday, December 7, 2009
  • Collected information Languages # pairs PT-EN Portuguese — English Ingredients 2112 Portuguese — English Actions 1658 5927 Portuguese — English 2157 Portuguese — Spanish 2157 Portuguese — French 2157 Portuguese — Italian 2157 Portuguese — German 2157 Total: 14555Monday, December 7, 2009
  • wine informationMonday, December 7, 2009
  • Cooking Coach an ontology-basedcoaching dialogue systemsMonday, December 7, 2009
  • Recipe descriptionMonday, December 7, 2009
  • Monday, December 7, 2009
  • Available ‘tasks’ - next - previous - repeat - how? (down) - continue (up)Monday, December 7, 2009
  • Cooking CorpusMonday, December 7, 2009
  • Monday, December 7, 2009
  • Corpus annotation Equivalences Actions Food Units & Recipes Utensils MeasuresMonday, December 7, 2009
  • Cooking Coach next steps...Monday, December 7, 2009
  • Explore semantic spectrum without automated Reasoning with automated Reasoning Complexity Catalog Terms / Term Formal Formal Frames Value Description General Thesaurus / ID Glossary Hierarchy Taxonomy Instance (props) Restrictions Logics LogicMonday, December 7, 2009
  • Reasoning ‘Caracóis’ (snails): broiled animal, a cake with snail shapeMonday, December 7, 2009
  • Tackle different usersMonday, December 7, 2009
  • Different coaching stylesMonday, December 7, 2009
  • System’s evaluation a lot Prior experience with the Dialog System some none none some a lot Knowledge of the Domain (Cooking prior experience)Monday, December 7, 2009
  • MIT’s CounterMonday, December 7, 2009
  • DFKI’s Smart KitchenMonday, December 7, 2009
  • Coaching another tasksMonday, December 7, 2009
  • Questions http://flickr.com/photos/jazz_dalek/1845705965/Monday, December 7, 2009
  • Thank you!http://flickr.com/photos/42304632@N00/232004616/Monday, December 7, 2009
  • Joana Paulo Pardal joana.paulo.pardal@gmail.com ladydoctorbug.blogspot.com www.l2f.inesc-id.pt ⁄!joanaMonday, December 7, 2009
  • Cooking an ontology-based coaching dialogue system Joana Paulo Pardal Quality and Usability Colloquium December 7th, 2009 Berlin, GermanyMonday, December 7, 2009