POWERPOINT SLIDESHOW

61,411 views

Published on

Published in: Technology, Business

POWERPOINT SLIDESHOW

  1. 1. MAGA: a Mobile Archaeological Guide at Agrigento Agnese Augello 1 , Antonella Santangelo 1 , Salvatore Sorce 1 , Giovanni Pilato 2 , Antonio Gentile 1 , Alessandro Genco 1 and Salvatore Gaglio 1,2 SpeechTek ’07 - New York City - 21 august 2007 2 ICAR - Istituto di CAlcolo e Reti ad alte prestazioni branch of Italian National Research Council Viale delle Scienze - 90128 Palermo - Italy 1 DINFO - Dipartimento di ingegneria Informatica University of Palermo Viale delle Scienze - 90128 Palermo - Italy
  2. 2. Summary <ul><ul><li>Aim of the work </li></ul></ul><ul><ul><li>Enabling Technologies </li></ul></ul><ul><ul><li>MAGA System architecture </li></ul></ul><ul><ul><li>Implementation details </li></ul></ul><ul><ul><li>Examples </li></ul></ul><ul><ul><li>Conclusions </li></ul></ul>
  3. 3. Aim of the work <ul><li>To build smart multimodal systems for information retrieval </li></ul><ul><ul><li>user-friendly </li></ul></ul><ul><ul><li>suitable for different mobile devices </li></ul></ul><ul><li>We propose the use of a verbal interface (for interactions) along with RFID (for in-door positioning purposes) </li></ul><ul><li>Requirements: </li></ul><ul><ul><li>Natural language interface </li></ul></ul><ul><ul><li>Availability of the system in environments where mobility of the user is essential ( e.g.: the Archaeological Museum of Agrigento ) </li></ul></ul>
  4. 4. Enabling technologies <ul><li>Personal Digital Assistants (PDAs) </li></ul><ul><li>Multimodal Browsers (Verbal Interaction) </li></ul><ul><li>Conversational Agents (Chatbots) </li></ul><ul><li>Semantic Networks (Reasoning Capabilities) </li></ul><ul><li>Radio Frequency Identification (RFID) </li></ul>
  5. 5. Personal Digital Assistant (PDA) <ul><li>Mobile devices and wireless communication allow people to access information everywhere </li></ul>Information retrieval on small devices is limited Boring interaction and large waste of time during information research!!! I/O interface
  6. 6. Verbal interaction <ul><li>Experimental results demonstrate that verbal interaction allows people to reduce the time spent on manual input (Lyons et al, 2005) </li></ul><ul><li>Vocal interaction is a more direct approach, but it comes with a set of constraints : </li></ul><ul><ul><li>processing power </li></ul></ul><ul><ul><li>speakers adaptation </li></ul></ul><ul><ul><li>narrow dictionaries </li></ul></ul><ul><ul><li>bounding grammars </li></ul></ul>
  7. 7. Conversational Agents <ul><li>A simple approach to build a dialogue system </li></ul><ul><li>Pattern-matching based rules </li></ul><ul><li>Chatbots can be used as interfaces for several applications (e.g.: entertaining, educational, e-learning …) </li></ul>
  8. 8. Semantic networks <ul><li>Allow to: </li></ul><ul><li>describe a specific domain </li></ul><ul><ul><li>Concepts, properties of concepts, relations between concepts are clearly defined in that domain </li></ul></ul><ul><li>represent c ommonsense knowledge covering aspects of everyday life </li></ul><ul><li>make inference to generate new knowledge from the existing one </li></ul><ul><li>share and reuse common understanding of the structure of information </li></ul><ul><li>Es: OpenCyc, OpenMind, WordNet… </li></ul>
  9. 9. RFID Passive tags: limited reading range -> limited position estimation error Useful in environments with spot-areas
  10. 10. MAGA <ul><li>Integration of chatbot , commonsense reasoning , speech recognition/synthesis and RFID technologies to build a virtual guide accessible: </li></ul><ul><ul><li>in a cultural heritage environment (such as the “ Archaelogical Museum of Agrigento ”) by means of different mobile devices </li></ul></ul><ul><ul><li>by simulation on a multimodal browser </li></ul></ul>
  11. 11. MAGA System Architecture
  12. 12. RFID tag detection Tag #1 Tag #2 Tag #3 Tag #4 Database
  13. 13. Verbal interaction engine
  14. 14. Verbal Interaction: X+V Technology <ul><ul><li>XHTML+Voice (X+V) as implementation language </li></ul></ul><ul><ul><li>Multimodal Tools 4.1.2.2 for WebSphere Studio V5.1.2: </li></ul></ul><ul><ul><ul><ul><li>IBM Multimodal Toolkit </li></ul></ul></ul></ul><ul><ul><ul><ul><li>IBM Multimodal Browser, (X+V interpreter) </li></ul></ul></ul></ul>X+V page
  15. 15. X+V Client-server architecture
  16. 16. X+V and Grammar Creation PHP for X+V creation php request (question) ChatBot XHTML code ChatBot (question) question in textual form VXML code with the reply replay in textual form VXML code with grammar XHTML code PHP for Grammar creation MXML Rule Pre php request (domain) Rule Key-Word Rule Post Domain Database GRXML info domain info domain info domain
  17. 17. Grammar example <rule id=&quot;pre&quot;> <one-of> <item repeat=&quot;0-1&quot;> tell me </item> <item repeat=&quot;0-1&quot;> i need information </item> <item repeat=&quot;0-1&quot;> to when </item> </one-of> <item repeat=&quot;0-1&quot;> about </item> <item repeat=&quot;0-1&quot;> the </item> </rule> <rule id=&quot;post&quot;> <item repeat=&quot;0-1&quot;> is dated </item> <item repeat=&quot;0-1&quot;> in Agrigento </item> </rule> <rule id=&quot;main_rule&quot;> <item repeat=&quot;0-1&quot;> <ruleref uri=&quot;#pre&quot;/> </item> <ruleref uri=&quot;#parola_chiave&quot;/> <item repeat=&quot;0-1&quot;> <ruleref uri=&quot;#post&quot;/> </item> </rule> <rule id=“parola_chiave&quot;> <one-of> <item> Archeological Museum </item> <item> Talamone </item> <item> Acropolis </item> </one-of> </rule>
  18. 18. Chatbot Knowledge Base
  19. 19. Chatbot technology : ALICE <ul><li>Chatbots systems: dialogue based on natural language </li></ul><ul><li>ALICE free software technology </li></ul>ALICE is an artificial intelligence, natural language chat robot The knowledge base is composed of question-answer modules, called categories and structured with Artificial Intelligence Mark-up Language ( AIML)
  20. 20. <ul><li>Each category is a rule for matching an input and converting to an output, and consists of a pattern , which represents the user input, and a template , which represents the chatbot’s answer. </li></ul><ul><li>< category > </li></ul><ul><ul><li>< pattern >WHAT IS A CIRCLE></ pattern > </li></ul></ul><ul><ul><li>< template >A circle is the set of points equidistant from a common point called the center</ template > </li></ul></ul><ul><li></ category > </li></ul><ul><li>The chatbots’ knowledge is stored in a tree called GraphMaster </li></ul>Chatbot technology : ALICE Alice’s brain
  21. 21. OpenCyc Ontology <ul><ul><li>Knowledge base : logic assertions, common sense rules and heuristics for the reasoning about quotidian life objects and events </li></ul></ul><ul><ul><li>Inference system : is based on a series of heuristic related to a reasoning techniques about microtheories </li></ul></ul><ul><ul><li>CycL : the language in which Cyc (and OpenCyc) has been written, is a formal language from Lisp language </li></ul></ul>
  22. 22. The “AgrigentoMuseumMt” created in OpenCyc
  23. 23. Example: a Microtheory
  24. 24. ALICE-KB and Cyc Inference capabilities <ul><li>We have written a java application that bridges the AIML interpreter with the OpenCyc inference engine (based on the CyN project). </li></ul><ul><li>The knowledge base of the chatbot is then enriched with new AIML tags that enable the ontology querying, executing and asserting CycL/SubL statements. </li></ul><ul><li>This allows (common sense) reasoning capabilities inside Alice chatbot. </li></ul><ul><li>A template can be seen as a meta-answer that must be processed to compose the appropriate response. </li></ul>
  25. 25. ALICE-KB and Cyc: Inference Capabilities <ul><li>The created ontology enables the composition of answers that are not present in the traditional AIML knowledge base. </li></ul><ul><li>It is possible to: </li></ul><ul><ul><li>extend the number of AIML categories to obtain a more fluent dialogue; </li></ul></ul><ul><ul><li>extend the ontology with new concepts, facts and relations of the specific domain to provide the chatbot a smarter behavior. </li></ul></ul>
  26. 26. An example of AIML Category… <ul><li><pattern> </li></ul><ul><li>I NEED INFORMATION ABOUT THE * </pattern> </li></ul><ul><li><template> </li></ul><ul><li><cycsystem>(cyc-query '(#$Comment <cycterm><star/></cycterm> </li></ul><ul><li></cycsystem> </li></ul><ul><li>It is located at </li></ul><ul><li><cycsystem>(cyc-query '(#$located_in <star/> ?X)) </cycsystem> </li></ul><ul><li>and has been created by </li></ul><ul><li><cycsystem>(cyc-query '(#$createdBy <star/> ?X)) </cycsystem> </li></ul><ul><li></template> </li></ul><ul><li>… and the resulting interaction </li></ul><ul><li>User: I need information about the Archeological Museum in Agrigento. </li></ul><ul><li>Chatbot: It is one of the most modern museums in Sicily and it contains archeological finds from the provinces of Agrigento and Caltanissetta, extending from prehistoric to Roman times. It is located at Agrigento, and has been created by the architect Franco Minissi. </li></ul>
  27. 27. An example of AIML Categories using the RFID information <ul><li><category><pattern> LOCALIZATION DETECTED </pattern><template> </li></ul><ul><li>You are in <set name=“showcase”> <system>java RFID “showcase”</system></set> </li></ul><ul><li><srai> INFO SHOWCASE <get name=“showcase”/> </srai> </li></ul><ul><li></template></category> </li></ul><ul><li><category><pattern>INFO SHOWCASE *</pattern><template> </li></ul><ul><li>The showcase contains <set name=“artifact”><cycsystem>(cyc-query '(#$Contains <star/> ?X)) </cycsystem></set>. </li></ul><ul><li><srai> INFO ARTIFACT <get name=“artifact”/> </srai> </li></ul><ul><li></template></category> </li></ul><ul><li><category> </li></ul><ul><li><pattern> INFO ARTIFACT </pattern> </li></ul><ul><li><template> belongs to </li></ul><ul><li><cycsystem>(cyc-query '(#$CreatedDuring <star/> ?X)) </cycsystem>, </li></ul><ul><li>it is attributed to </li></ul><ul><li><cycsystem>(cyc-query '(#$CreatedBy <star/>?X)) </cycsystem> and </li></ul><ul><li>and represents </li></ul><ul><li><cycsystem>(cyc-query '(#$Representing <star/>?X)) </cycsystem> </li></ul><ul><li></template> </li></ul><ul><li></category> </li></ul>
  28. 28. A dialogue example <ul><li>User: I need information about the Archeological Museum in Agrigento. </li></ul><ul><li>Chatbot: It is one of the most modern museums in Sicily and it contains archeological finds from neighbors of Agrigento and Caltanissetta, ranging from prehistoric to Roman times. </li></ul><ul><li>Chatbot: You are in room #3, in front of the showcase #17. Do you want more information? </li></ul><ul><li>User: Yes. </li></ul><ul><li>Chatbot: In this showcase you can see an Attic amphora. It belongs to Arcaic Greek period, it is attributed to Dikaios and represents Athena. </li></ul><ul><li>User: What is in the next room? </li></ul><ul><li>Chatbot: In the room #4 there are Capitals and gargoyles with lion's heads from various temples. </li></ul>
  29. 29. Demo
  30. 30. Conclusions and future work <ul><li>Chatbot with “reasoning capabilities” and speech recognition technologies have been embedded in a Personal Digital Assistant to assist user in visiting historical sites </li></ul><ul><li>Natural interaction overcomes the limitation of traditional pre-recorded audio/visual guides </li></ul><ul><li>Future work will regard the enhancement of the system: </li></ul><ul><ul><li>Enhance the chatbot reasoning capabilities and natural language interaction; </li></ul></ul><ul><ul><li>Use of different wireless technologies for context-aware service provision </li></ul></ul><ul><ul><li>Use of user profiles for ad-hoc service composition </li></ul></ul>
  31. 31. Thank you! <ul><li>Vocal User Interface, System Architecture Design: </li></ul><ul><ul><li>Antonella Santangelo { [email_address] } </li></ul></ul><ul><ul><li>Antonio Gentile { [email_address] } </li></ul></ul><ul><li>Information retrieval, reasoning: </li></ul><ul><ul><li>Agnese Augello { [email_address] } </li></ul></ul><ul><ul><li>Giovanni Pilato { [email_address] } </li></ul></ul><ul><ul><li>Salvatore Gaglio { [email_address] } </li></ul></ul><ul><li>RFID, pervasive access, HCI: </li></ul><ul><ul><li>Salvatore Sorce { [email_address] } </li></ul></ul><ul><ul><li>Alessandro Genco { [email_address] } </li></ul></ul>

×