Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dialogue systems and personal assistants


Published on

This presentation covers dialogue systems: their definition, basic structure (covering all modules: natural language understanding, dialogue manager, natural language generation), evaluation and the way they can be used. We also provide details about future directions and discusses current personal assistants: SIRI, S-Voice, Cortana, Maluuba etc.

Published in: Science, Technology
  • Be the first to comment

Dialogue systems and personal assistants

  1. 1. 1 Introduction to Dialogue SystemsIntroduction to Dialogue Systems Personal Assistants are becoming a realityPersonal Assistants are becoming a reality Dr Natalia Konstantinova University of Wolverhampton 11 April 2014
  2. 2. 2 OutlineOutline • What is a dialogue system? • System structure and classification; • Evaluation; • Examples of existing systems; • Future directions; • IQA;
  3. 3. 3 DefinitionDefinition • Artificial intelligence – idea to teach machines to think and act as humans. • NLP – give machines the ability to read, understand and use natural language. • Dialogue systems – part of artificial intelligence challenge.
  4. 4. 4 Optimistic viewOptimistic view • Hollywood and Artificial Intelligence (robots that can think and act like humans) • Video
  5. 5. 5 Realistic viewRealistic view • Talk to Alan (or to some other HAL personalities) • Chat with ALICE • Three bots talking
  6. 6. 6 What is a dialogue system?What is a dialogue system? Ideas? • • • •
  7. 7. 7 DefinitionDefinition • Editors of the Journal of Dialogue Systems : “A dialogue system is a computational device or agent that • (a) engages in interaction with other human and/or computer participant(s); • (b) uses human language in some form such as speech, text, or sign; • and (c) typically engages in such interaction across multiple turns or sentences.”
  8. 8. 8 Other termsOther terms • Conversational agents (Jurafsky and H.Martin, 2006), (Lester, Branting, and Mott, 2004) • “Chatterbot” or “chatbot”, first coined by Mauldin (1994): • simple dialogue systems, primarily based on simple analysis of keywords in the input and usage of different templates
  9. 9. 9 Where are they used?Where are they used? Usually embedded in such applications as: • customer service, • help desks, • website navigation, • guided selling, • technical support (Lester, Branting, and Mott, 2004)
  10. 10. 10 ““Body” for a dialogue systemBody” for a dialogue system • Embodied conversational agents (Cassell et al., 2000): • has a “body”, where both verbal and nonverbal devices advance and regulate the dialogue between the user and the computer. • Financial advisers, sales agents at online shops
  11. 11. 11 Embodied conversational agentsEmbodied conversational agents
  12. 12. 12 System structureSystem structure • They generally consist of 5 main components (Jurafsky and H.Martin, 2006): 1. speech recognition; 2. natural language understanding (NLU); 3. dialogue management; 4. natural language generation (NLG); 5. speech synthesis.
  13. 13. 13 System structureSystem structure • Some modules are optional: • e.g. speech recognition and speech synthesis • Dialogue systems involving speech are more complicated: • need to deal with errors in speech recognition • Speech recognition can be dialogue-state dependant
  14. 14. 14 NLUNLU • Aim of NLU module: • produce a semantic representation appropriate for a dialogue task.
  15. 15. 15 Dialogue managerDialogue manager • One of the most important parts of DS (Dale, Moisi, and Somers, 2000): • interpret the speech acts; • carry out problem-solving actions; • formulate response; • in general maintain the system's idea of the state of the discourse (e.g. dialogue move tree)
  16. 16. 16 Dialogue managerDialogue manager • Interlink of NLU and NLG • Responsible for the content generation • (taking decisions about what to say and how)
  17. 17. 18 NLGNLG • Chooses syntactic structures and words to express the intended meaning, which was formulated by a dialogue manager. • How?: • Templates to generate “prompts” (generated outputs) • Advanced natural language generators
  18. 18. 19 Speech synthesisSpeech synthesis • Is optional • Uses output on NLG module to generate natural speech
  19. 19. 20 System classificationSystem classification • Jurafsky and H.Martin (2006): 4 main types of dialogue management (DM) architectures: 1. finite-state DM; 2. frame/form based DM; 3. information-state DM; 4. plan-based DM.
  20. 20. 21 Finite-state DMFinite-state DM • A set of states • System totally controls the conversation with the user
  21. 21. 22 Frame/form based DMFrame/form based DM • Simple and the most widely used • Asks questions to fill in the slots in the frame • Perform a database query • E.g. booking a holiday
  22. 22. 23 Information-state DMInformation-state DM • More complicated • Incorporates several ways to achieve a result. • Components: • the information state (the “discourse context” or “mental model”); • dialogue act interpreter (or “interpretation engine”); • dialogue act generator (or “generation engine”); • set of update rules (to update information state); • control structure to select needed update rule.
  23. 23. 24 Plan-based DMPlan-based DM • The most sophisticated one • It interprets conversation as creation of a plan and then interprets a plan “in reverse” • Is often referred as BDI (belief, desire and intentions) model.
  24. 24. 25 Other classificationsOther classifications • system-initiative (or single initiative systems)  mixed initiative systems • spoken dialogue systems  text dialogue systems • multi-modal dialogue systems  unimodal dialogue systems • domain restricted dialogue systems  Open domain dialogue systems
  25. 25. Example of architectureExample of architecture 26
  26. 26. 27 EvaluationEvaluation • How to make an objective evaluation? • Task-based evaluation (Dale, Moisi, and Somers, 2000): • task completion success; • efficiency cost; • quality costs.
  27. 27. 28 EvaluationEvaluation • Asking people to complete a question list and rank the quality of the system giving grades: • E.g. evaluate naturalness • Maybe not very objective
  28. 28. 29 DifficultiesDifficulties • Necessity to collect training corpus: • Wizard-of-Oz experiments • Prompting experiments • Error handling
  29. 29. 30 ChatbotsChatbots •
  30. 30. Competitors of SIRICompetitors of SIRI • Cortana by Microsoft; • Voice Mate by LG; • S-Voice by Samsung; • Google Now; • E.g. Android versions: Maluuba; Robin; Iris; Vlingo; Skyvi; • More similar apps; 31
  31. 31. Further directionsFurther directions • Currently DM in all commercial systems is rule- based; • What can be used? • Reinforcement learning (hierarchical RL); • Online learning; • Dialogue manager based on partially observable Markov decision process (POMDP); • Quality-adaptive DM; 32
  32. 32. 33 ReferencesReferences • Cassell, Justine, Joe Sullivan, Scott Prevost, and Elizabeth F. Churchill, editors. 2000. Embodied Conversational Agents. Cambridge, MA: MIT Press. • Dale, Robert, Hermann Moisi, and Harold Somers, editors. 2000. Handbook of Natural Language Processing. Marcel Dekker, Inc. • Jurafsky, Daniel and James H.Martin. 2006. Speech and language processing an introduction to natural language processing, computational linguistics, and speech recognition. Prentice-Hall, Inc. • Lester, James, Karl Branting, and Bradford Mott. 2004. Conversational agents. In Munindar P Singh, editor, The Practical Handbook of Internet Computing. Chapman & Hall. • Mauldin, Michael L. 1994. Chatterbots, Tinymuds, and the Turing test: Entering the Loebner prize competition. In Proceedings of the Eleventh National Conference on Artificial Intelligence. AAAI Press. • Mitkov, Ruslan, editor. 2003. Handbook of Computational linguistics. Oxford University Press, USA. • Sacks, H., E. A. Schegloff, and G. Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language, 50(4):696-735. • Varges, S., F. Weng, and H. Pon-Barry. 2007. Interactive question answering and constraint relaxation in spoken dialogue systems. Natural Language Engineering, 15(1):9-30. • Webb, Nick and Bonnie Webber. 2009. Special issue on interactive question answering: Introduction. Natural Language Engineering, 15(1):1-8, January.