Successfully reported this slideshow.
Your SlideShare is downloading. ×

Spoken Dialogue Systems and Social Talk - Emer Gilmartin

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 21 Ad

More Related Content

Similar to Spoken Dialogue Systems and Social Talk - Emer Gilmartin (20)

More from Sebastian Ruder (20)

Advertisement

Recently uploaded (20)

Spoken Dialogue Systems and Social Talk - Emer Gilmartin

  1. 1. SPOKEN DIALOGUE SYSTEMS AND SOCIAL TALK Emer Gilmartin Speech Communication Lab Trinity College Dublin
  2. 2. What? • Spoken dialogue systems attempt to create a spoken interaction with a user • Dialogue systems, Intelligent Virtual Agents (IVA’s), Embodied Conversational Agents (ECA’s), Chatbots • Dream (Turing, 1950 ) vs Practical Progress (Allen, 2000) • AI – early chat – pattern matching – ELIZA • Practical Dialogues – task to be performed - Practical Dialogue Hypothesis (Allen, 2000)
  3. 3. What’s out there? • Command and Control – voice commands • Information Retrieval – Siri • Interactive Voice Response – IVR • Chatbots • Embodied Conversational Agents (ECA) • Intelligent Virtual Agents
  4. 4. Casual conversation – the unmarked case • Ordering a pizza (transactional) • performing a well-defined task • content (‘What?’) vital for success • Chat with neighbour (interactional) • building/maintaining social bonds • social (‘How?’) very important • ‘continuing state of incipient talk’
  5. 5. What is a Spoken Dialogue System?
  6. 6. Multimodality • Expression and Recognition • Audio, visual, verbal, vocal, non-verbal, facial expression, gesture, posture… • Presence, affect, attitude...
  7. 7. • The Problem: Building social dialogue systems entails understanding of casual social dialogue but… • Much linguistic theory is based on language similar to writing but highly unlike talk • regards spoken interaction as debased, chaotic • SDS technology based on • Practical Dialogue Hypothesis (Allen, 2000) • Constraint introduced to make dialogue modelling tractable • Much corpus study of spoken interaction based on Task-based Dialogue • Information gap activities – MapTask (HCRC), DiaPix (Lucid) • Meetings – AMI, ICSI • These are not corpora of casual or social talk
  8. 8. Social Talk • Spoken interaction as social activity • Malinowski, Dunbar, Jakobsen, Brown and Yule • Structure and Content • Smalltalk at the margins (Laver) • Chat and chunks (Slade & Eggins) • Bouts – gossip, narrative • Bouts end with ‘idling’ (Schneider) • Phases – greetings, approach, centre, leavetaking (Ventola) • Multiparty (Slade) • Problems: • much of this is theory, analysis by example • based on orthographical transcriptions • corpus based studies on transactional dyadic interaction, phonecalls… January 15, 2016 IWSDS 2016
  9. 9. Genre differences in spoken interaction? • Spoken interaction is situated • ‘speech-exchange systems’ (SSJ), • communicative activities (Allwood) • Some low level mechanisms may follow universal patterns • It is also possible that even basic interaction mechanisms such as turn-taking vary with the type and parameters of different interactions • What might vary? • Utterance/turn characteristics • Distribution of pauses/gaps/overlaps • ‘Disfluencies’, VSU’s, laughter… • Explore different genres and use knowledge to inform design of interfaces
  10. 10. Anatomy of casual conversation January 15, 2016 IWSDS 2016 L A C G
  11. 11. 10 minutes from a 5-party casual conversation showing chat (240s-480s and chunk 480 – end) phases Red-speech, yellow-laughter, grey-silence January 15, 2016 IWSDS 2016
  12. 12. Chunk to Chunk Transition – more interaction and laughter at end of chunks January 15, 2016 IWSDS 2016
  13. 13. Chat/Chunk • Significant differences in • Length – chat very variable, chunk ~ 30s • Phrase final prosody • Gap lengths • Important because; • Need different timing modules for different phases • Useful to know which phase we’re in • Current Work • Stochastic model – simple bigram HMM can classify chat/chunk • Goal - online classifier – knowledge additional to ASR to inform dialogue management January 15, 2016 IWSDS 2016
  14. 14. How do I make a spoken dialogue system? • Virtual Human Toolkit (https://vhtoolkit.ict.usc.edu/) • Pandorabots (www.pandorabots.com) • CSLU Toolkit (http://www.cslu.ogi.edu/toolkit/) • Voxeo Prophecy (http://voxeo.com/prophecy/) • AT&T Speech Mashupshttps://service.research.att.com/smm/login.jsp
  15. 15. SPOKEN AND MULTIMODAL DIALOGUE APPLICATIONS IN HEALTHCARE Emer Gilmartin Speech Communication Lab SCSS
  16. 16. ELIZA – text-based Rogerian therapist • Weizenbaum - 1966 • http://www.masswerk.at/elizabot/eliza_test.html
  17. 17. NWU - Relational Agents Group • Relational agents group • https://www.youtube.com/watch?v=Jx5Tsn9wFw • Presentation • https://www.youtube.com/watch?v=lcb_rMhJQTI • RAISE • https://www.youtube.com/watch?v=ttBMG-F1HS0
  18. 18. ICT – Virtual Humans • Simsensei • https://www.youtube.com/watch?v=I2aBJ6LjzMw • https://www.youtube.com/watch?v=ejczMs6b1Q4
  19. 19. Simulations and Virtual Reality • ICT – MILES – Training for Therapists • https://www.youtube.com/watch?v=KNGVRePdEL8 • ICT - Standard Patient Hospital
  20. 20. The future • ICTnarrative • https://www.youtube.com/watch?v=4IBoO2IKFMI • Bickmore’s warning • https://www.youtube.com/watch?v=V20KEjIjFl8
  21. 21. • http://relationalagents.com/index.html

×