Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Art and Science of Conversation Applied to Alexa Skills - ALX303 - re:Invent 2017

1,205 views

Published on


It used to be the case that we only spoke to computers in their language. But more and more often, we’re interacting with them in ours. We are moving quickly into a world of computer conversation, and one in which, for many applications, the most natural interactions will be through spoken language. But how do you create engaging narrative and compelling, organic conversational interactions using the imprecise tools of speech recognition and intent resolution? In this session, we look at the experience as a whole and take you through key learnings that you can use when building your skills. We cover issues like knowing your audience, creating compelling storylines, using a cast of characters, integrating voiceover, designing a soundscape, and finding those “magic moments”. For each of these, we share the design pattern, the backing AI or physiological science, and how to implement the experience with Alexa.

  • Be the first to comment

The Art and Science of Conversation Applied to Alexa Skills - ALX303 - re:Invent 2017

  1. 1. AWS re:INVENT THE AR T AND S C IENC E OF C ONV ER S ATION AP P LIED TO ALEXA S KIL LS A M I T J O T W A N I ( A M A Z O N A L E X A ) A L X 3 0 3 : B R E A K O U T D A N I E L L E F R I M E R ( P U L L S T R I N G ) L U C A S I V E S ( P U L L S T R I N G ) J A S O N H A B E R ( P U L L S T R I N G )
  2. 2. 4.54 billion years
  3. 3. 4.54 billion years
  4. 4. 200,000 years
  5. 5. 200,000 years
  6. 6. 100,000 years
  7. 7. 100,000 years
  8. 8. CONVERSATION
  9. 9. CREATING ENGAGING CONVERSATIONAL INTERFACES
  10. 10. ALEXA SKILL FOR BOSCH BOSCH: A Detective's Case
  11. 11. CONVERSATIONAL DESIGN: PROCESS AND PRACTICES DANIELLE FRIMER CONVERSATIONAL DESIGNER, PULLSTRING LUCAS IVES HEAD OF CONVERSATION ENGINEERING, PULLSTRING JASON HABER PROGRAM DIRECTOR, PULLSTRING
  12. 12. CASE STUDY: BOSCH From prompt • who’s your audience? • what’s your source material? • do your research To pitch
  13. 13. PITCH (EXCERPT) “Annabelle Crowe leaves a voicemail for Bosch: her friend Nora is in trouble. Nora was having some issues with… a client, a married guy who said he wanted to leave his family to be with her. […] Nobody’s been able to track Nora down for days, and Annabelle’s worried something really bad has happened. Could Bosch look into it? You (the user) are a new recruit at the bureau. You’ve been there just a few weeks. Bosch briefs you on the situation and asks you to handle it – discreetly.”
  14. 14. STORY STRUCTURE Act One Setup Act Two Confrontation Act Three Resolution Beginning Act One Climax Midpoint Act Two Climax End
  15. 15. GAME MECHANIC Nora’s House Bedroom Kitchen Bathroom WIN LOSE LOSE
  16. 16. NARRATIVE SKILL DESIGN Nora’s House Bedroom Kitchen Bathroom Climax WIN Climax Climax LOSE LOSE
  17. 17. DESIGN CONSIDERATIONS • Keep choices simple • Avoid content creep • Your ears are your best design tool • Alexa as a character
  18. 18. EXPECT THE UNEXPECTED • User as scene partner
  19. 19. EXPECT THE UNEXPECTED • User as scene partner • The Zen of Fallbacks • What they are • How to use them
  20. 20. EXPECT THE UNEXPECTED • User as scene partner • The Zen of Fallbacks • What they are • How to use them • Global Interjections • What they are • How to use them
  21. 21. TESTING STRATEGY • Debug while you write • ASR specific testing • Testing & scope • Value of user testing
  22. 22. PROJECT MANAGEMENT CONSIDERATIONS
  23. 23. SCOPING CONSIDERATIONS SkillLaunch StartDate Often projects come with hard launch dates, leaving us a short amount of time for development
  24. 24. SCOPING CONSIDERATIONS Alexa skill development time, with TTS Alexa skill development time, with Voice Talent & Sound Design Development Time (TTS) Development Time (TTS) +Additional Time (Voice Talent) SkillLaunch StartDate
  25. 25. SCOPING CONSIDERATIONS Alexa skill development time, with TTS Alexa skill development time, with Voice Talent & Sound Design Alexa skill development time, with Voice Talent & Sound Design - Optimized Development Time (TTS) Development Time (TTS) +Additional Time (Voice Talent) Development Time (TTS) Scoping & Optimization SkillLaunch StartDate
  26. 26. BUDGETING CONSIDERATIONS ● Writing ● Engineering Support/Solutions Architect ● QA ● Submission Support ● Production ● Writing Engineering Support/Solutions Architect QA Submission Support Production ● + Audio Engineering and Editing ● + Audio Design (Music, Sound Effects) ● + Voice Talent ● + Additional Overhead ● + Additional QA Alexa skill with Voice TalentAlexa skill with TTS
  27. 27. SCHEDULING CONSIDERATIONS Alexa skill with TTS Alexa skill with Voice Talent & Sound Design Design Production QA Submission Design Audio Design + Music/Sfx Production QA Submission Recording Audio Editing/Polish
  28. 28. SCHEDULING CONSIDERATIONS Design Audio Design + Music/Sfx Production QA Submission Recording Audio Editing/Polish Locking VO Script Timing of Recording Sessions Submission vs Launch AssetsAudio Design Input
  29. 29. TECHNICAL CONSIDERATIONS
  30. 30. GOAL IS TO UNIFY… • Content • Intent • Context
  31. 31. DEEPLY INTEGRATED CONTEXT • Context on Alexa • flat intent list • usually match something • no way to disable intents
  32. 32. • Context on Alexa • flat intent list • usually match something • no way to disable intents • Better scenario: • aware of exact location in flow, not just previous turn • only a handful of intents are active (+ globals) • context-specific fallbacks DEEPLY INTEGRATED CONTEXT
  33. 33. AUDIO CONSIDERATIONS
  34. 34. AUDIO CONSIDERATIONS
  35. 35. AUDIO CONSIDERATIONS • non-AudioPlayer SSML responses fairly limited: • limited to 16kHz (actually, 22.05kHz .. shh!) • 5 audio files per response max • single-track • 90sec limit (as of 9.2017)
  36. 36. AUDIO CONSIDERATIONS • non-AudioPlayer SSML responses fairly limited: • limited to 16kHz (actually, 22.05kHz .. shh!) • 5 audio files per response max • single-track • 90sec limit (as of 9.2017) • pre-bake audio for … • timing • layering
  37. 37. AUDIO CONSIDERATIONS • non-AudioPlayer SSML responses fairly limited: • limited to 16kHz (actually, 22.05kHz .. shh!) • 5 audio files per response max • single-track • 90sec limit (as of 9.2017) • pre-bake audio for … • timing • layering • localization can be an issue when using Alexa voice (en-US & en-GB)
  38. 38. PERSONNEL “your programmer should probably not also be your writer”
  39. 39. www.pullstring.com
  40. 40. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. alexa.design/guide alexa.design/standout alexa.design/officehours alexa.design/devrewards RESOURCES
  41. 41. THANK YOU! A L X 3 0 3 : B R E A K O U T A M I T J O T W A N I ( A M A Z O N A L E X A ) D A N I E L L E F R I M E R ( P U L L S T R I N G ) L U C A S I V E S ( P U L L S T R I N G ) J A S O N H A B E R ( P U L L S T R I N G )

×