Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1 / 78 Multimodal Voice Development with Amazon Alexa
2 / 78 Ralf Eggert CEO Travello GmbH Alexa Skill Developer Advisory Board Voice Con Alexa Champion
3 / 78 Portfolio (excerpt)
4 / 78 Three questions
5 / 78 Question 1: Who owns an Echo device with a display?
6 / 78 Question 2: Who has developed a Multimodal Skill for Alexa?
7 / 78 Question 3: Who has an APL based Multimodal Skill for Alexa live?
8 / 78 A definition of multimodal
9 / 78 What is this multimodal stuff anyway?
10 / 78 Multimodal Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, an...
11 / 78 Textual Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and v...
12 / 78 Aural Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and vis...
13 / 78 Linguistic Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, an...
14 / 78 Spatial Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and v...
15 / 78 Visual Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and vi...
16 / 78 Textual Aural Linguistic Visual Let's focus on
17 / 78 Interlude
18 / 78 OTIS REM Who knows this smart voice assistant? Is it multimodal?
19 / 78 Alexa & Multimodal
20 / 78 OK! How about Alexa?
21 / 78 AWS Lambda / HTTPS Endpoint Server Alexa Voice Service   Headless
22 / 78 Cards Support headless Echo devices with visuals. Are they multimodal?
23 / 78 Echo Buttons Another input media. Are they multimodal? What do you think?
24 / 78 AWS Lambda / HTTPS Endpoint Server Alexa Voice Service  Multimodal
25 / 78 Echo Dot Echo Echo Show Echo Spot Headless vs. Multimodal
26 / 78 Display devices Only 5.9 % of Alexa users in the US own an Echo device with a display Voicebot.ai, June 24, 2018, ...
27 / 78 More numbers 56.2 % own Echo Spot 25.0 % own Echo Show 18.7 % own both devices Voicebot.ai, June 24, 2018, see htt...
28 / 78 Mind the trap! Many Alexa Skills rather focus on Echo Show than on Echo Spot. More than twice as much Echo Spot th...
29 / 78 History: display interface
30 / 78 Echo Show 1st generation Echo Show was introduced in May 2017.
31 / 78 Echo Spot 1st generation Echo Spot was introduced in September 2017.
32 / 78 Display Interface Support for displays added to Alexa developer console
33 / 78 Reference Learn how to build multimodal Alexa Skills with the Display Interface Reference https://goo.gl/eeSQah
34 / 78 Display Interface Templates
35 / 78 Body Template 1 Echo Show Echo Spot
36 / 78 Body Template 2 Echo Show Echo Spot
37 / 78 Body Template 3 Echo Show Echo Spot
38 / 78 Body Template 6 Echo Show Echo Spot
39 / 78 Body Template 7 Echo Show Echo Spot
40 / 78 List Template 1 Echo Show Echo Spot
41 / 78 List Template 2 Echo Show Echo Spot
42 / 78 Wait! What happened to Body Templates 4 and 5? Does anyone know?
43 / 78 Template Docs Learn to choose the best template for your needs. https://goo.gl/pp4STX
44 / 78 { "version": "1.0", "session": {}, "context": { "Display": {}, "System": { "device": { "deviceId": "...", "support...
45 / 78 { "body": { "version": "1.0", "response": { "outputSpeech": {}, "card": {}, "directives": [ { "type": "Display.Ren...
46 / 78 { "type": "BodyTemplate6", "token": "launch", "backgroundImage": { "contentDescription": "Hello", "sources": [ { "...
47 / 78 Cons of Display Interface Only 7 templates Distinction Show vs. Spot Templates on Spot suck Static layouts Hard to...
48 / 78 Alexa Presentation Language
49 / 78 Alexa Presentation Language For the win
50 / 78 APL schedule Private beta June 2018 Public beta October 2018 General availability soon
51 / 78 Echo Show 2nd generation Echo Show was presented in September 2018.
52 / 78 Alexa Presentation Language Support for APL added to Alexa developer console
53 / 78 Reference Learn how to get started with the Alexa Presentation Language https://goo.gl/Zw9m62
54 / 78 Authoring Tool Design your own templates https://goo.gl/jbC535
55 / 78 Authoring Tool Pick from the predefined samples, start from scratch oder upload your code
56 / 78 APL terminology Document, package, layout, resources, styles, and components https://goo.gl/ufq6SH
57 / 78 APL document JSON file containing list of packages, resources, layouts, and styles. Works like a container and is ...
58 / 78 APL package Packages contain APL documents and images. Can easily be reused and are cached on the device.
59 / 78 APL layout Hierarchy set of components for rendering one the display. Can contain text, images, scrolling regions ...
60 / 78 APL resources Defined constants to be used for drawing text or images on the screen. For example font sizes, colou...
61 / 78 APL styles Collection of grouped resources to build a style. Defines size, background colour, text colour, borders...
62 / 78 APL components Components are primitive types to be added to a layout. Examples are containers, text, images, sequ...
63 / 78 { "version": "1.0", "session": {}, "context": { "Display": {}, "System": { "device": { "deviceId": "...", "support...
64 / 78 { "body": { "version": "1.0", "response": { "outputSpeech": {}, "card": {}, "directives": [ { "type": "Alexa.Prese...
65 / 78 { "type": "APL", "version": "1.0", "import": [], "resources": [], "styles": {}, "layouts": {}, "mainTemplate": {} ...
66 / 78 Authoring tool hands-on
67 / 78 Pros of APL Start with samples Build from scratch Build reusable packages Authoring tool Easy to test Split displa...
68 / 78 Cons of APL Response is more complex No saving option in the authoring tool Not fully rolled out yet No graphical ...
69 / 78 SDKs / tools with APL support Third party toolsOffical SDKs December 2018 January 2019 Alexa.NET
70 / 78 Compare display interface and APL
71 / 78 Display interface start screen Thanks toDanielMittendorf
72 / 78 APL start screen Thanks toDanielMittendorf
73 / 78 Display interface list screen Thanks toDanielMittendorf
74 / 78 APL list screen Thanks toDanielMittendorf
75 / 78 Conclusion
76 / 78 Start with APL Public beta phase You could start today Consider the numbers of sold display devices!
77 / 78 Need more motivation? Alexa Skills Challenge $150K in total prizes Bonus prize for Germany Enter til 22th of Janua...
78 / 78 Any questions? ralf@travello.audio https://www.travello.audio
Upcoming SlideShare
Loading in …5
×

Multi-Modal Voice Development with Amazon Alexa

159 views

Published on

Building a voice application for Amazon Alexa requires the Voice First approach. But with the growing device family with displays like the Echo Spot, the Echo Show, or the Fire TV, you are able to support your voice experience with photos, illustrations, or videos. This session concentrates on how to build a Multi-Modal application with Amazon Alexa. We will have a closer look on the best-practices as well as some tools and techniques to help you to create richer voice applications.

Published in: Technology
no profile picture user

  • Be the first to comment

Multi-Modal Voice Development with Amazon Alexa

  1. 1. 1 / 78 Multimodal Voice Development with Amazon Alexa
  2. 2. 2 / 78 Ralf Eggert CEO Travello GmbH Alexa Skill Developer Advisory Board Voice Con Alexa Champion
  3. 3. 3 / 78 Portfolio (excerpt)
  4. 4. 4 / 78 Three questions
  5. 5. 5 / 78 Question 1: Who owns an Echo device with a display?
  6. 6. 6 / 78 Question 2: Who has developed a Multimodal Skill for Alexa?
  7. 7. 7 / 78 Question 3: Who has an APL based Multimodal Skill for Alexa live?
  8. 8. 8 / 78 A definition of multimodal
  9. 9. 9 / 78 What is this multimodal stuff anyway?
  10. 10. 10 / 78 Multimodal Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  11. 11. 11 / 78 Textual Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  12. 12. 12 / 78 Aural Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  13. 13. 13 / 78 Linguistic Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  14. 14. 14 / 78 Spatial Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  15. 15. 15 / 78 Visual Multimodality describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages. Murray, Joddy (2013) / Wikipedia
  16. 16. 16 / 78 Textual Aural Linguistic Visual Let's focus on
  17. 17. 17 / 78 Interlude
  18. 18. 18 / 78 OTIS REM Who knows this smart voice assistant? Is it multimodal?
  19. 19. 19 / 78 Alexa & Multimodal
  20. 20. 20 / 78 OK! How about Alexa?
  21. 21. 21 / 78 AWS Lambda / HTTPS Endpoint Server Alexa Voice Service   Headless
  22. 22. 22 / 78 Cards Support headless Echo devices with visuals. Are they multimodal?
  23. 23. 23 / 78 Echo Buttons Another input media. Are they multimodal? What do you think?
  24. 24. 24 / 78 AWS Lambda / HTTPS Endpoint Server Alexa Voice Service  Multimodal
  25. 25. 25 / 78 Echo Dot Echo Echo Show Echo Spot Headless vs. Multimodal
  26. 26. 26 / 78 Display devices Only 5.9 % of Alexa users in the US own an Echo device with a display Voicebot.ai, June 24, 2018, see https://goo.gl/7WSkjD
  27. 27. 27 / 78 More numbers 56.2 % own Echo Spot 25.0 % own Echo Show 18.7 % own both devices Voicebot.ai, June 24, 2018, see https://goo.gl/7WSkjD
  28. 28. 28 / 78 Mind the trap! Many Alexa Skills rather focus on Echo Show than on Echo Spot. More than twice as much Echo Spot than Echo Show devices sold.
  29. 29. 29 / 78 History: display interface
  30. 30. 30 / 78 Echo Show 1st generation Echo Show was introduced in May 2017.
  31. 31. 31 / 78 Echo Spot 1st generation Echo Spot was introduced in September 2017.
  32. 32. 32 / 78 Display Interface Support for displays added to Alexa developer console
  33. 33. 33 / 78 Reference Learn how to build multimodal Alexa Skills with the Display Interface Reference https://goo.gl/eeSQah
  34. 34. 34 / 78 Display Interface Templates
  35. 35. 35 / 78 Body Template 1 Echo Show Echo Spot
  36. 36. 36 / 78 Body Template 2 Echo Show Echo Spot
  37. 37. 37 / 78 Body Template 3 Echo Show Echo Spot
  38. 38. 38 / 78 Body Template 6 Echo Show Echo Spot
  39. 39. 39 / 78 Body Template 7 Echo Show Echo Spot
  40. 40. 40 / 78 List Template 1 Echo Show Echo Spot
  41. 41. 41 / 78 List Template 2 Echo Show Echo Spot
  42. 42. 42 / 78 Wait! What happened to Body Templates 4 and 5? Does anyone know?
  43. 43. 43 / 78 Template Docs Learn to choose the best template for your needs. https://goo.gl/pp4STX
  44. 44. 44 / 78 { "version": "1.0", "session": {}, "context": { "Display": {}, "System": { "device": { "deviceId": "...", "supportedInterfaces": { "Display": { "templateVersion": "1.0", "markupVersion": "1.0" } } } } }, "request": {} } Alexa Request Very simplified
  45. 45. 45 / 78 { "body": { "version": "1.0", "response": { "outputSpeech": {}, "card": {}, "directives": [ { "type": "Display.RenderTemplate", "template": {} }, { "type": "Hint", "hint": {} } ], "reprompt": {}, "shouldEndSession": false } } } Alexa Response Also very simplified
  46. 46. 46 / 78 { "type": "BodyTemplate6", "token": "launch", "backgroundImage": { "contentDescription": "Hello", "sources": [ { "url": "https://host.com/image.jpg", "widthPixels": 0, "heightPixels": 0 } ] }, "textContent": { "primaryText": { "type": "RichText", "text": "<font size="7">Hello</font>" } }, "backButton": "HIDDEN" } RenderTemplate As well simplified
  47. 47. 47 / 78 Cons of Display Interface Only 7 templates Distinction Show vs. Spot Templates on Spot suck Static layouts Hard to test
  48. 48. 48 / 78 Alexa Presentation Language
  49. 49. 49 / 78 Alexa Presentation Language For the win
  50. 50. 50 / 78 APL schedule Private beta June 2018 Public beta October 2018 General availability soon
  51. 51. 51 / 78 Echo Show 2nd generation Echo Show was presented in September 2018.
  52. 52. 52 / 78 Alexa Presentation Language Support for APL added to Alexa developer console
  53. 53. 53 / 78 Reference Learn how to get started with the Alexa Presentation Language https://goo.gl/Zw9m62
  54. 54. 54 / 78 Authoring Tool Design your own templates https://goo.gl/jbC535
  55. 55. 55 / 78 Authoring Tool Pick from the predefined samples, start from scratch oder upload your code
  56. 56. 56 / 78 APL terminology Document, package, layout, resources, styles, and components https://goo.gl/ufq6SH
  57. 57. 57 / 78 APL document JSON file containing list of packages, resources, layouts, and styles. Works like a container and is send to the device.
  58. 58. 58 / 78 APL package Packages contain APL documents and images. Can easily be reused and are cached on the device.
  59. 59. 59 / 78 APL layout Hierarchy set of components for rendering one the display. Can contain text, images, scrolling regions and even other layouts. Can be used to build libraries.
  60. 60. 60 / 78 APL resources Defined constants to be used for drawing text or images on the screen. For example font sizes, colours or spacing.
  61. 61. 61 / 78 APL styles Collection of grouped resources to build a style. Defines size, background colour, text colour, borders, etc.
  62. 62. 62 / 78 APL components Components are primitive types to be added to a layout. Examples are containers, text, images, sequences, scroll views, or touch wrappers
  63. 63. 63 / 78 { "version": "1.0", "session": {}, "context": { "Display": {}, "System": { "device": { "deviceId": "...", "supportedInterfaces": { "Alexa.Presentation.APL": { "runtime": { "maxVersion": "1.0" } } } } } }, "request": {} } Alexa Request With active APL support
  64. 64. 64 / 78 { "body": { "version": "1.0", "response": { "outputSpeech": {}, "card": {}, "directives": [ { "type": "Alexa.Presentation. APL.RenderDocument", "version": "1.0", "document": {}, "datasource": {} } ], "reprompt": {}, "shouldEndSession": false } } } Alexa Response Also very simplified
  65. 65. 65 / 78 { "type": "APL", "version": "1.0", "import": [], "resources": [], "styles": {}, "layouts": {}, "mainTemplate": {} } APL Document Just the structure
  66. 66. 66 / 78 Authoring tool hands-on
  67. 67. 67 / 78 Pros of APL Start with samples Build from scratch Build reusable packages Authoring tool Easy to test Split display design from code
  68. 68. 68 / 78 Cons of APL Response is more complex No saving option in the authoring tool Not fully rolled out yet No graphical editor yet
  69. 69. 69 / 78 SDKs / tools with APL support Third party toolsOffical SDKs December 2018 January 2019 Alexa.NET
  70. 70. 70 / 78 Compare display interface and APL
  71. 71. 71 / 78 Display interface start screen Thanks toDanielMittendorf
  72. 72. 72 / 78 APL start screen Thanks toDanielMittendorf
  73. 73. 73 / 78 Display interface list screen Thanks toDanielMittendorf
  74. 74. 74 / 78 APL list screen Thanks toDanielMittendorf
  75. 75. 75 / 78 Conclusion
  76. 76. 76 / 78 Start with APL Public beta phase You could start today Consider the numbers of sold display devices!
  77. 77. 77 / 78 Need more motivation? Alexa Skills Challenge $150K in total prizes Bonus prize for Germany Enter til 22th of January 2019 https://goo.gl/EETRu5
  78. 78. 78 / 78 Any questions? ralf@travello.audio https://www.travello.audio

×