As Alexa has grown, how we approach innovation has had to evolve as well. From feature launches to machine learning to managing dependencies, learn how AWS contributes to Alexa’s ability to deliver innovative and delightful experiences and what it takes to build scalable innovation in a service with millions of live customers.

1. S U M M I T
SYDNEY

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Innovating at scale - lessons
learned growing Alexa
Leopold Bushkin
Principal Engineer
Digital Devices & Services, Alexa

3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
What is innovation at scale?
Deliver even as complexity increases.
Expand reach and create new customer value.
Evolve customer experiences without
breaking what works.

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Innovation challenges change as you grow
Scale  Noise / Failures
Team Size  Communication
Tech Debt  Opportunity Cost
Risk Aversion  Feature Velocity
Dependencies  Engineering Velocity

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alexa high level system architecture
Alexa Voice Service
Speech Recognition (ASR)
Language Understanding (NLU)
Skill Arbitration and Ranking (A/R)
RequestOrchestration
1P Skills 3P Interface
Shared Services & Capabilities
3P Skills/APIs
Internal
Systems/APIs
Inbound &
Outbound
Gateway
Offline Modelling &
Model Training

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alexa and innovation at scale
Stories from the world of Alexa...
Cortex – Federating data across Alexa services
Hadron – Rapid experimentation for NLU modelers
SkillsKit – Growth through partnership and extension

8. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How can teams across Alexa collaborate effectively?
Simplifying
Dependencies

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Request processing in Alexa
• Relies on contextual data vended by different services
• Most new features rely on multiple context providers
• Multiple requests to context providers impacts latency
• Changing context data representation impacts consumers

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dependencies accrue over time

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dependencies accrue over time

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dependencies accrue over time

14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Dependencies accrue over time

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Cortex service overview
Query Executor
Provider A
Provider B
Provider C
Provider N
⋮
Provider
Registry
Schema
Registry
{
session(id: "XYZ") {
dialog(id: "ABC") {
frames
}
}
}
{
request(id: $current) {
interpretation(type: Intent,
intentType: PlayMusic)
}
Distributed Dynamic
In-Memory Cache
GraphQL Query API

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Cortex as a federated data decoupling mechanism
• Providers introduced w/o impacting existing consumers
• Provider services can change independent of consumers
• Data from multiple providers retrieved in one round trip
• Schemas help ensure providers/consumers stay in sync
• Distributed cache reduces demand pressure on providers
• Security and access control policy centralised and enforced

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

18. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How can we improve NLU continuously and quickly?
Rapid
Experimentation

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
NLU modelling challenges
• NLU processing pipeline is complex and difficult to modify directly
• Modelling hard to scale when central bottlenecks exist downstream
• Modelling is experimentation – not every experiment succeeds
• Testing models shouldn’t require rebuilding from scratch

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Natural language modelling

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Hadron
Natural language modelling
Hoverboard

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
NLU modelling post Hadron
• Declarative pipeline allows side-by-side evaluation of different models
• Self-service approach accelerated experimentation WEEKS  DAYS
• Automated collection of metrics improves reliability and accuracy
• NLU engineers freed to work on other problems

24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T

25. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
How can we deliver more value to Alexa customers?
Grow Skills
and Capabilities

27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alexa Voice Service
How to grow to 100,000 skills
Speech Recognition (ASR)
Language Understanding (NLU)
Skill Arbitration and Ranking (A/R)
RequestOrchestration
Weather Flash Briefing
Music Smart Home
Video Recipes
Knowledge/QA Shopping
...

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Skills as extension points
Alexa Voice Service
Speech Recognition (ASR)
Language Understanding (NLU)
Skill Arbitration and Ranking (A/R)
RequestOrchestration
Weather Flash Briefing
Music Smart Home
Video Recipes
Knowledge/QA Shopping
...
Directives SPIs Lifecycle

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alexa skills using AWS Lambda
• Substantially simplifies development of new skills
• Allows skill interface to evolve independently of consumers
• Allows Alexa to host skills on behalf of smaller developers; reduces cost
• Allowed developers to use any language supported by AWS Lambda
• Accelerated adoption of ASK; ~70% of skills built on AWS Lambda

30. 80,000+ SKILLS
and growing every day

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staying innovative even as you grow...

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Staying innovative even as you grow...
• Don’t accept dependency jungles – there are always ways to simplify
• Look for bottlenecks in your business processes and eliminate them
• Leverage self-service approaches to enable creativity and independence
• Make it easier for your partners to succeed – find win/win opportunities
• Measure everything and then decide where you need to invest

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Alexa Skills Workshop at AWS Summit Sydney
Time: Thursday, 2 May, 2:45pm – 5:30pm
Location: Workshop Room A, Expo Floor
In this workshop you will learn how to build your first voice skill with Amazon Alexa. We will cover how
to build, test, and publish an Alexa skill and provide tips on how to make your Alexa skill more
engaging to customers with voice user design best practices.
Online Resources
• alexa.design/tutorials
• github.com/alexa
• developer.amazon.com/alexa-skills-kit/anz
• alexa.uservoice.com
• twitch.tv/amazonalexa

35. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Leopold Bushkin
leopoldb@amazon.com