SlideShare a Scribd company logo
AWS re:Invent
It’s All in the Data
T h e M a c h i n e L e a r n i n g b e h i n d A l e x a ’ s A I S y s t e m s
R a j i v R e d d y
N o v e m b e r 2 8 , 2 0 1 7
A L X 3 1 9
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What is Alexa Skills Kit?
Overview
• Alexa under the hood
• Data, data, and more data!
• Building great skills with the right data
• Utterances
• Intents
• Slots
• Synonyms
• Entity resolution
• Invocation name
• Dialog management
Under the hood
What's in there?
User communication with Alexa
Alexa sends
Customer Intent to
Skill
Audio stream is sent up to
Alexa
Skill
processes
Request
User makes
a request
Alexa Identifies Skill &
Recognizes Intent
Through ASR & NLU
User communication with Alexa
Respond to Intent
through Text & Visual
Alexa sends
Customer Intent to
Skill
Audio stream is sent up to
Alexa
Skill
processes
Request
User makes
a request
Alexa Identifies Skill &
Recognizes Intent
Through ASR & NLU
Alexa Converts Text-to-
Speech (TTS) & Renders
Graphical Component
Textual
or Audio
Response
Respond to Intent
through Text & Visual
Alexa sends
Customer Intent to
Skill
Audio stream is sent up to
Alexa
Skill
processes
Request
User makes
a request
Graphical
Response
Alexa Identifies Skill &
Recognizes Intent
Through ASR & NLU
Alexa Converts Text-to-
Speech (TTS) & Renders
Graphical Component
User communication with Alexa
Under the hood
On-Device Processing Cloud Processing
Wake Word
Detection
Automatic
Speech
Recognition
Natural
Language
Understanding
Skill Text to Speech
Signal
Processing
User: “Alexa, ask weather
info if it is hot in Las Vegas?”
Audio
Echo: “The
weather in Las
Vegas is:”Beamformed Signal
Audio
Stream
Speech to Text
Intent: GetWeather; Location=Las Vegas
Location:
Las Vegas,
NV
How is data used in machine learning?
TRAINING
MODELS
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
INFERENCE
How is data used in machine learning?
TRAINING
ASK Data:
• Utterances
• Intents
• Slots
• Synonyms
• Invocation name
• Dialog management
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
INFERENCE
MODELS
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Is there good surfing in Sydney
How is data used in machine learning?
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ASK Data:
• Utterances
• Intents
• Slots
• Synonyms
• Invocation name
• Dialog management
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
MODELS
Utterances
Intents
Slots
Synonyms
Utterances
An utterance is what a user may say to Alexa.
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Is there good surfing in Sydney
Wake Word Launch Invocation Name Connector Word Utterance
Activity Slot City Slot
Utterance
Activity Slot City Slot
Utterances
An utterance is what a user may
say to Alexa.
Intents
• What users ask your skill to do
Slots
• Specify variable parts of an utterance
like a city or an activity
One-Shot: Alexa, ask travel buddy about surfing in Sydney
In Skill: Is there good surfing in Sydney
Wake
Word
Launch Invocation Name Connector
Word
Utterance
Activity Slot City Slot
Utterance
Utterances
Intent Slots
ActivityInfoIntent {activity: “surfing”}
{city: “Sydney”}
Activity Slot City Slot
Utterances
Synonyms
ActivityType:
“Surfing” <- surf, boogie boarding
“Scuba” <- dive, diving, scuba diving
One-Shot: Alexa, ask travel buddy about surfing in Sydney
In Skill: Is there good surfing in Sydney
Wake
Word
Launch Invocation Name Connector
Word
Utterance
Activity Slot City SlotUtterances
Intent Slots
ActivityInfoIntent {activity: “surfing”}
{city: “Sydney”}
Activity Slot City Slot
Utterance
An utterance is what a user may
say to Alexa.
Intents
• What users ask your skill to do
Slots
• Specify variable parts of an utterance
like a city or an activity
Synonyms
• Specify variants to a slot value that
map to the same canonical form
ASK data for machine learning
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Is there good surfing in Sydney
MODELS
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ASK Data:
• Utterances
• Intents
• Slots
• Synonyms
• Invocation name
• Dialog management
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
MODELS
Intent training
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Is there good surfing in Sydney
MODELS
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent good {activity} in {city}
ActivityInfoIntent in {city} how is the {activity}
ActivityInfoIntent is there {activity} in {city}
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
Intent training: Not enough data
ASK user utterance:
In-Skill: Is there good surfing in Sydney
MODELS
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
ActivityInfoIntent {activity} in {city}
BookActivityIntent book {activity} in {city}
Intent: BookActivityIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
Input (Training
Data)
Answers (Truth)
Intent training: Statistical matches
ASK user utterance:
In-Skill: Is there good surfing in Sydney
MODELS
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent good {activity} in {city}
ActivityInfoIntent how is {activity} in {city}
ActivityInfoIntent is there {activity} in {city}
BookActivityIntent book {activity} in {city}
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
^
Intent training: Fixing errors
MODEL
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent good {activity} in {city}
ActivityInfoIntent how is {activity} in {city}
ActivityInfoIntent is there good {activity} in
{city}
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
^
ASK user utterance:
In-Skill: Is there good surfing in Sydney
MODELS
Intent training: Unplanned responses
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent good {activity} in {city}
AMAZON.HelpIntent
AMAZON.CancelIntent
Intent: ActivityInfoIntent
Slots: {city: “Sydney”}
Prompt: What do you want to do next?
ASK user utterance:
In-Skill: Book a trip to Sydney
MODELS
Intent training: Unnecessary connector words
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Surfing in Sydney
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent about {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent good {activity} in {city}
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
MODELS
Intent training: Multiple labels
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent {activity} in Sydney
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
MODELS
Intent training: Prompts matter
PROMPT PROMPT
Which activity and city are you interested in? Is there an activity and city you are interested in?
Intent training: Prompts matter
PROMPT PROMPT
USER
Diving in Seattle
Interested in diving in Sydney
How is surfing in San Diego
USER
Yes, diving in Seattle
Yes, I am interested in diving in Sydney
Yes, how is surfing in San Diego
No
Which activity and city are you interested in? Is there an activity and city you are interested in?
Intent training: Prompts matter
PROMPT
Which activity and city are you interested in?
PROMPT
Is there an activity and city you are interested in?
USER
Diving in Seattle
Interested in diving in Sydney
How is surfing in San Diego
USER
Yes, diving in Seattle
Yes, I am interested in diving in Sydney
Yes, how is surfing in San Diego
No
TRAINING
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent how is {activity} in {city}
Intent training: Prompts matter
PROMPT
Which activity and city are you interested in?
PROMPT
Is there an activity and city you are interested in?
USER
Diving in Seattle
Interested in diving in Sydney
How is surfing in San Diego
USER
Yes, diving in Seattle
Yes, I am interested in diving in Sydney
Yes, how is surfing in San Diego
No
TRAINING
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent how is {activity} in {city}
TRAINING
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent how is {activity} in {city}
ActivityInfoIntent yes {activity} in {city}
ActivityInfoIntent yes interested in {activity} in {city}
ActivityInfoIntent yes how is {activity} in {city}
AMAZON.YesIntent
AMAZON.NoIntent
Intent training: Prompts matter
PROMPT
Which activity and city are you interested in?
PROMPT
Is there an activity and city you are interested in?
USER
Diving in Seattle
Interested in diving in Sydney
How is surfing in San Diego
USER
Yes, diving in Seattle
Yes, I am interested in diving in Sydney
Yes, how is surfing in San Diego
No
TRAINING
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent how is {activity} in {city}
ActivityInfoIntent interested in {activity} in {city}
ActivityInfoIntent interested in {activity}
TRAINING
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent activities in {city}
ActivityInfoIntent how is {activity} in {city}
ActivityInfoIntent yes {activity} in {city}
ActivityInfoIntent yes interested in {activity} in {city}
ActivityInfoIntent yes how is {activity} in {city}
AMAZON.YesIntent
AMAZON.NoIntent
How does Alexa hear and understand you?
COMPONENT INPUT OUTPUT EXAMPLE
Automatic Speech
Recognition (ASR)
Speech Text “Is there good diving in Sydney”
Natural Language
Understanding (NLU)
- Entity Resolution (ER)
Text Intent and Slot labels
Intent/slot classification of input tokens:
Intent: ActivityInfoIntent
Slots: Activity: diving -> ER -> Scuba
City: Sydney -> ER -> SYD
Skill
Labels and
Context
Dialog Actions/Prompts
Ask the application to provide information on
the activity for the specified location
Text-to-Speech (TTS) Text Speech
“The Sydney surf season is best during…” or
“in which city?” if the user did not specify
Dialog
Manager
ASR NLU TTS
User
Speech
Text
Intents/
Slots
Actions/
Prompts
Speech
Output
How does skill data get used in Alexa?
ASR
Text
Intents/
Slots
Skill-specific
Language
Model
General
Language
Model
NLU
Deterministic
(exact match)
Model
Statistical slot
and intent
models
Entity
Resolution /
Slot
Resolution
Acoustic
Model
Slot training: Coverage
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Playa Grande
In-Skill: Is there good surfing in Playa Grande
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Activity:
Diving
Hiking
Skiing
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Playa Grande”}
City:
AMAZON.US_CITY
MODELS
Slot training: Coverage
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Playa Grande
In-Skill: Is there good surfing in Playa Grande
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Activity:
Diving
Hiking
Skiing
Surfing
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Playa Grande”}
City:
AMAZON.US_CITY
Playa Grande
MODELS
Slot training: Coverage
How do you get good coverage?
• Built-ins
• Find proxies for high frequency data
• Sales figures
• Census data
• Data from other applications (website, cell phone app, chat bot, and so on)
• For extensions of built-ins
• Bug fix
• Or comprehensive data sets
Slot training: Extending built-ins
ASK user utterance:
In-Skill: Send a message to Daniel
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Name:
AMAZON.US_FIRST_NAME
Daneal
Intent: MessageIntent
Slots: {name: “Daneal”}
MODELS
Slot training: Error handling
ASK user utterance:
One-shot: Alexa, ask travel buddy about sharks in San Diego
In-Skill: Is there good sharks in San Diego
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Activity:
Diving
Hiking
Skiing
Surfing
Intent: ActivityInfoIntent
Slots: {activity: “sharks”}
{city: “San Diego”}
City:
AMAZON.US_CITY
MODELS
Slot training: Different slot same values
ASK user utterance:
One-shot: Alexa, ask travel buddy about surfing in Sydney
In-Skill: Is there good surfing in Sydney
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Activity:
Diving
Hiking
Skiing
Surfing
Intent: ActivityInfoIntent
Slots: {activity: “surfing”}
{city: “Sydney”}
Water Activity:
Diving
Surfing
ActivityInfoIntent {activity} in {city}
ActivityInfoIntent {water_activity} in {city}
MODELS
Slot training: Data pre-processing
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Slot:
PJ Harvey
MotorBiking
Example Corp.
R&B
Fire HD 7
Spoken-form:
p. j. harvey
motor biking
Example corporation
r. and b.
fire h. d. seven
MODELS
Slot training: Using entity resolution
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Airport Slot:
Seattle
Las Vegas
London
Tokyo
Synonyms:
Seatac, s. e. a.
l. a. s.
Heathrow, l. h. r.
t. y. o.
ID:
SEA
LAS
LHR
TYO
Canonical slot:
Seattle
Las Vegas
London
Tokyo
ID:
SEA
LAS
LHR
TYO
Input:
Seatac
Las Vegas
Heathrow
t. y. o.
MODELS
Slot training: Slot anchor words
ASK user utterance:
In-Skill: Book travel on twenty fifth May
at seven pm from Seattle to Las Vegas
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Intent:
I want to travel on {Date} at {TIME} from
{FromCity} to {ToCity}
Intent: TravelIntent
Slots: {Date: “05-25-2018”}
{Time: “19:00”}
{FromCity: “Seattle”}
{ToCity: “Las Vegas”}
MODELS
Slot training: Utterance slot context words
ASK user utterance:
In-Skill: Book travel to twenty fifth may
from seven pm at Seattle on Las Vegas
MODEL
S
TRAINING
INFERENCE
DECODERInput Answers
TRAINER
Input (Training
Data)
Answers (Truth)
Intent:
I want to travel {Connector} {Date} {Connector}
{TIME} {Connector} {FromCity} {Connector}
{ToCity}
Slots:
Connector = on | at | from | in
Intent: TravelIntent
Slots: {FromCity: “7 pm”}
{ToCity: “25th May”}
MODELS
Utterances: Summary
• Test, test, test!
• Increase coverage of existing intents and slots
• Iterate on prompts to get less ambiguous responses
• Update new intents/slots to match responses to prompts
• Create new intents/slots for unhandled functionality for graceful error messages
• Limit generation of unrealistic utterances due to multiple consecutive slots
• Use synonyms and entity resolution
Invocation names
Invocation name: Key elements
Pick an invocation name that is:
• Easy to remember
• Relates to your skill
• Natural for users to say with invocation patterns
Alexa, ask travel buddy about surfing in Sydney
Wake Word Launch Invocation Name Connector Word Utterance
Activity Slot City Slot
Invocation name: Requirements
Requirements to provide users a better experience with Alexa
• Not confusable with common Alexa commands like the built-in “weather” commands.
• No names or places (for example, “molly,” “Seattle”)
• No one-word invocation names unless unique to your brand or IP
• No two-word names with definite articles (“the”), indefinite article (“a,” “an”) or
preposition (“for,” “to,” “of”). For example, “a bicycle,” “an espresso,” “to amuse,”
“for fun.”
Invocation name: Requirements
Requirements for Alexa to hears your invocation name correctly
• Easy to pronounce correctly and phonetically distinct to avoid being misinterpreted
as other, similar-sounding words (for example, avoid “alveolar sounds”)
• Lower-case alphabetic characters (“travel buddy” instead of “Travel Buddy”)
• Spaces between words (for example, “editor in chief” instead of “editor-in-chief”)
• Possessive apostrophes (for example, “sam’s science trivia”)
• Periods in abbreviations (for example, “a. b. c.”)
• Numbers must be spelled out (for example, “twenty one” instead of “21”)
Invocation name: Testing
• Test, test, test!
• Try various invocation patterns like:
• Open <invocation name>
• Launch <invocation name>
• Tell <invocation name> to do <utterance>
• Ask <invocation name> about <utterance>
• Example utterances you provided in skill
• Ask others to try it out (beta testing, pizza party…)
• Check if:
• Your invocation name is natural for users to use
• Alexa is recognizing and invoking your skill often (invocation accuracy)
• You can see how Alexa interpreted your invocation name by reviewing the history
in the Amazon Alexa App (in the app, navigate to Settings and then History).
Alexa, ask travel buddy about surfing in Sydney
Wake
Word
Launch Invocation Name Connector
Word
Utterance
City SlotActivity Slot
Invocation name: Some insights
• Alexa hears your invocation name more accurately with supported sample utterances
• More likely to misrecognize uncommon words for invocation names
• For invocation issues
• Increase coverage of sample utterances
• Consider another invocation name
• Publish with preferred invocation name if at least medium accuracy (>70%)
• Active learning
Alexa, ask travel buddy about surfing in Sydney
Wake
Word
Launch Invocation Name Connector
Word
Utterance
Activity Slot City Slot
Dialog management
Key features
• Slot filling
• Slot confirmation
• Intent confirmation
Advantages
• Simplifies collecting and confirming slot values and intents
• Improved accuracy in slot filling because Alexa now has context as to which slot you are filling
• Use it for a better-performing skill!
Dialog management
Summary
Insights
Test your experience with real users!
Pick an invocation name that users remember and Alexa understands well.
Add more data for slot values and utterance patterns that customers actually use.
Watch out for data pre-processing issues in your slots data.
Use entity resolution for all your slots.
Use dialog management features for handling slot filling.
Use alexa.design/guide
Test your experience with real users!
Thank you!
R a j i v R e d d y
A L X 3 1 9

More Related Content

What's hot

Amazon Alexa: our successes and fails
Amazon Alexa: our successes and failsAmazon Alexa: our successes and fails
Amazon Alexa: our successes and failsVyacheslav Lyalkin
 
Alexa in 10 minutes
Alexa in 10 minutesAlexa in 10 minutes
Alexa in 10 minutes
James Burt
 
Develop Alexa Skills for Amazon Echo with PHP
Develop Alexa Skills for Amazon Echo with PHPDevelop Alexa Skills for Amazon Echo with PHP
Develop Alexa Skills for Amazon Echo with PHP
Ralf Eggert
 
Alexa bootcamp - Skill building 101
Alexa bootcamp - Skill building 101Alexa bootcamp - Skill building 101
Alexa bootcamp - Skill building 101
Amazon Web Services
 
ALX402_Oh No, I Got Featured
ALX402_Oh No, I Got FeaturedALX402_Oh No, I Got Featured
ALX402_Oh No, I Got Featured
Amazon Web Services
 
Amazon Echo
Amazon EchoAmazon Echo
Amazon Echo
antimo musone
 
Building Voice Apps & Experiences For Amazon Echo
Building Voice Apps & Experiences For Amazon EchoBuilding Voice Apps & Experiences For Amazon Echo
Building Voice Apps & Experiences For Amazon Echo
Amazon Appstore Developers
 
Amazon Alexa Workshop
Amazon Alexa WorkshopAmazon Alexa Workshop
Amazon Alexa Workshop
Amazon Web Services
 
How to create a Voice – Enabled IoT solution for Alexa
How to create a Voice – Enabled IoT solution for AlexaHow to create a Voice – Enabled IoT solution for Alexa
How to create a Voice – Enabled IoT solution for Alexa
Amazon Web Services
 
FSI300 Bringing the Brains Behind Alexa to Financial Services
FSI300 Bringing the Brains Behind Alexa to Financial ServicesFSI300 Bringing the Brains Behind Alexa to Financial Services
FSI300 Bringing the Brains Behind Alexa to Financial Services
Amazon Web Services
 
Introduction to building alexa skills and putting your amazon echo to work
Introduction to building alexa skills and putting your amazon echo to workIntroduction to building alexa skills and putting your amazon echo to work
Introduction to building alexa skills and putting your amazon echo to work
Abe Diaz
 
Enabling New Voice Experiences with Amazon Alexa and AWS Lambda
Enabling New Voice Experiences with Amazon Alexa and AWS LambdaEnabling New Voice Experiences with Amazon Alexa and AWS Lambda
Enabling New Voice Experiences with Amazon Alexa and AWS Lambda
Amazon Web Services
 
Please meet Amazon Alexa and the Alexa Skills Kit
Please meet Amazon Alexa and the Alexa Skills KitPlease meet Amazon Alexa and the Alexa Skills Kit
Please meet Amazon Alexa and the Alexa Skills Kit
Amazon Web Services
 
Amazon Alexa Development Overview
Amazon Alexa Development OverviewAmazon Alexa Development Overview
Amazon Alexa Development Overview
John Brady
 
Amazon Alexa and AWS Lambda
Amazon Alexa and AWS LambdaAmazon Alexa and AWS Lambda
Amazon Alexa and AWS Lambda
Amazon Web Services
 
Building Boston Rail - An Alexa Skill
Building Boston Rail - An Alexa SkillBuilding Boston Rail - An Alexa Skill
Building Boston Rail - An Alexa SkillCharles J Christina
 
An Introduction to Using AWS and ASK to Build Voice Driven Experiences
An Introduction to Using AWS and ASK to Build Voice Driven ExperiencesAn Introduction to Using AWS and ASK to Build Voice Driven Experiences
An Introduction to Using AWS and ASK to Build Voice Driven Experiences
Amazon Web Services
 
Amazon Alexa
Amazon AlexaAmazon Alexa
Amazon Alexa
Vishal Prajapati
 
Intro to Alexa skills development
Intro to Alexa skills developmentIntro to Alexa skills development
Intro to Alexa skills development
Sahil Khosla
 
Alexa Smart Home Skill
Alexa Smart Home SkillAlexa Smart Home Skill
Alexa Smart Home Skill
Jun Ichikawa
 

What's hot (20)

Amazon Alexa: our successes and fails
Amazon Alexa: our successes and failsAmazon Alexa: our successes and fails
Amazon Alexa: our successes and fails
 
Alexa in 10 minutes
Alexa in 10 minutesAlexa in 10 minutes
Alexa in 10 minutes
 
Develop Alexa Skills for Amazon Echo with PHP
Develop Alexa Skills for Amazon Echo with PHPDevelop Alexa Skills for Amazon Echo with PHP
Develop Alexa Skills for Amazon Echo with PHP
 
Alexa bootcamp - Skill building 101
Alexa bootcamp - Skill building 101Alexa bootcamp - Skill building 101
Alexa bootcamp - Skill building 101
 
ALX402_Oh No, I Got Featured
ALX402_Oh No, I Got FeaturedALX402_Oh No, I Got Featured
ALX402_Oh No, I Got Featured
 
Amazon Echo
Amazon EchoAmazon Echo
Amazon Echo
 
Building Voice Apps & Experiences For Amazon Echo
Building Voice Apps & Experiences For Amazon EchoBuilding Voice Apps & Experiences For Amazon Echo
Building Voice Apps & Experiences For Amazon Echo
 
Amazon Alexa Workshop
Amazon Alexa WorkshopAmazon Alexa Workshop
Amazon Alexa Workshop
 
How to create a Voice – Enabled IoT solution for Alexa
How to create a Voice – Enabled IoT solution for AlexaHow to create a Voice – Enabled IoT solution for Alexa
How to create a Voice – Enabled IoT solution for Alexa
 
FSI300 Bringing the Brains Behind Alexa to Financial Services
FSI300 Bringing the Brains Behind Alexa to Financial ServicesFSI300 Bringing the Brains Behind Alexa to Financial Services
FSI300 Bringing the Brains Behind Alexa to Financial Services
 
Introduction to building alexa skills and putting your amazon echo to work
Introduction to building alexa skills and putting your amazon echo to workIntroduction to building alexa skills and putting your amazon echo to work
Introduction to building alexa skills and putting your amazon echo to work
 
Enabling New Voice Experiences with Amazon Alexa and AWS Lambda
Enabling New Voice Experiences with Amazon Alexa and AWS LambdaEnabling New Voice Experiences with Amazon Alexa and AWS Lambda
Enabling New Voice Experiences with Amazon Alexa and AWS Lambda
 
Please meet Amazon Alexa and the Alexa Skills Kit
Please meet Amazon Alexa and the Alexa Skills KitPlease meet Amazon Alexa and the Alexa Skills Kit
Please meet Amazon Alexa and the Alexa Skills Kit
 
Amazon Alexa Development Overview
Amazon Alexa Development OverviewAmazon Alexa Development Overview
Amazon Alexa Development Overview
 
Amazon Alexa and AWS Lambda
Amazon Alexa and AWS LambdaAmazon Alexa and AWS Lambda
Amazon Alexa and AWS Lambda
 
Building Boston Rail - An Alexa Skill
Building Boston Rail - An Alexa SkillBuilding Boston Rail - An Alexa Skill
Building Boston Rail - An Alexa Skill
 
An Introduction to Using AWS and ASK to Build Voice Driven Experiences
An Introduction to Using AWS and ASK to Build Voice Driven ExperiencesAn Introduction to Using AWS and ASK to Build Voice Driven Experiences
An Introduction to Using AWS and ASK to Build Voice Driven Experiences
 
Amazon Alexa
Amazon AlexaAmazon Alexa
Amazon Alexa
 
Intro to Alexa skills development
Intro to Alexa skills developmentIntro to Alexa skills development
Intro to Alexa skills development
 
Alexa Smart Home Skill
Alexa Smart Home SkillAlexa Smart Home Skill
Alexa Smart Home Skill
 

Similar to ALX319_It’s All in the Data

Voice enable all the things with Alexa
Voice enable all the things with AlexaVoice enable all the things with Alexa
Voice enable all the things with Alexa
Mark Bate
 
Developing alexa Skill using Java With AWS Lambda
Developing alexa Skill using Java With AWS LambdaDeveloping alexa Skill using Java With AWS Lambda
Developing alexa Skill using Java With AWS Lambda
Buddha Jyothiprasad
 
WKS403 Build an Alexa Skill using AWS Lambda
WKS403   Build an Alexa Skill using AWS LambdaWKS403   Build an Alexa Skill using AWS Lambda
WKS403 Build an Alexa Skill using AWS Lambda
Amazon Web Services
 
Building complex UI on Android
Building complex UI on AndroidBuilding complex UI on Android
Building complex UI on Android
Maciej Witowski
 
TrailheaDX Global Gathering Developer track
TrailheaDX Global Gathering Developer trackTrailheaDX Global Gathering Developer track
TrailheaDX Global Gathering Developer track
RajaKakayadi
 
IT Camp 2019: How to build your first Alexa skill in under one hour
IT Camp 2019: How to build your first Alexa skill in under one hourIT Camp 2019: How to build your first Alexa skill in under one hour
IT Camp 2019: How to build your first Alexa skill in under one hour
Ionut Balan
 
Darin Briskman_Amazon_June_9_2017_Presentation
Darin Briskman_Amazon_June_9_2017_PresentationDarin Briskman_Amazon_June_9_2017_Presentation
Darin Briskman_Amazon_June_9_2017_Presentation
TriNimbus
 
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
Amazon Web Services
 
SVC101 Building Search into Your App - AWS re: Invent 2012
SVC101 Building Search into Your App - AWS re: Invent 2012SVC101 Building Search into Your App - AWS re: Invent 2012
SVC101 Building Search into Your App - AWS re: Invent 2012
Amazon Web Services
 
Microservices, Events, and Breaking the Data Monolith with Kafka
Microservices, Events, and Breaking the Data Monolith with KafkaMicroservices, Events, and Breaking the Data Monolith with Kafka
Microservices, Events, and Breaking the Data Monolith with Kafka
VMware Tanzu
 
Data Visualization Strategies & Open Source Tools
Data Visualization Strategies & Open Source ToolsData Visualization Strategies & Open Source Tools
Data Visualization Strategies & Open Source Tools
Phase2
 
Amazon Alexa workshop
Amazon Alexa workshopAmazon Alexa workshop
Amazon Alexa workshop
Maha Ali
 
Presentation1219
Presentation1219Presentation1219
Presentation1219
joyousone1104
 
Reinventing the Transaction Script (NDC London 2020)
Reinventing the Transaction Script (NDC London 2020)Reinventing the Transaction Script (NDC London 2020)
Reinventing the Transaction Script (NDC London 2020)
Scott Wlaschin
 
Speed Grooming Requirements with SAFe
Speed Grooming Requirements with SAFeSpeed Grooming Requirements with SAFe
Speed Grooming Requirements with SAFe
TechWell
 
Behavior Driven Development with Cucumber
Behavior Driven Development with CucumberBehavior Driven Development with Cucumber
Behavior Driven Development with Cucumber
Asheesh Mehdiratta
 
Yesplan: 10 Years later
Yesplan: 10 Years laterYesplan: 10 Years later
Yesplan: 10 Years later
Pharo
 
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
HostedbyConfluent
 
Chat Bots with Amazon Lex - AWS Summit SG 2017
Chat Bots with Amazon Lex  - AWS Summit SG 2017Chat Bots with Amazon Lex  - AWS Summit SG 2017
Chat Bots with Amazon Lex - AWS Summit SG 2017
Amazon Web Services
 
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , KongAPIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
apidays
 

Similar to ALX319_It’s All in the Data (20)

Voice enable all the things with Alexa
Voice enable all the things with AlexaVoice enable all the things with Alexa
Voice enable all the things with Alexa
 
Developing alexa Skill using Java With AWS Lambda
Developing alexa Skill using Java With AWS LambdaDeveloping alexa Skill using Java With AWS Lambda
Developing alexa Skill using Java With AWS Lambda
 
WKS403 Build an Alexa Skill using AWS Lambda
WKS403   Build an Alexa Skill using AWS LambdaWKS403   Build an Alexa Skill using AWS Lambda
WKS403 Build an Alexa Skill using AWS Lambda
 
Building complex UI on Android
Building complex UI on AndroidBuilding complex UI on Android
Building complex UI on Android
 
TrailheaDX Global Gathering Developer track
TrailheaDX Global Gathering Developer trackTrailheaDX Global Gathering Developer track
TrailheaDX Global Gathering Developer track
 
IT Camp 2019: How to build your first Alexa skill in under one hour
IT Camp 2019: How to build your first Alexa skill in under one hourIT Camp 2019: How to build your first Alexa skill in under one hour
IT Camp 2019: How to build your first Alexa skill in under one hour
 
Darin Briskman_Amazon_June_9_2017_Presentation
Darin Briskman_Amazon_June_9_2017_PresentationDarin Briskman_Amazon_June_9_2017_Presentation
Darin Briskman_Amazon_June_9_2017_Presentation
 
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
Activision Blizzard: Giving Call of Duty Gamers an Edge with Alexa and AWS (G...
 
SVC101 Building Search into Your App - AWS re: Invent 2012
SVC101 Building Search into Your App - AWS re: Invent 2012SVC101 Building Search into Your App - AWS re: Invent 2012
SVC101 Building Search into Your App - AWS re: Invent 2012
 
Microservices, Events, and Breaking the Data Monolith with Kafka
Microservices, Events, and Breaking the Data Monolith with KafkaMicroservices, Events, and Breaking the Data Monolith with Kafka
Microservices, Events, and Breaking the Data Monolith with Kafka
 
Data Visualization Strategies & Open Source Tools
Data Visualization Strategies & Open Source ToolsData Visualization Strategies & Open Source Tools
Data Visualization Strategies & Open Source Tools
 
Amazon Alexa workshop
Amazon Alexa workshopAmazon Alexa workshop
Amazon Alexa workshop
 
Presentation1219
Presentation1219Presentation1219
Presentation1219
 
Reinventing the Transaction Script (NDC London 2020)
Reinventing the Transaction Script (NDC London 2020)Reinventing the Transaction Script (NDC London 2020)
Reinventing the Transaction Script (NDC London 2020)
 
Speed Grooming Requirements with SAFe
Speed Grooming Requirements with SAFeSpeed Grooming Requirements with SAFe
Speed Grooming Requirements with SAFe
 
Behavior Driven Development with Cucumber
Behavior Driven Development with CucumberBehavior Driven Development with Cucumber
Behavior Driven Development with Cucumber
 
Yesplan: 10 Years later
Yesplan: 10 Years laterYesplan: 10 Years later
Yesplan: 10 Years later
 
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
You Put *What* in Your Stream?! Patterns and Practices for Event Design with ...
 
Chat Bots with Amazon Lex - AWS Summit SG 2017
Chat Bots with Amazon Lex  - AWS Summit SG 2017Chat Bots with Amazon Lex  - AWS Summit SG 2017
Chat Bots with Amazon Lex - AWS Summit SG 2017
 
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , KongAPIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
APIdays Paris 2019 - Adopting Service Mesh by Marco Palladino , Kong
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
Amazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

ALX319_It’s All in the Data

  • 1. AWS re:Invent It’s All in the Data T h e M a c h i n e L e a r n i n g b e h i n d A l e x a ’ s A I S y s t e m s R a j i v R e d d y N o v e m b e r 2 8 , 2 0 1 7 A L X 3 1 9 © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  • 2.
  • 3. What is Alexa Skills Kit?
  • 4. Overview • Alexa under the hood • Data, data, and more data! • Building great skills with the right data • Utterances • Intents • Slots • Synonyms • Entity resolution • Invocation name • Dialog management
  • 7. User communication with Alexa Alexa sends Customer Intent to Skill Audio stream is sent up to Alexa Skill processes Request User makes a request Alexa Identifies Skill & Recognizes Intent Through ASR & NLU
  • 8. User communication with Alexa Respond to Intent through Text & Visual Alexa sends Customer Intent to Skill Audio stream is sent up to Alexa Skill processes Request User makes a request Alexa Identifies Skill & Recognizes Intent Through ASR & NLU Alexa Converts Text-to- Speech (TTS) & Renders Graphical Component
  • 9. Textual or Audio Response Respond to Intent through Text & Visual Alexa sends Customer Intent to Skill Audio stream is sent up to Alexa Skill processes Request User makes a request Graphical Response Alexa Identifies Skill & Recognizes Intent Through ASR & NLU Alexa Converts Text-to- Speech (TTS) & Renders Graphical Component User communication with Alexa
  • 10. Under the hood On-Device Processing Cloud Processing Wake Word Detection Automatic Speech Recognition Natural Language Understanding Skill Text to Speech Signal Processing User: “Alexa, ask weather info if it is hot in Las Vegas?” Audio Echo: “The weather in Las Vegas is:”Beamformed Signal Audio Stream Speech to Text Intent: GetWeather; Location=Las Vegas Location: Las Vegas, NV
  • 11. How is data used in machine learning? TRAINING MODELS DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) INFERENCE
  • 12. How is data used in machine learning? TRAINING ASK Data: • Utterances • Intents • Slots • Synonyms • Invocation name • Dialog management DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) INFERENCE MODELS
  • 13. ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Is there good surfing in Sydney How is data used in machine learning? TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ASK Data: • Utterances • Intents • Slots • Synonyms • Invocation name • Dialog management Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} MODELS
  • 15. Utterances An utterance is what a user may say to Alexa. One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Is there good surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance Activity Slot City Slot Utterance Activity Slot City Slot
  • 16. Utterances An utterance is what a user may say to Alexa. Intents • What users ask your skill to do Slots • Specify variable parts of an utterance like a city or an activity One-Shot: Alexa, ask travel buddy about surfing in Sydney In Skill: Is there good surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance Activity Slot City Slot Utterance Utterances Intent Slots ActivityInfoIntent {activity: “surfing”} {city: “Sydney”} Activity Slot City Slot
  • 17. Utterances Synonyms ActivityType: “Surfing” <- surf, boogie boarding “Scuba” <- dive, diving, scuba diving One-Shot: Alexa, ask travel buddy about surfing in Sydney In Skill: Is there good surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance Activity Slot City SlotUtterances Intent Slots ActivityInfoIntent {activity: “surfing”} {city: “Sydney”} Activity Slot City Slot Utterance An utterance is what a user may say to Alexa. Intents • What users ask your skill to do Slots • Specify variable parts of an utterance like a city or an activity Synonyms • Specify variants to a slot value that map to the same canonical form
  • 18. ASK data for machine learning ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Is there good surfing in Sydney MODELS TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ASK Data: • Utterances • Intents • Slots • Synonyms • Invocation name • Dialog management Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} MODELS
  • 19. Intent training ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Is there good surfing in Sydney MODELS TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent good {activity} in {city} ActivityInfoIntent in {city} how is the {activity} ActivityInfoIntent is there {activity} in {city} Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”}
  • 20. Intent training: Not enough data ASK user utterance: In-Skill: Is there good surfing in Sydney MODELS TRAINING INFERENCE DECODERInput Answers TRAINER ActivityInfoIntent {activity} in {city} BookActivityIntent book {activity} in {city} Intent: BookActivityIntent Slots: {activity: “surfing”} {city: “Sydney”} Input (Training Data) Answers (Truth)
  • 21. Intent training: Statistical matches ASK user utterance: In-Skill: Is there good surfing in Sydney MODELS TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent good {activity} in {city} ActivityInfoIntent how is {activity} in {city} ActivityInfoIntent is there {activity} in {city} BookActivityIntent book {activity} in {city} Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} ^
  • 22. Intent training: Fixing errors MODEL TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent good {activity} in {city} ActivityInfoIntent how is {activity} in {city} ActivityInfoIntent is there good {activity} in {city} Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} ^ ASK user utterance: In-Skill: Is there good surfing in Sydney MODELS
  • 23. Intent training: Unplanned responses MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent good {activity} in {city} AMAZON.HelpIntent AMAZON.CancelIntent Intent: ActivityInfoIntent Slots: {city: “Sydney”} Prompt: What do you want to do next? ASK user utterance: In-Skill: Book a trip to Sydney MODELS
  • 24. Intent training: Unnecessary connector words ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Surfing in Sydney MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent about {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent good {activity} in {city} Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} MODELS
  • 25. Intent training: Multiple labels ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) ActivityInfoIntent {activity} in {city} ActivityInfoIntent {activity} in Sydney Intent: ActivityInfoIntent Slots: {activity: “surfing”} MODELS
  • 26. Intent training: Prompts matter PROMPT PROMPT Which activity and city are you interested in? Is there an activity and city you are interested in?
  • 27. Intent training: Prompts matter PROMPT PROMPT USER Diving in Seattle Interested in diving in Sydney How is surfing in San Diego USER Yes, diving in Seattle Yes, I am interested in diving in Sydney Yes, how is surfing in San Diego No Which activity and city are you interested in? Is there an activity and city you are interested in?
  • 28. Intent training: Prompts matter PROMPT Which activity and city are you interested in? PROMPT Is there an activity and city you are interested in? USER Diving in Seattle Interested in diving in Sydney How is surfing in San Diego USER Yes, diving in Seattle Yes, I am interested in diving in Sydney Yes, how is surfing in San Diego No TRAINING ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent how is {activity} in {city}
  • 29. Intent training: Prompts matter PROMPT Which activity and city are you interested in? PROMPT Is there an activity and city you are interested in? USER Diving in Seattle Interested in diving in Sydney How is surfing in San Diego USER Yes, diving in Seattle Yes, I am interested in diving in Sydney Yes, how is surfing in San Diego No TRAINING ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent how is {activity} in {city} TRAINING ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent how is {activity} in {city} ActivityInfoIntent yes {activity} in {city} ActivityInfoIntent yes interested in {activity} in {city} ActivityInfoIntent yes how is {activity} in {city} AMAZON.YesIntent AMAZON.NoIntent
  • 30. Intent training: Prompts matter PROMPT Which activity and city are you interested in? PROMPT Is there an activity and city you are interested in? USER Diving in Seattle Interested in diving in Sydney How is surfing in San Diego USER Yes, diving in Seattle Yes, I am interested in diving in Sydney Yes, how is surfing in San Diego No TRAINING ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent how is {activity} in {city} ActivityInfoIntent interested in {activity} in {city} ActivityInfoIntent interested in {activity} TRAINING ActivityInfoIntent {activity} in {city} ActivityInfoIntent activities in {city} ActivityInfoIntent how is {activity} in {city} ActivityInfoIntent yes {activity} in {city} ActivityInfoIntent yes interested in {activity} in {city} ActivityInfoIntent yes how is {activity} in {city} AMAZON.YesIntent AMAZON.NoIntent
  • 31. How does Alexa hear and understand you? COMPONENT INPUT OUTPUT EXAMPLE Automatic Speech Recognition (ASR) Speech Text “Is there good diving in Sydney” Natural Language Understanding (NLU) - Entity Resolution (ER) Text Intent and Slot labels Intent/slot classification of input tokens: Intent: ActivityInfoIntent Slots: Activity: diving -> ER -> Scuba City: Sydney -> ER -> SYD Skill Labels and Context Dialog Actions/Prompts Ask the application to provide information on the activity for the specified location Text-to-Speech (TTS) Text Speech “The Sydney surf season is best during…” or “in which city?” if the user did not specify Dialog Manager ASR NLU TTS User Speech Text Intents/ Slots Actions/ Prompts Speech Output
  • 32. How does skill data get used in Alexa? ASR Text Intents/ Slots Skill-specific Language Model General Language Model NLU Deterministic (exact match) Model Statistical slot and intent models Entity Resolution / Slot Resolution Acoustic Model
  • 33. Slot training: Coverage ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Playa Grande In-Skill: Is there good surfing in Playa Grande MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Activity: Diving Hiking Skiing Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Playa Grande”} City: AMAZON.US_CITY MODELS
  • 34. Slot training: Coverage ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Playa Grande In-Skill: Is there good surfing in Playa Grande MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Activity: Diving Hiking Skiing Surfing Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Playa Grande”} City: AMAZON.US_CITY Playa Grande MODELS
  • 35. Slot training: Coverage How do you get good coverage? • Built-ins • Find proxies for high frequency data • Sales figures • Census data • Data from other applications (website, cell phone app, chat bot, and so on) • For extensions of built-ins • Bug fix • Or comprehensive data sets
  • 36. Slot training: Extending built-ins ASK user utterance: In-Skill: Send a message to Daniel MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Name: AMAZON.US_FIRST_NAME Daneal Intent: MessageIntent Slots: {name: “Daneal”} MODELS
  • 37. Slot training: Error handling ASK user utterance: One-shot: Alexa, ask travel buddy about sharks in San Diego In-Skill: Is there good sharks in San Diego MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Activity: Diving Hiking Skiing Surfing Intent: ActivityInfoIntent Slots: {activity: “sharks”} {city: “San Diego”} City: AMAZON.US_CITY MODELS
  • 38. Slot training: Different slot same values ASK user utterance: One-shot: Alexa, ask travel buddy about surfing in Sydney In-Skill: Is there good surfing in Sydney MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Activity: Diving Hiking Skiing Surfing Intent: ActivityInfoIntent Slots: {activity: “surfing”} {city: “Sydney”} Water Activity: Diving Surfing ActivityInfoIntent {activity} in {city} ActivityInfoIntent {water_activity} in {city} MODELS
  • 39. Slot training: Data pre-processing MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Slot: PJ Harvey MotorBiking Example Corp. R&B Fire HD 7 Spoken-form: p. j. harvey motor biking Example corporation r. and b. fire h. d. seven MODELS
  • 40. Slot training: Using entity resolution MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Airport Slot: Seattle Las Vegas London Tokyo Synonyms: Seatac, s. e. a. l. a. s. Heathrow, l. h. r. t. y. o. ID: SEA LAS LHR TYO Canonical slot: Seattle Las Vegas London Tokyo ID: SEA LAS LHR TYO Input: Seatac Las Vegas Heathrow t. y. o. MODELS
  • 41. Slot training: Slot anchor words ASK user utterance: In-Skill: Book travel on twenty fifth May at seven pm from Seattle to Las Vegas MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Intent: I want to travel on {Date} at {TIME} from {FromCity} to {ToCity} Intent: TravelIntent Slots: {Date: “05-25-2018”} {Time: “19:00”} {FromCity: “Seattle”} {ToCity: “Las Vegas”} MODELS
  • 42. Slot training: Utterance slot context words ASK user utterance: In-Skill: Book travel to twenty fifth may from seven pm at Seattle on Las Vegas MODEL S TRAINING INFERENCE DECODERInput Answers TRAINER Input (Training Data) Answers (Truth) Intent: I want to travel {Connector} {Date} {Connector} {TIME} {Connector} {FromCity} {Connector} {ToCity} Slots: Connector = on | at | from | in Intent: TravelIntent Slots: {FromCity: “7 pm”} {ToCity: “25th May”} MODELS
  • 43. Utterances: Summary • Test, test, test! • Increase coverage of existing intents and slots • Iterate on prompts to get less ambiguous responses • Update new intents/slots to match responses to prompts • Create new intents/slots for unhandled functionality for graceful error messages • Limit generation of unrealistic utterances due to multiple consecutive slots • Use synonyms and entity resolution
  • 45. Invocation name: Key elements Pick an invocation name that is: • Easy to remember • Relates to your skill • Natural for users to say with invocation patterns Alexa, ask travel buddy about surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance Activity Slot City Slot
  • 46. Invocation name: Requirements Requirements to provide users a better experience with Alexa • Not confusable with common Alexa commands like the built-in “weather” commands. • No names or places (for example, “molly,” “Seattle”) • No one-word invocation names unless unique to your brand or IP • No two-word names with definite articles (“the”), indefinite article (“a,” “an”) or preposition (“for,” “to,” “of”). For example, “a bicycle,” “an espresso,” “to amuse,” “for fun.”
  • 47. Invocation name: Requirements Requirements for Alexa to hears your invocation name correctly • Easy to pronounce correctly and phonetically distinct to avoid being misinterpreted as other, similar-sounding words (for example, avoid “alveolar sounds”) • Lower-case alphabetic characters (“travel buddy” instead of “Travel Buddy”) • Spaces between words (for example, “editor in chief” instead of “editor-in-chief”) • Possessive apostrophes (for example, “sam’s science trivia”) • Periods in abbreviations (for example, “a. b. c.”) • Numbers must be spelled out (for example, “twenty one” instead of “21”)
  • 48. Invocation name: Testing • Test, test, test! • Try various invocation patterns like: • Open <invocation name> • Launch <invocation name> • Tell <invocation name> to do <utterance> • Ask <invocation name> about <utterance> • Example utterances you provided in skill • Ask others to try it out (beta testing, pizza party…) • Check if: • Your invocation name is natural for users to use • Alexa is recognizing and invoking your skill often (invocation accuracy) • You can see how Alexa interpreted your invocation name by reviewing the history in the Amazon Alexa App (in the app, navigate to Settings and then History). Alexa, ask travel buddy about surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance City SlotActivity Slot
  • 49. Invocation name: Some insights • Alexa hears your invocation name more accurately with supported sample utterances • More likely to misrecognize uncommon words for invocation names • For invocation issues • Increase coverage of sample utterances • Consider another invocation name • Publish with preferred invocation name if at least medium accuracy (>70%) • Active learning Alexa, ask travel buddy about surfing in Sydney Wake Word Launch Invocation Name Connector Word Utterance Activity Slot City Slot
  • 51. Key features • Slot filling • Slot confirmation • Intent confirmation Advantages • Simplifies collecting and confirming slot values and intents • Improved accuracy in slot filling because Alexa now has context as to which slot you are filling • Use it for a better-performing skill! Dialog management
  • 53. Insights Test your experience with real users! Pick an invocation name that users remember and Alexa understands well. Add more data for slot values and utterance patterns that customers actually use. Watch out for data pre-processing issues in your slots data. Use entity resolution for all your slots. Use dialog management features for handling slot filling. Use alexa.design/guide Test your experience with real users!
  • 54. Thank you! R a j i v R e d d y A L X 3 1 9