Amazon Lex is a service for building conversational interfaces into any application using voice and text. With Amazon Lex, the same deep learning technologies that power Amazon Alexa are now available to any developer, enabling you to quickly and easily build sophisticated, natural language conversational chatbots. No deep learning experience is required to immediately start creating chatbots that understand voice or text, to ask questions, get answers, and complete sophisticated tasks. Lex enables you to easily publish your chatbots to mobile devices, web apps, services, and platforms such as Facebook Messenger, Twilio and Slack. This session will go over the features available with Amazon Lex and how they can be used to build and deploy chatbots. Join us for this introductory presentation and learn more about Amazon Lex! AWS Speaker: Herbert-John Kelly, Solutions Architect - Amazon Web Services

1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Herbert-John Kelly – Solutions Architect AWS
- herbertjkelly
July 2017
Getting started with Amazon Lex
Cape Town Summit 2017

2. Amazon Lex
Why did we build Amazon Lex?
What is Amazon Lex?
How do I build conversational apps using Amazon Lex?
A Quick look at Amazon Polly & Amazon Alexa
Q&A

3. Why did we build
Amazon Lex?

4. Advent of conversational interactions
1st Gen:
Punch Cards & Memory Registers
2nd Gen:
Pointers & Sliders
3nd Gen:
Conversational Interfaces

5. We have come a long way…

6. Developer challenges
Conversational interfaces need to combine a large number of
sophisticated algorithms and technologies
Speech
Recognition Language
Understanding
Business Logic
Disparate
Systems
Authentication
Messaging
platforms
Scale Testing
Security
Availability
Mobile

7. Amazon Lex:
A new service for building
conversational interfaces
using voice and text

8. Complete solution
End to End
Speech to Intent
ASR+NLU
integrated into
one API
Dialog Management
Native support &
maintains context
Text to Speech
Amazon Polly integrated
into API
Business Logic
Native integration with
AWS Lambda
Deployment
One click
deployment
Security
Encrypted data in
transit & at rest
Scale
Completely managed
service
Analytics
Monitor and improve

9. Text and speech language understanding: powered by the
same technology as Alexa
Enterprise Ready: connect to enterprise systems via
SaaS connectors
Versioning and alias support
Build once and deploy to multiple platforms
Designed for builders: efficient and intuitive tools
to build conversations; scales automatically
Amazon Lex - features
Continuous Learning: monitor and improve your
bot

10. Text and speech language understanding
Speech
Recognition
Natural Language
Understanding
Powered by the same deep learning technology as Alexa

11. Multi-platform
Mobile Messaging
Platforms
Web IoT
SDKs: iOS & Android
Mobile Hub
Facebook, Twilio
SMS and Slack
SDKs: Java, JavaScript,
Python, CLI, .NET, Ruby on
Rails, PHP, Go, and CPP
Integrated with AWS
IoT via AWS Lambda
Build once and deploy to multiple platforms

12. Designed for builders
Efficient and intuitive tools to build conversations

13. AWS Mobile Hub integration
Authenticate users
Analyze user behavior
Store and share media
Synchronize data
More ….
Track retention
Conversational bots
Amazon LexAWS Mobile SDKs
AWS Mobile Hub

14. Enterprise SaaS connectors with Mobile Hub
Amazon Lex
Mobile App
Mobile Hub
SaaS connector
Amazon API
Gateway
AWS
Lambda
1: Understand
user intent
Amazon API
Gateway
AWS
Lambda
3: REST
response into
natural language
2: Invoke a SaaS
application or existing
business application
Business
application
Firewall
User input
Mobile Hub
custom connector

15. Versioning and Alias support
AliasVersioning
• Supported for Intents, Slots, and Bots
• Enables multi-developer environment
• Rollback to previous versions
• Deploy different aliases to different platforms
• Run different stacks for dev, stage and prod environments
• Target different user groups with different aliases
v1 v2 v3 latest
v1 dev
v2 stage
v3 prod

16. Continuous learning
Monitor
AWS CloudWatch metrics
Missed utterances
Improve
Add to intent
Analyze

17. Amazon Lex – use cases
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….

18. Amazon Lex benefits
High quality text and speech language
understanding
Built-in integration with the AWS platform
Seamlessly deploy and scale
Easy to use
Cost effective

19. Building your bot with Amazon
Lex.

20. Amazon Lex
Utterances
Spoken or typed phrases that invoke
your intent
BookHotel
Intents
An intent performs an action in
response to natural language user
input
Slots
Slots are input data required to fulfill
the intent
Fulfillment
Fulfillment mechanism for your intent

21. “Book a hotel”
Book Hotel
NYC
“Book a Hotel in
NYC”
Automatic Speech
Recognition
Hotel Booking
New York City
Natural Language
Understanding
Intent/Slot
Model
UtterancesHotel Booking
City New York City
Check In April 19th
Check Out April 21st
“Your hotel is booked for
April 19th”
Amazon Polly
Confirmation: “Your hotel
is booked for Nov 30th”
“Can I go ahead
with the booking?
a
in

22. Slot elicitation
Check In
4/19/2017
City
New York
City
I would like to book a hotel
Sure, which city will you be
traveling to?
New York City
What date do you want to
check in?
April 19th

23. Dynamic conversation flow
Conversatio
n
Session
Attributes
Second
Intent
Switch
Intents
First Intent
Session
Attributes
Conversatio
n
Chain
Intents
Takeout
Dine In
Dine In or
Take out?
Anything
else today?
Book a Car

24. Dialog management
I would like to book a hotel
Sure, which city will you be
traveling to?
New York City
What date do you want to
check in?
Tomorrow
… And for how many nights
is this for?
City
Simple Declarative Model
Check-In Date
Check-Out Date
Slots
Which city will you be travelling to?
What date do you want to check in?
How many nights is this for?
Prompts
Build Multi-turn Conversations
Easy Setup in Console

25. Customize conversations
I would like to book a hotel
Would you prefer to stay in
Downtown this time as well?
Yes
What date do you want to
check in?
April 19th
Sorry no availability. Would
a different location work?
Personalize conversation
based on user preferences
Validate user input and re-
prompt as necessary

26. NATURAL
Amazon Lex the same
technology that powers
Alexa
DYNAMIC
Answer customer
questions before they
are even asked
PERSONAL
Contact flows adapt on
a per customer basis
Ok, you are now
booked for a
9:00AM departure
tomorrow out of
San Francisco,
arriving in Seattle
at 11:45AM.
Can you
please
rebook me
for the
same flight
tomorrow?
Great
Thank you!Data
Dip
CRM
content
Hi Nikki Wolf,
I see your flight
was cancelled
today. How can
I help you?
Incoming
customer
call
Contact Flow Engine – Customer Experience Example

27. Error handling
I want to …. {garbled} …..
Sorry can you please repeat
that?
I am having trouble understanding
Can you please say that again?
Sorry I am not able to assist
you at this time
Clarify by requesting user to
repeat
Uses a different prompt
every time
Hang up phrase to end the
conversation

28. Rich message formatting
I would like to rent a car
Sure. What type of car are
you looking for?
Compact
• Formatted for messaging platforms
• Multiple cards supported
• Preview capability
• Test in console

29. Fulfillment
AWS Lambda
Integration
Return to
Client
User input parsed to derive
intents and slot values.
Output returned to client
for further processing
Intents and slots passed
to AWS Lambda function
for business logic
implementation

30. Save, build and publish
Save Build
Saving your bot
preserves the current
state on the server
Building your bot
creates versions
that you can test
Publish
Publishing your app will create a
version of your bot and provide
an alias to your clients
Test
Test your bot in a
chat window on the
console

31. Programmatically build bots
… build, test and deploy bots using SDKs
Add Utterances Add Slot Values
SDKs: Java, JavaScript, Python,
CLI, .NET, Ruby on Rails, PHP, Go,
and CPP

32. Pre-defined resources
Built-in Slot Types
AMAZON.DATE
AMAZON.TIME
AMAZON.NUMBER
……
Ready to use slot types that are already trained
with sample values
Do not need to spend the time enumerating sample
values for these slot types
Benefit from continuously expanding pre-defined resources

33. Messaging platforms
Facebook Slack Twilio SMS

34. Customer testimonials

35. Amazon Lex pricing
Text Speech
Price per 1000 requests $0.75 $4.00
Free Tier*
(requests per month) 10,000 5,000
*Available for the first year upon sign-up to new Amazon Lex customers

36. Amazon Lex - technology
Amazon Lex
Automatic Speech
Recognition (ASR)
Natural Language
Understanding (NLU)
Same technology that powers Alexa
Amazon Cognito Amazon CloudWatch
AWS Services
Action
AWS Lambda
Authentication
& Visibility
Speech
API
Language
API
Fulfillment
End-Users
Developers
Console
SDK
Intents,
Slots,
Prompts,
Utterances
Input:
Speech
or Text
Multi-Platform Clients:
Mobile, IoT, Web,
Chat
API
Output:
Speech (via Amazon Polly TTS)
or Text

37. Amazon Polly – get your bot
talking.

38. Amazon Polly: life-like speech service
Converts text
to life-like speech
47 voices 24 languages Low latency,
real time
Fully managed

39. What is Polly
• A service that converts text into lifelike speech
• Offers 47 lifelike voices and 24 languages
• Low latency responses enable developers to build
real-time systems
• Developers can store, replay and distribute
generated speech

40. Polly – Quality
Natural sounding speech
A subjective measure of how close TTS output is to human speech.
Accurate text processing
Ability of the system to interpret common text formats such as abbreviations, numerical
sequences, homographs etc.
12°C is the same as, 53.6°F. If you enjoyed this summit be sure to go to our conference
"Re:Invent" in Las Vegas NV
"We live for the music", live from the Madison Square Garden.
Highly intelligibile
A measure of how comprehensible speech is.
”Peter Piper picked a peck of pickled peppers.”

41. Polly features: SSML
Speech Synthesis Markup Language
is a W3C recommendation, an XML-based markup language for speech
synthesis applications
<speak>
My name is Kuklinski. It is spelled
<prosody rate='x-slow'>
<say-as interpret-as="characters">Kuklinski</say-as>
</prosody>
</speak>

42. Polly features: Lexicons
Enables developers to customize the pronunciation of
words or phrases
My daughter’s name is Kaja.
<lexeme>
<grapheme>Kaja</grapheme>
<grapheme>kaja</grapheme>
<grapheme>KAJA</grapheme>
<phoneme>"kaI.@</phoneme>
</lexeme>

43. First app
from boto3 import Session
from contextlib import closing
polly = Session().client("polly")
response = polly.synthesize_speech(
Text="Hello world!",
OutputFormat="mp3",
VoiceId="Joanna")
with closing(response["AudioStream"]) as stream:
with open("speech.mp3", "wb") as file:
file.write(stream.read())

44. Polly is cost-effective
• Pay-as-you-go
• $4 for 1M characters
• Free Tier of 5M characters/month - first year
• You can store and reuse generated speech

45. Amazon Polly customers
Media and Entertainment
Education
Accessibility
Content Creation
Telephony/Contact Center
Internet of Things
Gaming

46. Demo Time! – Moviepedia bot
https://github.com/awslabs/aws-lex-convo-bot-example

47. Moviepedia bot

48. Moviepedia bot

49. Moviepedia bot

50. Moviepedia bot

51. Moviepedia bot

52. Moviepedia bot

53. Moviepedia bot

54. Demo Time! – Darren’s Prank
https://github.com/alexa/skill-sample-nodejs-audio-player

55. Darren’s Prank

56. Darren’s Prank

57. Darren’s Prank

58. Darren’s Prank

59. Darren’s Prank

60. Herbert-John Kelly
- herbertjkelly