Build Intelligent Apps with Amazon ML - Language Services - BDA302 - Chicago AWS Summit

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Niranjan Hira
AWS Solution Architect (Amazon Lex)
BDA302
Build Intelligent Apps with Amazon ML –
Language Services

The AWS machine learning stack
FRAMEWORKS / INTERFACES KERAS
PLATFORMS
APPLICATION SERVICES
A M A Z O N
R E K O G N I T I O N
A M A Z O N
R E K O G N I T I O N
V I D E O
A M A Z O N
P O L L Y
A M A Z O N
T R A N S C R I B E
A M A Z O N
T R A N S L A T E
A M A Z O N
C O M P R E H E N D
A M A Z O N
L E X
AMAZON SAGEMAKER AMAZON MECHANICAL TURK

Powerful language capabilities
Amazon Transcribe
Automatic conversion of speech into accurate, grammatically
correct text
Amazon Translate Natural and fluent language translation
Amazon Polly Turn text into lifelike speech using deep learning
Amazon Comprehend Discover insights and relationships in text
Amazon Lex
Conversational interfaces for text-based and voice-based
applications

Common language use cases
Information Bots
Education
Accessibility
Knowledge Management
Voice of Customer Applications
Customer Service/
Call Centers
Enterprise
Digital Assistant
Semantic Search
Captioning Workflows
LocalizationPersonalization

How do you make your applications
conversational?

Amazon Lex
A service for building conversational interfaces into your
applications using voice and text

Amazon Lex
End to End
Speech to Intent
ASR+NLU
integrated into
one API
Dialog Management
Native support &
maintains context
Text to Speech
Amazon Polly integrated into
API
Business Logic
Native integration with
AWS Lambda
Deployment
One click deployment
Security
Encrypted data in
transit & at rest
Scale
Completely managed
service
Analytics
Monitor and improve
End to End

Amazon Lex
Social Intelligence
Personality
Modality
Dynamic Conversation • Built-in error handling allows you to manage unexpected scenarios
• Intent chaining can be used to build a conversation flow based on user input
• Rich formatting capabilities can be leveraged for a more engaging experience
• Create conversation combining speech and text in the same session
• Select from multiple prompts to give your bot a personality
• Choose from a range of Amazon Polly voices depending on target audience
• Use AWS Lambda to personalize the conversation based on user’s social
profile
• Context is maintained allowing you to build intelligent conversations

Amazon Lex – use cases
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….
Contact Center Bots
Chatbots for customer service IVR
• Account inquiries
• Bill payment
• Service update ….

How do you give your applications a voice?

Amazon Polly
A text-to-speech (TTS) service that uses advanced deep
learning technologies to synthesize speech that sounds
like a human voice

Amazon Polly
• Content creation
• Mobile & desktop applications
• Internet of Things (IoT)
• Education & e-learning
• Telephony
• Game development
Use cases
• 53 voices across 26 languages
• Lip-syncing & text highlighting
• Fine-grained voice control
• Custom vocabularies
• Available in 16 AWS Regions
Key features

Amazon Polly – Use cases
Voiced
Notifications
Special Needs
AI Assistant
AI Radio
Announcer
Video
Creation
Amazon Polly
Navigation
Podcasting

How do you extract insights from
unstructured text?

Amazon Comprehend
A fully managed and continuously trained service that
helps you extract insights from unstructured text

Amazon Comprehend
Sentiment Entities LanguagesKeyphrases Topic modeling
Syntax

Amazon Comprehend – Natural Language Processing
A m a z o n . c o m , I n c . i s l o c a t e d i n
S e a t t l e , WA a n d w a s f o u n d e d
J u l y 5 , 1 9 9 4 b y J e f f B e z o s . O u r
c u s t o m e r s l o v e b u y i n g
e v e r y t h i n g f r o m b o o k s t o
b l e n d e r s a t g r e a t p r i c e s
N a m e d E n t i t i e s
• A m a z o n . c o m : O r g a n i z a t i o n
• S e a t t l e , W A : L o c a t i o n
• J u l y 5 t h , 1 9 9 4 : D a t e
• J e f f B e z o s : P e r s o n
K e y p h r a s e s
• O u r c u s t o m e r s
• b o o k s
• b l e n d e r s
• g r e a t p r i c e s
S e n t i m e n t
• P o s i t i v e
L a n g u a g e
• E n g l i s h

Amazon Comprehend – Syntax API
O u r c u s t o m e r s l o v e b u y i n g
e v e r y t h i n g f r o m b o o k s t o
b l e n d e r s a t g r e a t p r i c e s
Token (word) Part of Speech
customers Noun
love Verb
books Noun
great Adjective
prices Noun

Syntax detection
$ aws comprehend detect-syntax --language-code 'en' --text 'I love cloud!'
{
"SyntaxTokens": [
{
"TokenId": 1,
"Text": "I",
"BeginOffset": 0,
"EndOffset": 1,
"PartOfSpeech": {
"Tag": "PRON",
"Score": 0.9999802112579346
}
},
...

Supported parts of speech
ADJ – Adjective
ADP – Adposition
ADV – Adverb
AUX – Auxiliary
CCONJ – Coordinating Conjunction
DET – Determiner
INTJ - Interjection
NOUN - Noun
NUM – Numeral
O – Other
PART – Particle
PRON – Pronoun
PROPN – Proper Noun
PUNCT – Punctuation
SCONJ – Subordinating Conjunction
SYM – Symbol
VERB – Verb

Popular text analytics use cases
• Content Personalization
• Customers are using Comprehend NLP output to understand related documents based on entities, phrases
or even topic similarities for trends analysis, to drive content personalization and recommendations
• Semantic Search
• Customers using Amazon Comprehend to index entities for boosting and ranking search results
• Intelligent data warehouse
• Customers are using Amazon Comprehend to query unstructured data in relational databases, processing
data within the data lake (Amazon S3) and then inserting it back into the data warehouse
• Social Analytics
• Customers are using Amazon Comprehend to ingest, process and analyze trends from entities and
sentiment from social media posts across Twitter and Facebook

Text analytics workload
Amazon Kinesis
Amazon ES
Amazon Redshift
Amazon EMR
• Semantic
• Rich filtering
• Grouping, trends
• Joining, correlating
• Clustering
• Graph, search
• Near real-time
• Alerts
Amazon S3
Social media, support
Amazon Aurora
Articles, documents

How do you make your applications listen?

Amazon Transcribe
A fully managed and continuously trained automatic
speech recognition (ASR) service that takes in audio and
automatically generates accurate transcripts

Amazon Transcribe – automatic speech recognition
Time stamps and
confidence scores
Support for both
regular and telephony
audio
Punctuation
§
Amazon S3
integration
Hello
Hola
English and Spanish
with more to come
Detect multiple
speakers
Custom
vocabulary

SYD – Sydney, Australia
YUL – Montreal, Canada
Region Launches
Amazon Transcribe – new regions

Amazon Transcribe – use cases
Call centers Subtitles for
VOD
Broadcast closed
captions
Transcribe
meetings

AWS Lambda
Amazon S3
Amazon
Athena
Audio Input
Amazon
QuickSight
Amazon
Comprehend
Amazon Transcribe – media workloads

How do you make your applications
multilingual?

Amazon Translate
A fully managed and continuously trained neural machine
translation service that translates text from one language
to another

Amazon Translate – features
Translate text input Real-time Translation12 languages &
more to come
Language detection via
Amazon Comprehend
AWS security standards Available in us-east-1,
us-east-2, us-west-2, and
eu-west-1
Easy to use
(CLI & SDKs)
Cost effective

Amazon Translate – supported languages
Arabic
Simplified Chinese
French
German
Spanish
Portuguese
Japanese
Traditional Chinese
Italian
Russian
Turkish
Czech

Use for high volume and time-sensitive information
Share of worldwide digital content
Machine translation
• User-authored content
• Text Analytics
• Real-time communication
• Content discovery
Post-edited
machine
translation
• Legal documents
• Healthcare documents
• Advertising materials

Karin Tsai
Staff Software Engineer, Duolingo
BDA302
Duolingo – New Hindi Course

What is Duolingo?
• The largest language learning platform in the world
• Over 300 million users
• 25 million active monthly users
• 33 languages offered in 81 courses; 15 more in development
• Most downloaded education app on iTunes and Google Play
• Mission: Free, high-quality language education for the world

Who am I?
• Joined Duolingo 3 months after launch (2012)
• Leads the Core Learning team
• Built the Language Incubator for creating new courses
• Enables bilingual volunteers build courses on Duolingo
• Utilizes the crowd to scale our content creation

Why we rely on Text-to-Speech (TTS)
• Higher throughput than human recordings
• Simple automation
• Consistency; easy quality assurance
• Scalability
• Lower cost

Why Amazon Polly is amazing
• Users say it is:
• Pleasant to listen to
• Rarely mispronounces
• Sounds natural
• Polly beat other TTS providers in 6 out of 6 A/B tests
• We heavily use AWS for our infrastructure

Our TTS architecture
from Powering Language Learning on Duolingo with Amazon Polly

What it takes
from boto3 import client
import shutil
polly = client('polly', region_name='us-east-1')
response = polly.synthesize_speech(
Text=text, VoiceId=voice_id, OutputFormat='mp3')
with open(output_filename, 'wb') as output_file:
shutil.copyfileobj(response['AudioStream'], output_file)
from Powering Language Learning on Duolingo with Amazon Polly

Our new Hindi course using the Aditi voice
• Launched July 19, 2018
• 15,600+ sentences
• 37,400+ active users
• Still constantly improving!

Aditi demo

Thank you!

Build Intelligent Apps with Amazon ML - Language Services - BDA302 - Chicago AWS Summit

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Build Intelligent Apps with Amazon ML - Language Services - BDA302 - Chicago AWS Summit

Similar to Build Intelligent Apps with Amazon ML - Language Services - BDA302 - Chicago AWS Summit (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Build Intelligent Apps with Amazon ML - Language Services - BDA302 - Chicago AWS Summit