AWS offers a suite of AI and machine learning services including: - Rekognition for image and video analysis including object detection, facial recognition and analysis, and image moderation. - Polly for text-to-speech conversion with many voices and languages. - Lex for building conversational bots using voice and text across channels like Alexa, Slack, and Facebook Messenger. - Comprehend for natural language processing including keyword extraction, sentiment analysis, and topic modeling from text. - SageMaker as a fully managed platform for building, training, and deploying machine learning models at scale.

1. Artificial Intelligence & Machine Learning
Elena Ehrlich, PhD
eeehrlic@amazon.com

2. What is AI?
https://www.geospatialworld.net/blogs/difference-between-ai%EF%BB%BF-machine-learning-and-deep-learning/

3. Agenda
• Image & Video Recognition Rekognition
• Deep-Learning Enabled Video Cameras DeepLens
• Natural Language Understanding Comprehend
• Voice & Convseration Bots Polly, Lex, & Alexa
• Fully-Managed Machine Learning Sagemaker

4. Image Analysis
AWS Rekognition

5. Bay
Beach
Coast
Outdoors
Sea
Water
Palm_tree
Plant
Tree
Summer
Landscape
Nature
Hotel
99.18%
99.18%
99.18%
99.18%
99.18%
99.18%
99.21%
99.21%
99.21%
58.3%
51.84%
51.84%
51.24%
Category Confidence
Rekognition: Object & Scene Detection

6. Rekognition: Facial Analysis
"FaceDetails": [{
"BoundingBox": {
"Height": 0.22111110389232635 ,
"Left": 0.29600000381469727,
"Top": 0.08888889104127884,
"Width": 0.4000000059604645
},
"Confidence": 99.9970474243164,
"Emotions": [{
"Confidence": 98.48326110839844,
"Type": "HAPPY"
}, {
"Confidence": 15.214723587036133,
"Type": "CALM"
}, {
"Confidence": 1.2157082557678223,
"Type": "CONFUSED"
}],
"AgeRange": {
"High": 47,
"Low": 30
},
"Beard": {
"Confidence": 95.77610778808594,
"Value": false
},
"Eyeglasses": {
"Confidence": 99.68527221679688,
"Value": true
},
"EyesOpen": {
"Confidence": 99.99991607666016,
"Value": true
},
"Gender": {
"Confidence": 99.92896270751953,
"Value": ”Female"
},
"MouthOpen": {
"Confidence": 99.90928649902344,
"Value": true
},
"Mustache": {
smart cropping
& ad overlays
sentiment
capture
demographic
analysis
face editing
& pixelation
DetectFaces
{ "contentString":
{
"Attributes": [
"ALL"
],
"Image": {
"Bytes": "..."
}
}
}

7. Rekognition: Compare Faces
Face Comparision

8. Hierarchical taxonomy
Confidence score
"ModerationLabels": [
{
"Confidence": 82.7555923461914,
"Name": "Suggestive",
"ParentName": ""
},
{
"Confidence": 82.7555923461914,
"Name": "Female Swimwear or Underwear",
"ParentName": "Suggestive"
},
{
"Confidence": 50.11056137084961,
"Name": "Covered Nudity",
"ParentName": "Nudity and Sexuality"
},
{
"Confidence": 50.11056137084961,
"Name": "Nudity and Sexuality",
"ParentName": ""
},
]
Rekognition: Image Moderation
Suggestive 82.7%
Female Swimwear or Underwear 82.7%
Nudity and Sexuality 50.1%
Covered Nudity 50.1%

9. https://console.aws.amazon.com/rekognition/home?region=us-east-1#/label-detection
https://d3qtbfbtl5c95j.cloudfront.net/Main.html
http://iad-front.deepvideoanalysis.cloud/results.html#!?identifier=introducingamazongo.mp4
https://aws.amazon.com/rekognition/customers/
Interesting Demos…Time permitting

10. Deep-Learning Enabled
Video Cameras
AWS DeepLens

11. DeepLens: Deep-Learning Enabled Video Camera
A DL video camera uses deep convolutional neural networks (CNNs) to analyze visual imagery.
The device itself is a development environment to build computer vision applications.
AWS DeepLens communicates with the following ML endpoints:
• Amazon SageMaker, for model training and validation
• AWS Lambda, event-driven triggers run inference against CNN models
• AWS Greengrass, for deploying updates and functions to your device and other IoT devices
April 2018

12. Natural
Language Understanding
AWS Comprehend

13. https://www.ip-watch.org/2018/01/24/itu-4-5-people-ldcs-can-access-mobile-networks-not-using-internet/
Comprehend: Keyword, Sentiment, & Topic Modeling

14. Comprehend: Keyword, Sentiment, & Topic Modeling

15. Comprehend: Keyword, Sentiment, & Topic Modeling

16. Comprehend: Keyword, Sentiment, & Topic Modeling

17. Life-like Speech
AWS Polly

18. Polly: Life-like Speech Service
Plain Text SSML Lexicons
Plain Text SSML Lexicons
Speech Synthesis Markup Language
<speak> - Start Tag
<break> - Pause in Speech
<lang> - Specifies the language
<mark> - Tag Name for specific word
<p> - Indicates Paragraph
<phoneme>- phonetic pronunciation
<prosody> - Controls the volume
<s> - Indicates a sentence
<say-as>- Interpretation
<sub> - Alias words
<w> - Customize pronunciation
<amazon:effect name="whispered">
<lexeme>
<grapheme>espresso</grapheme
>
<alias>ess-press-oh</alias>
</lexeme>

19. Conversational Engines
AWS Lex

20. Lex: The Advent Of Conversational Interactions
1st Gen: Machine-oriented
interactions
2nd Gen: Control-oriented
& translated
3rd Gen:
Intent-oriented
Speech Recognition Language UnderstandingBusiness Logic
Disparate Systems Authentication
Messaging platforms
Scale
Testing
Security
Availability
Mobile

21. Lex – Converstaional Engines
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather
information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….
Operational Bots
Chatbots for IT automation
• Reset my
Password
• TCO analysis
• Productivity….

23. Most Common Algorithms Provided
•Linear Learner
•Factorization Machines
•XGBoost Algorithm
•Image Classification Algorithm
•Amazon SageMaker Sequence2Sequence
•K-Means Algorithm
•Principal Component Analysis (PCA)
•Latent Dirichlet Allocation (LDA)
•Neural Topic Model (NTM)
•DeepAR Forecasting
•BlazingText
import sagemaker
Sagemaker: Fully-Managed Machine Learning
10x
Performance
Single-Click Training
Train Models at Petabyte Scale
Deploy in Production
Auto-Scaling Cluster of AWS EC2 Instances
OpenSource tools
TensorFlow
Apache MXNet
A/B Testing
Built-in

24. AI/ML Adoption Benefits
CONVERTING
THE POWER OF
MACHINE
LEARNING INTO
BUSINESS VALUE
MAKING THE
BEST USE OF A
DATA SCIENTISTS
TIME
EMBEDDING
MACHINE
LEARNING INTO
THE FABRIC OF
YOUR BUSINESS
While the power of ML is unrivaled, “data scientists spend around
80% of their time on preparing and managing data for analysis” …
hence only 20% of their time is used to derive insights
The value of data science relies upon operationalizing models
within business applications and processes, yet “50% of the
predictive models [built] don’t get implemented”
While “60% of companies agree that big data will help improve their
decision making and competitiveness … only 28% indicate that they
are currently generating strategic value from their data”
1
2
3

25. Thank you
Elena Ehrlich, PhD
eeehrlic@amazon.com

26. Appendix

27. Services
Platforms
Frameworks
AWS AI/ML: The Stack
Apache
MXNet
KerasGluonPyTorch
Cognitive
Toolkit
Caffe2
& Caffe
Tensor-
Flow
AWS Deep Learning AMI
SageMaker
Mechanical
Turk
AWS
DeepLens
Amazon ML
Spark &
EMR
Speech:
Polly &
Transcribe
Vision:
Rekognition Image &
Rekognition Video
Language:
Lex, Translate &
Comprehend

28. AWS AI/ML: Notable Successes
Services
Platforms
Frameworks

29. AWS AI/ML: Solutions for Every Skill Level
• Designed for Developers & Data Scientists
• Solution-oriented Prebuilt Models Available via APIs
• Image Analysis, NLU, NLP, Translation, Text-to-Speech & Speech-to-Text
• Designed for Data Scientists to Address Common Needs
• Fully Managed Platform for Model Building
• Reduces the Heavy Lifting in Model Building & Deployment
• Designed for Data Scientists to Address Advanced / Emerging Needs
• Provides Maximum Flexibility to develop on the leading AI Frameworks
• Enables Expert AI Systems to be Developed & Deployed
Services
Platforms
Frameworks

30. Real-time &
batch image
analysis
Object & Scene
Detection
Facial Detection Face SearchFacial Analysis
Rekognition: Search & Understand Visual Content
Image Moderation Celebrity Recognition

31. Rekognition: Image Moderation
Manual
Review
Approved
Rejected
Picture posted
to end users
No inappropriate
content detected
Inappropriate
content detected
Object CreationUpload picture
Users
S3 Bucket Rekognition
Lambda
User
Notification

32. Rekognition: Video - Case Study Architecture

33. Polly: Life-like Speech Service
Converts text
to life-like speech
47 voices 27 languages Low latency,
real time
Fully managed

34. Lex: Build Natural, Conversational Interactions In Voice & Text
Voice & Text
“Chatbots”
Powers
Alexa
Voice interactions
on mobile, web
& devices
Text interaction
with Slack & Messenger
Enterprise
Connectors
(with more coming) Salesforce
Microsoft Dynamics
Marketo
Zendesk
Quickbooks
Hubspot

35. BOT Intent Slot & Slot type
BOT Intent Slot & Slot type
An intent represents an action that
the user wants to perform
Intent name– A descriptive name for
the intent.
Sample utterances – How a user
might convey the intent.
How to fulfill the intent – How you
want to fulfill the intent after the user
provides the necessary information
Slot - An intent can require zero or
more slots or parameters
Slot type – Each slot has a type.
You can create your custom slot
types or use built-in slot types
Lex: Build Natural, Conversational Interactions In Voice & Text
An Amazon Lex bot is powered
by Automatic Speech
Recognition (ASR) and Natural
Language Understanding
(NLU) capabilities

36. Response Cards
• Simplify interactions for your users
• Increase bot's accuracy
• Can be used with Facebook Messenger, Slack, and Twilio as well as your own client
applications.

37. DeepLens Architecture

38. IoT Anomaly Detection
AWS Kinesis Analytics

39. Kinesis Analytics: real-time insights from streaming data

40. Kinesis Analytics: real-time insights from streaming data

41. AI Inquisitors AI Adopters AI Experts
Interested in AI but
have limited expertise
and/or resources
Limited expertise
and/or use of AI for
one-off projects
Advanced expertise
and/or use of
embedded AI in apps
AI/ML Assessment

42. Business Value Ability to Execute Data Availability
Assessing POC Targets: Criteria

43. Prep Question Sample Answer
What Business or Operational benefits are you trying to drive? • Improve content personalization
How will you consume the outputs and put them into action?
• Content will be distributed at a
targeted level
What types of data is available today? Where does the data
reside?
• Content and subscription data
What types of analytics and/or machine learning are being
employed today?
• Business Intelligence
• Predictive Analytics
What staff and/or consultants currently support these activities?
• Data Engineers
• Data Scientists
What software currently supports these activities? • R / Python
What is your ideal scenario in tackling these business objectives? • One-to-one content for individuals
What challenges have you experienced when deploying AI?
• Prioritization of Targets
• Operationalization
AI/ML Assessment