With the advancement of machine learning applications, new business opportunities are rapidly emerging in media. In this session, you learn how the AWS Media2Cloud solution can save time and reduce costs by setting up a serverless end-to-end ingest workflow to move your video assets and associated metadata to the cloud. You gain insight into how to make those assets even more valuable by enabling searching and indexing on your video library and learn how to use Amazon Transcribe and Amazon Translate to take your live-streaming workflows to the next level with expert instruction on how to automatically create multilanguage subtitles.

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Increase the value of video with machine
learning & AWS Media Services
S V C 3 0 9
Liam Morrison
Principal Solutions Architect – AI in Media and Entertainment
AWS

2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Agenda
Why machine learning for media?
Use cases & relevant services
AWS solutions
• Media analysis solution
• Live captioning & translation solution

3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Content is no longer king
The viewer is king
Desired shift to a direct-to-consumer business model
Success relies on a rich viewer experience
Rich viewer experiences rely on data and analytics
Major shift in the Media and Entertainment industry

5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The new reality
Viewer
expectation
Content discoverability, recommendations,
personalization
Business
opportunity
Learn about viewers; use the data to increase
engagement
Customer
obsession
Customer satisfaction increases brand value and
drives revenue

6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Athena
Amazon EMR
Amazon Redshift
Amazon Elasticsearch Service
Amazon Kinesis
Amazon QuickSight
AnalyticsAmazon SageMaker
AWS Deep Learning AMIs
Amazon Rekognition
Amazon Lex
AWS DeepLens
Amazon Comprehend
Amazon Translate
Amazon Transcribe
Amazon Polly
Machine learning
AWS IoT Core
Amazon Kinesis Data Firehose
Amazon Kinesis Data Streams
Amazon Kinesis Video Streams
Real-time
data movement
AWS Direct Connect
AWS Snowball
AWS Snowmobile
AWS Database Migration Service
On-premises
data movement
Data lake on AWS
Storage | Archival Storage | Data Catalog
Incoming analytics data Existing asset metadata
Data lake + media assets = Content lake

7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Modern data architecture for media enrichment
Insightstoenhanceviewerengagement,personalization,andmonetization
Improve UX/QoS
Recommend
Monetize
Infer
Store
& Process
Content lake on AWS
Storage | Archival Storage | Data Catalog
Analyze
Viewer data
IoT sensors
Social media
Video
AMAZON
QUICKSIGHT
AMAZON
SAGEMAKER
Engage and personalize

8. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Sample ML use cases for media
Automate the creation of rich metadata index (object, scene, activity, faces, etc.), extracted from audiovisual
content and integrated into digital and media asset management systems
AWS services to build use case: Amazon Rekognition, Amazon SageMaker, Amazon Transcribe,
Amazon Comprehend
Media metadata tagging
Automated captions, transcription, and translation of audio content
AWS services to build use case: Amazon Transcribe, Amazon Translate, Amazon Comprehend
Closed captioning
Detect potentially inappropriate content to avoid compliance issues in global markets and to increase brand
safety for advertisers
AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker
Automated compliance
marking
Identify objects and emotion in content enabling users to integrate personalized ads into subscriber video streams
AWS services to build use case: Apache Spark on Amazon EMR, Amazon SageMaker, AWS Elemental MediaTailor
Ad personalization

10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Sample ML use cases for media
Identify and path player activity in the field or identify the precise timecodes when actors enter and leave a
scene, both visually and in spoken dialog (off-screen voice)
AWS services to build use case: Amazon Rekognition, Amazon Transcribe
Player/actor
identification & activity
pathing
Translate transcripts and metadata; improve localization workflows and search experience
AWS service to build use case: Amazon Translate
Language translation
Analyze disparate data and enable the consumer to make personalized content choices and the business to
predict behavior
AWS services to build use case: Amazon SageMaker, Amazon Personalize
Content
recommendations
Detect and pixelate faces captured incidentally to preserve the privacy of non–persons of interest in news feeds
and security footage
AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker
Automated redaction

11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Use case: From manual tagging to AI-based tagging
Background
• More than 200,000 hours of content
• Only half of content is manually tagged
• How can we enrich our metadata in AWS?
• How can we unleash the value of content we already own
once in AWS?
Challenge
• Large-scale video library
• High accuracy required
• Limited budget
• Ability to extract from video
• Keep up with daily increase in content
Results
• Solution developed within three weeks
• Live video frame–based analysis
• Established, searchable baseline archive
• All content is now tagged and indexed
• More than 99,000 faces indexed and searchable
• Saved ~9,000 hours a year in manual curation costs
C-SPAN

12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Use case: Event broadcasting enrichment

13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Live & post-production subtitling and translation

14. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
The Amazon ML stack
M L F R A M E W O R K S &
I N F R A S T R U C T U R E
A I S E R V I C E S
A M A Z O N
S A G E M A K E R
BUILD TRAIN DEPLOY
Pre-built algorithms & notebooks
Data labeling (Amazon SageMaker Ground Truth)
One-click model training & tuning
Optimization (Amazon SageMaker Neo)
One-click deployment & hosting
M L S E R V I C E S
F r a m e w o r k s I n t e r f a c e s I n f r a s t r u c t u r e
Hyperparameter optimization
Algorithms & models (AWS Marketplace for
machine learning)
Notebook hosting Reinforcement learning
Automatic scaling
Amazon Virtual Private Cloud
AWS PrivateLink
Amazon Elastic Inference integration
Amazon
Rekognition
Image
Amazon
Polly
Amazon
Transcribe
Amazon
Translate
Amaz o n
Co mprehend &
Amaz o n
Co mprehend Medical
Amazon
Lex
Amazon
Rekognition
Video
Vision Speech Language Chatbots Forecasting Recommendations
Amazon
Textract
Amazon
Forecast
Amazon
Personalize
Amazon
EC2 P3
& P3dn
instances
Amazon
EC2 C5
instances
FPGAs AWS IoT
Greengrass
Amazon
Elastic
Inference
AWS
Inferentia

16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Rekognition
Deep learning-based image analysis service
Object and scene
detection
Facial
analysis
Face search
& match
Text in imageContent moderation Celebrity recognition

17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Rekognition Video
Analysis for live or stored video content

18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Transcribe
Automatic speech recognition for stored or real-time audio
Time
stamps
Support for
both regular &
telephony audio
Punctuation
& formatting
Multiple
language
support
Recognition
of multiple
speakers
Custom
dictionaries
? !

19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
25 languages
595 combinations
Real-time
< 500ms / sentence on average
< 150ms / conversational / short form
Tag handling
XML tags placement maintains styling
and formatting through translation
< / >
Data security
Data ownership
Encryption
Access management
Ease of use
Simple API calls and partner solutions
$15/1M characters
Or $0.000075 per word;
Pay as you go, 2M characters monthly
free tier
HIPAA eligible
Amazon Translate
Features

20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon Polly
Turn text into lifelike speech
Use cases
Content creation
Mobile & desktop applications
Internet of Things (IoT)
Education & e-learning
Telephony
Game development
Key features
57 voices across 28 languages
Lip syncing & text highlighting
Fine-grained voice control
Custom vocabularies
Available in 17 AWS Regions

21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Custom machine learning for your business
Amazon SageMaker
Fast & accurate
data labeling
Built-in, high-
performance
algorithms &
notebooks
B U I LD
1
One-click
training and
tuning
T RAI N
Model optimization
2
Fully managed
hosting with
auto-scaling and
elastic inference
One-click
deployment
D EPLOY
3

22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Amazon SageMaker Ground Truth: Help with
dataset labeling
NEW

23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
NEW
Amazon SageMaker Ground Truth: Workflow

24. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS media solutions
Accelerate revenue
Get started with your project quickly and start
monetizing your content
One-click launch
Launch a fully tested solution starter kit in your
AWS account with one click
Well-architected
Rely on the architecture as it reflects AWS best
practices
Agility
Save the resources of developing a solution
from the beginning
Expand and add
Use the framework and available guides to expand the solution
starter kit based on your needs

26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
AWS media solutions
Go to: aws.amazon.com/media/solutions

27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Media analysis solution
Label & face detection
Celebrity detection
Face search
Activity tracking
VoD subtitle generation
Key entity & phrase detection
Automated
metadata
generationA m a z o n R e k o g n i t i o n
I m a g e
A m a z o n R e k o g n i t i o n
V i d e o
A m a z o n
T r a n s c r i b e
A m a z o n
T r a n s l a t e
A m a z o n
C o m p r e h e n d
Transcription
Context
Amazon SageMaker + Additional solutions for media use cases

29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Media analysis solution
Amazon Rekognition
Amazon Transcribe
Amazon Comprehend
Amazon S3 AWS Step
Functions
AWS Lambda
Amazon
Elasticsearch
Service
image video
audiotext
Amazon SageMaker

30. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Live subtitling solution
• Purpose-built to help customers create live
streaming subtitling workflows on the AWS
Cloud
• Serverless architecture
• Uses new Amazon Transcribe streaming to
extract speech for subtitles—then Amazon
Translate to convert into other subtitle
languages
• Deploy in minutes with a single click using an
AWS CloudFormation template
• Automatically spins up all resources needed
for the entire workflow
AWS Elemental
MediaLive
Lambda function
(Caption creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(Transcribe streaming)
AWS Elemental
MediaPackage
Amazon
Translate
manifests
live channel source
Amazon S3
bucket
multi-language (optional)
PCM audio
transcripts
subtitles, manifests, & video files
PCM audio
Amazon
CloudFront

33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Live subtitling solution
Ingest live video into AWS Elemental MediaLive
Amazon S3 events trigger subtitle creation
process with Amazon Transcribe streaming
Translate speech to other languages with
Amazon Translate
Live video with subtitles are ingested into AWS
Elemental MediaPackage
View your livestream in HLS, DASH, or Microsoft
Smooth (MSS)
1
2
3
4
5
AWS Elemental
MediaLive
Lambda function
(Caption creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(Transcribe streaming)
AWS Elemental
MediaPackage
Amazon
Translate
manifests
live channel source
Amazon S3
bucket
multi-language (optional)
PCM audio
transcripts
subtitles, manifests, & video files
PCM audio
Amazon
CloudFront
1 2
3
4
5

34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.S U M M I T
Live subtitling solution
• Purpose-built to help customers to create
live streaming subtitling workflows on the
AWS Cloud
• Serverless architecture
• Uses new Amazon Transcribe streaming to
extract speech for subtitles—then Amazon
Translate to convert into other subtitle
languages
• Deploy in minutes with a single click using an
AWS CloudFormation template
• Automatically spins up all resources needed
for the entire workflow
• Optionally add text to speech with Amazon
Polly or use Amazon Comprehend to
understand text sentiment.
AWS Elemental
MediaLive
Lambda function
(Caption Creation)
Amazon
Transcribe
Amazon S3
bucket
Lambda function
(Transcribe Streaming)
AWS Elemental
MediaPackage
Amazon
Translate
manifests
Amazon S3
bucket
multi-language (optional)
PCM audio
transcripts
subtitles, manifests, & video files
PCM audio
Amazon
CloudFront
Amazon
Polly
Amazon
Comprehend
Alternate Audio (optional) Insights & Sentiment (optional)

35. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

36. Thank you!
S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Liam Morrison