Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Increase the value of video using ML an...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Agenda
Why machine learning for media?
...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Major shift in the Media &
Entertainmen...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
The new reality
Viewer
expectation
Cont...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Athena
Amazon EMR
Amazon Redshif...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Modern data architecture for media enri...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Sample ML use cases for media
Automate ...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Sample ML use cases for media
Identify ...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Usecase: From manualtagging to AI-based...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Use case: Event broadcasting enrichment
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
The Amazon ML stack
M L f r a m e w o r...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Rekognition
Deep learning-based ...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Rekognition Video
Analysis for l...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Amazon Transcribe
Automatic speech reco...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
AWS media solutions
Accelerate revenue
...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
AWS media solutions
http://aws.amazon.c...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Media analysis solution
Label & face de...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Media analysis solution
Amazon Rekognit...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
• Purpose buil...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
Ingest live vi...
© 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T
Live subtitling solution
• Purpose buil...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
Thank you!
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
Bryan Samis
samisb@amazon.c...
S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
Upcoming SlideShare
Loading in …5
×

Increase the value of video using ML and AWS media services - SVC301 - Santa Clara AWS Summit

430 views

Published on

With the advancement of machine learning applications, new business opportunities are rapidly emerging in media. In this session, learn how the AWS Media2Cloud solution can save time and reduce costs through setting up a serverless end-to-end ingest workflow to move your video assets and associated metadata to the cloud. Gain insight into how to make those assets even more valuable by enabling searching and indexing on your video library, and learn how to use Amazon Transcribe and Amazon Translate to take your live-streaming workflows to the next level with expert instruction on how to enable automatically created multi-language subtitles.

  • Be the first to comment

  • Be the first to like this

Increase the value of video using ML and AWS media services - SVC301 - Santa Clara AWS Summit

  1. 1. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Increase the value of video using ML and AWS media services Bryan Samis Solutions Architect AWS S V C 3 0 1
  2. 2. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Agenda Why machine learning for media? Use cases & relevant services AWS solutions • Media analysis solution • Live captioning & translation solution
  3. 3. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  4. 4. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Major shift in the Media & Entertainment industry Content is no longer king The viewer is king Direct-to-consumer is a business model Success relies on a rich viewer experience Rich viewer experiences rely on data and analytics
  5. 5. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T The new reality Viewer expectation Content discoverability, recommendations, personalization Business opportunity Learn about viewers, use the data to increase engagement Customer obsession Customer satisfaction increases brand value and drives revenue
  6. 6. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Amazon Athena Amazon EMR Amazon Redshift Amazon Elasticsearch Service Amazon Kinesis Amazon QuickSight AnalyticsAmazon SageMaker AWS Deep LearningAMIs Amazon Rekognition Amazon Lex AWS DeepLens Amazon Comprehend Amazon Translate Amazon Transcribe Amazon Polly Machine learning AWS IoTCore Amazon KinesisData Firehose Amazon KinesisData Streams Amazon KinesisVideo Streams Real-time data movement AWS DirectConnect AWS Snowball AWS Snowmobile AWS DatabaseMigrationService On-premises data movement Data lake on AWS Storage | Archival storage | Data Catalog Incoming analytics data Existing asset metadata Data lake + media assets = Content lake
  7. 7. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Modern data architecture for media enrichment Insights toenhance viewer engagement, personalization, monetization Improve UX / QoS Recommend Monetize Infer Store & process Content lake on AWS Storage | Archival storage | Data Catalog Analyze Viewer data IoT sensors Social media Video Engage and personalize
  8. 8. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  9. 9. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Sample ML use cases for media Automate the creation of rich metadata index (object, scene, activity, faces etc.), extracted from audio visual content and integrated into digital and media asset management systems AWS services to build use case: Amazon Rekognition, Amazon SageMaker, Amazon Transcribe, Amazon Comprehend M e d i a m e t a d a t a t a g g i n g Automated captions, transcription, and translation of audio content AWS services to build use case: Amazon Transcribe, Amazon Translate, Amazon Comprehend C l o s e d c a p t i o n i n g Detect potentially inappropriate content to avoid compliance issues in global markets and increase brand safety for advertisers AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker A u t o m a t e d c o m p l i a n c e m a r k i n g Identify objects and emotion in content that enables users to integrate personalized ads into subscriber video streams AWS services to build use case: Spark/Amazon EMR, Amazon SageMaker, AWS Elemental MediaTailor A d p e r s o n al i z a t i o n
  10. 10. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Sample ML use cases for media Identify and path player activity in the field or identify the precise timecodes when actors enter and leave a scene, both visually and in spoken dialog (off-screen voice) AWS services to build use case: Amazon Rekognition, Amazon Transcribe P l a y e r / A c t o r i d e n t i f i c a t i o n & a c t i v i t y p a t h i n g Translate transcripts and metadata; improve localization workflows and search experience AWS services to build use case: Amazon Translate L a n g u a g e t r a n s l a ti o n Analyze disparate data and enable the consumer to make personalized content choices and the business to predict behavior AWS services to build use case: Amazon SageMaker, Amazon Personalize C o n t e n t r e c o m m e n d a ti o n s Detect and pixelate faces captured incidentally to preserve the privacy of nonpersons of interest in news feeds and security footage AWS services to build use case: Amazon Rekognition, Amazon Transcribe, Amazon SageMaker A u t o m a t e d r e d a c t i o n
  11. 11. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Usecase: From manualtagging to AI-based tagging Background • Over 200,000 hoursof content • Only half of contentis manually tagged • How can we enrich our metadatain AWS? • How can we unleash the value of content we already own once in AWS? Challenge • Large-scale video library • High accuracy required • Limited budget • Ability to extract from video • Keep up with daily increase in content Results • Solution developed withinthree weeks • Live video frame-based analysis • Established,searchable baseline archive • All contentis now tagged and indexed • Over 99,000 faces indexed and searchable • Saved ~9,000 hoursa year in manual curation costs
  12. 12. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Use case: Event broadcasting enrichment
  13. 13. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  14. 14. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T The Amazon ML stack M L f r a m e w o r k s & i n f r a s tr u c t u r e A I s e r v i c es A m a z o n R e k o g n i t i o n I m a g e A m a z o n P o l l y A m a z o n T r a n s c r i b e A m a z o n T r a n s l a t e A m a z o n C o m p r e h e n d & C o m p r e h e n d M e d i c a l A m a z o n L e x A m a z o n R e k o g n i t i o n V i d e o Vi si on Speech Chatbots A m a z o n S a g e M a k e r B u i l d T r a i n A m a z o n F o r e c a s t A m a z o n T e x t r a c t A m a z o n P e r s o n a l i z e D e p l o y Pre-bui l t al gori thms & notebooks Data l abel i ng (G r o u n d T r u t h ) One-cl i ck model trai ni ng & tuni ng Opti mi zati on (N e o ) One-cl i ck depl oyment & hosti ng M L s e r v i c e s F r a m e w o r k s I n t e r f a c e s I n fr a s t r u c t u r e A m a z o n E C 2 P 3 & P 3 d n A m a z o n E C 2 C 5 F P G A s A W S I o T G r e e n g r a s s A m a z o n E l a s t i c I n f e r e n c e Hyperparameter opti mi zati on Al go ri th ms & mo d el s ( A W S M a r k e t p l a c e f o r m a c h i n e l e a r n i n g ) Language Forecasti ng Recommendati ons No teb o o k h osti ng Rei nforcement l earni ng Automati c scal i ng Vi rtual pri vate cl oud AWS Pri vateLi nk El asti c Inference i ntegrati on
  15. 15. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Amazon Rekognition Deep learning-based image analysis service Object and scene detection Facial analysis Face search & match Text in imageContent moderation Celebrity recognition
  16. 16. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Amazon Rekognition Video Analysis for live or stored video content
  17. 17. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Amazon Transcribe Automatic speech recognition for stored or real-time audio TimestampsSupport for both regular & telephony audio Punctuation & formatting English, Spanish, French,Italian, Brazilian Portuguese Recognize multiple speakers Custom dictionaries ? !
  18. 18. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  19. 19. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T AWS media solutions Accelerate revenue Get started with your project quickly and start monetizing your content One-click launch Launch a fully tested solution starter kit in your AWS account with one click Well architected Rely on the architecture as it reflects AWS best practices Agility Save the resources of developing a solution from the beginning Expand and add Use the framework and available guides to expand the solution starter kit based on your needs
  20. 20. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T AWS media solutions http://aws.amazon.com/solutions/ Filter by: Industry > Media & Entertainment
  21. 21. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  22. 22. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Media analysis solution Label & face detection Celebrity detection Face search Activity tracking VoD subtitle generation Key entity & phrase detection Automated metadata generationA m a z o n R e k o g n i t i o n A m a z o n R e k o g n i t i o n V i d e o A m a z o n T r a n s c r i b e A m a z o n T r a n s l a t e A m a z o n C o m p r e h e n d Transcription Context Amazon SageMaker + Additional solutionsfor media use cases
  23. 23. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Media analysis solution Amazon Rekognition Amazon Transcribe Amazon Comprehend Amazon S3 AWS Step Functions AWS Lambda Amazon Elasticsearch Service Image Video AudioText Amazon SageMaker
  24. 24. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  25. 25. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  26. 26. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Live subtitling solution • Purpose built to help customers create live streaming subtitling workflows on the AWS Cloud • Serverless architecture • Uses new Amazon TranscribeStreaming to extract speech for subtitles, then uses Amazon Translateto convert into other subtitle languages • Deploy in minutes with a single click using an AWS CloudFormation template • Automatically spins up all resources needed for the entire workflow AWS Elemental MediaLive Lambda function (caption creation) Amazon Transcribe Amazon S3 bucket Lambda function (transcribe streaming) AWS Elemental MediaPackage Amazon Translate Manifests Live channel source Amazon S3 bucket Multi-language (optional) PCM audio Transcripts Subtitles,manifests,&videofiles PCM audio Amazon CloudFront
  27. 27. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Live subtitling solution Ingest live video into AWS Elemental MediaLive Amazon S3 events trigger subtitle creation process with Amazon Transcribe Streaming Translatespeech to other languages with Amazon Translate Live video with subtitles are ingested into AWS MediaPackager View your live stream in HLS, DASH, or Microsoft Smooth Streaming 1 2 3 4 5 AWS Elemental MediaLive Lambda function (caption creation) Amazon Transcribe Amazon S3 bucket Lambda function (transcribe streaming) AWS Elemental MediaPackage Amazon Translate Manifests Live channel source Amazon S3 bucket Multi-language (optional) PCM audio Transcripts Subtitles,manifests,&videofiles PCM audio Amazon CloudFront 1 2 3 4 5
  28. 28. © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.S UM M I T Live subtitling solution • Purpose built to help customers to create live streaming subtitling workflows on the AWS Cloud • Serverless architecture • Uses new Amazon TranscribeStreaming to extract speech for subtitles then Amazon Translateto convert into other subtitle languages • Deploy in minutes with a single click using an AWS CloudFormation template • Automatically spins up all resources needed for the entire workflow • Optionally add text to speech with Amazon Polly or use Amazon Comprehend to understand text sentiment AWS Elemental MediaLive Lambda function (caption creation) Amazon Transcribe Amazon S3 bucket Lambda function (transcribe streaming) AWS Elemental MediaPackage Amazon Translate Manifests Amazon S3 bucket Multi-language (optional) PCM audio Transcripts Subtitles,manifests,&videofiles PCM audio Amazon CloudFront Amazon Polly Amazon Comprehend Alternate audio(optional) Insights& sentiment(optional)
  29. 29. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.
  30. 30. Thank you! S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved. Bryan Samis samisb@amazon.com
  31. 31. S UM M I T © 2019, Amazon Web Services, Inc. orits affiliates. All rights reserved.

×