Analisi avanzata di video e immagini con i servizi AI di AWS Italy webinar series - June 2018 Speaker: Woody Borraccino, AWS Solutions Architect

1. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Analisi avanzata di video e immagini
con i servizi di Artificial Intelligence di AWS
Woody Borraccino, Solutions Architect

2. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
What is Machine Learning?
https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/

3. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
©NVIDIA
Raw Data Low Level Features M id Level Features High Level Features
Result
Application
Components
Task
Identify a Face
Training
10-100M images
Network
~ 10 layers
1B parameters
Learning
~ 30 Exaflops
~ 30 GPU days
© 2016 NVIDIA
Input
The 10,000ft Intro to Deep Learning

4. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Confidence Labels
99.2%
Animal
Dog
Chihuahua
98.6%
Food
Dessert
Muffin
97.9% Collage
Dog or Muffin?

5. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Keys to Success
Business ValueAbility to ExecuteData Availability

6. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
ML: Driving Outcomes
51% said AI initiatives goal
is to improve products quality

7. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.

8. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
https://news.sky.com/whoswho
Sky News - Royal Wedding

9. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AI/ML at Amazon
Investments in AI/ML for more than 20 years.
Thousands of engineers focused on AI/ML.
Search &
Discovery
Fulfilment
&
Logistics
Existing
Products
New
Initiatives
More AI/ML is built on AWS than any
other platform in the world.

10. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Solutions for Every Skill Level
• Designed for Developers & Data Scientists
• Solution-oriented Prebuilt Models Available via APIs
• Image Analysis, NLU, NLP, Translation, Text-to-Speech & Speech-to-Text
• Designed for Data Scientists to Address Common & Advanced ML Needs
• Fully Managed Platform for Model Building
• Reduces the Heavy Lifting in Model Building & Deployment
• Designed for Data Scientists to Address Advanced / Emerging Needs
• Provides Maximum Flexibility to develop on the leading AI Frameworks
• Enables Expert AI Systems to be Developed & Deployed
Services
Platforms
Frameworks

11. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
The ML Stack
Services
Language
Comprehend, Translate,
Transcribe, Polly
Vision
Rekognition Image &
Rekognition Video
Chatbots
Lex
Platforms
Frameworks
Infrastructure
Apache
MXNet
KerasGluonPyTorch
Cognitive
Toolkit
Caffe2
& Caffe
Tensor-
Flow
AWSDeepLearningAMI
GPU
(P3)
MobileCPU
IoT
(GreenGrass)
SageMaker
Mechanical
Turk
AWS
DeepLens
Amazon ML
Spark &
EMR

12. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
The ML Stack
Services
Language
Comprehend, Translate,
Transcribe, Polly
Vision
Rekognition Image &
Rekognition Video
Chatbots
Lex
Platforms
Frameworks
Infrastructure
Apache
MXNet
KerasGluonPyTorch
Cognitive
Toolkit
Caffe2
& Caffe
Tensor-
Flow
AWSDeepLearningAMI
GPU
(P3)
MobileCPU
IoT
(GreenGrass)
SageMaker
Mechanical
Turk
AWS
DeepLens
Amazon ML
Spark &
EMR

13. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
The ML Stack
Services
Language
Comprehend, Translate,
Transcribe, Polly
Vision
Rekognition Image &
Rekognition Video
Chatbots
Lex
Platforms
Frameworks
Infrastructure
Apache
MXNet
KerasGluonPyTorch
Cognitive
Toolkit
Caffe2
& Caffe
Tensor-
Flow
AWSDeepLearningAMI
GPU
(P3)
MobileCPU
IoT
(GreenGrass)
SageMaker
Mechanical
Turk
AWS
DeepLens
Amazon ML
Spark &
EMR

14. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition
Easy-to-use deep learning-based computer vision analysis
Fully
managed
service
Easy-to-use
API
Low costImage
analysis
Video
analysis

15. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Detect objects, scenes, and faces, extract text, recognize celebrities, and identify
unsafe content in images
Object and Scene
Detection
Face
Recognition
Unsafe Image
Detection
Celebrity
Recognition
Facial
Analysis
Text in Image Face
Comparison
Amazon Rekognition Image

16. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
{
"FaceMatches": [
{"Face": {"BoundingB
"Height":
0.2683333456516266,
"Left":
0.5099999904632568,
"Top":
0.1783333271741867,
"Width":
0.17888888716697693},
"
{
"FaceMatches": [
{"Face": {"BoundingB
"Height":
0.2683333456516266,
"Left":
0.5099999904632568,
"Top":
0.1783333271741867,
"Width":
0.17888888716697693},
"
Rekognition APIs
Overview
CompareFaces
DetectFaces
DetectLabels
DetectModerationLabels
GetCelebrityInfo
RecognizeCelebrities
Non-storage API Operations
CreateCollection
DeleteCollection
DeleteFaces
IndexFaces
ListCollections
SearchFaces
SearchFacesByImage
Storage-based API Operations
ListFaces
Detect-Text

17. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Complementary Services
Decoupling
Am azon
SQ S
Am azon
SNS
Am azon
KinesisProcessing
Am azon API
Gatew ay
AW S Batch
Am azon
EC2 Am azon
ECS
Compute
Applications
AW S
Lam bda
Storage
Am azon
EFSAm azon
S3
Persistence
Am azon
Dynam oDB
Am azon
ElasticSearch
AW S
Elem ental
M ediaConvert AW S
Elem ental
M ediaLive

18. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
moderation level = safe
Building Rich Metadata Indexes using Rekognition
Name: You?

19. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Sky News - Royal Wedding
Architecture

20. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Video Analysis
Limitations of traditional solutions
Tem poral inform ation lost M otion context lost

21. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Analyze activity, recognize, and track people in stored and live video
Facial
Analysis
Amazon Rekognition Video

22. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Amazon Kinesis Video Streaming
Capture, process, and store video streams for analytics/ML
Am azon AI
Services
Apache
M XNet
TensorFlow
O penCV
Custom
Video
Processing
Kinesis
Video
Stream s
Stream video from millions of devices
Easily build vision-enabled apps
Secure
Durable, searchable storage
Serverless

23. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Video
Media and entertainment
Create metadata for celebrities, emotions, key topics in video
with time segments for recommendation engines and ad
placement
Automatically detect unsafe
content, based on market requirements
Extract data in streaming mode to enhance
user engagement

24. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Automated Media Metadata Tagging
Use case : Video Search Index
Video AmazonS3 AWSLambda
Rekognition
Video
AmazonElasticsearch Asset Management System
1. Video is uploaded
and stored to S3
2. Create m etadata for
celebrities, em otions, scene
tim e, objects, voices in video
4. Lam bda also pushes
the m etadata and confidence
scores into Elasticsearch
3. The output is sent to the
digital/m edia asset
m anagem ent system
D y n a m i c s e a r c h i n d e x i n g
Transcribe

25. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.

26. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Soccer Media Analysis Example
START DEMO

27. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Person in Scene Detection
Amazon S3 Face collectionRekognition
1. Store im ages of recognized people 2. Index individuals into a collection
R e c o g n i t i o n a n d t r a c k i n g o f o n - s c r e e n p e r s o n s
Amazon S3 Rekognition Video
3. Store video files
4. Track persons on screen
and recognize individual
faces and voices
Videos AWS Lambda
Transcribe
5. Sum m arize tim e
on screen detail

28. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.

29. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Face Search—Public Safety

30. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Use-Case: C-SPAN
• Over 200,000 hours of content
• Only half of content is manually tagged
• How can we enrich our metadata in AWS?
• How can we unleash the value of content we already
own once in AWS?
Background
• Large scale video library
• High accuracy required
• Limited budget
• Ability to extract from video
• Keep up with daily increase in content
Challenge
• Solution developed within three weeks
• Live video frame based analysis
• Established, searchable baseline archive
• All content is now tagged and indexed
• Over 99,000 faced indexed and searchable
• Saved ~9,000 hours a year in manual curation costs
Results

31. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Amazon Rekognition Customers

32. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS Step Functions
Orchestrate video analysis
• Built-in error handling, with
automatic retry
• Automatic scaling (serverless)
• High availability (Multi-AZ)
• Secure (IAM based)
• Detailed event logs
• Pay per use

33. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Open Solution
A Good Starting Point: Media Analysis Solution
https://aws.amazon.com/it/answers/media-entertainment/media-analysis-solution/

34. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Thank You!
Woody Borraccino, Solutions Architect, AWS