Companies have to process, analyze, and extract meaning from ever-growing volumes of audio, image, and video data. Automating media workflows, such as image and video indexing or manual transcription for closed captions, can help you scale the growth of your media library and save time from manual, error-prone work. In this workshop, you learn how to automate workflows using the Media Analysis Solution, which includes Amazon Rekognition, Amazon Transcribe, and Amazon Comprehend. You learn how to extract metadata from media files and create a searchable library of metadata.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build a Searchable Media Library & Moderate
Content at Scale Using Machine Learning
Kashif Imran
Sr. Solutions Architect, Amazon Rekognition
Amazon Web Services -- AI
A I M 4 1 7

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition:
Deep Learning-Based Image and Video Analysis

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Features
Faces Celebrities Labels
Moderation ScenesActivities Paths
Text

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Benefits
Low cost
Your data
is your ownServerless
Rapid
integration
State of the
art capabilities
Continuous
improvement

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Customer
engagement Industrial
… and more
Safe content Public safety Mapping
Amazon Rekognition Customers

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Use cases and features

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Customer
engagement Industrial
…and more
Safe content Public safety Mapping
Amazon Rekognition Customers

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Live
events
Media
libraries
Social
media
Influencer
marketing

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Who
Celebrities
What
Labels and Activities
Where
Scenes and Text
Tools for media discovery

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity detection

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Object and scene detection

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
New features: object and scene detection

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Activities and paths

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Text in the wild

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tinder
Tinder’s mission is to make being single more fun and rewarding by
introducing our members to new people to connect with in real life

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Top Picks
Discover engaging curated potential matches
Available daily to Gold subscribers +
purchasable packs
More than 4M subscribing members
Highlighting unique characteristics via tags
creates a richer experience that stands out
Compelling tags show a 20% engagement lift

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tagging

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition – high pace of innovation
Recent changes provide richer data with
● Enhanced model accuracy
● Hierarchical taxonomy
● Bounding Boxes
These provide deeper knowledge of scenery
Amazon Rekognition now helps power tagging
for Top Picks in São Paulo, Brazil
We’re excited to incorporate these features
and further unlock the power of photos
everywhere on Tinder
The Future

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DetectLabels
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxLabels": number,
"MinConfidence": number
}
{
"Labels": [
{
"Confidence": number,
"Name": "string”
}
],
"OrientationCorrection": "string"
}
Image API – Request and Response

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GetLabelDetection
StartLabelDetection
{
“ClientRequestToken": "string",
"JobTag": "string",
"MinConfidence": number,
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string”
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string”
}
}
}
{
"JobStatus": string,
"StatusMessage": string,
"VideoMetadata": {
"Format": string,
"Codec": string,
"DurationMillis": number,
"FrameRate": float,
"FrameWidth": number,
"FrameHeight": number
},
"NextToken": string,
"Labels": [
{
"Timestamp": number,
"Label":
{
"Name": string,
"Confidence": float
}
}
],
...
JobId
Video API – Request and Response

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
30 minutes
Go to: https://github.com/darwaishx/media-library
Select: 1-MediaAnalysisSolution
Self-Paced Lab Instructions

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Customer
engagement Industrial Mapping
Safe content Public safety Aerial imagery …more
Focus of this module

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Top-level
category
Second-level
category
Confidence
score
Explicit nudity
Nudity XX.X%
Graphic male nudity XX.X%
Graphic female nudity XX.X%
Sexual activity XX.X%
Partial nudity XX.X%
Suggestive
Female swimwear
or underwear
XX.X%
Male swimwear
or underwear
XX.X%
Revealing clothes XX.X%
Rekognition for Content Moderation

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
“With Amazon Rekognition, we were able to reduce the need for human
involvement by 97%, which enabled our employees to focus on more
strategic business tasks and lowered our moderation costs by 72%.”
Maksim Gutman, Engineering Manager, Coffee Meets Bagel
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
>250M
+80%
+75% of content is from
a business or blog
On mobile
+175BPins
Monthly active users
+3BBoards
is a visual discovery engine

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Modify AWS Lambda and AWS Step Function

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Content Moderation Notification

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
60 minutes
Go to: https://github.com/darwaishx/media-library
Select: 2-Content Moderation
Self-Paced Lab Instructions

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What you need before starting
An AWS Account with permissions to create IAM role an other resources
Reference Images are provided

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Global Infrastructure

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy Media Analysis Solution
US East (N. Virginia): https://amzn.to/2FNnVRn
US East (Ohio): https://amzn.to/2TXbwNK
US West (Oregon) https://amzn.to/2DRY4oJ
EU (Ireland) https://amzn.to/2SlGVrK

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Stack Details

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS CloudFormation Stack Complete

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Email with User Credentials

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Deploy Media Analysis Solution
US East (N. Virginia): https://amzn.to/2FNnVRn
US East (Ohio): https://amzn.to/2TXbwNK
US West (Oregon): https://amzn.to/2DRY4oJ
EU (Ireland): https://amzn.to/2SlGVrK

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://github.com/darwaishx/media-library

44. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.