This document discusses Amazon Rekognition, Amazon's deep learning-based image and video analysis service. It provides an overview of Amazon Rekognition's capabilities for image and video analysis, including object and scene detection, facial analysis, face recognition, text detection in images, and more. It also discusses examples of customers like VidMob that use Amazon Rekognition for tasks like analyzing video content, moderating user-generated content, and powering face-based features.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Venkatesh Bagaria
Senior Product Manager, Amazon Rekognition
Joline McGoldrick
SVP, Data and Insights, VidMob
BDA303
Amazon Rekognition
Deep Learning-Based Image and Video Analysis
James Kupernik
CTO, VidMob

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
VideoImage &

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Image
Object and Scene
Detection
Facial
Analysis
Face
Recognition
Text in Image
Deep Learning-Based Image analysis service
Unsafe Image
Detection
Celebrity
Recognition

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Video
Deep Learning-Based Video analysis service
Object and Activity
Detection
Pathing Face Detection &
Recognition
Real-time Live
Stream
Unsafe Video
Detection
Celebrity
Recognition

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Console demo

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition customers

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Boat 99.3%
Plant 95.1%
Harbor 94.8%
Yacht 78.1%
Dock 75.7%
City 72.4%
Architecture 71.8%
Urban 63.9%
Building 62.3%
Marina 60.3%
Plaza 51.1%
Spire 50.8%
Neighborhood 50.7%
Flower 50.6%
Waterfront 94.8%
Object and Scene detection

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DetectLabels
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxLabels": number,
"MinConfidence": number
}
Image API – Request and Response
{
"Labels": [
{
"Confidence": number,
"Name": "string”
}
],
"OrientationCorrection": "string"
}

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GetLabelDetection
StartLabelDetection
{
“ClientRequestToken": "string",
"JobTag": "string",
"MinConfidence": number,
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string”
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string”
}
}
}
Video API – Request and Response
{
"JobStatus": string,
"StatusMessage": string,
"VideoMetadata": {
"Format": string,
"Codec": string,
"DurationMillis": number,
"FrameRate": float,
"FrameWidth": number,
"FrameHeight": number
},
"NextToken": string,
"Labels": [
{
"Timestamp": number,
"Label":
{
"Name": string,
"Confidence": float
}
}
],
...
JobId

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before After
Influencer Marketing
2.6x more insights, on 30% more posts

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Context from user-generated content
Person 99.2%
Dog 95.7%
Person 99.2%
Snowboarding 98.1%

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Blowing a candle Drinking
Object and Activity detection

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Joline McGoldrick
SVP, Data and Insights, VidMob
James Kupernik
CTO, VidMob
When every second counts,
every frame matters.
How Amazon helps VidMob power deeper insights
and stronger creative.
©2018, AmazonWebServices, Inc. or itsaffiliates. All rightsreserved.
Joline McGoldrick
SVP, Data and Insights, VidMob
James Kupernik
CTO, VidMob
When every second counts,
every frame matters.
How Amazon helps VidMob power deeper insights
and stronger creative.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
We AreVidMobisacreative technology
platformthat connectsbrands
withaglobal networkof expert
creative talent toproduce,analyze
andoptimize mobile video.

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
ADD INTEGRATION SLIDE (THAT WE’VE USED BEFORE,
HERE)

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Powerat your
ngertipswith
WatchDashboard.
Rekognition allowsclientsto understand
every frame of their creative.
By breaking down all of the components
of an ad –millisecond by millisecond –
creatorscan understand how each element
of the creative in uencesviewthrough and
how it differsacrossaudiences.
The insightspowered by Rekognition and
Transcribe give marketersthe toolsto re-
edit and evolve their videosto better hold
their audiences’ attention.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Format &Audience
DeepDive
Rolling up tagsover campaigns,quarters
or yearspowerstrendspotting.
Creatorscan understand how akey
visual or theme can have adifferent
impact depending on the format or
audience.
Thispowerssmarter versioning and
personalization by audience segment.

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
22,500
HOURS
21
DAYS
+80%
ROI

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Image and Video Moderation
Hierarchical taxonomy provides greater control for users
Top-Level Category Second-Level Category
Explicit Nudity
Nudity
Graphic Male Nudity
Graphic Female Nudity
Sexual Activity
Partial Nudity
Suggestive
Female Swimwear Or Underwear
Male Swimwear Or Underwear
Revealing Clothes

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reviewing user-generated content

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Image quality
Facial landmarks
Demographic data Emotions
General attributes
Facial pose
Brightness 24.0%
Sharpness 99.9%
EyeLeft,EyeRight,Nose
RightPupil,LeftPupil
MouthRight,LeftEyeBrowUp
Age Range 29–45
Gender: Male 91.6%
Happy 90.38%
Smile:True 99.8%
EyesOpen:True 99.7%
Beard:True 99.8%
Mustache:True 99.6%
Pitch 2.059
Roll 4.569
Yaw 2.970
Facial Detection and Analysis

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
"BoundingBox": {
"Height": 0.3449999988079071,
"Left": 0.09666666388511658,
"Top": 0.27166667580604553,
"Width": 0.23000000417232513
},
"Confidence": 100,
"Emotions": [
{"Confidence": 99.1335220336914,
"Type": "HAPPY" },
{"Confidence": 3.3275485038757324,
"Type": "CALM"},
{"Confidence": 0.31517744064331055,
"Type": "SAD"}
],
"Eyeglasses": {"Confidence": 99.8050537109375,
"Value": false},
"EyesOpen": {Confidence": 99.99979400634766,
"Value": true},
"Gender": {"Confidence": 100,
"Value": "Female”}
DetectFaces

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reviewing uploaded photos
No faces detected Eyes open: False 99.99% 10 faces detected

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face Search

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Finding missing people

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity recognition

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face-based authentication
• Facility access control
• Know your customer
• Remote password reset

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
‘Face Ticket’

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Collection
IndexFaces
SearchFacesbyImage
FaceID: 4c55926e-69b3-5c80-8c9b-78ea01d30690
Similarity: 97
FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d
Similarity: 92

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Device Camera
1. Images stored in
Amazon S3
AWS Lambda
Amazon DynamoDB
Amazon Rekognition
Amazon Cognito
Face collectionAmazon S3
Amazon Rekognition
2. Use Lambda to process
images with Amazon
Rekognition
3. Index faces into a face
collection, get FaceId
4. Create person name
and collection stats
metadata with FaceId5. SearchFacesByImage with
the collection using AWS SDK
6. Get back search results
with names and stats
Face Search architecture

42. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best practices for faces
• Face size: At least 24–32 pixels or ~5%
• Filter out poor quality faces when indexing
• Blurry
• Too small
• Extreme pose
• For interactive use cases, send image as bytes
• Reduce image resolution for faster response
• Multiple training images per identity

43. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Text in Image
Extract textual content from real-world images in various layouts, fonts, and styles

44. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reading road signs

45. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reading marathon bibs

46. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. Submit
Image
4 .DetectFaces 7. DetectText
1. Upload
3. Store image AWS Lambda AWS Step Functions
5. DetectLabels 6. DetectModerationLabels
8. Store metadata &
analysis Amazon
DynamoDB
Amazon ES
Content review system

47. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer resources for Amazon Rekognition
Homepage: https://aws.amazon.com/rekognition/
Amazon Machine Learning Blog – Amazon Rekognition:
https://aws.amazon.com/blogs/ai/tag/amazon-rekognition/
Serverless image recognition processing backend: https://github.com/awslabs/lambda-
refarch-imagerecognition
Reviewing user generated content demo code:
https://github.com/mbtaws/rekognition-reviewing-user-content

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Submit session feedback
1. Tap the Schedule icon.
2. Select the session you attended.
3. Tap Session Evaluation to submit your feedback.

49. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!