Learn how Amazon Rekognition is using deep learning-based image and video analysis to power more targeted influencer marketing and advertising, analysis of user-generated content on social platforms, real-time public safety alerts, and visual authentication in banking applications. In this session, we provide an overview of Amazon Rekognition image and video features, highlight customer stories from specific vertical use cases, such as influencer marketing, media, and public safety, and walk through some demonstrations and architectures for common use cases.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Venkatesh Bagaria
Senior Product Manager, Amazon Rekognition
Micky Dionisio
CTO, Open Influence
BDA303
Amazon Rekognition
Deep learning-based Image and Video Analysis

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
VideoImag
e
&

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Image
Object and Scene
Detection
Facial
Analysis
Face
Recognition
Text in Image
Deep learning-based image analysis service
Unsafe Image
Detection
Celebrity
Recognition

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Video
Deep learning-based video analysis service
Object and Activity
Detection
Person Tracking Face Detection &
Recognition
Real-time Live
Stream
Unsafe Video
Detection
Celebrity
Recognition

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Console Demo

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DetectLabels
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxLabels": number,
"MinConfidence": number
}
Image API – Request and Response
{
"Labels": [
{
"Confidence": number,
"Name": "string”
}
],
"OrientationCorrection": "string"
}

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GetLabelDetection
StartLabelDetection
{
“ClientRequestToken": "string",
"JobTag": "string",
"MinConfidence": number,
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string”
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string”
}
}
}
Video API - Request and Response
{
"JobStatus": string,
"StatusMessage": string,
"VideoMetadata": {
"Format": string,
"Codec": string,
"DurationMillis": number,
"FrameRate": float,
"FrameWidth": number,
"FrameHeight": number
},
"NextToken": string,
"Labels": [
{
"Timestamp": number,
"Label":
{
"Name": string,
"Confidence": float
}
}
],
...
JobId

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
Regions
US West (Oregon)
(us-west-2) US East (N. Virginia)
(us-east-1)
EU (Ireland)
(eu-west-1)
GovCloud (US)
(us-gov-west-1)
Asia Pacific (Tokyo)
(ap-northeast-1)
Asia Pacific (Sydney)
(ap-southeast-2)
US East (Ohio)
(us-east-2)

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Customers

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Influencer Marketing

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Micky Dionisio, CTO

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Influencer Marketing?
Marketing focused on
identifying individuals
who have influence
over potential buyers
Oversaturation of
information has created a
trust deficit
4-10x higher
conversion when
companies turn to
brand advocates

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is OpenInfluence?

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our Use Case: Pain Points with Hashtags
• 2015  Hashtags were extremely popular
• 2018  Influencers don’t use hashtags
• How do we differentiate our offering to gain an edge over the
competition?
Image Recognition
First party data consisting of
over 1B+ data points
+

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
After

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before & After Comparison

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Operational Efficiency
• Became a company that ran on technology-enabled
services
• Went from sourcing influencers in days/weeks to within
seconds
• Decreased influencer approval process time by 40%
• Over 60% of searches on our platform include some aspect
of label criteria

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dating and Social Platforms

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analyze User Generated Content
No faces detected
Age Range: 0-2 years
One face detected
10 faces detected

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analyze User Generated Content
Person 99.2%
Dog 95.7%
Person 99.2%
Snowboarding 98.1%
No Moderation labels
detected

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. Submit
Image
4. DetectFaces 7. DetectText
1. Upload
3. Store image Lambda Step Functions
5.DetectLabels 6.DetectModerationLabels
8. Store Metadata
& analysis
DynamoDB
ElasticSearch
Analyze User Generated Content

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public Safety

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Person Tracking

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
"Persons": [
{
"Timestamp": number,
"Person":
{
"Index": number,
"BoundingBox":
{
"Width": number,
"Top": number,
"Height": number,
"Left": number
},
"Face":
{
"BoundingBox": { ... },
"Landmarks": { ... },
"Pose": { ... },
"Quality": { ... },
"Confidence": number
}
},
...
GetPersonTracking
StartPersonTracking
Person Tracking

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Real-time face search against tens of millions of faces
Face Search

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Persons of Interest
300,000+ images of previous offenders indexed in 2 days
Match time reduced from multiple days to seconds

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Persons of Interest

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Immediate Response for Public Safety or Home Automation
Live Street Camera Amazon Kinesis Video Streams Amazon Rekognition Video Face collection
1. Camera-captured video
streams are processed by
Kinesis Video Streams
2. Rekognition Video analyses the
video and searches faces on screen
against a collection of millions of faces
L i v e r e c o g n i t i o n o f P e r s o n s o f I n t e r e s t
End User
3. End user is notified
in case of face matches
Amazon SNS AWS Lambda Amazon Kinesis
Streams

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Authentication

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face-based verification

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face-based verification
 Multi-factor authentication
 Identity Verification
 Use Cases:
 Access control
 New customer enrolment
 Remote password reset

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face v3
Improved Accuracy for real-time face search and verification
Up to 25% more accurate in search against millions
of faces
Up to 80% more accurate in rejecting similar faces
Up to 35% more accurate in accepting variations

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Real-time Face Search

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Device Camera
1. Images stored in S3
AWS Lambda
Amazon DynamoDB
Amazon Rekognition
Amazon Cognito
Face
collection
Amazon S3
Amazon Rekognition
2. Use Lambda to
process images with
Rekognition
3. Index faces into a
face collection, get
FaceId
4. Create person name
and collection stats
metadata with FaceId5.
SearchFacesByImage
with the collection using
AWS SDK
6. Get back search
results with names and
stats
Face Search Demo

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices for Faces
• Faces at least ~5% of shorter image dimension,
minimum 24-32 pixels
• For live use cases, send image as bytes (5MB limit)
• Reduce image resolution for faster response
• Add multiple reference images per identity
• Face Model Versioning

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer Resources for Amazon
Rekognition
Homepage: https://aws.amazon.com/rekognition/
AWS Machine Learning Blog – Amazon Rekognition:
https://aws.amazon.com/blogs/ai/tag/amazon-rekognition/
Serverless Image Recognition processing backend:
https://github.com/awslabs/lambda-refarch-imagerecognition
Reviewing User Generated Content Demo Code:
https://github.com/aws-samples/amazon-rekognition-policing-user-content