Deep learning continues to push the state of the art in domains such as computer vision, natural language understanding, and recommendation engines. In this session, you learn how to get started with the TensorFlow deep learning framework using Amazon SageMaker, a platform to easily build, train, and use to deploy models at scale. You learn how to build a model using TensorFlow by setting up a Jupyter Notebook to get started with image and object recognition. You also learn how to quickly train and deploy a model through Amazon SageMaker.

1. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Venkatesh Bagaria
Senior Product Manager, Amazon Rekognition
Micky Dionisio
CTO, Open Influence
BDA303
Amazon Rekognition
Deep Learning-Based Image and Video Analysis

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
Vide
o
Imag
e
&

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Image
Object and scene
detection
Facial
analysis
Face
recognition
Text in image
Deep learning-based image analysis service
Unsafe image
detection
Celebrity
recognition

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Video
Deep learning-based video analysis service
Object and activity
detection
Person tracking Face detection &
recognition
Real-time live
stream
Unsafe video
detection
Celebrity
recognition

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Console Demo

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
DetectLabels
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxLabels": number,
"MinConfidence": number
}
Image API – Request and Response
{
"Labels": [
{
"Confidence": number,
"Name": "string”
}
],
"OrientationCorrection": "string"
}

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
GetLabelDetection
StartLabelDetection
{
“ClientRequestToken": "string",
"JobTag": "string",
"MinConfidence": number,
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string”
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string”
}
}
}
Video API – Request and Response
{
"JobStatus": string,
"StatusMessage": string,
"VideoMetadata": {
"Format": string,
"Codec": string,
"DurationMillis": number,
"FrameRate": float,
"FrameWidth": number,
"FrameHeight": number
},
"NextToken": string,
"Labels": [
{
"Timestamp": number,
"Label":
{
"Name": string,
"Confidence": float
}
}
],
...
JobId

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition
Regions
US West (Oregon)
(us-west-2) US East (N. Virginia)
(us-east-1)
EU (Ireland)
(eu-west-1)
AWS GovCloud (US)
(us-gov-west-1)
Asia Pacific (Tokyo)
(ap-northeast-1)
Asia Pacific (Sydney)
(ap-southeast-2)
US East (Ohio)
(us-east-2)

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Customers

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Influencer Marketing

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Micky Dionisio, CTO

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Influencer Marketing?
Marketing focused on
identifying individuals
who have influence
over potential buyers
Oversaturation of
information has created a
trust deficit
4–10x higher
conversion when
companies turn to
brand advocates

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What is Open Influence?

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Our Use Case: Pain Points with Hashtags
• 2015  Hashtags were extremely popular
• 2018  Influencers don’t use hashtags
• How do we differentiate our offering to gain an edge over the
competition?
Image recognition
First-party data consisting of
over 1B+ data points
+

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
After

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Before & After Comparison

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Operational Efficiency
• Became a company that ran on technology-enabled
services
• Went from sourcing influencers in days or weeks to within
seconds
• Decreased influencer approval process time by 40%
• Over 60% of searches on our platform include some aspect
of label criteria

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dating and Social Platforms

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analyze User-Generated Content
No faces detected
Age Range: 0–2 years
One face detected
10 faces detected

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Analyze User-Generated Content
Person 99.2%
Dog 95.7%
Person 99.2%
Snowboarding 98.1%
No moderation labels
detected

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
2. Submit
image
4. DetectFaces 7. DetectText
1. Upload
3. Store image AWS Lambda AWS
Step Functions
5.DetectLabels 6.DetectModerationLabels
8. Store metadata
& analysis
Amazon
DynamoDB
Amazon
Elasticsearch
Service
Analyze User-Generated Content

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Public Safety

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Person Tracking

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
"Persons": [
{
"Timestamp": number,
"Person":
{
"Index": number,
"BoundingBox":
{
"Width": number,
"Top": number,
"Height": number,
"Left": number
},
"Face":
{
"BoundingBox": { ... },
"Landmarks": { ... },
"Pose": { ... },
"Quality": { ... },
"Confidence": number
}
},
...
GetPersonTracking
StartPersonTracking
Person Tracking

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Real-time face search against tens of millions of faces
Face Search

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Persons of Interest
300,000+ images of previous offenders indexed in 2 days
Match time is reduced from multiple days to seconds

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Persons of Interest

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Immediate Response for Public Safety or Home Automation
Live Street Camera Amazon Kinesis Video Streams Amazon Rekognition Video Face collection
1. Camera-captured video
streams are processed by
Kinesis Video Streams
2. Amazon Rekognition Video analyses
the video and searches faces on screen
against a collection of millions of faces
L i v e r e c o g n i t i o n o f p e r s o n s o f i n t e r e s t
User
3. User is notified
if face matches
Amazon SNS AWS Lambda Amazon Kinesis
Data Streams

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Authentication

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face-Based Verification

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Face-Based Verification
 Multi-factor authentication
 Identity verification
 Use cases:
 Access control
 New customer enrolment
 Remote password reset

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Real-Time Face Search

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Device camera
1. Images stored in S3
AWS Lambda
Amazon DynamoDB
Amazon Rekognition
Amazon Cognito
Face collectionAmazon S3
Amazon Rekognition
2. Use Lambda to
process images with
Amazon Rekognition
3. Index faces into a
face collection, get
FaceId
4. Create person name
and collection stats
metadata with FaceId5. SearchFacesByImage
with the collection
using AWS SDK
6. Get back search
results with names and
stats
Face Search Demo

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Best Practices for Faces
• Faces at least ~5% of shorter image dimension,
minimum 24–32 pixels
• For live use cases, send image as bytes (5-MB limit)
• Reduce image resolution for faster response
• Add multiple reference images per identity
• Face model versioning

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Developer Resources for Amazon
Rekognition
Homepage: https://aws.amazon.com/rekognition/
AWS Machine Learning Blog – Amazon Rekognition:
https://aws.amazon.com/blogs/ai/tag/amazon-rekognition/
Serverless Image Recognition Processing Backend:
https://github.com/awslabs/lambda-refarch-imagerecognition
Reviewing user-generated content demo code:
https://github.com/aws-samples/amazon-rekognition-policing-user-content

39. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Please complete the session survey in
the summit mobile app.

40. Submit Session Feedback
1. Tap the Schedule icon. 2. Select the session
you attended.
3. Tap Session Evaluation
to submit your feedback.

41. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Thank you!