by Mikhail Prudnikov, Solutions Architect, AWS This session will introduce you to Amazon Rekognition, a service that makes it easy to add image analysis to your applications. With Rekognition, you can detect objects, scenes, and faces in images. You can also search and compare faces. Rekognition’s API lets you easily build powerful visual search and discovery into your applications. With Amazon Rekognition, you only pay for the images you analyze and the face metadata you store. There are no minimum fees and there are no upfront commitments. To get started with Rekognition, simply log in to the Rekognition console to try the service with sample photos or your own photos. Join this session and learn more about Amazon Rekognition!

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
An Introduction to Amazon Rekognition
Deep learning-based image recognition
Mikhail Prudnikov, Senior Solutions Architect
Amazon Web Services
September 14, 2017

2. Amazon AI
Intelligent Services Powered By Deep Learning

3. Rekognition: Extract Metadata from Visual Content
objects, scenes, facial attributes, people
rich media
index

4. Boat 99.3%
Plant 95.1%
Harbor 94.8%
Yacht 78.1%
Dock 75.7%
City 72.4%
Architecture 71.8%
Urban 63.9%
Building 62.3%
Marina 60.3%
Plaza 51.1%
Spire 50.8%
Neighborhood 50.7%
Flower 50.6%
Waterfront 94.8%

5. DetectLabels
"Labels": [
{
"Confidence": 98.9294204711914,
"Name": "Moss"
},
{
"Confidence": 98.9294204711914,
"Name": "Plant"
},
{
"Confidence": 97.35887908935547,
"Name": "Creek"
},
{
"Confidence": 97.35887908935547,
"Name": "Outdoors"
},
{
"Confidence": 97.35887908935547,
"Name": "Stream"
},
{
"Confidence": 97.35887908935547,
"Name": "Water"
},

6. DetectLabels API
Request Parameters:
Image: Image Bytes or S3 Object
Maximum labels to return
Minimum confidence to return
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxLabels": number,
"MinConfidence": number
}

7. DetectLabels API
Response Parameters:
Array of detected Labels
Ø Confidence score
Ø Name of label
Orientation correction on
the entire image
{
"Labels": [
{
"Confidence": number,
"Name": "string"
}
],
"OrientationCorrection":
"string"
}

8. Age Range 38-59
Beard: False 84.3%
Emotion: Happy 86.5%
Eyeglasses: False 99.6%
Eyes Open: True 99.9%
Gender: Male 99.9%
Mouth Open: False86.2%
Mustache: False 98.4%
Smile: True 95.9%
Sunglasses: False 99.8%
Bounding Box
Height: 0.36716..
Left: 0.40222..
Top: 0.23582..
Width: 0.27222..
Landmarks
EyeLeft
EyeRight
Nose
MouthLeft
MouthRight
LeftPupil
RightPupil
LeftEyeBrowLeft
LeftEyeBrowRight
LeftEyeBrowUp
:
Quality
Brightness 52.5%
Sharpness 99.9%

9. "BoundingBox": {
"Height": 0.3449999988079071,
"Left": 0.09666666388511658,
"Top": 0.27166667580604553,
"Width": 0.23000000417232513
},
"Confidence": 100,
"Emotions": [
{"Confidence": 99.1335220336914,
"Type": "HAPPY" },
{"Confidence": 3.3275485038757324,
"Type": "CALM"},
{"Confidence": 0.31517744064331055,
"Type": "SAD"}
],
"Eyeglasses": {"Confidence": 99.8050537109375,
"Value": false},
"EyesOpen": {Confidence": 99.99979400634766,
"Value": true},
"Gender": {"Confidence": 100,
"Value": "Female”}
DetectFaces
smart cropping
& ad overlays
sentiment
capture
demographic
analysis
face editing
& pixelation

10. Sentiment and
Demographic
Analysis
"AgeRange": {
"High": 68,
"Low": 48 }
"Gender": {
"Confidence": 99.926…,
"Value": "Male“ }
"Emotions": [
{ "Confidence": 99.449…,
"Type": "HAPPY” } …
"Smile": {
"Confidence": 73.576…,
"Value": true }
"AgeRange": {
"High": 55,
"Low": 35 }
"Gender": {
"Confidence": 100,
"Value": “Female“ }
"Emotions": [
{ "Confidence": 99.885…,
"Type": "HAPPY” } …
"Smile": {
"Confidence": 99.075…,
"Value": true }
service desks
hotel lobbies
focus groups
pre-screenings

11. Similarity 93%
Similarity 0%

12. "FaceMatches": [
{"Face": {"BoundingBox": {
"Height": 0.2683333456516266,
"Left": 0.5099999904632568,
"Top": 0.1783333271741867,
"Width": 0.17888888716697693},
"Confidence": 99.99845123291016},
"Similarity": 96
},
{"Face": {"BoundingBox": {
"Height": 0.2383333295583725,
"Left": 0.6233333349227905,
"Top": 0.3016666769981384,
"Width": 0.15888889133930206},
"Confidence": 99.71249389648438},
"Similarity": 0
}
],
"SourceImageFace": {"BoundingBox": {
"Height": 0.23983436822891235,
"Left": 0.28333333134651184,
"Top": 0.351423978805542,
"Width": 0.1599999964237213},
"Confidence": 99.99344635009766}
}
CompareFaces

13. How AI Analyzes Faces
Face Detection Landmark Feature Extraction Identification/Recognition
Attributes Verification/Comparison
Index/Search
Estimated age range,
gender, and emotion;
facial hair, smiling++
Face comparison,
match, index and
search

15. Collection
IndexFaces
SearchFacesbyImage
Nearest neighbor
search
FaceID: 4c55926e-69b3-5c80-8c9b-78ea01d30690
Similarity: 97
FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d
Similarity: 92
FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d
Similarity: 85

16. Celebrity Recognition
Recognize hundreds of thousands of global, public figures
politics, sports, entertainment, business, and media
RecognizeCelebrities
"CelebrityFaces": [
{
"Id": "2oi1w8u",
"MatchConfidence": 95,
"Name": "Andy Jassy",
"Urls": []
}

17. Celebrity Recognition at Scale
• Build an index of public figures from archived media
• Sync with video for a dynamic second screen experience
AWS Batch
integration
Elasticsearch
integration

18. Image Moderation
Detect images with explicit or suggestive adult content
Automate and optimize manual review processes
Hierarchical taxonomy provides greater control for geo-sensitive content
"ModerationLabels": [
{
"Confidence": 83.55088806152344,
"Name": "Suggestive",
"ParentName": ""
},
{
"Confidence": 83.55088806152344,
"Name": "Female Swimwear Or Underwear",
"ParentName": "Suggestive"
}
]
}
DetectModerationLabels

19. Image Moderation
Detect images with explicit or suggestive adult content
Automate and optimize manual review processes
Hierarchical taxonomy provides greater control for geo-sensitive content
Top-Level Category Second-Level Category
Explicit Nudity
Nudity
Graphic Male Nudity
Graphic Female Nudity
Sexual Activity
Partial Nudity
Suggestive
Female Swimwear Or Underwear
Male Swimwear Or Underwear
Revealing Clothes

20. https://console.aws.amazon.com/rekognition/home

21. What can you do with
Amazon Rekognition?

22. Relevant APIs:
For people, objects, scenes, and concepts across millions of images
Search
DetectLabels SearchFacesByImage SearchFaces
AWS Lambda
integration

23. Integration With Lambda

24. For people, objects, scenes, and concepts across millions of images
Search
Washington County Sheriff (OR)
300,000+ images of previous offenders indexed in 2 days
Face match reduced from multiple days to seconds
First suspect identified within first week

26. Relevant APIs:
For inappropriate or specific content, target demographics, and sentiment
Filter
DetectModerationLabels DetectLabels DetectFaces

27. For inappropriate or specific content, target demographics, and sentiment
Filter

28. Analyze
Anonymous, high volume analysis of demographic and sentiment
DetectFaces

29. Relevant APIs:
Identify people within a set of millions of faces
Recognize
SearchFacesByImage SearchFaces

30. A u t o m a t i n g F o o t a g e
T a g g i n g w i t h A m a z o n
R e k o g n i t i o n
Built in 3 weeks
Index against 99,000 people
Saving ~9,000 hours a year in labor
Identify people within a set of millions of faces
Recognize

31. C-SPAN Indexing Architecture
Video feeds encoded from
8 locations (3 networks and
5 federal courthouses)
Frames extracted into
JPGs and hosted in
Amazon S3
Amazon SQS provides
asynchronous decoupling
Search Amazon Rekognition
collection for high similarity
matches
Results cache
drives search and
discovery requests
R3 hashing detects if a
scene significantly changes

33. Identify people within a set of millions of faces
Recognize

34. Relevant APIs:
Identities by matching against reference faces
Verify
CompareFaces

35. Relevant APIs:
Identities or inappropriate images
Redact
DetectFaces DetectModerationLabels

36. Amazon Rekognition
Customers
• Digital Asset Management
• Media and Entertainment
• Travel and Hospitality
• Influencer Marketing
• Systems Integration
• Digital Advertising
• Consumer Storage
• Law Enforcement
• Public Safety
• eCommerce
• Education

37. Amazon Rekognition Availability and Pricing
Free Tier: 5000 images processed per month for first 12 months
General Availability in 3 regions:
US East (N. Virginia), US West (Oregon); EU (Ireland)
Image Analysis Tiers Price per 1000
images processed
First 1 million images processed* per month $1.00
Next 9 million images processed* per month $0.80
Next 90 million images processed* per month $0.60
Over 100 million images processed* per month $0.40

38. Developer Resources and more…
https://aws.amazon.com/blogs/ai/
https://aws.amazon.com/rekognition

39. Thank You!