Video has become an increasingly successful medium for advertising, marketing, and engaging customers. However, many companies underutilize their substantial video assets because they are poorly indexed and cataloged. In this workshop, learn how to use machine learning services to gain more value from video by building a customer celebrity detection feature that can recognize mainstream celebrities and individuals from your own uploaded media files.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Build a "Who's Who" App for Your Media
Content
Kashif Imran
Sr. Solutions Architect, Amazon Rekognition
Amazon Web Services -- AI
A I M 4 0 9

3. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition:
Deep-Learning-Based Image and Video Analysis

4. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Features
Faces Celebrities Labels
Moderation ScenesActivities Paths
Text

5. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition benefits
Low cost
Your data
is your ownFully Managed
Rapid integrationState of the
art capabilities
Continuous
improvement

6. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Customer
engagement Industrial
… and more
Safe content Public safety Mapping
Amazon Rekognition Customers

7. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Media discovery
Customer
engagement Industrial
…and more
Safe content Public safety Mapping
Focus of Today’s Workshop

8. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity guests at the Royal Wedding

9. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity detection

10. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity Recognition – Image API
RecognizeCelebrities
{
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
}
}
{
"CelebrityFaces": [
{
"Face": {
"BoundingBox": {
...
},
"Confidence": number,
"Landmarks": [
{
...
}
],
"Pose": {
...
},
"Quality": {
...
}
},
"Id": "string",
"MatchConfidence": number,
"Name": "string",
"Urls": [ "string" ]
}
],
"OrientationCorrection": "string",
"UnrecognizedFaces": [
{
...
}
]
}

11. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

12. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity Recognition – Video API
GetCelebrityRecognition
StartCelebrityRecognition
{
“ClientRequestToken": "string",
"JobTag": "string",
"MinConfidence": number,
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string”
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string”
}
}
}
{
"Celebrities": [
{
"Celebrity": {
"BoundingBox": {
...
},
"Confidence": number,
"Face": {
"BoundingBox": {
...
},
"Confidence": number,
...
},
"Id": "string",
"Name": "string",
"Urls": [ "string" ]
},
"Timestamp": number
}
],
"JobStatus": "string",
"NextToken": "string",
"StatusMessage": "string",
"VideoMetadata": {
"Codec": "string",
"DurationMillis": number,
"Format": "string",
"FrameHeight": number,
"FrameRate": number,
"FrameWidth": number
}
}
JobId

13. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

14. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recognize other public figures in your media

15. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recognize other public figures in a photo or video using your own repository of face images
Search
Index
Collection
Custom Celebrity Recognition

16. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Facial Recognition – IndexFaces
IndexFaces
{
"CollectionId": "string",
"DetectionAttributes": [ "string" ],
"ExternalImageId": "string",
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxFaces”: number
”QualityFilter” : string
}
{
"FaceModelVersion": "string",
"FaceRecords": [
{
"Face": {
"BoundingBox": {
...
},
"Confidence": number,
"ExternalImageId": "string",
"FaceId": "string",
"ImageId": "string"
},
"FaceDetail": {
"AgeRange": {
"High": number,
"Low": number
},
...
"Pose": {
"Pitch": number,
"Roll": number,
"Yaw": number
},
"Quality": {
"Brightness": number,
"Sharpness": number
},
}
}
],
"OrientationCorrection": "string"
}

17. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Rekognition Collection to Amazon DynamoDB Mapping
{
"CollectionId": ”my-collection",
"DetectionAttributes": [ ”DEFAULT" ],
"ExternalImageId": ”2",
"Image": {
"S3Object": {
"Bucket": ”my-bucket",
"Name": ”Kashif-face.jpg",
}
}
}
{
"CollectionId": ”my-collection",
"DetectionAttributes": [ ”DEFAULT" ],
"ExternalImageId": ”1",
"Image": {
"S3Object": {
"Bucket": ”my-bucket",
"Name": ”chris-face.jpg",
}
}
}

18. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Facial Recognition – Image API
SearchFacesByImage
{
"CollectionId": "string",
"FaceMatchThreshold": number,
"Image": {
"Bytes": blob,
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
},
"MaxFaces": number
}
{
"FaceMatches": [
{
"Face": {
"BoundingBox": {
"Height": number,
"Left": number,
"Top": number,
"Width": number
},
"Confidence": number,
"ExternalImageId": "string",
"FaceId": "string",
"ImageId": "string"
},
"Similarity": number
}
],
"FaceModelVersion": "string",
"SearchedFaceBoundingBox": {
"Height": number,
"Left": number,
"Top": number,
"Width": number
},
"SearchedFaceConfidence": number
}

19. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

20. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Facial Recognition - Video API
GetFaceSearch
StartFaceSearch
{
"ClientRequestToken": "string",
"CollectionId": "string",
"FaceMatchThreshold": number,
"JobTag": "string",
"NotificationChannel": {
"RoleArn": "string",
"SNSTopicArn": "string"
},
"Video": {
"S3Object": {
"Bucket": "string",
"Name": "string",
"Version": "string"
}
}
}
JobId
{
"JobStatus": "string",
"NextToken": "string",
"Persons": [
{
"FaceMatches": [
{
"Face": {
"BoundingBox": {
...
},
"Confidence": number,
"ExternalImageId": "string",
"FaceId": "string",
"ImageId": "string"
},
"Similarity": number
}
],
"Person": {
"BoundingBox": {
...
},
"Face": {
"AgeRange": {
"High": number,
"Low": number
},
"BoundingBox": {
...
},
"Confidence": number,
...
},
"Index": number
},
"Timestamp": number
}
],
"StatusMessage": "string",
"VideoMetadata": {
...
}
}

21. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

22. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Recognize public figures in your media

23. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

24. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Architecture

25. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

26. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
What you need before starting
An AWS Account with permissions to create AWS Identity and Access
Management (IAM) role and other resources
Reference Images are provided

27. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Global Infrastructure

28. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Region for this lab
US East (N. Virginia)
US East 2 (Ohio)
US West 2 (Oregon)
EU West 1 (Ireland)

29. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Create Jupyter Notebook Instance

30. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

31. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

32. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

33. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

34. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

35. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

36. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Celebrity Recognition

37. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
https://github.com/darwaishx/celebrity-recognition

38. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AIM417: Build a Searchable Media Library & Moderate
Content at Scale Using Machine Learning
Friday, Nov 30, 8:30 AM - 10:45 AM
– Mirage, Mirage Event Center C1

39. Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

40. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.