Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Globant - Amazon recognition workshop - 2018


Published on

This was a workshop about recognition in Medellín.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Globant - Amazon recognition workshop - 2018

  1. 1. Javier Cristancho – PSA AWS Marzo - 2018 Amazon Rekognition Deep learning-based image recognition service
  2. 2. Images – Universal, Ubiquitous, & Essential There are 3,700,000,000 internet users in 2017 1,200,000,000 photos will be taken in 2017 (9% YoY Growth) Source: InfoTrends Worldwide
  3. 3. Amazon Rekognition Extract rich metadata from visual content Object and Scene Detection Activity Detection Facial Analysis Face Comparison Facial Recognition Person Tracking Real time live Stream Celebrity Recognition Image Moderation
  4. 4. Why use Rekognition? johnf
  5. 5. Object & Scene Detection Object and scene detection makes it easy for you to add features that search, filter, and curate large image libraries. Identify objects and scenes and provide confidence scores DetectLabels Flower Arrangement Chair Coffee Table Living Room Indoors Furniture Cushion Vase Maple Villa Plant Garden Water Swimming Pool Tree Potted Plant Backyard Patio
  6. 6. Facial Analysis Analyze facial characteristics in multiple dimensions DetectFaces Image Quality Facial Landmarks Demographic Data Emotion Expressed General Attributes Facial Pose Brightness 23.6% Sharpness 99.9% EyeLeft,EyeRight,Nose RightPupil,LeftPupil MouthRight,LeftEyeBrowUp Bounding Box... Age Range 29-45 Gender:Male 96.5% Happy 83.8% Surprised 0.65% Smile:True 23.6% EyesOpen:True 99.8% Beard:True 99.5% Mustache:True 99.9%... Pitch 1.446 Roll 5.725 Yaw 4.383
  7. 7. Face Comparison Measure the likelihood that faces are of the same person Similarity 93% Similarity 0% CompareFaces
  8. 8. Facial Recognition Find similar faces in a large collection of images SearchFacesByImage Search Index Collection
  9. 9. Celebrity Recognition & Image Moderation Newly released Rekognition features Detect explicit and suggestive contentRecognize thousands of famous individuals DetectModerationLabelsRecognizeCelebrities
  10. 10.
  11. 11. What Can You Do with Amazon Rekognition? • Search for people, objects, scenes, and concepts across millions of images • Filter inappropriate or specific content • Verify identities by matching against reference faces • Recognize individuals by matching faces to a collection • Analyze user traffic hotspots and journey paths by demographics and sentiment
  12. 12. Sentiment Analysis - Use Case (Retail – In-store and Online) Demographic and Sentiment Analysis Female Happy Smiling Male No Facial Hair Happy Female Sad No Eyeglasses
  13. 13. Sentiment Analysis Amazon RedshiftAmazon Quicksight Live Subject In-Store Camera Application Amazon S3 Analyze Faces Shoppers enter and browse in retail store In-store cameras capture live images of shoppers A Lambda function is triggered and calls Rekognition Rekognition analyzes the image and returns facial attributes detected, which include emotion and demographic detail Return data is normalized and staged in S3 en route to Redshift Marketing Reports Periodic ingest of data into Redshift Regular analysis to identify trends in demographic activity and in-store sentiment over time Trend reporting for retail store locations
  14. 14. Look Your Best All Day Time for A New Look? Facial Analysis - Use Case (Targeted Marketing) Demographic and Sentiment Analysis PersonAPersonB Sees Sees
  15. 15. Facial Analysis - Use Case (Targeted Marketing) Demographic and Sentiment Analysis demographic and sentiment attributes Look Your Best All Day display ad image Application AMAZON REDSHIFT AMAZON DYNAMODBAMAZON S3 log demographic profile updates retrieve ad image face image is collected and analyzed AMAZON REKOGNITION DetectFaces Store M etadata
  16. 16. Collections and Access Patterns Logging - public events; visitor logs; digital libraries • One large collection per event/time period • Wide searches Social Tagging - photo storage and sharing • One collection per application user • Automated friend tagging Person Verification - employee gate check • One collection for each person to be verified • Detection of stolen/shared IDs
  17. 17. { "FaceMatches": [ {"Face": {"BoundingB "Height": 0.2683333456516266, "Left": 0.5099999904632568, "Top": 0.1783333271741867, "Width": 0.17888888716697693}, " { "FaceMatches": [ {"Face": {"BoundingB "Height": 0.2683333456516266, "Left": 0.5099999904632568, "Top": 0.1783333271741867, "Width": 0.17888888716697693}, " Rekognition APIs – Overview Rekognition’s computer vision API operations can be grouped into Non-storage API operations, and Storage-based API operations CompareFaces DetectFaces DetectLabels DetectModerationLabels GetCelebrityInfo RecognizeCelebrities Non-storage API Operations CreateCollection DeleteCollection DeleteFaces IndexFaces ListCollections SearchFaces SearchFacesByImage Storage-based API Operations ListFaces
  18. 18. Rekognition APIs – Advanced Usage Decision trees and processing pipelines Why? • Many use cases require more than a single operation to arrive at actionable data How? • S3 event notifications, Lambda, Step Functions • DynamoDB for persistent pipeline storage • Augmenting results with 3rd Party AI/ML • OpenCV, MXNet, etc. on EC2 Spot, ECS, AI/ML AMI Sample Use Cases • Person of interest near a celebrity • Multi-pass motion detection enhancement • Subjects leaving a location without possessions IndexFaces DetectLabels “person”
  19. 19. Rekognition APIs – Advanced Usage Person of Interest Near a Celebrity
  20. 20. aws rekognition recognize-celebrities –image “S3Object={Bucket=mybucket,Name=cam.jpg}” aws rekognition search-faces-by-image –image “S3Object={Bucket=mybucket,Name=cam.jpg}” --collection-id “persons-of-interest" aws rekognition create-collection --collection-id “persons-of-interest” aws rekognition index-faces --image “S3Object={Bucket=mybucket,Name=subject.jpg}” --collection-id “persons-of-interest” Rekognition APIs – Advanced Usage Person of Interest Near a Celebrity { "FaceMatches": [ {"Face": {"BoundingB "Height": 0.2683333456516266, "Left": 0.5099999904632568, "Top": 0.1783333271741867, "Width": 0.17888888716697693}, " CompareFaces DetectFaces DetectLabels DetectModerationLabels GetCelebrityInfo RecognizeCelebrities 2 { "FaceMatches": [ {"Face": {"BoundingB "Height": 0.2683333456516266, "Left": 0.5099999904632568, "Top": 0.1783333271741867, "Width": 0.17888888716697693}, " CreateCollection DeleteCollection DeleteFaces IndexFaces ListCollections SearchFaces SearchFacesByImage ListFaces 3 1
  21. 21. • Built in 3 weeks • Indexed against 99,000 people • Index created in one day • Saved ~9,000 hours a year in manual curation costs • Live video with frame sampling Automating Footage Tagging with Amazon Rekognition Previously, only about half of all footage was indexed due to the immense time requirements required by manual processes
  22. 22. Automating Footage Tagging with Amazon Rekognition Solution Architecture EncodersStills Extraction & FeedsResults Cache Bucket R3 Amazon Rekognition users Stills Frames SQS Trigger 1 2 3 4
  23. 23. Quickly Identifying Persons of Interest with Amazon Rekognition • More than 300,000 mugshots indexed within 1-2 days • Identification process went from days (and weeks), to seconds – greatly increasing the ability for law enforcement to act quickly • Within 1 week of going live, the application helped identify a suspect in a case that had no leads, leading to an arrest There was no software on the market that allows users to quickly search hundreds of thousands of images using a face in another image
  24. 24. Quickly Identifying Persons of Interest with Amazon Rekognition Solution Architecture MS SQL Database ColdFusion Web Service Mugshot Bucket T2.Micro MySQL DB instance Amazon Rekognition Amazon Cognito iOSMobile Client client PHP 1 2 3 4
  25. 25. • Media and Entertainment • Digital Asset Management • Safety and Security • Law Enforcement • Consumer Electronics • Influencer Marketing • Consumer Storage • Travel and Hospitality • Digital Advertising • eCommerce • Education Rekognition Customers
  26. 26. Rekognition - Summary • Leverage Amazon internal experience with AI, ML and Computer Vision • Managed API service with embedded AI for maximum accessibility and simplicity • Integrates natively with other AWS Services • Extensible by design
  28. 28. Face-Based User Verification Authenticated User Image Capture Application Amazon S3 Compare Faces If the similarity score is over 92%, the application returns a green status. If not, an alert is issued to security staff. The application captures a live image of each employee as they scan their access card Rekognition compares the live image and the badge image – and returns a similarity score The application retrieves the user’s badge from S3 Confirm user identities by comparing their live image with a reference image
  29. 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. GRACIAS