Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Machine Learning Week SF: Build an Image-Based Automatic Alert System with Amazon Rekognition

117 views

Published on

AWS Machine Learning Week at the San Francisco Loft: Build an Image-Based Automatic Alert System with Amazon Rekognition

This hands-on workshop will walk through how to build a solution that listens and captures images from Twitter, and then compares those images against a reference image to automatically notify you about a new post featuring your favorite celebrity. Additionally, we will integrate sentiment analysis into this image-based automatic alert system in order to gauge whether the determined celebrities are happy, sad, etc. in the posted image.
Speaker: Sireesha Muppala - Solutions Architect, AWS

  • Be the first to comment

  • Be the first to like this

AWS Machine Learning Week SF: Build an Image-Based Automatic Alert System with Amazon Rekognition

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Pop-up Loft Build an Image-Based Automatic Alert System with Amazon Rekognition Sireesha Muppala, Solutions Architect
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved What to expect? • Goal: Learn how to build computer vision-based smart applications • Overview of Amazon Rekognition • Outline of the Workshop Scenario • Preview of the Lab/Instructions
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES PLATFORM SERVICES APPLICATION SERVICES
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES PLATFORM SERVICES APPLICATION SERVICES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Chainer Keras Gluon AWS Deep Learning AMIs
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Chainer Keras Gluon AWS Deep Learning AMIs Amazon SageMaker AWS DeepLens EDUCATION PLATFORM SERVICES Amazon Mechanical Turk APPLICATION SERVICES
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved The Amazon Machine Learning Stack FRAMEWORKS & INTERFACES Caffe2 CNTK Apache MXNet PyTorch TensorFlow Chainer Keras Gluon AWS Deep Learning AMIs Amazon SageMaker Rekognition Transcribe Translate Polly Comprehend Lex AWS DeepLens EDUCATION PLATFORM SERVICES APPLICATION SERVICES Amazon Mechanical Turk
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon Rekognition Image Facial Analysis Face Recognition Text in Image Deep learning-based image analysis service Unsafe Image Detection Object & Scene Detection Celebrity Recognition
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon Rekognition Video Object & Activity Detection Face Detection & Recognition Real-time Live Stream Deep learning-based video analysis service Unsafe Video DetectionCelebrity Recognition Pathing
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Boat 99.3% Plant 95.1% Harbor 94.8% Yacht 78.1% Dock 75.7% City 72.4% Architecture 71.8% Urban 63.9% Building 62.3% Marina 60.3% Plaza 51.1% Spire 50.8% Neighborhood 50.7% Flower 50.6% Waterfront 94.8% Object and Scene Detection
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Analyze facial characteristics in multiple dimensions Image Quality Facial Landmarks Demographic Data Emotion Expressed General Attributes Facial Pose Brightness 23.6% Sharpness 99.9% EyeLeft,EyeRight,Nose RightPupil,LeftPupil MouthRight,LeftEyeBrowUp Bounding Box... Age Range 29-45 Gender:Male 96.5% Happy 83.8% Surprised 0.65% Smile:True 23.6% EyesOpen:True 99.8% Beard:True 99.5% Mustache:True 99.9%... Pitch 1.446 Roll 5.725 Yaw 4.383 Facial Analysis
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Measure the likelihood that faces are of the same person Similarity 93% Similarity 0% Face Comparison
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Detect explicit and suggestive contentRecognize thousands of famous individuals DetectModerationLabelsRecognizeCelebrities Celebrity Recognition & Image Moderation
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved FaceID: 4c55926e-69b3-5c80-8c9b-78ea01d30690 Similarity: 97 FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d Similarity: 92 FaceID: 02e56305-1579-5b39-ba57-9afb0fd8782d Similarity: 85 Index Faces Collection Search Faces MaxFaces FaceMatchThreshold Face Search Real-time search against tens of millions of faces from your private face collection
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Celebrity Guests at the Royal Wedding • “Who’s Who Live” function • Celebrity guests identified in live stream • On Screen captions of relation to the royal couple
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Sports Media: Player Tracking and Recognition
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Image API – Request and Response DetectLabels { "Image": { "Bytes": blob, "S3Object": { "Bucket": "string", "Name": "string", "Version": "string" } }, "MaxLabels": number, "MinConfidence": number } { "Labels": [ { "Confidence": number, "Name": "string” } ], "OrientationCorrection": "string" }
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Video API – Request and Response GetLabelDetection StartLabelDetection { “ClientRequestToken": "string", "JobTag": "string", "MinConfidence": number, "NotificationChannel": { "RoleArn": "string", "SNSTopicArn": "string” }, "Video": { "S3Object": { "Bucket": "string", "Name": "string", "Version": "string” } } } { "JobStatus": string, "StatusMessage": string, "VideoMetadata": { "Format": string, "Codec": string, "DurationMillis": number, "FrameRate": float, "FrameWidth": number, "FrameHeight": number }, "NextToken": string, "Labels": [ { "Timestamp": number, "Label": { "Name": string, "Confidence": float } } ], ... JobId
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Other Services You Will Use Today
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon Elastic Compute Cloud (EC2) Match Capacity and Demand Global Footprint Elasticity Provision Servers in Minutes Infrastructure as Code Programmatic Networking
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon Kinesis Data Firehose
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved AWS Lambda No Servers to Manage Continuous Scaling Don’t Pay for Idle Capacity Lambda allows you to run application logic without provisioning servers or worrying about the health or security of underlying resources Lambda scales infrastructure beneath your application logic; just send requests and events and Lambda will automatically scale to accommodate it With Lambda, you’re billed in 100ms increments of execution time and number of requests and you’re never charged for anything when your code isn’t running
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon DynamoDB • Fast and flexible NoSQL database service for any scale Dead Simple • GetItem(primaryKey) • PutItem(item) Robust Depth • Fine-Grained Access Control • Streams • Triggers • Global Tables • Encryption • DynamoDB local • Free-text search • Strong consistency option • Atomic counters
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Amazon Simple Storage Service (S3) • Durable, massively scalable object storage • Designed for 99.999999999% durability and 99.99% availability • Stores trillions of objects and regularly handles millions of requests per second • Effectively infinite storage without provisioning capacity
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Workshop Overview
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Build an Image-Based Automatic Alert System • Identify images that match a reference image. – Images : From Twitter Feed • Need to have Twitter Account & Twitter Developer Account ( Prerequisite) • Need to create a Twitter App and gather Consumer/API keys (Prerequisite) • Listen to twitter feed and send to S3 via Kinesis Streams • Trigger Lambda to start a Rekognition Celebrity Detection API • Trigger SNS Notification when match is found. – Reference Image : Celebrity Image uploaded to S3 • Alert on matched image.
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Architecture
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Extension
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Workshop Step by Step Guide • https://github.com/aws-samples/aws-developer- workshop/blob/master/episode2/Instructions.md https://bit.ly/2tdSmH8
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved Pop-up Loft aws.amazon.com/activate Everything and Anything Startups Need to Get Started on AWS

×