Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MCL306_Making IoT Smarter with AWS Rekognition.pdf

241 views

Published on

Motion detection triggers have reduced the amount of video recorded by modern devices. But maybe you want to reduce that further—maybe you only care if a car or a person is on-camera before recording or sending a notification. Security cameras and smart doorbells can use Amazon Rekognition to reduce the number of false alarms. Learn how device makers and home enthusiasts are building their own smart layers of person and car detection to reduce false alarms and limit video volume. Learn too how you can use face detection and recognition to notify you when a friend has arrived.

  • Be the first to comment

MCL306_Making IoT Smarter with AWS Rekognition.pdf

  1. 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS re:Invent Mak ing IoT Smar ter w ith AW S R ek ognition N i c k R o b i n s o n , S o f t w a r e D e v e l o p e r M C L 3 0 6
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. What to Expect from the Session • How to leverage AWS to add intelligence to IoT devices • Smart Door Cam • Smart Object Detector • A reference architecture for performing real-time recognition across a number of categories
  3. 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Problem
  4. 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Reference Architecture AWSAWS AWS
  5. 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Detection Pipeline AWSAWS AWS
  6. 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT AWSAWS AWS
  7. 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. State of IoT • Connected Device Growth • 2016 - 6.38 Billion • 2017 – 8.38 Billion • 2020 Projections – 20.42 Billion • Rapid Growth of Sector • Growth in Consumer Spending • 2016 - $532B • 2017- $725B • 2020 Projections - $1.5T
  8. 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Common Problems in IoT • Security • Limited Compute Power • Limited Storage
  9. 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT Overview
  10. 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT Rules Engine
  11. 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT Rules Engine • Rule syntax • Example Rule Definition • SELECT resource_id as id, cycle as c_number FROM ‘pm/topic’ WHERE failure = 1
  12. 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS IoT Setup • Provision new device through the console • Download connection kit • Example Pub/Sub Application
  13. 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  14. 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Rekogntion AWSAWS AWS
  15. 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Rekognition Overview Object and Scene Detection Facial Analysis Face Comparison Facial Recognition Celebrity Recognition Image Moderation Text Recognition
  16. 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Facial Analysis Image Quality Facial Landmarks Demographic Data Emotion Expressed General Attributes Facial Pose Brightness 23.6% Sharpness 99.9% EyeLeft,EyeRight,Nose RightPupil,LeftPupil MouthRight,LeftEyeBrowUp Bounding Box... Age Range 29-45 Gender:Male 96.5% Happy 83.8% Surprised 0.65% Smile:True 23.6% EyesOpen:True 99.8% Beard:True 99.5% Mustache:True 99.9%... Pitch 1.446 Roll 5.725 Yaw 4.383 DetectFaces
  17. 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Facial Recognition Search Index Collection SearchFacesByImage
  18. 18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Object And Scene Detection Flower Arrangement Chair Coffee Table Living Room Indoors Furniture Cushion Vase Maple Villa Plant Garden Water Swimming Pool Tree Potted Plant Backyard Patio DetectLabels
  19. 19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Interface with Amazon Rekognition • S3 input for API calls - max image size of 15MB • 5MB limit for non-S3 (Base64 encoded) API calls • Minimum image resolution (x or y) of 80 pixels • Image data supported in PNG or JPG format • Face collections support tens of millions of faces • The max matching faces the search API returns is 4096 • Size of face should occupy 5%+ of image for detection • Collections are for faces!
  20. 20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Potential Use Cases • Smart Body Cameras with alerts for persons of interest • Dog bowl with embedded camera that sends you updates during the day when your pet is spotted • Adaptive digital advertising, real-time content decisions based on crowd demographics • Vision systems with configurable match criteria
  21. 21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS Lambda AWSAWS AWS
  22. 22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. IoT Message Processing • AWS Lambda invoked from IoT action based on topic • Receives base64 encoded jpeg image in JSON message • Sends image to Amazon Rekognition for label detection • Filters out accepted labels • Communicates back to IoT device
  23. 23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Setting Up Our Lambda Function • Creating Lambda Functions • Connect to AWS IoT • Creating topics • Configuring Actions
  24. 24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. object_list = ['Person', 'People', 'Human'] def lambda_handler(event, context): client = boto3.client('rekognition') response = client.detect_labels( Image={ 'Bytes': base64.b64decode(event['image']), }, MaxLabels=5 ) client = boto3.client('iot-data', region_name='us-east-1') for item in response['Labels']: if item['Name'] in object_list: client.publish( topic='sdk/objectDetected', qos=1, payload=json.dumps({"object":item['Name']}) ) Lambda Handler
  25. 25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  27. 27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Delivery Pipeline AWSAWS AWS
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customer Notifications – AWS SNS AWSAWS AWS
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Deliver notifications to your customers in a scalable way • Setup a topic • Subscribe your users • Publish notifications and have them multiplexed to subscribed users Amazon SNS
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deliver Video AWSAWS AWS
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Amazon S3 Events provides powerful tool for event generation • Deliver video content to bucket • Can trigger variety of events across a number of AWS services including AWS Lambda Amazon S3 Events
  32. 32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Setup steps • Generating Signed URLs • Sending notifications to our users Setting up our Notification Lambda
  33. 33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Modify our Lambda function to add Face Detection and metadata collection • Perform facial recognition against a face collection for more detailed notifications • Use for real-time content moderation Extending the Pipeline
  34. 34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  35. 35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Come out to the skills session! • SKL69 Learn More
  36. 36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Questions? Making IoT Smarter with Amazon Rekognition
  37. 37. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. THANK YOU!

×