Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

JavaZone 2017 : Building a smart security camera with raspberry pi zero, java and aws

326 views

Published on

In this session, I’ll share how I transformed a lowly Raspberry Pi Zero W webcam into a smart security camera (with motion detection, threat analysis and alert notifications) by combining open source software with cloud based image analysis.

Attendees can expect a short explanation of how to set up their own motion activated webcam, followed by a demonstration of how they can use Java and a range of AWS Services (including Rekognition, Lambda Functions and Step Functions) to help their camera distinguish between an unwanted guest and the neighbour’s cat.

Finally we'll compare Node.js and Java versions of this solution and compare them in terms of execution speed, operating cost and ease of development.

Video here: https://vimeo.com/233849443

Published in: Technology
  • Be the first to comment

JavaZone 2017 : Building a smart security camera with raspberry pi zero, java and aws

  1. 1. BUILDING A SMART SECURITY CAMERA WITH PIZERO, JAVA AND AWS @markawest
  2. 2. Who Am I? • IT Consultant at Bouvet. • Hacker and Maker. • Java, JavaScript, AI, Cloud & IoT. • Active member of javaBin - the Norwegian JUG. @markawest
  3. 3. Talk Outline Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java Lambda Functions @markawest
  4. 4. Motivation and Requirements Input from Stakeholder Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java @markawest
  5. 5. Initial Motivation @markawest
  6. 6. Project Requirements Functional • Monitor activity in the garden. • Send warning when activity detected. • Live video stream. Non-functional • In place as soon as possible. • Low cost. • Portable. @markawest
  7. 7. Functional Design 1. Activity in garden 2. Camera detects movement 3. Camera sends alert email with snapshot @markawest
  8. 8. Building a Motion Activated Camera Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java Lambda Functions @markawest
  9. 9. Pi Zero W Essentials Kit: NOK 285 Camera Module: NOK 244 Camera Adapter: NOK 41 ZeroView: NOK 71 Total Cost: NOK 641 Hardware Shopping List @markawest
  10. 10. Assembled Camera Rear View Front View @markawest
  11. 11. Implementing the Functional Design 1. Activity in garden 2. Camera detects movement 3. Camera sends alert email with snapshot @markawest
  12. 12. Motion (https://motion-project.github.io) • Open source motion detection software. • Excellent performance on the Raspberry Pi Zero. • Built-in Web Server for streaming video. • Detected activity or ‘motion’ triggers events. • Works out of the box. No need for additional programming. @markawest
  13. 13. Motion Streaming Demo @markawest
  14. 14. Example Alert Email 1 2 1 @markawest
  15. 15. How the Motion Software works 1 2 3 4 @markawest 1 2 3
  16. 16. How the Motion Software works 1 2 3 4 1 2 3 @markawest 1 2 3
  17. 17. How the Motion Software works 1 2 3 4 1 2 3 @markawest 1 2 3
  18. 18. Alarm Snapshots from Web Camera @markawest
  19. 19. Project Requirements : Evaluation Functional • Monitor activity in the garden. • Send warning when activity detected. • Live video stream. Non-functional • In place as soon as possible. • Low cost. • Portable. @markawest
  20. 20. False Alarms from Web Camera cat cloud @markawest
  21. 21. The Motion Software focuses on the amount of changed pixels, and not the cause of the changed pixels! @markawest
  22. 22. Adding Smart Image Analysis with AWS Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java Lambda Functions @markawest
  23. 23. Improving Alarm Accuracy with Smart Image Analysis person @markawest cat
  24. 24. Project Requirements Reloaded Functional • Monitor activity in our garden. • Send warning when activity detected. • Live video stream. Non-functional • In place as soon as possible. • Low cost. • Portable. @markawest
  25. 25. Project Requirements Reloaded Functional • Monitor activity in our garden. • Send warning when human activity detected. • Live video stream. Non-functional • In place as soon as possible. • Low cost. • Portable. @markawest
  26. 26. Finding an Image Analysis Solution OpenCV • Use Face Detection to find out if a human was in the snapshot. • Problem: What if the subject was facing away from the camera, or wearing a mask? TensorFlow • Train and use a Neural Network to find humans in the snapshot. • Problem: Potentially a fun project, but would take more time than I had available. @markawest
  27. 27. AWS Rekognition • Part of Amazon Web Services suite of Cloud based services. • Image Analysis as a Service, offering a range of API’s. • Built upon Deep Neural Networks. • Officially launched in November 2016. • Alternatives: Google Vision, MicroSoft Computer Vision, Clarafai. @markawest
  28. 28. AWS Rekognition Demo @markawest
  29. 29. Adding AWS to the Web Camera Amazon Web Services (AWS) Send warning email (if snapshot contains a person) Snapshot analysed via AWS Rekognition @markawest Camera pushes snapshot to AWS Email alert sent (with snapshot)
  30. 30. AWS IAM AWS Rekognition AWS Simple Email Service AWS S3 (storage) Amazon Web Service Flow Overview AWS Step Function (workflow) Upload Trigger 1 2 4 5 6 calls 3 uses uses @markawest
  31. 31. AWS IAM AWS Rekognition AWS Simple Email Service AWS S3 (storage) AWS Lambda Functions Code Building Blocks (aka Microservices) AWS Step Function (workflow) Upload Trigger calls uses uses @markawest
  32. 32. AWS Lambda Functions • Code units based on Java, C#, Python or Node.js • Serverless, no infrastructure to manage. • High availability out of the box. • Pay as you go model based on total invocations and duration - with a generous free tier. • Native AWS SDK gives easy access to other AWS services. @markawest
  33. 33. AWS Lambda Demo @markawest
  34. 34. AWS IAM AWS S3 (storage) AWS Step Functions Orchestration of Lambda Functions AWS Step Function (workflow) Upload Trigger calls AWS Rekognition AWS Simple Email Service uses uses @markawest
  35. 35. AWS Step Functions • Coordinate and orchestrate Lambda Functions into Workflows or State Machines. • Launched in December 2016. • Defined via JSON files, displayed as visual workflows. • Provide the same benefits as AWS Lambda (High Availability, Serverless, Scalable, “Pay as you go” pricing model). @markawest
  36. 36. Step Function for Image Processing Error Handler (sends Error Email via AWS SES) Sends snapshot to AWS Rekognition Evaluate AWS Rekognition response Send Alert Email? Send Alert Email via AWS SES Archive image in AWS S3 1 2 3 4 5 6 1 2 3 4 5 6 @markawest
  37. 37. Smart Camera (plus AWS Step Function) Demo @markawest
  38. 38. Evaluation and Lessons Learned Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java Lambda Functions @markawest
  39. 39. Project Requirements : Evaluation Functional • Monitor activity in the garden. • Send warning when human activity detected. • Live video stream. Non-functional • In place as soon as possible. • Low cost. • Portable. @markawest
  40. 40. Benefits of Using AWS • Very easy to get started with, due to Free Tier, automated infrastructure and good documentation. • Large ecosystem with a broad range of services. • Serverless - so hot right now! @markawest
  41. 41. Drawbacks of Using AWS • It’s not free. • So many services! • Step Functions need a graphical editor. • Potential for vendor lock-in. @markawest
  42. 42. AWS Rekognition : Positives • Consistent results. • Good results from 'partial’ images. • Handles poor quality images (motion blur, poor lighting). • No more false alarms from the camera! @markawest
  43. 43. AWS Rekognition : Negatives • Black box. • Unable to provide feedback. • Struggles with ‘noisy’ pictures. • Results are only as good as the pictures you upload. @markawest
  44. 44. Activity Timeline Improving AWS Rekognition Results @markawest Activity Event Start 1 Activity Event Stop 2 3 4 5
  45. 45. Activity Timeline Improving AWS Rekognition Results @markawest Activity Event Start 1 Activity Event Stop 2 3 4 5 Most Pixels Changed
  46. 46. Activity Timeline Improving AWS Rekognition Results @markawest Activity Event Start 1 Activity Event Stop 2 3 4 5 Most Central Activity
  47. 47. Activity Timeline Improving AWS Rekognition Results @markawest Activity Event Start 1 Activity Event Stop 2 3 4 5 Upload all snapshots
  48. 48. Improving AWS Rekognition Hit Rate Upload One Snapshot when activity detected • Lower cost, due to less calls to the AWS stack. • Potentially lower hit rate. Upload Many Snapshots when activity detected • Higher cost due to increased calls to the AWS stack. • Potentially higher hit rate. @markawest
  49. 49. AWS Monthly Cost : August 2017 @markawest Rekognition: $3.09 SES: $0.01S3: $0.49 Step Functions: $1.12 VAT: $1.17 (Based on 8093 images processed) Lambda: $0.00
  50. 50. “Fun project, but has it caught any real criminals?” @markawest
  51. 51. @markawest
  52. 52. Node.js vs Java Motivation and Requirements Building a Motion Activated Camera Adding Smart Image Analysis with AWS Evaluation and Tips Node.js vs Java Lambda Functions @markawest
  53. 53. Java 8 vs. Node.js 6.10 1. Artefact Sizes 2. Memory Requirements 3. Performance and Cost @markawest
  54. 54. AWS Lambda Functions @markawest Lambda Function Purpose AWS Service Calls rekognition-image- assessment Uses AWS Rekognition to generate a list of labels describing each uploaded picture. AWS Rekognition rekognition-evaluate-labels Evaluates labels to find out if an alarm email should be sent. - ses-send-notification Sends an alarm email via AWS SES when the smart security camera detects a person. AWS SES s3-archive-image Moves the processed image to the correct archive location in S3. AWS S3
  55. 55. Artefact Sizes 3 Reasons for the difference: 1. Java is generally more verbose. 2. Unused JAR’s in Java Deployments. 3. AWS SDK is natively available to Node.js Lambda Functions. @markawest Lambda Function Java Node rekognition-image- assessment 7.9 MB 813 bytes rekognition-evaluate- labels 7.6 MB 677 bytes ses-send-notification 8.7 MB 4.8 MB s3-archive-image 7.6 MB 943 bytes
  56. 56. Memory Requirements Memory Allocated AWS Free Seconds Per Month AWS Price Per Second 128 MB 3 200 000 $0.00000208 192 MB 2 133 333 $0.00000313 256 MB 1 600 000 $0.00000417 1536 MB 266 667 $0.00002501 @markawest • Node.js 6.10 happy with 128 MB memory. • Java 8 requires a minimum of 192 MB, and preferably 256 MB to avoid OutOfMemoryException errors.
  57. 57. Testing Cost & Performance : Method • Simulate 1500 alerts (not false alarms). • Triggered in batches of 30, with a 60 second pause. • Memory set to 256 MB for both Node.js and Java. • Prior to testing, Lambda Functions “rested” for 90 minutes. • Ran tests 5 times to ensure consistency of results. @markawest
  58. 58. Results from AWS CloudWatch • User defined graphical dashboards based on log data. • Example shows execution duration data for one of the Java AWS Lambda Functions. • Note the Cold Starts. @markawest
  59. 59. Test Results @markawest Lambda Function Average Duration Java Average Duration Node.js rekognition-image-assessment 2250 ms 1720 ms rekognition-evaluate-labels 6.57 ms 1.61 ms ses-send-notification 3040 ms 996 ms s3-archive-image 1050 ms 364 ms
  60. 60. Extrapolated Duration Costs • 256 MB memory. • $0.00000417 per second. • 1 600 000 free seconds. AWS Lambda Function Type 1 000 000 Alerts 5 000 000 Alerts 10 000 000 Alerts Java $23.39 $143.66 $293.99 Node.js $7.63 $64.84 $136.36 @markawest
  61. 61. Extrapolated Duration Costs 256 MB Memory $0.00000417 per second 1 600 000 free seconds AWS Lambda Function Type 1 000 000 Alerts 5 000 000 Alerts 10 000 000 Alerts Java $23.39 $143.66 $293.99 Node.js $7.63 $64.84 $136.36 @markawest
  62. 62. Why was Java slower on AWS Lambda? • Cold Starts (5-10 times longer on Java). • Lack of JVM tuning options on AWS. • Different underlying implementations. • Use Case for Project Jigsaw? @markawest
  63. 63. Is there a place for Java on AWS Lambda? • Of course there is! • Evaluate your use case and potential costs first. • Select languages based on overall value. • Try different memory settings to reduce costs. • Remember the Free Tier! @markawest
  64. 64. Would you like to know more? markwest1972/ smart-security- camera bouvet.no/bouvet- deler @markawest slideshare.net/ markawest Slides Blog Code
  65. 65. FOR LISTENING! @markawest Kraków, 17-19 May 2017

×