Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

IoT: Leprechaun Trap with Tessel & Salesforce Einstein


Published on

Can Salesforce help catch a Leprechaun for St Patrick's Day? See here for audio:

Published in: Technology
  • Be the first to comment

  • Be the first to like this

IoT: Leprechaun Trap with Tessel & Salesforce Einstein

  1. 1. IoT: Build a Leprechaun trap with Tessel and Salesforce Einstein
  2. 2.  Challenged by my daughter (age 6) to catch that pesky leprechaun About Leprechauns:  Nocturnal  Plays Pranks (glitter, funny drawings, etc.)  If caught, you get their gold! (who doesn’t need gold?)  Only visits between March 1 and March 17 St Patrick’s Day Background
  3. 3.  Catch leprechaun without harming him (Smart devices)  Avoid false alarms (AI / Computer Vision)  Send text message alert to minimize escape time (Apex / Twilio) Trap Requirements (& How to Meet Them)
  4. 4.  Proof-of-concept, not production-ready code  All images/code are used as part of “fair-use” for educational projects  A huge “thank you” to the previous projects that helped point me in the right direction: – Nforce Library: Kevin O’Hara – IoT Check-in System: Shruti Sridharan – Einstein Vision Quick Start: Trailhead – Tessel Review & Salesforce REST Example: Reid Carlberg Disclaimers
  5. 5. Trap Architecture Overview Leprechaun activates mag switch Tessel takes photo Motor springs trap Tessel sends photo to Salesforce SF requests Einstein Vision prediction If Leprechaun is detected in the image, an SMS alert is sent
  6. 6. Trap Architecture Overview
  7. 7.  Tessel2  Breadboard  Magnetic Door Switch  Hobby Gear Motor  Sparkfun Motor Driver  Logitech USB Webcam  Recommended: the Johnny Five Inventor Kit from Sparkfun Hardware
  8. 8.  Runs Node JS  Seamless Wifi connection  Very web service friendly About Tessel2
  9. 9.  See the Einstein Vision quick start:  Steps: – Connect to Einstein – Create model for leprechaun recognition – Train model using positive/negative images About Einstein Vision
  10. 10.  Detects magnetic switch movement  Powers motor to pull trap string  Uses webcam to take photo  Uses Nforce library to connect to Salesforce  Inserts Salesforce attachment record  Listens for Einstein prediction Tessel Code
  11. 11.  Einstein Vision Quickstart Classes: – JWT: handles Oauth between SF and Einstein – Vision: returns prediction objects  MyService Class: – Public web service that returns prediction results to Tessel – Also includes logic to send SMS messages  SendTwilio Class: – Uses the TwilioHelperClass to transmit message Apex Code
  12. 12.  Trap runs!  Some false positives in the vision model due to limited input data Results!
  13. 13.  Leprechaun was caught and forced to leave gold in order to escape!  A fun way to spend time with our child and dive into Salesforce Einstein. Results!
  14. 14.  Potential Future Enhancements: – Increased speed to return a prediction before springing the trap? – “Demo Mode” for parents – Fine tune Einstein model – Add photo within sms message – Switch to ContentVersion in SFDC – Re-purpose for animal classification, attendance tracking…or Easter Bunny!  Questions / Comments? – Reach out on LinkedIn: Final Considerations