Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nacs 2018 Office 365, Azure and a Raspberry Pi - a 21st century lunch bell

211 views

Published on

Session from the North American Collaboration Summit on using Azure Cognitive Services to check images taken from a Raspberry Pi for the presence of our food truck. If present it informs the user in MS Teams or Skype.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Nacs 2018 Office 365, Azure and a Raspberry Pi - a 21st century lunch bell

  1. 1. OFFICE 365, AZURE AND A RASPBERRY PI: A 21ST CENTURY LUNCH BELL NACS 2018, Branson, MO – Thomas Gölles
  2. 2. THOMAS GÖLLES Head of Modern Workplace Solutions Co-Organizer of SharePoint Saturday Vienna (#SPSVIE), Office 365 Meetup Graz, Office 365 Meetup Vienna http://www.modernworkplacesolutions.rocks/ @thomyg
  3. 3. MAKE SURE TO THANK OUR SPONSORS! WEAR YOUR NACS WRISTBAND FOR LUNCH SPECIALS ON BRANSON LANDING PLAY VENDOR BINGO FOR YOUR CHANCE TO WIN A i7 16GB 1TB SURFACE BOOK
  4. 4. TODAY‘S AGENDA 1 Little bit of background 2 “Bussiness Problem” 3 Tech geek out 4 DEMO
  5. 5. OUR OFFICE IN GRAZ, AUSTRIA
  6. 6. THE VIEW I USED TO HAVE …
  7. 7. EVERYDAY AT AROUND 10:45 …
  8. 8. SANDWICH TIME
  9. 9. BUT I GOT A NEW OFFICE AND A NEW VIEW…
  10. 10. POSSIBLE SOLUTIONS Create a recurrent event in Outlook “Ping” a colleague everyday Stand up and have a look Use technology to solve the problem
  11. 11. MOCADESYMO Penny: Well, you know, it’s the Cheesecake Factory. People order cheesecake, and I bring it to them. Leonard: So, you sort of act as a Carbohydrate Delivery System. Penny: Yeah, call it whatever you want, I get minimum wage. Mobile Monitor
  12. 12. FRONT DOOR VIEW
  13. 13. MOCADESYMO
  14. 14. ARCHITECTURE
  15. 15. MAKE SURE TO ATTEND THE ATTENDEE PARTY FRIDAY NIGHT AT 7:30PM AT BLACK OAK GRILL FOR GREATE LIVE MUSIC GET YOUR FREE DRINK TICKETS AT THE END OF THE DAY SESSION ON FRIDAY
  16. 16. CHATBOTS
  17. 17. Chatbots 101 “a computer program designed to have a conversation with a human being, especially over the internet” * * https://dictionary.cambridge.org/dictionary/english/chatbot
  18. 18. “HOW DO YOU PERCEIVE COMPANIES THAT OFFER A CHATBOT?” https://insights.ubisend.com/2017-chatbot-report
  19. 19. Microsoft Bot Framework
  20. 20. COGNITIVE SERVICES
  21. 21. Yes Similar image Query image
  22. 22.  Researchers took a traditional machine learning approach • Example: HoG Detectors - Histogram of oriented gradients (HoG) features - Sliding window detector - SVM Classifier - Very fast OpenCV implementation (<100ms)
  23. 23. Deep Neural Network for Computer Vision cat? YES dog? NO car? NO Convolutional Layers Fully Connected Layers Complex Objects & Scenes (people, animals, cars, beach scene, etc.) Image Low-Level Features (lines, edges, color fields, etc.) High-Level Features (corners, contours, simple shapes) Object Parts (wheels, faces, windows, etc.)
  24. 24. TECH GEEK OUT
  25. 25. AZURE / RASPBERRY PI
  26. 26. RASPBERRY PI + CAMERA MODULE
  27. 27. PARTS
  28. 28. PI SETUP & TOOLS Runs Raspbian Jessie Lan/Wlan connection raspistill for taking images mogrify for cropping the image Azure CLI 2.0 curl
  29. 29. AZURE COMPONENTS Blob Storage Azure Function Custom Vision API Storage Tables
  30. 30. BLOB STORAGE
  31. 31. AZURE FUNCTION • Min. Visual Studio 2017 Update 2 • Usage of custom library • Including PnP library • Gets triggered by the curl request from the Pi • Takes the image from the blob storage
  32. 32. AZURE FUNCTION • Calls Custom Vision API • Gets back the prediction result from http request • Stores data to log and state tables • Informs users in Teams by calling a connector if prediction is above a certain threshold • uploads pictures to a SharePoint Portal
  33. 33. CUSTOM VISION API • Project with sample data • Start with at least 30 images • Train the models • Use live data to learn in iterations • Be prepared to understand prediction results • Getting above 95% with ~450 images now
  34. 34. NORMAL STATE
  35. 35. 98,53% TRUE
  36. 36. 100% TRUE
  37. 37. 99,9% TRUE
  38. 38. Log table with every call and result from the Custom Vision API State Table with timestamps at arriving and leaving for easier integration of other systems in the future (Cortana, Alexa, bot framework) STORAGE TABLES
  39. 39. TEAMS CONNECTOR
  40. 40. LOG OUTPUT • Basic incoming webhook connector in Teams • Easy to test with Powershell request : Invoke-WebRequest -Uri http://localhost:7071/api /FoodTruckMonitorFunc ?image=https://snackcar monitor.blob.core.windo ws.net/snackcarimages/ 2017-07-26-10-49- 01.png
  41. 41. SHAREPOINT PORTAL
  42. 42. PICTURE LIBRARY AS A KIND OF ALBUM
  43. 43. DEMO TIME
  44. 44. DEMO CASE Blob Storage 1 2 3 4 5
  45. 45. PRIZE DRAWINGS HAPPEN AFTER THE END OF THE DAY SESSIONS ON FRIDAY AND SATURDAY. YOU MUST ATTEND THE END OF THE DAY SESSION TO WIN.

×