Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Google I/O 2018 iot ml_session_demo_inside_story

665 views

Published on

I/O Extendedで話した内容です。

Published in: Engineering
  • Be the first to comment

Google I/O 2018 iot ml_session_demo_inside_story

  1. 1. Google I/O 2018 ML model + IoT Demo Inside Story 2018/05/19 Satoru Nakamura Groovenauts, Inc.
  2. 2. Who am I ● Satoru Nakamura ● GDE(Cloud) ●
  3. 3. QueryIt Smart (Google Cloud Next ‘17 Playground Demo) https://queryit.magellanic-clouds.com/
  4. 4. Device Location Detection (Google Cloud Next ‘17 in Tokyo Groovenauts’ presentation)
  5. 5. Outline of session demo
  6. 6. Google I/O 2018 Session “ML models + IoT data = a smarter world” https://www.youtube.com/watch?v=avxpkFUXIfA
  7. 7. Regulations of this technical demo ● Adopt Cloud IoT Core ● Utilize ML model & Cloud ML Engine (training and inference) ● ML model inference on edge device is out of scope (It’s included in Laurence’s part of the session)
  8. 8. Smart Shopping Navigator
  9. 9. Demo dashboard Camera Image + Object Detection Results Recommended recipe according to cart contents+α Information about the foodstuffs required for the recipe
  10. 10. Smart Shopping Navigator
  11. 11. Demo development timeline
  12. 12. 3/27 initial offer from kaz-san 5/9 Google I/O Session
  13. 13. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/27 5/9 Google I/O Session Device (Raspberry Pi) software and ML model development deadline.
  14. 14. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/13 4/27 Pivot!! 5/9 Google I/O Session Device (Raspberry Pi) software and ML model development deadline.
  15. 15. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/13 Pivot!! 4/27 Device (Raspberry Pi) software and ML model development deadline. 5/9 Google I/O Session GW (Japanese Holidays) kaz-san’s departures to US
  16. 16. In fact only 2 weeks are available for development ● Choose and purchase hardware setup (Raspberry Pi, camera module, touch display etc...) ● Purchase imitation foodstuff (onion, tomato, potato, etc...) ● Device setup & demo application development ● Capture images for ML model training (Object Detection) ● Thinking of recipes ● Collect images for each recipe ● Generate dummy foodstuff sequence of shopper’s pickup. ● Training ML models (Object Detection & Next item prediction) ● Dashboard application
  17. 17. Technical Internal
  18. 18. Smart Shopping Navigator
  19. 19. Cloud IoT Core + ML Engine (TensorFlow) Raspberry Pi Camera Touch display ● upload telemetry (camera image) ● download configuration (including recipe data) ● Object Detaction ● Next shopping item prediction
  20. 20. System Architecture Raspberry Pi dashboard - Object Detection - Cart Items Prediction (Ingest Object Detection results) touch display
  21. 21. ML models ● Based on Tensorflow Object Detection API https://github.com/tensorflow/models/tree/master/research/object_detection ○ SSD + mobilenet v1 ○ Transfer learning (upon COCO-trained model) ● Next shopping item prediction ○ 1D-Conv + MLP model ML models are trained using
  22. 22. Smart Shopping Navigator
  23. 23. Smart Shopping Navigator
  24. 24. Smart Shopping Navigator
  25. 25. Smart Shopping Navigator
  26. 26. Difficulties (except schedule)
  27. 27. ML inference should be executed on cloud ● We should think of reasons justify cloud inference ○ Edge inference was optimum in most scinario
  28. 28. The demo should be attractive and easy to understand at glance ● “Small sensor device” is typical and realistic IoT system scenario ○ Temperature, Air pressure, Air dust etc.. ● Attractive visualization is hard to develop ● Image recognition is intuitive and easy to visualize! (But is it IoT thing?)
  29. 29. All images should be licensed under CC0 https://github.com/groovenauts/SmartShoppingNavigator/issues/11 ● Collecting CC0 or Public Domain attractive images is time consuming stuff ● Deeply thanks to pixabay (https://pixabay.com/)
  30. 30. Recipes should be those which US people are familiar with ● Culture gap? ● There are less typical cuisine with common name (really?) ● Cookpad (https://cookpad.com/us) show variety of recipes But they are like “Bacon and potato”, “○○’s speciality”...
  31. 31. Smart Shopping Navigator https://github.com/groovenauts/SmartShoppingNavigator/

×