Recommended
More Related Content
What's hot
What's hot (6)
Similar to Google I/O 2018 iot ml_session_demo_inside_story
Similar to Google I/O 2018 iot ml_session_demo_inside_story (20)
More from さとる なかむら
Recently uploaded
Recently uploaded (20)
Google I/O 2018 iot ml_session_demo_inside_story
- 1. Google I/O 2018 ML model + IoT Demo Inside Story 2018/05/19 Satoru Nakamura Groovenauts, Inc.
- 2. Who am I ● Satoru Nakamura ● GDE(Cloud) ●
- 3. QueryIt Smart (Google Cloud Next ‘17 Playground Demo) https://queryit.magellanic-clouds.com/
- 4. Device Location Detection (Google Cloud Next ‘17 in Tokyo Groovenauts’ presentation)
- 5. Outline of session demo
- 6. Google I/O 2018 Session “ML models + IoT data = a smarter world” https://www.youtube.com/watch?v=avxpkFUXIfA
- 7. Regulations of this technical demo ● Adopt Cloud IoT Core ● Utilize ML model & Cloud ML Engine (training and inference) ● ML model inference on edge device is out of scope (It’s included in Laurence’s part of the session)
- 9. Demo dashboard Camera Image + Object Detection Results Recommended recipe according to cart contents+α Information about the foodstuffs required for the recipe
- 12. 3/27 initial offer from kaz-san 5/9 Google I/O Session
- 13. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/27 5/9 Google I/O Session Device (Raspberry Pi) software and ML model development deadline.
- 14. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/13 4/27 Pivot!! 5/9 Google I/O Session Device (Raspberry Pi) software and ML model development deadline.
- 15. 3/27 initial offer from kaz-san 〜4/5 Draw rough blueprint of demo 〜4/13 Pivot!! 4/27 Device (Raspberry Pi) software and ML model development deadline. 5/9 Google I/O Session GW (Japanese Holidays) kaz-san’s departures to US
- 16. In fact only 2 weeks are available for development ● Choose and purchase hardware setup (Raspberry Pi, camera module, touch display etc...) ● Purchase imitation foodstuff (onion, tomato, potato, etc...) ● Device setup & demo application development ● Capture images for ML model training (Object Detection) ● Thinking of recipes ● Collect images for each recipe ● Generate dummy foodstuff sequence of shopper’s pickup. ● Training ML models (Object Detection & Next item prediction) ● Dashboard application
- 20. Cloud IoT Core + ML Engine (TensorFlow) Raspberry Pi Camera Touch display ● upload telemetry (camera image) ● download configuration (including recipe data) ● Object Detaction ● Next shopping item prediction
- 21. System Architecture Raspberry Pi dashboard - Object Detection - Cart Items Prediction (Ingest Object Detection results) touch display
- 22. ML models ● Based on Tensorflow Object Detection API https://github.com/tensorflow/models/tree/master/research/object_detection ○ SSD + mobilenet v1 ○ Transfer learning (upon COCO-trained model) ● Next shopping item prediction ○ 1D-Conv + MLP model ML models are trained using
- 28. ML inference should be executed on cloud ● We should think of reasons justify cloud inference ○ Edge inference was optimum in most scinario
- 29. The demo should be attractive and easy to understand at glance ● “Small sensor device” is typical and realistic IoT system scenario ○ Temperature, Air pressure, Air dust etc.. ● Attractive visualization is hard to develop ● Image recognition is intuitive and easy to visualize! (But is it IoT thing?)
- 30. All images should be licensed under CC0 https://github.com/groovenauts/SmartShoppingNavigator/issues/11 ● Collecting CC0 or Public Domain attractive images is time consuming stuff ● Deeply thanks to pixabay (https://pixabay.com/)
- 31. Recipes should be those which US people are familiar with ● Culture gap? ● There are less typical cuisine with common name (really?) ● Cookpad (https://cookpad.com/us) show variety of recipes But they are like “Bacon and potato”, “○○’s speciality”...