Successfully reported this slideshow.
Your SlideShare is downloading. ×

[DevDay2019] Developing IoTs application combines Google Assistant with the most natural experience - By Le Cong Vinh Khai, IoTs R&D Leader at TAPIT Co., Ltd

[DevDay2019] Developing IoTs application combines Google Assistant with the most natural experience - By Le Cong Vinh Khai, IoTs R&D Leader at TAPIT Co., Ltd

In this session, you will learn how to build an Internet of Things application that can monitor and control devices via voice and understand step by step building applications based on the following parts: Dialogflow platform, Firebase Realtime Database, Actions on Google platform and connect microcontroller (Arduino, ESP8266, ESP32,...) to Firebase Realtime Database.

In this session, you will learn how to build an Internet of Things application that can monitor and control devices via voice and understand step by step building applications based on the following parts: Dialogflow platform, Firebase Realtime Database, Actions on Google platform and connect microcontroller (Arduino, ESP8266, ESP32,...) to Firebase Realtime Database.

More Related Content

More from DevDay.org

Related Books

Free with a 30 day trial from Scribd

See all

[DevDay2019] Developing IoTs application combines Google Assistant with the most natural experience - By Le Cong Vinh Khai, IoTs R&D Leader at TAPIT Co., Ltd

  1. 1. Internet of Things and Google Assistant Le Cong Vinh Khai TAPIT Co., Ltd Director IoT R&D Leader 1 SMARTHOME ACTIONS
  2. 2. CONTENT  About us  Internet of Things and VUIs  Why Google Assistant?  Actions on Google: Smart home Actions  Steps 2
  3. 3. 3 About us Name: Lê Công Vĩnh Khải Date of Birth: December 4, 1995 Work: IoTs R&D Leader at TAPIT Co., Ltd Email: vinhkhai95@gmail.com Phone: +84787412885 Experience: More than 2 years experience in IoTs research.
  4. 4. 4 CÔNG TY TNHH KỸ THUẬT TAPIT MST: 0401918051 SĐT: 0981001119 Website: https://tapit.vn Địa chỉ: 146 Trương Định, TP.Đà Nẵng About us Internet of Things Mr. Lê Công Vĩnh Khải Mr. Ngô Văn Trung Embedded System Ms. Trần Thụy Ngọc Hằng Mr. Đậu Đức Thắng Mr. Huỳnh Ngọc Thương Hardware Design Mr. Ngô Thanh Liêm Mr. Nguyễn Huỳnh Nhật Thương CỘNG ĐỒNG KỸ THUẬT TAPIT 300+ thành viên + 6 trường đại học từ TP. Đà Nẵng, TP. Hà Nội, TP. Hồ Chí Minh + 24% kỹ sư, 19% sinh viên năm 5, 27% sinh viên năm 4, 23% sinh viên năm 3,... + 10 chuyên ngành, lĩnh vực khác nhau Training -Vi điều khiển: STM32, MSP430, Arduino. - Internet of Things. - C/C++ fb.com/tapit.vn0981001119 https://tapit.vntapitlrs@gmail.com
  5. 5. Internet of Things and VUIs 5 VUIs: Voice User Interfaces The Next Disruption
  6. 6. Alexa Google Assistant Siri Cortana Bixby 6
  7. 7. 7 Why is Google Assistant? Asia Pacific
  8. 8. 8
  9. 9. 9 Compatibility
  10. 10. Sonoff Philips Hue Belkin WeMo Lutron 10 Broadlink Xiaomi
  11. 11. Actions on Google Smart Home Actions Conversation Actions 11 Actions on Google  Smart Home Actions: Control IoT devices (connect, query and control through existing cloud infrastructure).  Conversation Actions: Invoke Actions to build two-way interactive conversations between users and assistant (News, Shopping, Quiz Game, IoTs…)
  12. 12. Smart home Actions Sonoff Philips Hue Belkin WeMo Lutron 12 Broadlink Xiaomi
  13. 13. 13 Smart homeActions Google Home App Manufacturer/Personal Cloud auth/token server smarthome server 2. SYNC intent 3.EXEC/QUERY intent Request Response Response Request 1. Account linking OAuth Credentials  SYNC: Requests the list of devices that the user has connected and are available for use.  EXEC: Requests a command to execute on smart home devices “Ok Google, change the TAPIT light to blue !”  QUERY: Queries for the current states of devices “Hey Google, is the TAPIT fan on?”
  14. 14. Flow for device setup 14
  15. 15. Google Assistant 15 Flow for execution/query Device Cloud Service HomeGraph EXEC intent QUERY intent Response User say
  16. 16. 16 Device definition DEVICE DEVICE TYPE DEVICE TRAITS Light action.devices.types.LIGHT action.devices.traits.Brightness action.devices.traits.ColorSetting action.devices.traits.OnOff Fan action.devices.types.FAN action.devices.traits.OnOff action.devices.traits.FanSpeed action.devices.traits.Modes action.devices.traits.Toggles Washer action.devices.types.WASHER action.devices.traits.OnOff action.devices.traits.StartStop action.devices.traits.Modes action.devices.traits.Toggles action.devices.traits.RunCycle ………. …………… …………….. Supported more than 20 different device types
  17. 17. 17 SYNC intent for LIGHT { "requestId": "ff36a3cc-…", "inputs": [{ "intent":"action.devices.SYNC" }] } { "requestId": "ff36a3cc….", "payload": { "agentUserId": "1836", "devices": [ { "id": "123", "type": "action.devices.types.LIGHT", traits: [ “action.devices.traits.OnOff”, “action.devices.traits.Brightness”, “action.devices.traits.ColorSetting” ], “name”: { name: ‘tapit light', }, ………………. } Request Response
  18. 18. 18 EXEC intent for LIGHT { "requestId": "ff36a3cc-….", "inputs": [{ "intent": "action.devices.EXECUTE", "payload": { "commands": [{ "devices": [{ "id": "123" }], "execution": [{ "command": "action.devices.commands.OnOff", "params": { "on": true } }] } { "requestId": "ff36a3cc-…", "payload": { "commands": [ { "ids": [ "123" ], "status": "SUCCESS", "states": { "on": true, "online": true } } ] } } Request Response
  19. 19. 19 QUERY intent for LIGHT { "requestId": "ff36a3cc…", "inputs": [{ "intent": “action.devices.QUERY”, "payload": { "devices": [{ "id": "123", }] } }] } { "requestId": "ff36a3cc-…", "payload": { "devices": { "123": { "on": true, "online": true } } } } Request Response
  20. 20. 2020 Steps 1. Set up an server for Account Linking (AUTH, TOKEN) and Smart home intents (SYNC, EXEC, QUERY). 2. Set up database for devices state storage 3. Create an Actions on Google developer project. 4. Provide server URL for requesting. Using Cloud Functions for Firebase Using Firebase Realtime Database
  21. 21. 21

Editor's Notes

  • https://developers.google.com/actions/smarthome/

×