Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Internet of Things and Google
Assistant
Le Cong Vinh Khai
TAPIT Co., Ltd Director
IoT R&D Leader
1
SMARTHOME ACTIONS
CONTENT
 About us
 Internet of Things and VUIs
 Why Google Assistant?
 Actions on Google: Smart home Actions
 Steps
2
3
About us
Name: Lê Công Vĩnh Khải
Date of Birth: December 4, 1995
Work: IoTs R&D Leader at TAPIT Co., Ltd
Email: vinhkhai...
4
CÔNG TY TNHH KỸ THUẬT TAPIT
MST: 0401918051
SĐT: 0981001119
Website: https://tapit.vn
Địa chỉ: 146 Trương Định, TP.Đà Nẵ...
Internet of Things and VUIs
5
VUIs: Voice User Interfaces The Next Disruption
Alexa
Google Assistant
Siri
Cortana Bixby
6
7
Why is Google Assistant?
Asia Pacific
8
9
Compatibility
Sonoff Philips Hue
Belkin WeMo Lutron
10
Broadlink
Xiaomi
Actions
on
Google
Smart Home Actions
Conversation Actions
11
Actions on Google
 Smart Home Actions: Control IoT devices (...
Smart home
Actions
Sonoff
Philips Hue
Belkin WeMo
Lutron
12
Broadlink
Xiaomi
13
Smart homeActions
Google Home App Manufacturer/Personal Cloud
auth/token server
smarthome server
2. SYNC intent
3.EXEC/...
Flow for device setup
14
Google Assistant
15
Flow for execution/query
Device Cloud Service
HomeGraph
EXEC intent
QUERY intent
Response
User say
16
Device definition
DEVICE DEVICE TYPE DEVICE TRAITS
Light action.devices.types.LIGHT action.devices.traits.Brightness
ac...
17
SYNC intent for LIGHT
{
"requestId": "ff36a3cc-…",
"inputs": [{
"intent":"action.devices.SYNC"
}]
}
{
"requestId": "ff3...
18
EXEC intent for LIGHT
{
"requestId": "ff36a3cc-….",
"inputs": [{
"intent": "action.devices.EXECUTE",
"payload": {
"comm...
19
QUERY intent for LIGHT
{
"requestId": "ff36a3cc…",
"inputs": [{
"intent": “action.devices.QUERY”,
"payload": {
"devices...
2020
Steps
1. Set up an server for Account Linking (AUTH, TOKEN) and
Smart home intents (SYNC, EXEC, QUERY).
2. Set up dat...
21
Upcoming SlideShare
Loading in …5
×

[DevDay2019] Developing IoTs application combines Google Assistant with the most natural experience - By Le Cong Vinh Khai, IoTs R&D Leader at TAPIT Co., Ltd

44 views

Published on

In this session, you will learn how to build an Internet of Things application that can monitor and control devices via voice and understand step by step building applications based on the following parts: Dialogflow platform, Firebase Realtime Database, Actions on Google platform and connect microcontroller (Arduino, ESP8266, ESP32,...) to Firebase Realtime Database.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

[DevDay2019] Developing IoTs application combines Google Assistant with the most natural experience - By Le Cong Vinh Khai, IoTs R&D Leader at TAPIT Co., Ltd

  1. 1. Internet of Things and Google Assistant Le Cong Vinh Khai TAPIT Co., Ltd Director IoT R&D Leader 1 SMARTHOME ACTIONS
  2. 2. CONTENT  About us  Internet of Things and VUIs  Why Google Assistant?  Actions on Google: Smart home Actions  Steps 2
  3. 3. 3 About us Name: Lê Công Vĩnh Khải Date of Birth: December 4, 1995 Work: IoTs R&D Leader at TAPIT Co., Ltd Email: vinhkhai95@gmail.com Phone: +84787412885 Experience: More than 2 years experience in IoTs research.
  4. 4. 4 CÔNG TY TNHH KỸ THUẬT TAPIT MST: 0401918051 SĐT: 0981001119 Website: https://tapit.vn Địa chỉ: 146 Trương Định, TP.Đà Nẵng About us Internet of Things Mr. Lê Công Vĩnh Khải Mr. Ngô Văn Trung Embedded System Ms. Trần Thụy Ngọc Hằng Mr. Đậu Đức Thắng Mr. Huỳnh Ngọc Thương Hardware Design Mr. Ngô Thanh Liêm Mr. Nguyễn Huỳnh Nhật Thương CỘNG ĐỒNG KỸ THUẬT TAPIT 300+ thành viên + 6 trường đại học từ TP. Đà Nẵng, TP. Hà Nội, TP. Hồ Chí Minh + 24% kỹ sư, 19% sinh viên năm 5, 27% sinh viên năm 4, 23% sinh viên năm 3,... + 10 chuyên ngành, lĩnh vực khác nhau Training -Vi điều khiển: STM32, MSP430, Arduino. - Internet of Things. - C/C++ fb.com/tapit.vn0981001119 https://tapit.vntapitlrs@gmail.com
  5. 5. Internet of Things and VUIs 5 VUIs: Voice User Interfaces The Next Disruption
  6. 6. Alexa Google Assistant Siri Cortana Bixby 6
  7. 7. 7 Why is Google Assistant? Asia Pacific
  8. 8. 8
  9. 9. 9 Compatibility
  10. 10. Sonoff Philips Hue Belkin WeMo Lutron 10 Broadlink Xiaomi
  11. 11. Actions on Google Smart Home Actions Conversation Actions 11 Actions on Google  Smart Home Actions: Control IoT devices (connect, query and control through existing cloud infrastructure).  Conversation Actions: Invoke Actions to build two-way interactive conversations between users and assistant (News, Shopping, Quiz Game, IoTs…)
  12. 12. Smart home Actions Sonoff Philips Hue Belkin WeMo Lutron 12 Broadlink Xiaomi
  13. 13. 13 Smart homeActions Google Home App Manufacturer/Personal Cloud auth/token server smarthome server 2. SYNC intent 3.EXEC/QUERY intent Request Response Response Request 1. Account linking OAuth Credentials  SYNC: Requests the list of devices that the user has connected and are available for use.  EXEC: Requests a command to execute on smart home devices “Ok Google, change the TAPIT light to blue !”  QUERY: Queries for the current states of devices “Hey Google, is the TAPIT fan on?”
  14. 14. Flow for device setup 14
  15. 15. Google Assistant 15 Flow for execution/query Device Cloud Service HomeGraph EXEC intent QUERY intent Response User say
  16. 16. 16 Device definition DEVICE DEVICE TYPE DEVICE TRAITS Light action.devices.types.LIGHT action.devices.traits.Brightness action.devices.traits.ColorSetting action.devices.traits.OnOff Fan action.devices.types.FAN action.devices.traits.OnOff action.devices.traits.FanSpeed action.devices.traits.Modes action.devices.traits.Toggles Washer action.devices.types.WASHER action.devices.traits.OnOff action.devices.traits.StartStop action.devices.traits.Modes action.devices.traits.Toggles action.devices.traits.RunCycle ………. …………… …………….. Supported more than 20 different device types
  17. 17. 17 SYNC intent for LIGHT { "requestId": "ff36a3cc-…", "inputs": [{ "intent":"action.devices.SYNC" }] } { "requestId": "ff36a3cc….", "payload": { "agentUserId": "1836", "devices": [ { "id": "123", "type": "action.devices.types.LIGHT", traits: [ “action.devices.traits.OnOff”, “action.devices.traits.Brightness”, “action.devices.traits.ColorSetting” ], “name”: { name: ‘tapit light', }, ………………. } Request Response
  18. 18. 18 EXEC intent for LIGHT { "requestId": "ff36a3cc-….", "inputs": [{ "intent": "action.devices.EXECUTE", "payload": { "commands": [{ "devices": [{ "id": "123" }], "execution": [{ "command": "action.devices.commands.OnOff", "params": { "on": true } }] } { "requestId": "ff36a3cc-…", "payload": { "commands": [ { "ids": [ "123" ], "status": "SUCCESS", "states": { "on": true, "online": true } } ] } } Request Response
  19. 19. 19 QUERY intent for LIGHT { "requestId": "ff36a3cc…", "inputs": [{ "intent": “action.devices.QUERY”, "payload": { "devices": [{ "id": "123", }] } }] } { "requestId": "ff36a3cc-…", "payload": { "devices": { "123": { "on": true, "online": true } } } } Request Response
  20. 20. 2020 Steps 1. Set up an server for Account Linking (AUTH, TOKEN) and Smart home intents (SYNC, EXEC, QUERY). 2. Set up database for devices state storage 3. Create an Actions on Google developer project. 4. Provide server URL for requesting. Using Cloud Functions for Firebase Using Firebase Realtime Database
  21. 21. 21

×