iot cocktail mixer with firebase and google actions dialogflow

1. Google Assistant and Firebase:
Cocktail Mixer

2. lim shang yi
software developer @ EZYPAY
java spring angular ionic firebase
malaysia
https://github.com/shangyilim

4. Introducing
Google Assistant

5. Introducing Google Assistant

6. Introducing Google Assistant

8. Introducing
Google Actions

10. Building for Google Actions

11. Demo
Lets talk to dev fest
cebu!

13. Presenting
Soda Mixer!

14. Soda Mixer

15. Hey Google, talk to my soda mixer
Hello! How can I help you?
Get me ice lemon tea
Sure!

16. Soda Mixer
• Using Google Actions to crate an app to
• Dispense mix of various soda together to get unique flavours

17. Ask for the
menu

18. What’s the menu?
Sprite
Apple Juice
Fruit Tea
Fizzy Iced Lemon Tea
Lemon Tea
Fizzy Apple Sparkling Fruit Tea

19. Integrating Google
Actions App with
Firebase

20. What’s Firebase

22. Integrating Google
Actions App with
Firebase

24. Firebase Real Time Database

25. Firebase Real Time Database

27. Traditional Approach in Backend

28. Simplify with Cloud Functions
• Upload complex functions without creating applications servers
• Scales automatically
• Create APIs directly
• Handle heavy processing
asynchronously.

29. Welcome emails upon signup

30. Process data in the background

31. Move heavy processing to Functions

32. Cloud Functions to publish events

33. Make a
selection

34. Demo
Lets talk to soda
mixer

35. Get me the
menu!
Get me the menu
Soda Mixer
We have a
selection of …

36. Get me the
menu!
Get me the menu
Soda Mixer
DialogFlow
Cloud Functions
We have a
selection of …

37. Asking for a drink

38. Get me aple
juice!
Get me the menu
Soda Mixer
Here you go!

39. Get me apple
juice!
Get me the drink
Soda Mixer
DialogFlow
Cloud Functions
Coming right up!
PubSub

40. Demo
Lets get a drink

41. Software
Architecture

42. Architecture Overview
Subscribe to
topic event
Talks to Google
Action App
“Soda Mixer”
Process intent
in API.AI
Fire webhook
with intent
details
Update intent details
to Assistant
Talks to
Google
Assistant
Process
event data,
send serial
command
Turn on
relay 2
Turn on
relay 1
Process and
send topic
event
Change this section to
any IOT functionality

43. Hardware
Architecture

44. Serial from Raspberry Pi->

45. Google Actions
• Create app to interact with Google Assistant
• Create interactions via api.ai
• Trigger word to start the application “Soda Mixer”
• Simulator to test conversations

46. DialogFlow
• Design intents, entities, conversation responses
• Send webhooks once intent is fulfilled
• Integrates with Google Assistant, and others like Skype, Slack etc.

47. Webhook: Google Cloud Function
• GFC Webhook to receive responses from DialogFlow
• Process data, send SodaMixerMessages topic event
• Change intent result if needed
• Do anything you want

48. Subscribe to Event
• Python script to receive SodaMixerMessages event
• Process event payload
• Send commands via Serial Interface to Arduino

49. Arduino
• Read serial commands
• Turn on relay pins
• Relay turns on motor to pump soda

50. Questions?