This tutorial will get you started with creating applications for the Google Home by helping you create your first "Action" which you can deploy on your device.
2. 2
Overview
• Computer interaction via natural language (voice, chatbot) is growing
– U.S.: 20% use phone voice assistants; 10% use stand-alone devices (Home, Echo)
• Machine learning drives success & growth in this space
– Free tools exist to learn & build, with no programming experience required
• Google Home and Amazon echo have two types of voice applications:
– Built in: Check the weather, set a timer, play music, etc.
– 3rd Party Applications:
• Check my bank balance
• Order an Uber
• Anything else you come up with – We’ll learn how today !
3. 3
Tools to get started
• Go to DialogFlow, create an account, and log in - https://dialogflow.com/
• We’ll cover the basics today, and this video is also a great walkthrough
https://www.youtube.com/watch?v=9SUAuy9OJg4
• Basic Terminology:
– Agent – Voice applications are known as ‘Agents’ in API.AI
– Intent – A certain thing you want your agent to do
– Entity – A list of objects that your agent will recognize as inputs from the user
• Example: For this tutorial, we’ll create an Agent that tells you what
meeting rooms are near your team
– Agent Name: “example_rooms_near_me”
– Intent example: “list rooms near 3500 block”
– Entity example: “teams_in_3500”
• e.g. Marketing, Operations, Design, etc.
4. 4
Create Agent: Click the drop down in top left, create new agent, give
it a name and description, then save
6. 6
Create an intent request: Give it a name, and type an example user
request. Note that API.AI recognizes and defines the entity from user
7. 7
Create the intent response: Scroll down and define how you want
your agent to respond when this intent is requested. You can call the
user’s input parameter using the dollar sign “$”
8. 8
Push live & test: You can push your agent live across many
applications. Click “actions on google” and follow prompts to test on
the Google Home simulator
9. 9
Your Agent is working! Now you can enhance it with training, pre-
built intent additions, and backend fulfillment
10. 10
Training: Constantly improve your agent by assigning the right
intent or entity when it encounters a question it can’t answer
11. 11
Prebuilt Agents: Instantly add functionality to your agent with
API.AI’s suit of pre-built intents – add weather, small talk, and more
12. 12
Backend Fulfillment: Connect your agent to a database, website, or anything
accessible via API. You can use the ‘webhook’ feature to make an API call
for additional info (aka ‘connect the backend’)
**I recommend getting your first agent running and tested without using a webhook (focus of this training).
After that, connecting to backend via webhook is a great step 2 – we’ll tackle this in a separate training session.
• Or, check out this link for a detailed tutorial: https://docs.api.ai/docs/webhook#webhook-example
13. 13
As we’ve seen, you can create an agent in less than an hour;
Try creating one of these and impress your family & friends
• Chores agent:
– Intent: “Who should take out the trash?”
– Text Response: Return a random family members name
• Office agent:
– Intent: “When does the morning bus run?”
– Intent: “Which cafeteria should we go to for lunch?”
• Fun fact:
– Intent: “Tell me a fun fact about $family_members_name”
– Text Response: Return a random fact about the family member requested
• Other ideas for work, home, fun ??
15. 15
User Request
Interaction Model
(for your Application)
Backend
How does it work?
• Voice (Echo, Home)
• Chat-bot
– Facebook
– Slack
• Intent
– What user wants to do
• Sample Utterances
– How they request it
• Execution Logic
– Code to fulfill request
• Database
• Website
• Anything connected to
the internet..
16. 16
User Request Interaction Model Backend
How do we do it?
• Voice
• Chat-bot
• Intent
• Sample Utterances
• Execution Logic
• Anything on the
internet
Machine Learning API ConnectionsKeys to success:
API.AI Web InterfaceTools for this demo: “Webhook” API call
17. 17
20% of Americans leverage voice on mobile phone, while 10%
leverage in-home voice devices; large YoY growth seen for in-home
Mobile Phone: https://www.emarketer.com/Article/Alexa-Say-What-Voice-Enabled-Speaker-Usage-Grow-Nearly-130-This-Year/1015812
Mobile Search: 20% of Mobile Queries are generated by Voice.
Echo / Home Users: http://www.businessinsider.com/amazon-echo-vs-google-home-sales-estimates-chart-2017-5
Echo Growth: http://www.geekwire.com/2016/amazon-holiday-sales-numbers/ ; http://www.geekwire.com/2016/amazon-echo-sales-reach-5m-two-
years-research-firm-says-google-competitor-enters-market/