Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Actions On Google - How? Why?

2,393 views

Published on

How to develop your actions on google? In these slides we covered the main aspects: API.AI, Voice UI, google assistant and more.

Published in: Technology

Actions On Google - How? Why?

  1. 1. Developing Conversation Actions for the Google Assistant +GreenIdo @greenido Developer Advocate June 2017
  2. 2. Google Home — voice-activated speaker powered The Google Assistant — A conversation between you and Google that helps you get more done in your world. Actions on Google — How developers can extend the assistant (via Conversation Actions)
  3. 3. “Ok Google, talk to personal chef” Conversation API, Actions SDK Invoke “personal chef” action “Sure, here’s personal chef. Hi, I’m your personal chef, what are you in the mood for?” Speech to Text “What protein would you like to use?” “Well, it’s kind of cold outside, so I’d like something to …” Text to Speech “Sure, here’s your personal chef” Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech
  4. 4. Webhook (optional) “Ok Google, talk to personal chef” api.ai Invoke “personal chef” action “Well, it’s kind of cold outside, so I’d like something to …” “Sure, here’s your personal chef” Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech “Sure, here’s personal chef. Hi, I’m your personal chef, what are you in the mood for?” “What protein would you like to use?” Speech to Text Text to Speech
  5. 5. Intent Triggered via a series of “user says” phrases Share a common set of entities - Create new Intent otherwise
  6. 6. Intent name Name to use in API.AI console Only used by the developer, not referenced in your code
  7. 7. Action name String passed to the webhook that indicates what action is being performed Can share the same action name across different Intents API.AI specific use of “Action” compared to Google Assistant “Conversation Action”
  8. 8. Input and Output Contexts You can require a context to be available before an Intent is enabled Intent can set context to enable other Intents Implement flow control between some Intents
  9. 9. Entities Values that we are trying to capture from the user phrases Can specify a parameter name and a type of value Values can be optional Values can be a list of fixed values
  10. 10. Text response Spoken when the Intent is complete Embed entity values if desired - “I found a recipe for $protein and $vegetable!” Webhook can dynamically adjust this response
  11. 11. Prompts Specify follow up questions if a user doesn’t specify certain values Read out in random order to make it more natural
  12. 12. Webhook with HTTPS POST request
  13. 13. Node.js client library https://developers.google.com/actions/apiai/fulfillment 'use strict'; // Boilerplate setup let ApiAiAssistant = require('actions-on-google').ApiAiAssistant; let express = require('express'); let bodyParser = require('body-parser'); let app = express(); app.set('port', (process.env.PORT || 8080)); app.use(bodyParser.json({type: 'application/json'})); // Create an instance of ApiAiAssistant app.post('/', function (request, response) { const assistant = new ApiAiAssistant( {request: request, response: response}); }); // Create functions to handle requests here // Start the server let server = app.listen(app.get('port'), function () { console.log('App listening on port %s', server.address().port); console.log('Press Ctrl+C to quit.'); });
  14. 14. Node.js client library https://developers.google.com/actions/apiai/fulfillment const assistant = new ApiAiAssistant({request: request, response: response}); const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent function welcomeIntent (assistant) { assistant.ask('Welcome to action snippets! Say a number.'); } function numberIntent (assistant) { let number = assistant.getArgument(NUMBER_ARGUMENT); assistant.tell('You said ' + number); } let actionMap = new Map(); actionMap.set(WELCOME_INTENT, welcomeIntent); actionMap.set(NUMBER_INTENT, numberIntent); assistant.handleRequest(actionMap);
  15. 15. Facts About Google sample (context and deep links) https://developers.google.com/actions/apiai/fulfillment
  16. 16. Requesting user’s name and location https://developers.google.com/actions/assistant/helpers Q: "Recommend me a local bookstore" A: "To find bookstores near you, I'll just need to get your zip code from Google. Is that okay?" NAME (Registered device user’s full name) DEVICE_PRECISE_LOCATION (Coordinates and street address) DEVICE_COARSE_LOCATION (Zip code and city) // assistant is an instance of ApiAiAssistant or ActionsSdkAssistant let permission = assistant.SupportedPermissions.NAME; assistant.askForPermission('To address you by name', permission);
  17. 17. Account linking with OAuth 2.0 https://developers.google.com/actions/identity/account-linking "It looks like your <invocation name> account is not linked yet. You can link <invocation name> to your Google Account from the Google Home app." Google Home app provides a link to your login page. OAuth token is presented to your webhook. Will only connect to a login page on your domain, not a 3rd party login.
  18. 18. Speech Synthesis Markup Language (SSML) & Earcons https://developers.google.com/actions/reference/ssml <speak> <!-- Must be at the start of the string --> <say-as interpet-as="characters">SSML</say-as> <break time="3s"/>. <audio src="https://example.com/file.mp3"></audio> <say-as interpret-as="cardinal">10</say-as>. <say-as interpret-as="ordinal">10</say-as> <say-as interpret-as="characters">10</say-as>. <sub alias="World Wide Web">WWW</sub>. <p><s>This is one.</s><s>This is two.</s></p> </speak> <!-- Must be at the end of the string --> “S S M L” [3 second pause] [audio file plays] “Ten” “Tenth” “One Oh” World Wide Web [two sentences]
  19. 19. g.co/actionswebsim
  20. 20. Action naming and policies https://developers.google.com/actions/policies/general-policies ● Policies about what Conversation Actions are allowed ● See the guidelines for more information ● Invocation names must be unique since it is a global name ● Direct Actions not widely available to build and deploy, register for interest
  21. 21. Conversation Design
  22. 22. GUI :: VUI What’s the difference?
  23. 23. Top 3 Design Tips
  24. 24. Create a persona 1
  25. 25. Persona is conveyed through: ● Tone ● Word and phrase choices ● Functional design ● Style ● Technique ● Voice ...and based on: 1. Your user population and their needs 2. The imagery & qualities associated with your brand
  26. 26. Preserve and reinforce your persona by engaging the user as a separate entity from the Google Assistant. Own it. Hey! This is ___ Welcome to ___ Ready to play ____ Hi! ___ here. Hello. I’m ___ Greetings, human. Welcome back to ___ Hey again. ___ Let’s play ___ Here’s your ___ Brought to you by ___ Hi there, ___ Let’s get started. Ready for your ___ ___, here to… Live from ___ This is ___ What’s up, ___ and more...
  27. 27. Think outside the box. Literally. 2
  28. 28. Confidential and proprietary code code code code code code code code code code code “ dialog string” code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code “dialog string” code code code code code code code code code code code code code code code code code code code code code code code code code STRUCTURE + CODE
  29. 29. Confidential and proprietary DIALOG + STRUCTURE
  30. 30. In conversations, there are no “errors”. 3
  31. 31. real life user conditions{
  32. 32. “I don’t understand you”
  33. 33. When a so-called “error” occurs in a conversation, it should be treated simply as a new turn in the dialog, only with different conditions.
  34. 34. User’s have intent
  35. 35. g.co/dev/ActionsDesign
  36. 36. Actions on Google Developer Community https://g.co/actionsdev Actions on Google Developers https://developers.google.com/actions @ActionsOnGoogle
  37. 37. Thank you! +GreenIdo @greenido Developer Advocate June 2017

×