Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Google Assistant app (Actions on Google) with Microsoft Azure Functions in C# (Serverless service by Microsoft)

3,030 views

Published on

https://github.com/chomado/GoogleHomeHack
Please use "Webhook" in FulFillment page in Dialogflow to use Azure

Published in: Technology
  • Thanks for the help. I also ordered from www.HelpWriting.net
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Finally found a service provider which actually supplies an essay with an engaging introduction leading to the main body of the exposition Here is the site ⇒⇒⇒WRITE-MY-PAPER.net ⇐⇐⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! I can recommend a site that has helped me. It's called ⇒ www.WritePaper.info ⇐ They helped me for writing my quality research paper.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Did you know some people make ✄✄✄ https://dwz1.cc/v5Fcq3Qr
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi there! I just wanted to share a list of sites that helped me a lot during my studies: .................................................................................................................................... www.EssayWrite.best - Write an essay .................................................................................................................................... www.LitReview.xyz - Summary of books .................................................................................................................................... www.Coursework.best - Online coursework .................................................................................................................................... www.Dissertations.me - proquest dissertations .................................................................................................................................... www.ReMovie.club - Movies reviews .................................................................................................................................... www.WebSlides.vip - Best powerpoint presentations .................................................................................................................................... www.WritePaper.info - Write a research paper .................................................................................................................................... www.EddyHelp.com - Homework help online .................................................................................................................................... www.MyResumeHelp.net - Professional resume writing service .................................................................................................................................. www.HelpWriting.net - Help with writing any papers ......................................................................................................................................... Save so as not to lose
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Google Assistant app (Actions on Google) with Microsoft Azure Functions in C# (Serverless service by Microsoft)

  1. 1. Developed Google Home app (Actions on Google) using Microsoft Azure Functions
  2. 2.  Everyone calls me “Chomado”.  Based in Tokyo, Japan. Love anime/manga  Before joining Microsoft, I’d been a mobile app developer using Xamarin + Microsoft Azure  I’m not only an engineer but also a manga artist:
  3. 3. My manga work: “Run! Coding Academy”
  4. 4. 37.4K Twitter : @chomado
  5. 5. I love my Grandma She has presbyopia (aged eyes) and difficult to read. So I bought a smart speaker for her. I think VUI (Voice UI) is good for old-aged people.
  6. 6. What is “Smart Speaker”? Wireless speaker with AI assistant!
  7. 7. Virtual Assistant (logo) Virtual Assistant Owned by Devices Languages/Markets Microsoft Cortana Harman Kardon INVOKE English for US, Canada, Australia, India Amazon Alexa Amazon Echo English for US, UK, Ireland, Canada, India, Australia, New Zealand; German (Germany, Austria); Japanese (Japan); Google Assistant Google Home English (U.S., U.K., Canada, Australia), French (France, Canada), German, Japanese Siri Apple HomePod English for US, UK, Australia Clova Clova Friends, Clova WAVE Japanese and Korean for Japan and South Korea markets
  8. 8. Virtual Assistant (logo) Virtual Assistant Owned by Devices Languages/Markets Microsoft Cortana Harman Kardon INVOKE English for US, Canada, Australia, India Amazon Alexa Amazon Echo English for US, UK, Ireland, Canada, India, Australia, New Zealand; German (Germany, Austria); Japanese (Japan); Google Assistant Google Home English (U.S., U.K., Canada, Australia), French (France, Canada), German, Japanese Siri Apple HomePod English for US, UK, Australia Clova Clova Friends, Clova WAVE Japanese and Korean for Japan and South Korea markets (ToT) Google Home
  9. 9. How Microsoft Cortana skill works?
  10. 10. How's the weather like in Tokyo today? voice How's the weather like in Tokyo today? text Weather Service When you say a command which Google Assistant knows How Google Assistant works Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech Devices Google Assistant Weather Service
  11. 11. Connect to HelloAzureFunctio ns service voice HelloAzureFunctions I don’t know such service. I don’t know what to do When you say a command which Google Assistant does NOT know Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech Devices Google Assistant Connect to HelloAzureFunction s service text How Google Assistant works
  12. 12. When you say a command which Google Assistant does NOT know Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech Devices Google Assistant How Google Assistant works So, to make Google Assistant do your own command, you need to create an Google Assistant app called “Actions on Google” (same as Cortana/Alexa skill development motivation)
  13. 13. You need to implement this function (Web API) called by Webhook Speech to Text, NLP, Knowledge Graph, ML Ranking, User Profile, Text to Speech Devices Google Assistant Create Actions on Google to run your own command on your Google Assistant
  14. 14. Hi, I’m Madoka (voice) Hi, I’m Madoka (in cool voice) I made a voice changer app (action) working like this!
  15. 15. I post its demo video to Twitter two weeks ago (16 March) I made this app using Microsoft Azure Functions (C#) and it works like this … 47.5 K views in 2 weeks! It means many Google developers got interested in Microsoft technology. https://twitter.com/chomado/status/974534165419892736
  16. 16. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  17. 17. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook Azure Functions (where my code running) “I’m Madoka” (json data) Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  18. 18. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook Azure Functions (where my code running) “I’m Madoka” (json data) VoiceText Web API (text to speech service by HOYA, Inc) “I’m Madoka” (string) Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  19. 19. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook Azure Functions (where my code running) “I’m Madoka” (json data) VoiceText Web API (text to speech service by HOYA, Inc) “I’m Madoka” (string) Azure Storage (Blob) Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  20. 20. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook Azure Functions (where my code running) “I’m Madoka” (json data) VoiceText Web API (text to speech service by HOYA, Inc) “I’m Madoka” (string) Azure Storage (Blob) mp3 data’s URL + Request to play the mp3 file (written in SSML) Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  21. 21. Actions on Google “I’m Madoka” She says “I’m Madoka”. Invoke Webhook I’m Madoka (in cool voice) Azure Functions (where my code running) “I’m Madoka” (json data) VoiceText Web API (text to speech service by HOYA, Inc) “I’m Madoka” (string) Azure Storage (Blob) mp3 data’s URL + Request to play the mp3 file (written in SSML) Microsoft Azure Dialogflow (Natural language user interface like MS LUIS)
  22. 22. Trigger: HttpTrigger Output: CloudBlockBlob (I chose this because it can directly access the stored mp3 file’s URL) Azure Function (v1) which is called by Google’s Webhook Google Assistant users’ input text (“I’m Madoka!”) is stored in this JSON property named “ResolvedQuery”, so we use it.
  23. 23. In the Function In speech execution, it’s better to use SSML format. (Speech Synthesis Markup Language: an XML-based markup language for speech synthesis applications) Not only Google Assistant but also our Cortana and Amazon Alexa support it. You can access the URL which stores the mp3 file in Azure Blob Storage As you know, Azure Functions allows us to connect & post data to Azure Storage sooooo easily! Azure Functions is awesome. Everyone knows it.
  24. 24. It got 58 stars and forked 10 times

×