Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Voice Applications - Next Generation Intelligent Interfaces

56 views

Published on

Have you ever wanted to build a Star Trek communicator, Knight Rider’s KITT, or Star Wars’ C-3PO? Now, with the introduction of Siri, Amazon Echo, or Google Home, voice interfaces are part of our everyday lives. Come and learn what open source tools we at Salesforce have built to achieve that. Walk away from the session with an understanding of the tools and how to create your own intuitive voice experience with the power of Salesforce.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Voice Applications - Next Generation Intelligent Interfaces

  1. 1. Voice Applications Next Generation Intelligent Interfaces @VineetSinha Vineet Sinha, Director R&D Executive Immersion
  2. 2. Voice is a natural interface for us to work with Computers
  3. 3. Voice is a natural interface for us to work with Computers
  4. 4. Voice Applications Next Generation Intelligent Interfaces @VineetSinha Vineet Sinha, Director R&D Executive Immersion
  5. 5. Voice can be a Natural User Interface Voice Interactions Today Which restaurants are nearby? Here are a few popular restaurants: ... Which restaurants nearby serve alcohol? Here are a few popular restaurants: ... Typical Human Interactions Where can I get dinner? We have a few great Italian, Korean, and Indian restaurants close by. Are you interested in any of them? Yes, but which ones they have alcohol? Among restaurants that serve alcohol, we have … Voice apps today are primarily designed to just work in a command-response manner In part because APIs focused around low-level implementation details They end up having low Voice User Experience (VUX)
  6. 6. Building a Voice Application Easily
  7. 7. Getting Started var violet = require( 'violet-conversations/lib/violet').script(); violet.respondTo({ expecting: "What next on my todo", resolve: (response) => { var nextItem = quipSvc.getNextItem(); response.out(`Next is ${nextItem}`); }}); module.exports = violet; 1. Create Script (in Node.js) - framework code - input/output bits - business logic 2. Run Locally – Use Web interface for debugging 3. Follow registration instructions to use with Amazon Echo, etc
  8. 8. Getting Started var violet = require( 'violet-conversations/lib/violet').script(); violet.respondTo({ expecting: "What next on my todo", resolve: (response) => { var nextItem = quipSvc.getNextItem(); response.out(`Next is ${nextItem}`); }}); module.exports = violet; 1. Create Script (in Node.js) - framework code - input/output bits - business logic 2. Run Locally – Use Web interface for debugging 3. Follow registration instructions to use with Amazon Echo, etc
  9. 9. Getting Started var violet = require( 'violet-conversations/lib/violet').script(); violet.respondTo({ expecting: "What next on my todo", resolve: (response) => { var nextItem = quipSvc.getNextItem(); response.out(`Next is ${nextItem}`); }}); module.exports = violet; 1. Create Script (in Node.js) - framework code - input/output bits - business logic 2. Run Locally – Use Web interface for debugging 3. Follow registration instructions to use with Amazon Echo, etc
  10. 10. Getting Started var violet = require( 'violet-conversations/lib/violet').script(); violet.respondTo({ expecting: "What next on my todo", resolve: (response) => { var nextItem = quipSvc.getNextItem(); response.out(`Next is ${nextItem}`); }}); module.exports = violet; 1. Create Script (in Node.js) - framework code - input/output bits - business logic 2. Run Locally – Use Web interface for debugging 3. Follow registration instructions to use with Amazon Echo, etc
  11. 11. Getting Started var violet = require( 'violet-conversations/lib/violet').script(); violet.respondTo({ expecting: "What next on my todo", resolve: (response) => { var nextItem = quipSvc.getNextItem(); response.out(`Next is ${nextItem}`); }}); module.exports = violet; 1. Create Script (in Node.js) - framework code - input/output bits - business logic 2. Run Locally – Use Web interface for debugging 3. Follow registration instructions to use with Amazon Echo, etc
  12. 12. How do you get data from users?
  13. 13. Getting data from users is easy violet.addInputTypes({ 'todoItem': LITERAL }); violet.respondTo({ expecting: 'Remind me [[todoItem]]'], response: (response) => { todoList.add(response.get('todoItem')); response.say('Added to your list'); }}); 1. Declare variables 2. Build the script
  14. 14. Getting data from users is easy violet.addInputTypes({ 'todoItem': LITERAL }); violet.respondTo({ expecting: 'Remind me [[todoItem]]'], response: (response) => { todoList.add(response.get('todoItem')); response.say('Added to your list'); }}); 1. Declare variables 2. Build the script
  15. 15. So, What is Violet? A framework for building voice applications • Based on Node.js and Express With Violet you get • Ease for building basic ‘skills’ • Support for sophisticated conversational bots • Development able to focus on the Voice User Experience • Plugins to make it easier to extend the framework, for example – integration with Salesforce.com • Collection of pre-built easy-to-customize templates to get started quickly
  16. 16. Sophistication in Voice Experiences
  17. 17. Build Sophisticated Voice Apps using Conversational Goals What if we could group a set of user and app responses? • Essentially functions/methods for voice Lets call these goals • And conversation become a set of changing shared goals between the user and the app Classic speech systems call these Dialogs • Are focus is on primitives to make creating great Voice User Experience’s
  18. 18. Example: Adding an item to one of two categories violet.respondTo({ expecting: 'Remind me [[todoItem]]', resolve: (response) => { if (quipSvc.getCategoryCount() > 1) return findCategory(); //... }}); var findCategory = () => { response.prompt(’Which category?'); response.addGoal('getCategory'); } violet.respondTo({ expecting: 'Category [[categoryNo]]', resolve: (response) => { if (response.hasGoal('getCaegory')) //... //... }}); 1. Check when things need to be complex 2. Prompt and trigger the goal 3. When getting a response validate the goal
  19. 19. Example: Adding an item to one of two categories violet.respondTo({ expecting: 'Remind me [[todoItem]]', resolve: (response) => { if (quipSvc.getCategoryCount() > 1) return findCategory(); //... }}); var findCategory = () => { response.prompt(’Which category?'); response.addGoal('getCategory'); } violet.respondTo({ expecting: 'Category [[categoryNo]]', resolve: (response) => { if (response.hasGoal('getCaegory')) //... //... }}); 1. Check when things need to be complex 2. Prompt and trigger the goal 3. When getting a response validate the goal
  20. 20. Example: Adding an item to one of two categories violet.respondTo({ expecting: 'Remind me [[todoItem]]', resolve: (response) => { if (quipSvc.getCategoryCount() > 1) return findCategory(); //... }}); var findCategory = () => { response.prompt(’Which category?'); response.addGoal('getCategory'); } violet.respondTo({ expecting: 'Category [[categoryNo]]', resolve: (response) => { if (response.hasGoal('getCategory')) //... //... }}); 1. Check when things need to be complex 2. Prompt and trigger the goal 3. When getting a response validate the goal
  21. 21. Example: Adding an item to one of two categories (better) violet.respondTo({ expecting: 'Remind me [[todoItem]]', resolve: (response) => { if (quipSvc.getCategoryCount() > 1) return findCategory(); //... }}); var findCategory = () => { response.prompt(’Which category?'); response.addGoal('getCategory'); } violet.respondTo({ expecting: 'Category [[categoryNo]]', resolve: (response) => { if (response.hasGoal('getCategory')) //... //... }}); violet.respondTo({ expecting: 'Remind me [[todoItem]]', resolve: (response) => { if (quipSvc.getCategoryCount() > 1) response.addGoal('getCategory'); //... }}); violet.defineGoal({ goal: 'getCategory', prompt: 'Which category?', respondTo: [{ expecting: 'Category [[categoryNo]]', resolve: (response) => { if (response.hasGoal('getCategory')) //... //... }]});
  22. 22. A Little Magic to Get Results Quick
  23. 23. Easy Salesforce Integration var sfStore = require(’violetStoreSF.js')(v); sfStore.store.propOfInterest = { 'Lead*': ['Name*', 'Company*'], 'Case*': ['CaseNumber*', 'Contact*.Name*’ 'Status*', 'Priority*']} 1. Load the Salesforce Store Plugin 2. Configure fields that you care about 3. Use Salesforce as a store: - Query: For eg, Check for any new leads - Store Data: For eg, Create a new lead - Update: For eg, Change priority on a case
  24. 24. Easy Salesforce Integration var sfStore = require(’violetStoreSF.js')(v); sfStore.store.propOfInterest = { 'Lead*': ['Name*', 'Company*'], 'Case*': ['CaseNumber*', 'Contact*.Name*’ 'Status*', 'Priority*']} var results = yield response.load( 'Lead*', 'CreatedDate = TODAY'); if (results.length == 0) { response.say(‘You have no new leads.'); } 1. Load the Salesforce Store Plugin 2. Configure fields that you care about 3. Use Salesforce as a store: - Query: For eg, Check for any new leads - Store Data: For eg, Create a new lead - Update: For eg, Change priority on a case
  25. 25. Easy Salesforce Integration var sfStore = require(’violetStoreSF.js')(v); sfStore.store.propOfInterest = { 'Lead*': ['Name*', 'Company*'], 'Case*': ['CaseNumber*', 'Contact*.Name*’ 'Status*', 'Priority*']} response.store('Lead*', { 'FirstName*': names[0], 'LastName*': names[1], 'Company*': leadCompany}); 1. Load the Salesforce Store Plugin 2. Configure fields that you care about 3. Use Salesforce as a store: - Query: For eg, Check for any new leads - Store Data: For eg, Create a new lead - Update: For eg, Change priority on a case
  26. 26. Easy Salesforce Integration var sfStore = require(’violetStoreSF.js')(v); sfStore.store.propOfInterest = { 'Lead*': ['Name*', 'Company*'], 'Case*': ['CaseNumber*', 'Contact*.Name*’ 'Status*', 'Priority*']} response.update('Case*', 'CaseNumber*', caseNumber, { 'Priority*': casePriority }); 1. Load the Salesforce Store Plugin 2. Configure fields that you care about 3. Use Salesforce as a store: - Query: For eg, Check for any new leads - Store Data: For eg, Create a new lead - Update: For eg, Change priority on a case
  27. 27. And a collection of templates to get you moving faster Launch & configure a pre-built voice script for your org • See the violet-templates project Customize it to your needs • Use the templates as example code and put them together Create new templates • We would love to see what you come up with
  28. 28. What’s Next
  29. 29. We want to see what you build… Check out Violet and try: 1. Deploying one of the existing voice-script and using it. 2. Customize a script to your needs. 3. Extend Violet: • Support Google Home or other devices • Try supporting sentiment analysis or chat-bots with it Let us know how it goes For more info: https://github.com/salesforce/violet-conversations http://HelloViolet.ai
  30. 30. Ashita Saluja Gina Nguyen John Brunswick Built with incredible insights from … Nilan Fernando

×