Successfully reported this slideshow.
Your SlideShare is downloading. ×

How Our Team Uses Rasa to Learn from Real Conversations | Rasa Summit

More Related Content

More from Rasa Technologies

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

How Our Team Uses Rasa to Learn from Real Conversations | Rasa Summit

  1. 1. From Chatbot to Platform Nikhil Mane Autodesk
  2. 2. Problem Landscape (2017-18) ▪ 1 million cases/year and growing ▪ 2 days average resolution time ▪ 3-10 systems accessed by agents for resolution ▪ $13-200+ cost/case range ▪ Customers lose time, create duplicate cases, agents swamped.
  3. 3. Alternative Solutions Option Pros Cons Decision Tree Good for issue triaging Structure is often representative of the back-office organization and not of customer intent. Content Wide coverage of customer problems Dependent on search for discovery Human Help Will get you to the right answer Costly in terms of resources, time, effort Forums/Community Other users can help Needs time, active involvement of resources to be useful
  4. 4. Decision Tree https://www.vabulous.com/20-simple-ways-that-small-ecommerce-businesses-can-compete-with-amazon/4-customer-support-decision-tree/
  5. 5. Decision Tree
  6. 6. Solution ▪ For Autodesk customers, the Autodesk Virtual Agent (AVA) is a customer service option that provides a seamless blend of digital and human support across all Autodesk help channels. ▪ It aims to reduce customer effort, provide instant access to products and services, and scale support.
  7. 7. Classic AVA (1.0) ▪ Single web page, standalone monolithic application ▪ Focus on providing easy access to automated transactional use cases ▪ Three high-level capabilities: ▪ Recognition (IBM Watson) ▪ Decision-making (IBM Watson) ▪ Conversational (Custom) ▪ Beta voice/video support use cases in partnership with Soul Machines
  8. 8. AVA Metrics
  9. 9. Research: Virtual Agents ▪ Context-awareness & Urgency ▪ “Well, if I had a serious issue where I felt like I really had to speak to someone and then got the result of speaking to a robot, I'd be a little disappointed.” ▪ Solutions, not instructions ▪ “It's very direct and I like it because instead of giving the instructions, it just gives me the solution.” ▪ Avatar + Conversation = Positive experience ▪ "It was a nice personal touch; I will say that. I don't know if that image (AVA avatar) is just some computer simulated image. But it is a more personable experience."
  10. 10. Research: Conversation vs Decision Tree ▪ A “conversational interface” suggests to people a more “friendly, comfortable, personable” engagement as opposed to the more mechanical interaction of clicking through a decision tree. ▪ Participants want to ask their question early in the flow. ▪ Free-form text entry is perceived as a direct and efficient way to ask specific questions. ▪ Participants modify their style of text input: Human agents get details; virtual agents get keywords. ▪ “If AVA gets a user quickly and easily to a human agent who solves their problem, […] will score points in the eyes of the user.”
  11. 11. Redesign Goals ▪ Reduce customer effort ▪ Buy ▪ Download ▪ Manage Access ▪ Self-help ▪ “Allow customers to ask a question in their own words” ▪ Natural for customers ▪ Improve routing efficiency using NLP ▪ Increase understanding of customer intent from collected data
  12. 12. Modular AVA (v2) Ideal State: ▪ Reach customers where they are experiencing issues ▪ Powered by a set of Machine Learning capabilities that automate natural language processing ▪ Independent of the interface ▪ Consumable by other ADSK self-help ▪ Increased observability
  13. 13. High-Level Plan ▪ Recognition ▪ Scale recognition of customer intent using multiple Data Science models ▪ Decision-making ▪ Decouple dialog design from application orchestration ▪ Conversational ▪ Support customers across different interfaces and different environments with consistent solutions ▪ Technical ▪ Build for re-use by other Digital Help teams
  14. 14. Roadmap ▪ Modularize front-end using React ▪ Reduce cost of developing & maintaining custom components ▪ Extract existing application logic out of Conversation ▪ Build components that can serve multiple use cases ▪ Build workflows that represent business processes ▪ And assist customers in their journey ▪ Expand presence with integrations to reduce customer effort ▪ Support existing use cases (download) in new destinations (Account) ▪ Improve observability with event driven architecture & instrumentation
  15. 15. Platform Capabilities Natural Language input Orchestration Layer Next Best Action Layer Skills model Intent model Edu model …ML model Self service widget Content Recommendation Option to contact Via ContactApp Conversatio nal workflow ▪ Customer input along with behavioral data as input ▪ Personalized solution to problem as output
  16. 16. Diagram Solution personalized to Customer input Self-service help Conversation Content Escalation
  17. 17. Planned Architecture Provides more flexibility and control over infrastructure needs.
  18. 18. Updates ▪ Input bar expansion ▪ Quick links
  19. 19. Updates ▪ Customer retains agency ▪ Escalation always available ▪ No forced path based on single prediction
  20. 20. Customer Effort Reduction ▪ Localized context-specific workflows on other ADSK properties ▪ Contributed to 5% improvements in score ▪ Benefits newly WFH users with quicker access to home use licenses
  21. 21. Customer Effort Reduction ▪ Converted existing articles & manual review process to automated workflow ▪ Simplified process led to improvements in lead generation & ultimately revenue
  22. 22. Early results ▪ Aggregate case creation from education support page vs. total page visits ▪ Blue line is when AVA was added
  23. 23. Qualitative Metrics
  24. 24. Evaluative Research Finding Learning Action Item In most cases customer provide multiple entries, iterating on what they write. Customers re-iterate their initial query to see if the solution improves. Preserve input bar after first input. Customer mentioned being ‘disappointed’, or not finding content helpful if there was no self-service solution. Self-service solutions will be preferred by customers if available. Improve article prediction model training. Some customers expected more interaction from a digital agent, rather than ‘just’ the recommendation of articles. Customers consider ‘search’ as an unfavorable comparison benchmark. Increase coverage of walkthroughs and interactive solutions.
  25. 25. Demand Gen ▪ Product discovery via personalized conversations between prospective customers and sales. ▪ Reach different customers on the same platform. ▪ Provide education about services and products.
  26. 26. Future Work ▪ Transition use cases to new Rasa Architecture with an eye on ▪ Scalability ▪ Maintainability ▪ Observability ▪ New use cases ▪ Demand generation ▪ In-product help ▪ Intelligent agent hand-off ▪ Personalization & data-driven experimentation
  27. 27. Autodesk and the Autodesk logo are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. All other brand names, product names, or trademarks belong to their respective holders. Autodesk reserves the right to alter product and services offerings, and specifications and pricing at any time without notice, and is not responsible for typographical or graphical errors that may appear in this document. © 2019 Autodesk. All rights reserved.

×