Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Watson DevCon 2016 - Engage and Be Engaging: Building Compassionate and Personalized Conversational Systems


Published on

Presentation at Watson DevCon 2016, given by Rama Akkiraju.
Getting to know each other, tailoring our communication style, conversing with each other naturally — humans do this everyday. Now, with the advent of cognitive technologies, computers can, too. Find out how you can create apps that can engage and delight users with APIs that predict personality from writings, analyze linguistic tones from communications, and automate conversations. See firsthand how easy it can be to make digital interactions at massive scale personal and productive.

Published in: Technology

Watson DevCon 2016 - Engage and Be Engaging: Building Compassionate and Personalized Conversational Systems

  1. 1. Engage  and  be  Engaging:   Building  Compassionate  and   Personalized  Conversational   Systems Rama  Akkiraju IBM  Distinguished  Engineer
  2. 2. Please  note IBM’s statements  regarding  its  plans,  directions,  and  intent  are  subject  to change  or withdrawal without  notice  and   at  IBM’s sole  discretion. Information  regarding potential  future  products  is  intended  to  outline  our  general  productdirection  and  it  should   not  be  relied  on  in  making  a  purchasing  decision. The  information  mentionedregarding  potential  future  products  is  not  a  commitment,  promise,  or legal  obligation  to   deliver  any  material,  code  or functionality.  Information  about  potential  futureproducts  may not  be  incorporated  into   any  contract. The development,  release,  and  timing  of  any  future  features  or functionality  described  for  our  products  remains  at   our  sole  discretion. Performance  is  based  on measurements  and  projections  using  standard  IBM  benchmarks  in  a controlled   environment.  The  actual  throughput  or  performance  that  any user  will  experience  will vary  depending  upon  many   factors,  including  considerations such  as  the  amount  of  multiprogramming  in  the  user’s  job  stream,  the I/O   configuration,  the  storage  configuration,  and  theworkload processed.  Therefore,  no  assurance  can  be  given  that   an  individual user  will  achieve  results  similar  to  those  stated  here. 11/21/16World  of  Watson  2016  2
  3. 3. Consider  this   conversation Presentation  Title 8/11/16 Watson Developer Conference 2016  3 What  kind  of  cuisine  do   you  prefer? What’s  a  good  place  to   eat  in  San  Francisco? Are  you  a  vegetarian? Italian,  or  Mexican,  or   Indian,  or  Chinese Do  you  like  a  fine  dining   experience  or  fast  food? Yes Are  you  driving,  taking  a   taxi  or  using  public   transportation? Fine  dining Do  you  prefer  a   restaurant  that  has  a   parking  lot  of  its  own,  or   are  you  okay  to  find   street  parking? Driving I  can  recommend  three   choices:  Greens  on   Marina,  Gracious  Madre   on  Mission,  and  Enjoy   Chinese  on  Kearney… Parking  lot  would  be   good Works!  But  takes  too   many  turns  to  arrive  at   a  solution  and  solution   is  not  personalized
  4. 4. Consider  this revised conversation Presentation  Title 8/11/16 Watson Developer Conference 2016  4 What’s  a  good  place  to   eat  in  San  Francisco? Better!  Takes  far  fewer   turns  and  offers  much   more  personalized   experience What  kind  of  cuisine  do   you  prefer? Joan,  because  you  are   a  vegetarian,  open  to   experiences,  prefer  fine   dining,   and  are  likely  to   drive  given  you  are   currently  renting  a  car,   I’d  recommend  Greens,   a  fusion  vegetarian   Restaurant  on  2  Marina   Blvd.  It  is  a  4.1/5-­rated   vegetarian  restaurant   with  ample  parking  and   is  within  1  mile   from   your  current  location,   so  you  could  even  walk   to  it. Italian,  or  Mexican,  or   Indian,  or  Chinese Cognitive  assistant  is   demonstrating   understanding  of   Joan’s  preferences,   personality,  life  events   and  personalizing   answers  accordingly.
  5. 5. Consider  this   conversation Presentation  Title 8/11/16 Watson Developer Conference 2016  5 There  are  proven,   effective  ways  to  deal   with  bullying  and  bullies.   Would  you  like  me  to   help  you? I’m  getting  bullied   in   school.  I’m  so  upset. Are  you  facing  verbal,   physical  or  social   bullying? Yes.  Please. Where  has  this  been   happening? I  guess,  social  and   verbal  bullying.   I  can’t   focus  on  my  studies.  My   situation  is  so  hopeless. Why  do  you  think  they   are  bullying  you? On  the  school  bus,   during  recess,  after   school,  during  projects  – everywhere Have  you  tried  avoiding   those  who  are  bullying   you?   I  think  it’s  because  I’m   overweight.  They  call   me  names.  It’s  so   demeaning. Here  is  a  short  video  on   how  to  deal  with  bullying.   This  can  help  you. I  can’t.  they  are  my   teammates.  I  have  to  do   a  project  with  them. Works!  But  lacks   compassion
  6. 6. Consider  this revised conversation Presentation  Title 8/11/16 Watson Developer Conference 2016  6 Kathy,  I’m  so  sorry  to   hear  that  you  are  facing   bullying   in  school.   Don’t  feel  distressed.   You  are  not  alone  in   this  fight.  There  are   proven,  effective  ways   to  deal  with  bullying   and  bullies.   I’m  here  to   listen  and  to  help  you  in   any  way  I  can.  Would   you  like  me  to  help   you?   I’m  getting  bullied   in   school.  I’m  so  upset. Are  you  facing  verbal,   physical  or  social   bullying? Yes.  Please. Oh!  That  is  indeed   unfortunate!  Don’t  lose   hope,  Kathy.  You  can   manage  the  situation.   I’m  here  to  help  you.   First,  let  me  ask  you   this.  Where  has  the   bullying   been   happening?   I  guess,  social  and   verbal  bullying.   I  can’t   focus  on  my  studies.  My   situation  is  so  hopeless. Better!  Compassionate   companion
  7. 7. Consider  this revised conversation Presentation  Title 8/11/16 Watson Developer Conference 2016  7 I  see.  Why  do  you  think   they  are  bullying  you? On  the  school  bus,   during  recess,  after   school,  during  projects  – everywhere I’m  sorry  to  hear  that.   That  is  indeed  very   insensitive  of  them  to   say  that.  Have  you  tried   avoiding  those  who  are   bullying   you?   I  think  it’s  because  I’m   overweight.  They  call   me  names.  It’s  so   demeaning. I  see.  Kathy,  I  can  help   you  in  the  following   ways.  I  have  access  to   some  videos  that  may   help  you.    After  you  had   a  chance  to  watch  them   and  grasp  the   techniques  suggested,   if  you  wish,  we  can  also   do  some  role  play.  That   way,  you  can  be   prepared  to  deal  with   them  next  time  you  face   them.   I  can’t.  They  are  my   teammates.  I  have  to  do   a  project  with  them. Cognitive  assistant  is   demonstrating   understanding  of   Kathy’s  emotional  state   and  is  working  with  her   in  a  compassionate   manner. Better!  Compassionate   companion
  8. 8. To  build  compassionate  and   personalized  conversational  systems,   three  core  models  are  needed: Presentation  Title 8/11/16 Watson Developer Conference 2016  8 Naturally3 Interact2 People1 Understand  people  at  a  deeper  level   Understand  and  respond  in   various  mediums  in  which   interactions  can  occur Interact2 naturally3 with  people1 Input  types:  Text,  speech,  gestures Mediums:  Computers,  mobile   devices,  robots,  avatars Understand  styles  of  human  interaction   and  optimize  human-­computer  interaction
  9. 9. Building  block  1:   People  modeling/ user  modeling Naturally3 Interact2 People1
  10. 10. People  are   complex  beings   with  many  facets “Who  are  these   people?” “How   do  they   make   decisions?” “What  do  they   value?” “How   do  they   think?” “What  is  their   context?” “What  are   their   preferences?” “What  are   their   emotions?” “How   do  they   act?” “How   are  they   feeling?” “What  are  their   skills?” “How   do  they   express   themselves?”
  11. 11. User  modeling:   Our  framework An  individual  takes  action  based  on  the  combination  of  his/her  unique   being  and  environment   Act Be Feel Context Think Options Explore & Decide Inner  State Environment Outer  State
  12. 12. User  modeling:  Our  framework Act Search Preferences Communications Decisions Commitments Purchases Context Life  Style,  Events Sociological Economic Political Technological Options Price Promotions Products/Services Place Feel Perceptions Emotions Sensations Attitudes Influences Sentiments Be Personality Needs,   Values Beliefs Motives Identity Goals,  Ambitions Interests Think Knowledge Skills Opinions Cognitive   Style Explore & Decide Choices Consequences Session Intent Time
  13. 13. 11/21/16 Watson Developer Conference 201613 Ready  to  use Still  in  R&D
  14. 14. Use  Personality   Insights  to  engage   with  individuals  at   personalized  level 11/21/16 Watson Developer Conference 201614 Watson-­pi-­ Source:
  15. 15. Emotional   Analysis  helps   build  empathetic   systems 11/21/16 Watson Developer Conference 201615 sentiment-­and-­
  16. 16. Use  Tone   Analyzer  to   understand  and   fine  tune  your   message 11/21/16 Watson Developer Conference 201616 tone-­analyzer-­
  17. 17. Building  block  2:   Human  interaction   patterns Naturally3 Interact2 People1
  18. 18. Natural   interactions   among  people 11/21/16 Watson Developer Conference 201618 Verbal  (expressive,  aggressive,  passive)  ,   Non-­verbal  (gestures,  facial  expressions,  postures)
  19. 19. Dialog  strategies   Start Giving   an   extra Acknowledging   Need Description   of   Need Anger Acknowledging   w/o  encouraging Refocus   statements Active   Listening Possibility   of   mistake Admitting   mistake Allowing   venting Apology Smiles Arranging   Follow-­up Need   cannot   be  fulfilled   on   the  spot Assurance   of   effort Assurance   of   result Mistake  has   been   made Bonus   buyoff Broken   record Uncooperative   customer Closing   positively Common   Courtesy Completing   Follow-­up Contact   Security Aggressiveness Disengaging Distraction Frustration Empathy   statement Expediting Expert   Recommend ation Explain   Reasoning   or   action Embarrassment Face-­Saving   Out Conflict Finding Agreement   Points Following   up Helpless Offering   Choice Empowering Preventive   strike Privacy   insurance Privacy   concern Probing   question Pros  and   Cons Providing   Alternatives Providing   Takeaway Confusion Providing Explanation Questioning   instead   of   stating Referral   to   supervisor Referral   to  3rd party Lost  focus Refocus Inappropriate   behavior Setting  Limits Critical Neutral   mode Summarize   the   conversation Silence Thank-­you Timeout Use  customer   name Verbal   Softeners When  questionYou’re   right Action Negative   Emotion Monologue End External Giving Emotions General States Gratitude Statement Happiness Work  by  Michal  Shmueli-­Scheuer,   Jonathan   Herzig,   Guy  Feigenblat,   David   Konopnicki
  20. 20. Tone-­aware   Expressions  in   Conversations 11/21/16 Watson Developer Conference 201620 food-­coach.
  21. 21. 11/21/16World  of  Watson  2016  Page  21
  22. 22. Auto-­generated   response  in   customer  care   domain 11/21/16 Watson Developer Conference 201622 Agent   responses   are  auto-­ generated.
  23. 23. Bot  personas   and  expressive   paraphrasing 11/21/16 Watson Developer Conference 201623
  24. 24. Personalized   conversational   systems 11/21/16 Watson Developer Conference 201624 Based  on  customer’s  emotion  and  personality  the  cognitive  bot   personalizes  responses Detect   user   emotion Bot   understands   user’s   personality   and   preferences   and   provides   recommendations Change   bot   expression   based   on  user   emotion
  25. 25. Building  block  3:   Understand  various  mediums   in  which  human-­computer   interaction  can  occur Naturally3 Interact2 People1
  26. 26. Tone-­aware   Expressions  in   Conversations 11/21/16 Watson Developer Conference 201626 Speech-­to-­Text Text-­to-­Speech speech-­to-­text-­ text-­to-­speech-­
  27. 27. Deriving  human   expressions  from   speech,  images   and  video Speech – Non-­verbal  clues:  pauses,   volume,  intonation,  pitch   11/21/16 Watson Developer Conference 201627 Video,  images – Gestures – Facial  expressions – Eye  contact – Posture – Tone  of  voice – Distance
  28. 28. Bringing  it  all  together: Building  compassionate  and   personalized  conversational   systems Naturally3 Interact2 People1
  29. 29. Deep-­dive:   Watson  Food-­Coach  App   Demonstrates  how  to   incorporate  tone  and  emotion   detection  into  Conversations   and  to  design  compassionate   dialog  flow 29
  30. 30. 11/21/16World  of  Watson  2016  Page  30 Intents Entities
  31. 31. Food  Coach  Dialog  Flow 31
  32. 32. Building  Compassionate  and   Personalized  Conversational  Agents 11/21/16 Watson Developer Conference 201632 Naturally3 Interact2 People1 • We  are  developing  the  platform  and  the   building  blocks  for  implementing   compassionate  and  personalized   conversational  agents. • Checkout  our  services,  Build  your   conversational  apps  and  give  us  your   feedback!
  33. 33. Notices  and   disclaimers   continued   Information   concerning   non-­IBM   products   was  obtained   from  the  suppliers   of  those   products,   their  published   announcements   or  other   publicly  available   sources.    IBM  has   not  tested   those   products   in  connection   with  this  publication   and   cannot   confirm  the   accuracy  of   performance,   compatibility  or  any   other  claims  related   to  non-­IBM   products.    Questions   on  the   capabilities   of  non-­IBM  products   should   be   addressed   to  the  suppliers   of  those   products.   IBM  does  not   warrant  the  quality   of  any  third-­party   products,   or  the   ability  of  any  such  third-­ party  products   to  interoperate   with  IBM’s  products.    IBM  EXPRESSLY   DISCLAIMS  ALL   WARRANTIES,  EXPRESSED   OR  IMPLIED,   INCLUDING  BUT  NOT  LIMITED  TO,  THE  IMPLIED   WARRANTIES  OF  MERCHANTABILITY  AND  FITNESS   FOR  A  PARTICULAR   PURPOSE.   The  provision   of  the  information   contained   herein   is  not  intended   to,  and   does  not,   grant   any  right  or  license   under  any   IBM  patents,   copyrights,   trademarks   or  other   intellectual   property   right.   IBM,  the  IBM  logo,,  Aspera®,   Bluemix,  Blueworks  Live,   CICS,  Clearcase,   Cognos®,   DOORS®,  Emptoris®,  Enterprise   Document   Management   System™,   FASP®,  FileNet®,  Global  Business   Services  ®,  Global   Technology   Services  ®,  IBM  ExperienceOne™,   IBM   SmartCloud®,   IBM  Social  Business®,   Information   on  Demand,   ILOG,   Maximo®,  MQIntegrator®,   MQSeries®,  Netcool®,   OMEGAMON,   OpenPower,   PureAnalytics™,   PureApplication®,   pureCluster™,   PureCoverage®,   PureData®,   PureExperience®,   PureFlex®,  pureQuery®,   pureScale®,   PureSystems®,   QRadar®,  Rational®,   Rhapsody®,   Smarter  Commerce®,  SoDA,   SPSS,   Sterling  Commerce®,  StoredIQ,   Tealeaf®,   Tivoli®,  Trusteer®,   Unica®,  urban{code}®,   Watson,   WebSphere®,   Worklight®,  X-­Force®  and   System  z®  Z/OS,  are  trademarks   of   International   Business   Machines  Corporation,   registered   in  many  jurisdictions  worldwide.  Other   product   and   service  names   might  be   trademarks  of  IBM  or  other   companies.   A  current  list  of  IBM  trademarks  is  available   on  the   Web  at  "Copyright  and   trademark  information"   at: 33 11/21/16World  of  Watson  2016  
  34. 34. Thank  you 11/21/16 Watson Developer Conference 201634