Interaction Mining: the new frontier of Call Center Analytics

1,032 views

Published on

Paper presented at DART2011 workshop in Palermo, Italy.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,032
On SlideShare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
25
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Interaction Mining: the new frontier of Call Center Analytics

  1. 1. Interac(on  Mining:  the  new  fron(er  of  Call  Center  Analy(cs     Vincenzo  Pallo:a     Rodolfo  Delmonte     Lammert  Vrieling     David  Walker     ©  2011  interAnaly(cs   1  
  2. 2. Outline  •  Call  Center  Analy(cs  •  Automa(c  Argumenta(ve  Analysis  for   Interac(on  Mining  •  Experiments  with  Call  Center  Data  •  Conclusions   ©  2011  interAnaly(cs   2  
  3. 3. CALL  CENTER  ANALYTICS   ©  2011  interAnaly(cs   3  
  4. 4. Call  Center  Analy(cs  •  Call  centers  data  represent  a  valuable  asset  for   companies,  but  it  is  oOen  underexploited  for   business  purposes  because:   –  it  is  highly  dependent  on  quality  of  speech  recogni(on   technology   –  it  is  mostly  based  on  text-­‐based  content  analysis.  •  Interac(on  Mining  as  a  viable  alterna(ve:   –   more  robust   –  tailored  for  the  conversa(onal  domain   –  slanted  towards  pragma&c  and  discourse  analysis     ©  2011  interAnaly(cs   4  
  5. 5. Mainstream  Call  Center  Analy(cs    re al   v eil er   ot  un stom oe s  n ut  cu D  abo n   hts sfac=o sig a= in s ©  2011  interAnaly(cs   5  
  6. 6. Call  Center  Analy(cs:  metrics  and  KPIs  •  Agent  Performance  Sta(s(cs:     –  Average  Speed  of  Answer,  Average  Hold  Time,  Call  Abandonment  Rate,   A<ained  Service  Level,  and  Average  Talk  Time.     –  Quan(ta(ve  measurements  that  can  be  obtained  directly  through  ACD   (Automa(c  Call  Distribu(on),  Switch  Output  and  Network  Usage  Data.  •  Peripheral  Performance  Data:       –  Cost  Per  Call,  First-­‐Call  Resolu&on  Rate,  Customer  Sa,sfac,on,  Account   Reten&on,  Staff  Turnover,  Actual  vs.  Budgeted  Costs,  and  Employee  Loyalty.     –  Quan(ta(ve,  with  the  excep(on  of  Customer  Sa&sfac&on  that  is  usually   obtained  through  Customer  Surveys.    •  Performance  Observa(on:     –  Call  Quality,  Accuracy  and  Efficiency,  Adherence  to  Script,   Communica,on  E,que;e,  and  Corporate  Image  Exemplifica,on.     –  Qualita=ve  metrics  based  on  analysis  of  recorded  calls  and  session  monitoring   by  a  supervisor.   ©  2011  interAnaly(cs   6  
  7. 7. Four  objec(ves  1.  Iden(fy  Customer  Oriented  Behaviors,     –  which  are  highly  correlated  to  posi(ve  customer  ra(ngs   (Rafaeli  et  al.  2007);  2.  Iden(fy  Root  Cause  of  Problems     –  by  looking  at  controversial  topics  and  how  agents  are  able   to  deal  with  them;  3.  Iden(fy  customers  who  need  par(cular  a:en(on     –  based  on  history  of  problema(c  interac(ons;  4.  Learn  best  prac(ces  in  dealing  with  customers     –  by  iden(fying  agents  able  to  carry  coopera(ve   conversa(ons.     ©  2011  interAnaly(cs   7  
  8. 8. ARGUMENTATIVE  ANALYSIS  FOR  INTERACTION  MINING     ©  2011  interAnaly(cs   8  
  9. 9. Argumenta(ve  Structure  of   Conversa(ons  DISCUSS(issue) <- PROPOSE(alternative)1702.95 David: so - so my question is should we go ahead and get na- -nine identical head mounted crown mikes ? {qy} 61a REJECT(alternative) 1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a PROVIDE(justification) 1714.09 B: because theres no point in doing that if its John: because theres no point in doing that if its going to to be better . {s} {s} 61b+ not not goingbe anyany better . 61b+ ACCEPT(justification) 1712.69 David: okay . {s^bk} 62b PROPOSE(alternative) 1716.85 John: so why dont we get one of these with the crown with a different headset ? {qw^cs} 63a PROVIDE(justification) 1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if its preferable and if it is then well get more . {s^cs^2} 64b 1725.47 Mark: comfort . {s} ACCEPT(alternative) 1721.56 David: yeah . {s^bk} 63b 1726.05 Lucy: yeah . {b} Why  was  David’s  proposal  on  microphones  rejected?   1727.34 John: yeah . {b} ©  2011  interAnaly(cs   9  
  10. 10. Automa(c  Argumenta(ve  Analysis  •  Based  on  the  GETARUNS  system1.  •  Clauses  in  Turns  are  labelled  with  Primi(ve  Discourse   Rela(ons:     –  statement,  narra,on,  adverse,  result,  cause,  mo,va,on,   explana,on,  ques,on,  hypothesis,  elabora,on,   permission,  incep,on,  circumstance,  obliga,on,   evalua,on,  agreement,  contrast,  evidence,  hypoth,   seCng,  prohibi,on.  •  And  then  Turns  are  labelled  with  Argumenta(ve  labels:   –  ACCEPT,  REJECT/DISAGREE,  PROPOSE/SUGGEST,   EXPLAIN/JUSTIFY,    REQUEST  EXPLANATION/ JUSTIFICATION.  1  Delmonte  R.,    Bistrot  A.,  Pallo:a  V.,Deep  Linguis(c  Processing  with  GETARUNS  for  spoken  dialogue  Understanding.  Proceedings  LREC  2010  (P31  Dialogue  Corpora).   ©  2011  interAnaly(cs   10  
  11. 11. Evalua(on   ICSI  corpus  of  mee(ngs  (Janin  et  al.,  2003)   Precision:  81.26%  Recall:  97.53%   Total   Correct Incorrect Precision Found Accept 662 16 678 98% Reject 64 18 82 78% Propose 321 74 395 81% Request 180 1 181 99% Explain 580 312 892 65% Total 1826 421 2247 81.26%Delmonte  R.,    Bistrot  A.,  Pallo:a  V.,Deep  Linguis(c  Processing  with  GETARUNS  for  spoken  dialogue  Understanding.  Proceedings  LREC  2010  (P31  Dialogue  Corpora).   ©  2011  interAnaly(cs   11  
  12. 12. EXPERIMENTS  WITH  CALL  CENTER  DATA   ©  2011  interAnaly(cs   12  
  13. 13. Ra(onale:  implement  the  four   objec(ves  1.  Iden(fy  Customer  Oriented  Behaviors,    2.  Iden(fy  Root  Cause  of  Problems    3.  Iden(fy  customers  who  need  par(cular   a:en(on    4.  Learn  best  prac(ces  in  dealing  with   customers     ©  2011  interAnaly(cs   13  
  14. 14. The  Data  •  Corpus  of  213  manually  transcribed   conversa(ons  of  a  help  desk  call  center  in  the   banking  domain.    •  Average  of  66  turns  per  conversa(on.  •  Average  of  1.6  calls  per  agent.    •  Collected  for  a  study  aimed  at  iden(fying   customer  oriented  behaviors  that  could  favor   sa(sfactory  interac(on  with  customers   (Rafaeli  et  al.  2007).     ©  2011  interAnaly(cs   14  
  15. 15. Iden(fy  Customer  Oriented  Behaviors  •  Based  on  the  work  of  Rafaeli  et  al.  2006.  •  Customer  Oriented  Behaviors     –  an(cipa(ng  customers  requests  22,45%   –  educa(ng  the  customer  16,91%   –  offering  emo(onal  support  21,57%   –  offering  explana(ons  /  jus(fica(ons  28,57%   –  personaliza(on  of  informa(on  10,50%   ©  2011  interAnaly(cs   15  
  16. 16. Significant  correla(on  with   argumenta(ve  labels   ©  2011  interAnaly(cs   16  
  17. 17. Iden(fy  Root  Cause  of  Problems  •  Coopera(veness  score     Argumenta=ve  Categories Coopera=veness Accept  explana(on 5 –  a  measure  obtained  by   averaging  the  score   Suggest 4 obtained  by  mapping   Propose 3 argumenta(ve  labels  of   Provide  opinion 2 each  turn  in  the   Provide  explana(on/jus(fica(on 1 conversa(on  into  a  [-­‐5   Request  explana(on/jus(fica(on 0 +5]  scale.     Ques(on -­‐1 Raise  issue -­‐2 •  Sen(ment  Analysis   Provide  nega(ve  opinion -­‐3 module.   Disagree -­‐4 Reject  explana(on  or  jus(fica(on -­‐5 ©  2011  interAnaly(cs   17  
  18. 18. Top  20  Controversial  Topics  with  average   coopera(veness  scores  and  sen(ment   ©  2011  interAnaly(cs   18  
  19. 19. Coopera(veness  of  speakers  on  top   discussed  topics   ©  2011  interAnaly(cs   19  
  20. 20. Iden(fy  problema(c  customers   ©  2011  interAnaly(cs   20  
  21. 21. Select  a  specific  customer   ©  2011  interAnaly(cs   21  
  22. 22. Visualize  a  selected  call   ©  2011  interAnaly(cs   22  
  23. 23. CONCLUSIONS   ©  2011  interAnaly(cs   23  
  24. 24. Conclusions  •  New  Genera(on  Call  Center  Analy(cs  requires   Interac(on  Mining   –  Call  Center  Qualita(ve  metrics  and  KPIs  can  be  only   implemented  with  a  full  understanding  of  the   customer  interac(on  dynamics  •  Argumenta(on  is  pervasive  in  conversa(ons.   –  In  order  to  recognize  argumenta(ve  acts,  advanced   Natural  Language  Understanding  is  necessary.  •  Future  work:   –  Scalability:  need  to  process  millions  of  call  per  day!   –  Mul(-­‐language:  call  centers  all  over  the  world.   ©  2011  interAnaly(cs   24  
  25. 25. The  Team  Dr. Lammert Vrieling(1968) - Chief Executive Officer ‣ 15 years in both profit and not-for-profit organizations as consultant, trainer/coach and as executive. ‣ Experience in the steel and aluminium industry, multimedia publishing and newspaper, financial services and in the not-for-profit sector.David E. Walker(1964) - Chief Operating Officer ‣ 25+ years in IT as Software engineer, developer, project manager, and architect. ‣ Senior Software Solutions Architect with extensive experience in designing, developing and delivering enterprise solutions for payment processing, human resource, healthcare, marketing, manufacturing and scientific research environments.Prof. Dr. Rodolfo Delmonte(1946) - Chief Science Officer ‣ Since 1993 head of Computational Linguistics Laboratory of the University of Venice, Italy ‣ 30+ years experience in computational linguistics ‣ From 1986 to 1992 he worked with the Department of Engineering of the University of Parma. From 1978 to 1986 worked with the Department ofDr. Vincenzo Pallotta(1966) - Chief Technology Officer ‣ 30 years in ICT ‣ 10 years in R&D ‣ Human-Language Technology, Digital Libraries, Artificial Intelligence, Ubiquitous Computing, Human-Computer Interaction, Usability Engineering, Information Retrieval, Web Search Engines, Semantic Web, Computational Logics, Training and Education, e-learning. 23 ©  2011  interAnaly(cs   25  
  26. 26. …find  us  at   www.interanaly=cs.ch  

×