Interaction Mining: the new frontier of Call Center Analytics


Published on

Paper presented at the DART 2011 workshop in Palermo. The paper introduces a new type of call center analytics based on interaction mining. It shows how advanced metrics and KPIs for call center quality management can be implemented through interAnalytics NLP technology.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Standard metrics with nice visualizations…
  • Interaction Mining: the new frontier of Call Center Analytics

    1. 1. Interaction Mining: the new frontier of Call Center Analytics <br />Vincenzo Pallotta <br />Rodolfo Delmonte<br />LammertVrieling<br />David Walker<br />© 2011 interAnalytics<br />1<br />
    2. 2. Outline<br />Call Center Analytics<br />Automatic Argumentative Analysis for Interaction Mining<br />Experiments with Call Center Data<br />Conclusions<br />© 2011 interAnalytics<br />2<br />
    3. 3. Call Center Analytics<br />© 2011 interAnalytics<br />3<br />
    4. 4. Call Center Analytics<br />Call centers data represent a valuable asset for companies, but it is often underexploited for business purposesbecause:<br />it is highly dependent on quality of speech recognition technology<br />it is mostly based on text-based content analysis.<br />Interaction Mining as a viable alternative:<br />more robust<br />tailored for the conversational domain<br />slanted towards pragmatic and discourse analysis <br />© 2011 interAnalytics<br />4<br />
    5. 5. Mainstream Call Center Analytics<br />© 2011 interAnalytics<br />5<br />Does not unveil real insights about customer satisfaction<br />
    6. 6. Call Center Analytics: metrics and KPIs<br />Agent Performance Statistics: <br />Average Speed of Answer, Average Hold Time, Call Abandonment Rate, Attained Service Level, and Average Talk Time. <br />Quantitative measurements that can be obtained directly through ACD (Automatic Call Distribution), Switch Output and Network Usage Data.<br />Peripheral Performance Data: <br />Cost Per Call, First-Call Resolution Rate, Customer Satisfaction, Account Retention, Staff Turnover, Actual vs. Budgeted Costs, and Employee Loyalty. <br />Quantitative, with the exception of Customer Satisfaction that is usually obtained through Customer Surveys. <br />Performance Observation: <br />Call Quality, Accuracyand Efficiency, Adherence to Script, Communication Etiquette, and Corporate Image Exemplification. <br />Qualitativemetrics based on analysis of recorded calls and session monitoring by a supervisor.<br />© 2011 interAnalytics<br />6<br />
    7. 7. Four objectives<br />Identify Customer Oriented Behaviors, <br />which are highly correlated to positive customer ratings (Rafaeli et al. 2007);<br />Identify Root Cause of Problems <br />by looking at controversial topics and how agents are able to deal with them;<br />Identify customers who need particular attention <br />based on history of problematic interactions;<br />Learn best practices in dealing with customers <br />by identifying agents able to carry cooperative conversations. <br />© 2011 interAnalytics<br />7<br />
    8. 8. Argumentative Analysis for Interaction Mining <br />© 2011 interAnalytics<br />8<br />
    9. 9. Argumentative Structure of Conversations<br />REJECT(alternative)<br />1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a<br />PROVIDE(justification)<br />1714.09 B: because there's no point in doing that if it's not going to be any better . {s} 61b+<br />PROVIDE(justification)<br />1714.09 John: because there's no point in doing that if it's not going to be any better . {s} 61b+<br />ACCEPT(justification)<br />1712.69 David: okay . {s^bk} 62b<br />PROPOSE(alternative)<br />1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a<br />PROVIDE(justification)<br />1722.4 John: and - and see if that works . {s^cs} 63a+.64a <br />1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b<br />1725.47 Mark: comfort . {s}<br />ACCEPT(alternative)<br />1721.56 David: yeah . {s^bk} 63b<br />1726.05 Lucy: yeah . {b} <br />1727.34 John: yeah . {b}<br />DISCUSS(issue) <- PROPOSE(alternative)<br />1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a<br />Why was David’s proposal on microphones rejected?<br />© 2011 interAnalytics<br />9<br />
    10. 10. Automatic Argumentative Analysis<br />Based on the GETARUNS system1.<br />Clauses in Turns are labelled with Primitive Discourse Relations: <br />statement, narration, adverse, result, cause, motivation, explanation, question, hypothesis, elaboration, permission, inception, circumstance, obligation, evaluation, agreement, contrast, evidence, hypoth, setting, prohibition.<br />And then Turns are labelled with Argumentative labels:<br />ACCEPT, REJECT/DISAGREE, PROPOSE/SUGGEST, EXPLAIN/JUSTIFY, REQUEST EXPLANATION/JUSTIFICATION.<br />1 DelmonteR., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogue<br />Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora).<br />© 2011 interAnalytics<br />10<br />
    11. 11. Evaluation<br />ICSI corpus of meetings (Janin et al., 2003)<br />Precision: 81.26% Recall: 97.53%<br />Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogue<br />Understanding. Proceedings LREC 2010 (P31 Dialogue Corpora).<br />© 2011 interAnalytics<br />11<br />
    12. 12. Experiments with Call Center data<br />© 2011 interAnalytics<br />12<br />
    13. 13. Rationale: implement the four objectives<br />Identify Customer Oriented Behaviors, <br />Identify Root Cause of Problems <br />Identify customers who need particular attention <br />Learn best practices in dealing with customers <br />© 2011 interAnalytics<br />13<br />
    14. 14. The Data<br />Corpus of 213 manually transcribed conversations of a help desk call center in the banking domain. <br />Average of 66turns per conversation.<br />Average of 1.6 calls per agent. <br />Collected for a study aimed at identifying customer oriented behaviors that could favor satisfactory interaction with customers (Rafaeli et al. 2007). <br />© 2011 interAnalytics<br />14<br />
    15. 15. Identify Customer Oriented Behaviors<br />Based on the work of Rafaeli et al. 2006.<br />Customer Oriented Behaviors <br />anticipating customers requests 22,45%<br />educating the customer 16,91%<br />offering emotional support 21,57%<br />offering explanations / justifications 28,57%<br />personalization of information 10,50%<br />© 2011 interAnalytics<br />15<br />
    16. 16. Significant correlation with argumentative labels<br />© 2011 interAnalytics<br />16<br />
    17. 17. Identify Root Cause of Problems<br />Cooperativeness score <br />a measure obtained by averaging the score obtained by mapping argumentative labels of each turn in the conversation into a [-5 +5] scale. <br />Sentiment Analysis module.<br />© 2011 interAnalytics<br />17<br />
    18. 18. Top 20 Controversial Topics with average cooperativeness scores and sentiment<br />© 2011 interAnalytics<br />18<br />
    19. 19. Cooperativeness of speakers on top discussed topics<br />© 2011 interAnalytics<br />19<br />
    20. 20. Identify problematic customers<br />© 2011 interAnalytics<br />20<br />
    21. 21. Select a specific customer<br />© 2011 interAnalytics<br />21<br />
    22. 22. Visualize a selected call<br />© 2011 interAnalytics<br />22<br />
    23. 23. Conclusions<br />© 2011 interAnalytics<br />23<br />
    24. 24. Conclusions<br />New Generation Call Center Analytics requires Interaction Mining<br />Call Center Qualitative metrics and KPIs can be only implemented with a full understanding of the customer interaction dynamics<br />Argumentation is pervasive in conversations.<br />In order to recognize argumentative acts, advanced Natural Language Understanding is necessary.<br />Future work:<br />Scalability: need to process millions of call per day!<br />Multi-language: call centers all over the world.<br />© 2011 interAnalytics<br />24<br />
    25. 25. The Team<br />© 2011 interAnalytics<br />25<br />
    26. 26. …find us at<br /><br />