Paper presented at DART2011 workshop in Palermo, Italy.

1. Interac(on
Mining:
the
new
fron(er
of
Call
Center
Analy(cs
Vincenzo
Pallo:a
Rodolfo
Delmonte
Lammert
Vrieling
David
Walker
©
2011
interAnaly(cs
1

2. Outline
• Call
Center
Analy(cs
• Automa(c
Argumenta(ve
Analysis
for
Interac(on
Mining
• Experiments
with
Call
Center
Data
• Conclusions
©
2011
interAnaly(cs
2

3. CALL
CENTER
ANALYTICS
©
2011
interAnaly(cs
3

4. Call
Center
Analy(cs
• Call
centers
data
represent
a
valuable
asset
for
companies,
but
it
is
oOen
underexploited
for
business
purposes
because:
– it
is
highly
dependent
on
quality
of
speech
recogni(on
technology
– it
is
mostly
based
on
text-­‐based
content
analysis.
• Interac(on
Mining
as
a
viable
alterna(ve:
–
more
robust
– tailored
for
the
conversa(onal
domain
– slanted
towards
pragma&c
and
discourse
analysis
©
2011
interAnaly(cs
4

5. Mainstream
Call
Center
Analy(cs
re al
v eil er
ot
un stom
oe s
n ut
cu
D
abo n
hts sfac=o
sig a=
in s
©
2011
interAnaly(cs
5

6. Call
Center
Analy(cs:
metrics
and
KPIs
• Agent
Performance
Sta(s(cs:
– Average
Speed
of
Answer,
Average
Hold
Time,
Call
Abandonment
Rate,
A<ained
Service
Level,
and
Average
Talk
Time.
– Quan(ta(ve
measurements
that
can
be
obtained
directly
through
ACD
(Automa(c
Call
Distribu(on),
Switch
Output
and
Network
Usage
Data.
• Peripheral
Performance
Data:
– Cost
Per
Call,
First-­‐Call
Resolu&on
Rate,
Customer
Sa,sfac,on,
Account
Reten&on,
Staff
Turnover,
Actual
vs.
Budgeted
Costs,
and
Employee
Loyalty.
– Quan(ta(ve,
with
the
excep(on
of
Customer
Sa&sfac&on
that
is
usually
obtained
through
Customer
Surveys.
• Performance
Observa(on:
– Call
Quality,
Accuracy
and
Efficiency,
Adherence
to
Script,
Communica,on
E,que;e,
and
Corporate
Image
Exemplifica,on.
– Qualita=ve
metrics
based
on
analysis
of
recorded
calls
and
session
monitoring
by
a
supervisor.
©
2011
interAnaly(cs
6

7. Four
objec(ves
1. Iden(fy
Customer
Oriented
Behaviors,
– which
are
highly
correlated
to
posi(ve
customer
ra(ngs
(Rafaeli
et
al.
2007);
2. Iden(fy
Root
Cause
of
Problems
– by
looking
at
controversial
topics
and
how
agents
are
able
to
deal
with
them;
3. Iden(fy
customers
who
need
par(cular
a:en(on
– based
on
history
of
problema(c
interac(ons;
4. Learn
best
prac(ces
in
dealing
with
customers
– by
iden(fying
agents
able
to
carry
coopera(ve
conversa(ons.
©
2011
interAnaly(cs
7

8. ARGUMENTATIVE
ANALYSIS
FOR
INTERACTION
MINING
©
2011
interAnaly(cs
8

9. Argumenta(ve
Structure
of
Conversa(ons
DISCUSS(issue) <- PROPOSE(alternative)
1702.95 David: so - so my question is should we go ahead and get na- -
nine identical head mounted crown mikes ? {qy} 61a
REJECT(alternative)
1708.89 John: not before having one come here and have
some people try it out . {s^arp^co} 61b.62a
PROVIDE(justification)
1714.09 B: because there's no point in doing that if it's
John: because there's no point in doing that if
it's going to to be better . {s} {s} 61b+
not not goingbe anyany better . 61b+
ACCEPT(justification)
1712.69 David: okay . {s^bk} 62b
PROPOSE(alternative)
1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs}
63a
PROVIDE(justification)
1722.4 John: and - and see if that works . {s^cs} 63a+.64a
1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b
1725.47 Mark: comfort . {s}
ACCEPT(alternative)
1721.56 David: yeah . {s^bk} 63b
1726.05 Lucy: yeah . {b}
Why
was
David’s
proposal
on
microphones
rejected?
1727.34 John: yeah . {b}
©
2011
interAnaly(cs
9

10. Automa(c
Argumenta(ve
Analysis
• Based
on
the
GETARUNS
system1.
• Clauses
in
Turns
are
labelled
with
Primi(ve
Discourse
Rela(ons:
– statement,
narra,on,
adverse,
result,
cause,
mo,va,on,
explana,on,
ques,on,
hypothesis,
elabora,on,
permission,
incep,on,
circumstance,
obliga,on,
evalua,on,
agreement,
contrast,
evidence,
hypoth,
seCng,
prohibi,on.
• And
then
Turns
are
labelled
with
Argumenta(ve
labels:
– ACCEPT,
REJECT/DISAGREE,
PROPOSE/SUGGEST,
EXPLAIN/JUSTIFY,
REQUEST
EXPLANATION/
JUSTIFICATION.
1
Delmonte
R.,
Bistrot
A.,
Pallo:a
V.,Deep
Linguis(c
Processing
with
GETARUNS
for
spoken
dialogue
Understanding.
Proceedings
LREC
2010
(P31
Dialogue
Corpora).
©
2011
interAnaly(cs
10

11. Evalua(on
ICSI
corpus
of
mee(ngs
(Janin
et
al.,
2003)
Precision:
81.26%
Recall:
97.53%
Total
Correct Incorrect Precision
Found
Accept 662 16 678 98%
Reject 64 18 82 78%
Propose 321 74 395 81%
Request 180 1 181 99%
Explain 580 312 892 65%
Total 1826 421 2247 81.26%
Delmonte
R.,
Bistrot
A.,
Pallo:a
V.,Deep
Linguis(c
Processing
with
GETARUNS
for
spoken
dialogue
Understanding.
Proceedings
LREC
2010
(P31
Dialogue
Corpora).
©
2011
interAnaly(cs
11

12. EXPERIMENTS
WITH
CALL
CENTER
DATA
©
2011
interAnaly(cs
12

13. Ra(onale:
implement
the
four
objec(ves
1. Iden(fy
Customer
Oriented
Behaviors,
2. Iden(fy
Root
Cause
of
Problems
3. Iden(fy
customers
who
need
par(cular
a:en(on
4. Learn
best
prac(ces
in
dealing
with
customers
©
2011
interAnaly(cs
13

14. The
Data
• Corpus
of
213
manually
transcribed
conversa(ons
of
a
help
desk
call
center
in
the
banking
domain.
• Average
of
66
turns
per
conversa(on.
• Average
of
1.6
calls
per
agent.
• Collected
for
a
study
aimed
at
iden(fying
customer
oriented
behaviors
that
could
favor
sa(sfactory
interac(on
with
customers
(Rafaeli
et
al.
2007).
©
2011
interAnaly(cs
14

15. Iden(fy
Customer
Oriented
Behaviors
• Based
on
the
work
of
Rafaeli
et
al.
2006.
• Customer
Oriented
Behaviors
– an(cipa(ng
customers
requests
22,45%
– educa(ng
the
customer
16,91%
– offering
emo(onal
support
21,57%
– offering
explana(ons
/
jus(fica(ons
28,57%
– personaliza(on
of
informa(on
10,50%
©
2011
interAnaly(cs
15

16. Significant
correla(on
with
argumenta(ve
labels
©
2011
interAnaly(cs
16

17. Iden(fy
Root
Cause
of
Problems
• Coopera(veness
score
Argumenta=ve
Categories
Coopera=veness
Accept
explana(on
5
– a
measure
obtained
by
averaging
the
score
Suggest
4
obtained
by
mapping
Propose
3
argumenta(ve
labels
of
Provide
opinion
2
each
turn
in
the
Provide
explana(on/jus(fica(on
1
conversa(on
into
a
[-­‐5
Request
explana(on/jus(fica(on
0
+5]
scale.
Ques(on
-­‐1
Raise
issue
-­‐2
• Sen(ment
Analysis
Provide
nega(ve
opinion
-­‐3
module.
Disagree
-­‐4
Reject
explana(on
or
jus(fica(on
-­‐5
©
2011
interAnaly(cs
17

18. Top
20
Controversial
Topics
with
average
coopera(veness
scores
and
sen(ment
©
2011
interAnaly(cs
18

19. Coopera(veness
of
speakers
on
top
discussed
topics
©
2011
interAnaly(cs
19

20. Iden(fy
problema(c
customers
©
2011
interAnaly(cs
20

21. Select
a
specific
customer
©
2011
interAnaly(cs
21

22. Visualize
a
selected
call
©
2011
interAnaly(cs
22

23. CONCLUSIONS
©
2011
interAnaly(cs
23

24. Conclusions
• New
Genera(on
Call
Center
Analy(cs
requires
Interac(on
Mining
– Call
Center
Qualita(ve
metrics
and
KPIs
can
be
only
implemented
with
a
full
understanding
of
the
customer
interac(on
dynamics
• Argumenta(on
is
pervasive
in
conversa(ons.
– In
order
to
recognize
argumenta(ve
acts,
advanced
Natural
Language
Understanding
is
necessary.
• Future
work:
– Scalability:
need
to
process
millions
of
call
per
day!
– Mul(-­‐language:
call
centers
all
over
the
world.
©
2011
interAnaly(cs
24

25. The
Team
Dr. Lammert Vrieling
(1968) - Chief Executive Officer
‣ 15 years in both profit and not-for-profit organizations as consultant, trainer/coach and as
executive.
‣ Experience in the steel and aluminium industry, multimedia publishing and newspaper,
financial services and in the not-for-profit sector.
David E. Walker
(1964) - Chief Operating Officer
‣ 25+ years in IT as Software engineer, developer, project manager, and architect.
‣ Senior Software Solutions Architect with extensive experience in designing, developing and
delivering enterprise solutions for payment processing, human resource, healthcare,
marketing, manufacturing and scientific research environments.
Prof. Dr. Rodolfo Delmonte
(1946) - Chief Science Officer
‣ Since 1993 head of Computational Linguistics Laboratory of the University of
Venice, Italy
‣ 30+ years experience in computational linguistics
‣ From 1986 to 1992 he worked with the Department of Engineering of the
University of Parma. From 1978 to 1986 worked with the Department of
Dr. Vincenzo Pallotta
(1966) - Chief Technology Officer
‣ 30 years in ICT
‣ 10 years in R&D
‣ Human-Language Technology, Digital Libraries, Artificial Intelligence,
Ubiquitous Computing, Human-Computer Interaction, Usability Engineering,
Information Retrieval, Web Search Engines, Semantic Web, Computational
Logics, Training and Education, e-learning. 23
©
2011
interAnaly(cs
25

26. …find
us
at
www.interanaly=cs.ch