AWS Community Day CPH - Three problems of Terraform
ReComment: Towards Critiquing-based Recommendation with Speech Interaction
1. S C I E N C E
P A S S I O N
T E C H N O L O G Y
Critiquing-based
Recommendation
with Speech Interaction
Peter Grasch (peter.grasch@student.tugraz.at)
Alexander Felfernig (afelfern@ist.tugraz.at)
Florian Reinfrank (freinfra@ist.tugraz.at),
Institute for Software Technology
October 15, 2013
www.tugraz.at
2. www.tugraz.at
Speech Interaction in Recommender Systems
(Written) natural language input has shown promise
¨
˚
in (Shimazu 2001; arnestal, 2004)
Constraint satisfaction using spoken language
presented by Thompson et al in 2004
2
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
3. www.tugraz.at
Critiquing-based Recommender Systems
Pioneering work as early as 1984: M. Williams’
RABBIT [Williams, 1984]
Seminal work by Burke et al: FindMe
[Burke et al., 1997]
Continued, active research, especially in the areas of
advanced critiques
[McCarthy et al., 2004, Zhang and Pu, 2006] and
user modeling
[Reilly et al., 2005a, McCarthy et al., 2010]
3
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
7. www.tugraz.at
ReComment: Rationale
A speech-based natural language interface can allow
more expressive feedback, thus reducing session
length.
7
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
8. www.tugraz.at
ReComment: Recommendation Strategy
Incremental unit critiquing-based system
[Burke, 2000, Reilly et al., 2005a]
Prior probability based on sales rank
No initial search, relaxed similarity constraint
Custom utility function
8
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
9. www.tugraz.at
ReComment: Recommendation Strategy
P ← {p ∈ P |p satisfies last given critique};
maxUtility ← −∞; bestOffer ← rold ;
for p ∈ P do
thisUtility ← ∞ ;
for c ∈ C do
c .age
thisUtility ← thisUtility + (1 − MaxAge ) ∗ c .utility (p) ;
end
if thisUtility > maxUtility then
maxUtility ← thisUtility ; bestOffer ← p ;
end
end
return bestOffer
9
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
10. www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
distance = distance(a.value, p[a.id ].value) ∗ r .direction;
perfectDist = metaModifier ∗ 0.5;
if critiqueViolated then
return −abs(distance − perfectDist );
else
if distance < perfectDist then
return
distance
perfectDist ;
else
return max (perfectDist − distance + 1, 0.0001);
end
end
10
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
Algorithm 1: Schematic utility calculation.
11. www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of the critique x > 50.
11
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
12. www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of subsequent critiques.
12
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
13. www.tugraz.at
ReComment: Utility Function
Control rate of change: Implicit goals
Figure : Utility function of subsequent critiques.
13
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
14. www.tugraz.at
ReComment: Speech Processing
Speech recognition solution based on CMU SPHINX
and Simon [sph, 2013, sim, 2013]
Adapted to recommender situation
Keyword parser
14
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
15. www.tugraz.at
Experiment
Comparison with traditional interface
80 participants
Measuring:
Interaction cycles
Perceived recommendation quality
Usability (adapted SUS)
15
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
16. www.tugraz.at
Experiment: Traditional User Interface
16
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
Figure : ReComment: Mouse-based user interface.
17. www.tugraz.at
Experiment: Speech-based User Interface
17
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
Figure : ReComment: Speech-based user interface.
18. www.tugraz.at
Experiment: Speech-based User Interface
Sentence
I am looking for a camera with 12 megapixel and
a weight of around 200 gram.
This camera with the same properties just
smaller.
An even smaller camera.
Optical zoom of 14 times would be better.
More optical zoom.
[...]
Table : Sample user interaction session.
18
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
19. www.tugraz.at
Results: Feedback Strategies
Category
Discarded
Unit critique
Compound critique (2 attributes)
Compound critique (3 attributes)
Compound critique (5 attributes)
Count
49 (12.8%)
329 (85.7%)
3 (0.8%)
2 (0.5%)
1 (0.3%)
Table : Types of used commands.
74 sentences (20 %) referred to explicit values.
12 sentences (3 %) used modifiers.
19
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
20. www.tugraz.at
"Recomment understands
my voice input"
Results: Speech Processing
25
24
20
15
10
10
5
0
3
0
1
2
3
4
Figure : Participants’ perception of the speech-recognition
accuracy ([1, 4], higher is better).
20
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
21. www.tugraz.at
Results: Usability
Figure : Usability evaluation (adapted SUS scores).
21
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
25. www.tugraz.at
Future Work
Explore more natural user interfaces
Advanced sentiment analysis
Use of prosodic features, timing information, etc. to
infer certainty, frustration, etc.
Compare different recommender systems (e.g.,
constraint based approaches)
25
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
26. www.tugraz.at
Thank you for your attention.
Q&A
26
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
27. www.tugraz.at
Sources I
[sim, 2013] (2013).
About Simon — Simon.
http://simon.kde.org.
[sph, 2013] (2013).
CMU Sphinx - Speech Recognition Toolkit.
http://cmusphinx.sf.net.
27
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
28. www.tugraz.at
Sources II
[Burke, 2000] Burke, R. (2000).
Knowledge-based recommender systems.
In Encyclopedia of Library and Information
Systems. Marcel Dekker.
[Burke et al., 1997] Burke, R. D., Hammond, K. J.,
and Yound, B. (1997).
The findme approach to assisted browsing.
IEEE Expert, 12(4):32–40.
28
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
29. www.tugraz.at
Sources III
[McCarthy et al., 2004] McCarthy, K., Reilly, J.,
McGinty, L., and Smyth, B. (2004).
On the dynamic generation of compound critiques
in conversational recommender systems.
In Adaptive Hypermedia and Adaptive Web-Based
Systems, pages 176–184. Springer.
29
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
30. www.tugraz.at
Sources IV
[McCarthy et al., 2010] McCarthy, K., Salem, Y., and
Smyth, B. (2010).
Experience-based critiquing: reusing critiquing
experiences to improve conversational
recommendation.
In Case-Based Reasoning. Research and
Development, pages 480–494. Springer.
30
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
31. www.tugraz.at
Sources V
[Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty,
L., and Smyth, B. (2005a).
Incremental critiquing.
Knowledge-Based Systems, 18(4):143–151.
31
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
32. www.tugraz.at
Sources VI
[Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L.,
and McCarthy, K. (2005b).
Critiquing with confidence.
In Case-Based Reasoning Research and
Development, pages 436–450. Springer.
32
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
33. www.tugraz.at
Sources VII
[Williams, 1984] Williams, M. D. (1984).
What makes rabbit run?
International Journal of Man-Machine Studies,
21(4):333–352.
33
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
34. www.tugraz.at
Sources VIII
[Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006).
A comparative study of compound critique
generation in conversational recommender
systems.
In Adaptive Hypermedia and Adaptive Web-Based
Systems, pages 234–243. Springer.
34
Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology
October 15, 2013
