S C I E N C E

P A S S I O N

T E C H N O L O G Y

Critiquing-based
Recommendation
with Speech Interaction
Peter Grasch (p...
www.tugraz.at

Speech Interaction in Recommender Systems
(Written) natural language input has shown promise
¨
˚
in (Shimaz...
www.tugraz.at

Critiquing-based Recommender Systems
Pioneering work as early as 1984: M. Williams’
RABBIT [Williams, 1984]...
www.tugraz.at

Traditional Critiquing-based Recommender

Figure : QwikShop [Reilly et al., 2005b]
4

Peter Grasch (peter.g...
www.tugraz.at

ReComment: Concept

Preference Elicitation

Preference Model

Recommendation
Strategy

Recommendation

5

P...
www.tugraz.at

ReComment: Concept

Preference Elicitation

Preference Model

Recommendation
Strategy

Recommendation

6

P...
www.tugraz.at

ReComment: Rationale
A speech-based natural language interface can allow
more expressive feedback, thus red...
www.tugraz.at

ReComment: Recommendation Strategy
Incremental unit critiquing-based system
[Burke, 2000, Reilly et al., 20...
www.tugraz.at

ReComment: Recommendation Strategy
P ← {p ∈ P |p satisfies last given critique};
maxUtility ← −∞; bestOffer ...
www.tugraz.at

ReComment: Utility Function
Control rate of change: Implicit goals
distance = distance(a.value, p[a.id ].va...
www.tugraz.at

ReComment: Utility Function
Control rate of change: Implicit goals

Figure : Utility function of the critiq...
www.tugraz.at

ReComment: Utility Function
Control rate of change: Implicit goals

Figure : Utility function of subsequent...
www.tugraz.at

ReComment: Utility Function
Control rate of change: Implicit goals

Figure : Utility function of subsequent...
www.tugraz.at

ReComment: Speech Processing
Speech recognition solution based on CMU SPHINX
and Simon [sph, 2013, sim, 201...
www.tugraz.at

Experiment
Comparison with traditional interface
80 participants
Measuring:
Interaction cycles
Perceived re...
www.tugraz.at

Experiment: Traditional User Interface

16

Peter Grasch (peter.grasch@student.tugraz.at), Institute for So...
www.tugraz.at

Experiment: Speech-based User Interface

17

Peter Grasch (peter.grasch@student.tugraz.at), Institute for S...
www.tugraz.at

Experiment: Speech-based User Interface
Sentence
I am looking for a camera with 12 megapixel and
a weight o...
www.tugraz.at

Results: Feedback Strategies

Category
Discarded
Unit critique
Compound critique (2 attributes)
Compound cr...
www.tugraz.at

"Recomment understands
my voice input"

Results: Speech Processing
25

24
20
15
10

10

5
0

3
0
1

2

3

4...
www.tugraz.at

Results: Usability

Figure : Usability evaluation (adapted SUS scores).
21

Peter Grasch (peter.grasch@stud...
www.tugraz.at

Results: Recommendation Quality
25

20
Score
1

15

2
3

10

4
5

0
Mouse−based interface

Speech−based int...
www.tugraz.at

Session length (cycles)

Results: Recommender Efficiency

50
40

47.7
34.5

30
20
10

9.025

7

Speech inter...
www.tugraz.at

Conclusion

Spoken language recommender systems
are worth exploring!

24

Peter Grasch (peter.grasch@studen...
www.tugraz.at

Future Work
Explore more natural user interfaces
Advanced sentiment analysis
Use of prosodic features, timi...
www.tugraz.at

Thank you for your attention.

Q&A

26

Peter Grasch (peter.grasch@student.tugraz.at), Institute for Softwa...
www.tugraz.at

Sources I
[sim, 2013] (2013).
About Simon — Simon.
http://simon.kde.org.
[sph, 2013] (2013).
CMU Sphinx - S...
www.tugraz.at

Sources II
[Burke, 2000] Burke, R. (2000).
Knowledge-based recommender systems.
In Encyclopedia of Library ...
www.tugraz.at

Sources III
[McCarthy et al., 2004] McCarthy, K., Reilly, J.,
McGinty, L., and Smyth, B. (2004).
On the dyn...
www.tugraz.at

Sources IV
[McCarthy et al., 2010] McCarthy, K., Salem, Y., and
Smyth, B. (2010).
Experience-based critiqui...
www.tugraz.at

Sources V
[Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty,
L., and Smyth, B. (2005a).
Incremental ...
www.tugraz.at

Sources VI
[Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L.,
and McCarthy, K. (2005b).
Critiquing ...
www.tugraz.at

Sources VII
[Williams, 1984] Williams, M. D. (1984).
What makes rabbit run?
International Journal of Man-Ma...
www.tugraz.at

Sources VIII
[Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006).
A comparative study of compound critique
gen...
Upcoming SlideShare
Loading in …5
×

ReComment: Towards Critiquing-based Recommendation with Speech Interaction

224 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
224
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ReComment: Towards Critiquing-based Recommendation with Speech Interaction

  1. 1. S C I E N C E P A S S I O N T E C H N O L O G Y Critiquing-based Recommendation with Speech Interaction Peter Grasch (peter.grasch@student.tugraz.at) Alexander Felfernig (afelfern@ist.tugraz.at) Florian Reinfrank (freinfra@ist.tugraz.at), Institute for Software Technology October 15, 2013 www.tugraz.at
  2. 2. www.tugraz.at Speech Interaction in Recommender Systems (Written) natural language input has shown promise ¨ ˚ in (Shimazu 2001; arnestal, 2004) Constraint satisfaction using spoken language presented by Thompson et al in 2004 2 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  3. 3. www.tugraz.at Critiquing-based Recommender Systems Pioneering work as early as 1984: M. Williams’ RABBIT [Williams, 1984] Seminal work by Burke et al: FindMe [Burke et al., 1997] Continued, active research, especially in the areas of advanced critiques [McCarthy et al., 2004, Zhang and Pu, 2006] and user modeling [Reilly et al., 2005a, McCarthy et al., 2010] 3 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  4. 4. www.tugraz.at Traditional Critiquing-based Recommender Figure : QwikShop [Reilly et al., 2005b] 4 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  5. 5. www.tugraz.at ReComment: Concept Preference Elicitation Preference Model Recommendation Strategy Recommendation 5 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  6. 6. www.tugraz.at ReComment: Concept Preference Elicitation Preference Model Recommendation Strategy Recommendation 6 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  7. 7. www.tugraz.at ReComment: Rationale A speech-based natural language interface can allow more expressive feedback, thus reducing session length. 7 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  8. 8. www.tugraz.at ReComment: Recommendation Strategy Incremental unit critiquing-based system [Burke, 2000, Reilly et al., 2005a] Prior probability based on sales rank No initial search, relaxed similarity constraint Custom utility function 8 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  9. 9. www.tugraz.at ReComment: Recommendation Strategy P ← {p ∈ P |p satisfies last given critique}; maxUtility ← −∞; bestOffer ← rold ; for p ∈ P do thisUtility ← ∞ ; for c ∈ C do c .age thisUtility ← thisUtility + (1 − MaxAge ) ∗ c .utility (p) ; end if thisUtility > maxUtility then maxUtility ← thisUtility ; bestOffer ← p ; end end return bestOffer 9 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  10. 10. www.tugraz.at ReComment: Utility Function Control rate of change: Implicit goals distance = distance(a.value, p[a.id ].value) ∗ r .direction; perfectDist = metaModifier ∗ 0.5; if critiqueViolated then return −abs(distance − perfectDist ); else if distance < perfectDist then return distance perfectDist ; else return max (perfectDist − distance + 1, 0.0001); end end 10 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013 Algorithm 1: Schematic utility calculation.
  11. 11. www.tugraz.at ReComment: Utility Function Control rate of change: Implicit goals Figure : Utility function of the critique x > 50. 11 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  12. 12. www.tugraz.at ReComment: Utility Function Control rate of change: Implicit goals Figure : Utility function of subsequent critiques. 12 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  13. 13. www.tugraz.at ReComment: Utility Function Control rate of change: Implicit goals Figure : Utility function of subsequent critiques. 13 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  14. 14. www.tugraz.at ReComment: Speech Processing Speech recognition solution based on CMU SPHINX and Simon [sph, 2013, sim, 2013] Adapted to recommender situation Keyword parser 14 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  15. 15. www.tugraz.at Experiment Comparison with traditional interface 80 participants Measuring: Interaction cycles Perceived recommendation quality Usability (adapted SUS) 15 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  16. 16. www.tugraz.at Experiment: Traditional User Interface 16 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013 Figure : ReComment: Mouse-based user interface.
  17. 17. www.tugraz.at Experiment: Speech-based User Interface 17 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013 Figure : ReComment: Speech-based user interface.
  18. 18. www.tugraz.at Experiment: Speech-based User Interface Sentence I am looking for a camera with 12 megapixel and a weight of around 200 gram. This camera with the same properties just smaller. An even smaller camera. Optical zoom of 14 times would be better. More optical zoom. [...] Table : Sample user interaction session. 18 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  19. 19. www.tugraz.at Results: Feedback Strategies Category Discarded Unit critique Compound critique (2 attributes) Compound critique (3 attributes) Compound critique (5 attributes) Count 49 (12.8%) 329 (85.7%) 3 (0.8%) 2 (0.5%) 1 (0.3%) Table : Types of used commands. 74 sentences (20 %) referred to explicit values. 12 sentences (3 %) used modifiers. 19 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  20. 20. www.tugraz.at "Recomment understands my voice input" Results: Speech Processing 25 24 20 15 10 10 5 0 3 0 1 2 3 4 Figure : Participants’ perception of the speech-recognition accuracy ([1, 4], higher is better). 20 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  21. 21. www.tugraz.at Results: Usability Figure : Usability evaluation (adapted SUS scores). 21 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  22. 22. www.tugraz.at Results: Recommendation Quality 25 20 Score 1 15 2 3 10 4 5 0 Mouse−based interface Speech−based interface Figure : User score of last recommended item ([1, 4], higher is better). 22 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  23. 23. www.tugraz.at Session length (cycles) Results: Recommender Efficiency 50 40 47.7 34.5 30 20 10 9.025 7 Speech interface (mean) Speech interface (median) 0 Mouse interface (mean) Mouse interface (median) Figure : Session length (lower is better). 23 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  24. 24. www.tugraz.at Conclusion Spoken language recommender systems are worth exploring! 24 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  25. 25. www.tugraz.at Future Work Explore more natural user interfaces Advanced sentiment analysis Use of prosodic features, timing information, etc. to infer certainty, frustration, etc. Compare different recommender systems (e.g., constraint based approaches) 25 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  26. 26. www.tugraz.at Thank you for your attention. Q&A 26 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  27. 27. www.tugraz.at Sources I [sim, 2013] (2013). About Simon — Simon. http://simon.kde.org. [sph, 2013] (2013). CMU Sphinx - Speech Recognition Toolkit. http://cmusphinx.sf.net. 27 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  28. 28. www.tugraz.at Sources II [Burke, 2000] Burke, R. (2000). Knowledge-based recommender systems. In Encyclopedia of Library and Information Systems. Marcel Dekker. [Burke et al., 1997] Burke, R. D., Hammond, K. J., and Yound, B. (1997). The findme approach to assisted browsing. IEEE Expert, 12(4):32–40. 28 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  29. 29. www.tugraz.at Sources III [McCarthy et al., 2004] McCarthy, K., Reilly, J., McGinty, L., and Smyth, B. (2004). On the dynamic generation of compound critiques in conversational recommender systems. In Adaptive Hypermedia and Adaptive Web-Based Systems, pages 176–184. Springer. 29 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  30. 30. www.tugraz.at Sources IV [McCarthy et al., 2010] McCarthy, K., Salem, Y., and Smyth, B. (2010). Experience-based critiquing: reusing critiquing experiences to improve conversational recommendation. In Case-Based Reasoning. Research and Development, pages 480–494. Springer. 30 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  31. 31. www.tugraz.at Sources V [Reilly et al., 2005a] Reilly, J., McCarthy, K., McGinty, L., and Smyth, B. (2005a). Incremental critiquing. Knowledge-Based Systems, 18(4):143–151. 31 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  32. 32. www.tugraz.at Sources VI [Reilly et al., 2005b] Reilly, J., Smyth, B., McGinty, L., and McCarthy, K. (2005b). Critiquing with confidence. In Case-Based Reasoning Research and Development, pages 436–450. Springer. 32 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  33. 33. www.tugraz.at Sources VII [Williams, 1984] Williams, M. D. (1984). What makes rabbit run? International Journal of Man-Machine Studies, 21(4):333–352. 33 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013
  34. 34. www.tugraz.at Sources VIII [Zhang and Pu, 2006] Zhang, J. and Pu, P. (2006). A comparative study of compound critique generation in conversational recommender systems. In Adaptive Hypermedia and Adaptive Web-Based Systems, pages 234–243. Springer. 34 Peter Grasch (peter.grasch@student.tugraz.at), Institute for Software Technology October 15, 2013

×