Licia Capra (Imperial College London) presentation at Workshop on Self-Awareness in Computing June 2010

1. Self-Adaptation of
Online Recommender Systems
via Feed-Forward Controllers
Licia Capra
University College London
Workshop on Self-Awareness in Computing
June 27th, 2010

2. RECOMMENDER SYSTEMS

3. SCENARIO
Web 2.0 Recommender Systems (CiteULike)

4. RECOMMENDER SYSTEMS
items
ratings
users

5. RECOMMENDER SYSTEMS
items
ratings
users

6. WEB 2.0 RECOMMENDER SYSTEMS

7. RECOMMENDER SYSTEMS (Web 2.0)
items
tags
users

8. RECOMMENDER SYSTEMS
i j k l m
a
b
c
d
User x Item

9. PROBLEM

10. DOMAIN PROBLEM ANALYSIS
 Dataset: Bibsonomy
• 1.3k users; 24k items; 12k tags; 73k bookmarks
5/+72<"
)!!!!"
(!!!!"
*++,-./,0"
123-0"
'!!!!"
4.50"
603/0"
&!!!!"
%!!!!"
123-0"5/+7"8.023/"
$!!!!"
#!!!!"
4.50"5/+7"8.023/"
!"
#" (" ##" #(" $#" $(" %#" %(" &#" &(" '#"
9-3":-+;2<0="

11. DOMAIN PROBLEM ANALYSIS
 Method of Assessment
Time
User
Tag
Item
Period
T1
6%
17%
2%
T2
6%
1%
11%
T3
6%
2%
3%
Ti
1500
tests
Training
Ti(1)
Training
Ti(2)

12. DOMAIN PROBLEM ANALYSIS
 Method of Assessment
Time
User
Tag
Item
Accuracy
Loss
Accuracy
Loss
Accuracy
Loss
Period
(25-­‐75)
(50-­‐50)
(75-­‐25)
T1
6%
17%
2%
24%
32%
45%
T2
6%
1%
11%
12%
20%
23%
T3
6%
2%
3%
3%
10%
14%
Ti
1500
tests
Training
Ti(1)
Training
Ti(2)

13. DOMAIN PROBLEM ANALYSIS
 Method of Assessment
Time
User
Tag
Item
Accuracy
Loss
Accuracy
Loss
Accuracy
Loss
Period
(25-­‐75)
(50-­‐50)
(75-­‐25)
T1
6%
17%
2%
24%
32%
45%
T2
6%
1%
11%
12%
20%
23%
T3
6%
2%
3%
3%
10%
14%
Ti
1500
tests
Training
Ti(1)
Training
Ti(2)

14. IDEA

15. DYNAMIC UPDATE METHODOLOGY
 Recommender Systems as Self-Adaptive
Systems
x
[users,items,tags]
[Recommender
] y
System
[recommendaFon
list]
u
[update
frequency]
Feed-­‐Back

16. DYNAMIC UPDATE METHODOLOGY
 Feed-Forward Controller for Dynamic
Updating of Recommender Systems
x
[users,items,tags]
[Recommender
] y
[recommendaFon
u
System
list]
[update
frequency]
Feed-­‐Forward

17. DOES IT WORK?

18. EVALUATION
 Empirical Estimate of Performance Loss
./01232456/01788#8433#
(&"#
(%"#
($"#
(!"#
+*"-$*"#
'"# *!"-*!"#
$*"-+*"#
&"#
%"#
$"#
!"#
("# $"# )"# %"# *"# &"# +"# '"# ,"# (!"# 9/4:;<#/7;0#

19. EVALUATION
 Cumulative Error (50-50 test set)
'#"
'!"
&"
+,-./01"
%" 2345678")9"
2345678"#9"
:88;01"
$"
#"
!"
'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"

20. EVALUATION
 Cumulative Error & N. of Updates
Technique N. Of Updates
'#"
'!"
Weekly 80
&"
Adaptive 2% 29
+,-./01"
%" 2345678")9"
2345678"#9"
:88;01"
$"
#"
Adaptive 5% 13
!"
'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"
Monthly 18

21. EVALUATION
 Cumulative Error & N. of Updates
Technique N. Of Updates
'#"
'!"
Weekly 80
&"
Adaptive 2% 29
+,-./01"
%" 2345678")9"
2345678"#9"
:88;01"
$"
#"
Adaptive 5% 13
!"
'" %" ''" '%" #'" #%" ('" (%" $'" $%" )'" )%" %'" %%" *'" *%" &'"
Monthly 18

22. CONCLUSIONS
 Accuracy vs Cost Tradeoff may
lead to suboptimal choices
 Recommender Systems as Self-
Adaptive Systems
 Feed-Forward Control Theory for
Unobservable Outputs

23. … & FUTURE WORK
 Automation of Empirical Evaluation
 Beyond Accuracy and Cost (diversity,
surprise, serendipity)

24.  On self-adaptation
• B.H. Cheng, et al. Software Engineering for Self-Adaptive Systems: A Research Roadmap.
In Software Engineering for Self-Adaptive Systems, pages 1-26, 2009. Springer-Verlag
• Y. Brun, et al. Engineering Self-Adaptive Systems through Feedback Loops. In Software
Engineering for Self-Adaptive Systems, pages 48-70, 2009. Springer-Verlag.
 On recommender-systems
• J. Herlocker, et al. An Algorithmic Framework for Performing Collaborative Filtering. In
Proc. of the 22nd Annual International Conference on Research and Development in
Information Retrieval, pages 230-237, New York, NY, USA, 1999. ACM.
• G. Adomavicius and A. Tuzhilin. Context-Aware Recommender Systems. In Proc. of the ACM
Conference on Recommender Systems, 2008.
 From my group
• V. Zanardi and L. Capra. Social Ranking: Uncovering Relevant Content using Tag-based
Recommender Systems. In Proc. of the Conference on Recommender Systems, pages
51-58, 2008. ACM.
• V. Zanardi and L. Capra. "Dynamic Updating of Online Recommender Systems via Feed-
Forward Controllers". In 6th Intl. Symposium on Software Engineering for Adaptive and
Self-Managing Systems (SEAMS 2011). Waikiki, Honolulu, Hawaii, USA. May 2011
THANK YOU!