A hybrid recommender system user profiling from keywords and ratings-a hybrid web recommendation systems

1. Hybrid Web Recommender Systems
Robin Burke
Presentation by Jae-wook Ahn
10/04/05

2. 10/5/05 Hybrid Web Recommender Systems2
References
• Entrée system & dataset
• Burke, R. (2002). Semantic ratings and heuristic similarity for
collaborative filtering. AAAI Workshop on Knowledge-based
Electronic Markets 2000.
• Feature augmentation, mixed hybrid example
• Torres, R., McNee, S., Abel, M., Konstan J., & Riedl J. (2004).
Enhancing Digital Libraries with TechLens+. Proceedings of the
2004 Joint ACM/IEEE Conference on Digital Libraries.
• Hybrid recommender system UI issue
• Schafer, J. (2005). DynamicLens: A Dynamic User-Interface for a
Meta-Recommendation System. Workshop: Beyond
Personalization 2005, IUI’05.
• Collaborative filtering algorithm
• Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based
collaborative filtering recommendation algorithms. In Proceedings
of the 10th international conference on World Wide Web.

3. Concepts and Techniques

4. 10/5/05 Hybrid Web Recommender Systems4
Hybrid Recommender Systems
• Mix of recommender systems
• Recommender system classification – knowledge source
• Collaborative (CF)
• User’s ratings “only”
• Content-based (CN)
• Product features, user’s ratings
• Classifications of user’s likes/dislikes
• Demographic
• User’s ratings, user’s demographics
• Knowledge-based (KB)
• Domain knowledge, product features, user’s need/query
• Inferences about a use’s needs and preferences

5. 10/5/05 Hybrid Web Recommender Systems5
CF vs. CN
• User-based CF
• Searches for similar users
in user-item “rating”
matrix
• Item-based CF
• Searches for similar items
in user-item “rating”
matrix
• CN
• Searches for similar items
in item-feature matrix
• Example – TF*IDF term
weight vector for news
recommendation
Items
Users Ratings

6. 10/5/05 Hybrid Web Recommender Systems6
Recommender System Problems
• Cold-start problem
• Learning based techniques
• Collaborative, content-based, demographic
 Hybrid techniques
• Stability vs. plasticity problem
• Difficulty to change established user’s profile
 Temporal discount – older rating with less influence
 KB – fewer cold start problem (no need of historical data)
 CF/Demographic – cross-genre niches, jump outside of the
familiar (novelty, serendipity)

7. 10/5/05 Hybrid Web Recommender Systems7
Strategies for Hybrid Recommendation
• Combination of multiple recommendation techniques
together for producing output
• Different techniques of different types
• Most common implementations
• Most promise to resolve cold-start problem
• Different techniques of the same type
• Ex) NewsDude – naïve Bayes + kNN

8. 10/5/05 Hybrid Web Recommender Systems8
Seven Types of Recommender Systems
• Taxonomy by Burke (2002)
1. Weighted
2. Switching
3. Mixed
4. Feature combination
5. Feature augmentation
6. Cascade
7. Meta-level

9. 10/5/05 Hybrid Web Recommender Systems9
Weighted Hybrid
• Concept
• Each component of the hybrid scores a given item and
the scores are combined using a linear formula
• When recommenders have consistent relative accuracy
across the product space
• Uniform performance among recommenders (otherwise
 other hybrids)

10. 10/5/05 Hybrid Web Recommender Systems10
Weighted Hybrid Procedure
1. Training
2. Joint rating
• Intersection –
candidates shared
between the candidates
• Union – case with no
possible rating 
neutral score (neither
liked nor disliked)
3. Linear combination

11. 10/5/05 Hybrid Web Recommender Systems11
Mixed Hybrid
• Concepts
• Presentation of different components side-by-side in a
combined list
• If lists are to be combined, how are rankings to be
integrated?
• Merging based on predicted rating or on recommender
confidence
• Not fit with retrospective data
• Cannot use actual ratings to test if right items ranked highly
• Example
• CF_rank(3) + CN_rank(2)  Mixed_rank(5)

12. 10/5/05 Hybrid Web Recommender Systems12
Mixed Hybrid Procedure
1. Candidate generation
2. Multiple ranked lists
3. Combined display

13. 10/5/05 Hybrid Web Recommender Systems13
Switching Hybrid
• Concepts
• Selects a single recommender among components based
on recommendation situation
• Different profile  different recommendation
• Components with different performance for some types
of users
• Existence of criterion for switching decision
• Ex) confidence value, external criteria

14. 10/5/05 Hybrid Web Recommender Systems14
Switching Hybrid Procedure
1. Switching decision
2. Candidate generation
3. Scoring
• No role for unchosen
recommender

15. 10/5/05 Hybrid Web Recommender Systems15
Feature Combination Hybrid
• Concepts
• Inject features of one source into a different source for
processing different data
• Features of “contributing recommender” are used as a
part of the “actual recommender”
• Adding new features into the mix
• Not combining components, just combining knowledge
source

16. 10/5/05 Hybrid Web Recommender Systems16
Feature Combination Hybrid Procedure
1. Feature combination
 In training stage
2. Candidate generation
3. Scoring

17. 10/5/05 Hybrid Web Recommender Systems17
Feature Augmentation Hybrid
• Concepts
• Similar to Feature Combination
• Generates new features for each item by contributing
domain
• Augmentation/combination – done offline
• Comparison with Feature Combination
• Not raw features (FC), but the result of computation
from contribution (FA)
• More flexible to apply
• Adds smaller dimension

18. 10/5/05 Hybrid Web Recommender Systems18
Feature Augmentation Hybrid Procedure

19. 10/5/05 Hybrid Web Recommender Systems19
Cascade Hybrid
• Concepts
• Tie breaker
• Secondary recommender
• Just tie breaker
• Do refinements
• Primary recommender
• Integer-valued scores – higher probability for ties
• Real-valued scores – low probability for ties
• Precision reduction
• Score: 0.8348694  0.83

20. 10/5/05 Hybrid Web Recommender Systems20
Cascade Hybrid Procedure
• Procedure
1. Primary recommender
2. Ranks
3. Break ties by secondary
recommender

21. 10/5/05 Hybrid Web Recommender Systems21
Meta-level Hybrid
• Concepts
• A model learned by contributing recommender
 input for actual recommender
• Contributing recommender completely replaces the
original knowledge source with a learned model
• Not all recommenders can produce the intermediary
model

22. 10/5/05 Hybrid Web Recommender Systems22
Meta-level Hybrid Procedure
• Procedure
1. Contributing
recommender
 Learned model
2. Knowledge Source
Replacement
3. Actual Recommender

23. Experiments

24. 10/5/05 Hybrid Web Recommender Systems24
Testbed – Entrée Restaurant Recommender
• Entrée System
• Case-based reasoning
• Interactive critiquing dialog
• Ex) Entry  Candidates  “Cheaper”  Candidates  “Nicer”
 Candidates  Exit
• Not “narrowing” the search by adding constrains, but changing the
focus in the feature space

25. 10/5/05 Hybrid Web Recommender Systems25
Testbed – Entrée Restaurant Recommender (cont’d)
• Entrée Dataset
• Rating
• Entry, ending point – “positive” rating
• Critiques – “negative” rating
• Mostly negative ratings
• Validity test for positive ending point assumption –
strong correlation between original vs. modified
(entry points with positive ratings)
• Small in size

26. 10/5/05 Hybrid Web Recommender Systems26
Evaluation Methodology
• Measures
• ARC (Average Rank of the Correct recommendations)
• Accuracy of retrieval
• At different size retrieval set
• Fraction of the candidate set (0 ~ 1.0)
• Training & Test set
• 5 fold cross validation – random partition of training/test set
• “Leave one out” methodology – randomly remove one item and
check whether the system can recommend it
• Sessions Sizes
• Single visit profiles – 5S, 10S, 15S
• Multiple visit profiles – 10M, 20M, 30M

27. 10/5/05 Hybrid Web Recommender Systems27
Baseline Algorithms
• Collaborative Pearson (CFP)
• Pearson’s correlation coefficient for similarity
• Collaborative Heuristic (CFH)
• Heuristics for calculating distances between critiques
• “nicer” and “cheaper”  dissimilar
• “nicer” & “quieter”  similar
• Content-based (CN)
• Naïve Bayes algorithm – compute probability that a item is “liked” /
“disliked”
• Too few “liked” items  modified candidate generation
• Retrieve items with common features with the “liked” vector of the
naïve Bayes profile
• Knowledge-based (KB)
• Knowledge-based comparison metrics of Entrée
• Nationality, price, atmosphere, etc.

28. 10/5/05 Hybrid Web Recommender Systems28
Baseline Evaluations
• Techniques vary in
performance on the Entrée
data
• Content-based (CN) –
weak
• Knowledge-based (KB) –
better on single-session
than multi-session
• Heuristic collaborative
(CFH) – better than
correlation-based (CFP) for
short profiles
• Room for improvement
• Multi-session profiles

29. 10/5/05 Hybrid Web Recommender Systems29
Baseline Evaluations

30. 10/5/05 Hybrid Web Recommender Systems30
Hybrid Comparative Study
• Missing components
• Mixed hybrid
• Not possible with retrospective data
• Demographic recommender
• No demographic data

31. 10/5/05 Hybrid Web Recommender Systems31
Results – Weighted
• Hybrid performance
better in only 10 of 30
• CN/CFP – consistent
synergy (5 of 6)
• Lacks uniform
performance
• KB, CFH
• Linear weighting
scheme assumption –
fault

32. 10/5/05 Hybrid Web Recommender Systems32
Results – Switching
• KB hybrids – best switching hybrids

33. 10/5/05 Hybrid Web Recommender Systems33
Results – Feature Combination
• CN/CFH, CN/CFP
• Contributing CN
• Identical to CFH, CFP
• CFH maintains accuracy with reduced dataset
• CF/CN Winnow – modest improvement

34. 10/5/05 Hybrid Web Recommender Systems34
Results – Feature Augmentation
• Best performance so far
• Particularly CN*/CF*
• Good for multi-session profiles

35. 10/5/05 Hybrid Web Recommender Systems35
Results – Cascade
• CFP/KB, CFP/CN
• Great improvement
• Also good for multi-profile sessions

36. 10/5/05 Hybrid Web Recommender Systems36
Results – Meta-level Hybrids
• CN/CF, CN/KB, CF/KB, CF/CN
• Not effective
• No synergy
• Weakness of KB/CN in Entrée dataset
• Both components should be strong

37. 10/5/05 Hybrid Web Recommender Systems37
Discussion
• Dominance of the
hybrids over basic
recommenders
• Synergy was found
under
• Smaller profile size
• Sparse
recommendation
density
  hybridization
conquers cold start
problem

38. 10/5/05 Hybrid Web Recommender Systems38
Discussion (cont’d)
• Best hybrids
• Feature augmentation, cascade
• FA allows a contributing recommender to make a
positive impact
• without interfering with the performance of the better
algorithm

39. 10/5/05 Hybrid Web Recommender Systems39
Conclusions
• Knowledge-based recommendation is not limited
• Numerously combined to build hybrids
• Good for secondary or contributing components
• Cascade hybrids are effective
• Though rare in literatures
• Effective for combining recommender with different
strengths
• Different performance characteristics
• Six hybridization techniques
• Relative accuracy & consistency of hybrid components

40. System Example & Related Issues

41. 10/5/05 Hybrid Web Recommender Systems41
System Example – TechLens+
• Hybrid recommender system
• Recommenders – CF, CN
• Hybrid algorithms – CF/CN FA, CN/CF FA, Fusion (Mixed)
• Corpus
• CiteSeer
• Title, abstract (CN), citations (CF)
• Methodology
• Offline experiment, Online user study with questionnaire (by asking
satisfaction on the recommendation)
• Results
• Fusion was the best
• Some FA were not good due the their sequential natures
• Different algorithms should be used for recommending different papers
• Users with different levels of experiences perceive recommendations
differently

42. 10/5/05 Hybrid Web Recommender Systems42
Meta-recommender – DynamicLens
• Can user provided
information improve hybrid
recommender system
output?
• Meta-recommender
• Provide users with
personalized control over
the generation of a
recommendation list from
hybrid recommender
system
• MetaLens
• IF (Information Filtering),
CF

43. 10/5/05 Hybrid Web Recommender Systems43
Meta-recommender – DynamicLens (cont’d)
• Dynamic query
• Merges preference & recommendation interfaces
• Immediate feedback
• Discover why a given set of ranking recommendations were made

44. Questions & Comments