Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Do not blame it on the algorithm - An
empirical assessment
of multiple recommender systems and their
impact
on content div...
Does algorithmic recommendation
lead to less diversity?
2
News recommendation algorithms
Optimize sets of recommended news articles by sorting
them according to
• Popularity
• Coll...
Diversity
Functional approach: Diversity has a different meaning/
significance depending on which value or objective it is...
Democratic
participation
Ratio of
news/public
affairs
Different
political/ideo
logical
viewpoints
Tolerance
Ratio of
conte...
Democratic
participation
Ratio of
news/public
affairs
Different
political/ideo
logical
viewpoints
Tolerance
Ratio of
conte...
Research question
 Which algorithmic setting in news recommender system
produces the largest and smallest amount of diver...
Methods
Data: 1000 Simulated recommendation sets in different
algorithmic settings based of data on Volkskrant.nl publishe...
9
Methods
Operationalization (Preliminary): Diversity for democratic
participation: Overlap in topic; Diversity for autonomy...
How to measure overlap
 Naïve approach:
(1) map: same feature (topic)  1; different feature (topic)  0
(2) sum
In a rec...
Results
Benchmark Popularity Col. filter Semantic
Participation 1.237 1.094 1.286 1.251
Autonomy 0.38 0.29 0.38 0.43
Delib...
How to measure overlap
 But this naïve approach oversimplifies.
 Better: Instead of binary [0/1] have [0;1] interval
 D...
14
Spo
rtDome
stic
politic
s
Internat
ional
Relation
s
15
Spo
rtDome
stic
politic
s
Internat
ional
Relation
s
d
(1) Use MDS to represent each topic t by coordinates (x,y) in
two-dimensional space
(2) Each document D is represented by ...
(1) We multiply M with a and , for document 1 and 2 ,
resulting in two matrices:
These represent the topic dissimilarity m...
Apply to recommendation sets
Given that each document in the dataset generated three
recommendations, we propose to calcul...
Results
20
Future work
 Evaluate topic distance measure: crowdcoding?
 More diverse corpora
 Include minority voice diversity (Wik...
Work in progress… optimize diversity
in recommender systems.
Thanks
Want to know more?
@judith_moeller, @damian0604
Personalised-communication.net
Upcoming SlideShare
Loading in …5
×

Do not blame it on the algorithm - An empirical assessment of multiple recommender systems and their impact on content diversity

773 views

Published on

Presentation at IC2C2, Cologne, 11-7-2017

Published in: Science
  • Be the first to comment

  • Be the first to like this

Do not blame it on the algorithm - An empirical assessment of multiple recommender systems and their impact on content diversity

  1. 1. Do not blame it on the algorithm - An empirical assessment of multiple recommender systems and their impact on content diversity Möller, Trilling, Helberger, van Es The Amsterdam Personalised Communication Project
  2. 2. Does algorithmic recommendation lead to less diversity? 2
  3. 3. News recommendation algorithms Optimize sets of recommended news articles by sorting them according to • Popularity • Collaborative filtering • Semantic match 3
  4. 4. Diversity Functional approach: Diversity has a different meaning/ significance depending on which value or objective it is connected to 4
  5. 5. Democratic participation Ratio of news/public affairs Different political/ideo logical viewpoints Tolerance Ratio of content dedicated to ethnic, linguistic, national groups Different languages Autonomy Choice between different formats, topics, genres, sources Little variance, in the sense of distance from personal preferences Deliberation Reconcillary tone, share of articles presenting various perspectives Diversity of emotions, range of different authors, sources Contestation Minority voices Share of content that is purposefully biased 5
  6. 6. Democratic participation Ratio of news/public affairs Different political/ideo logical viewpoints Tolerance Ratio of content dedicated to ethnic, linguistic, national groups Different languages Autonomy Choice between different formats, topics, genres, sources Little variance, in the sense of distance from personal preferences Deliberation Reconcillary tone, share of articles presenting various perspectives Diversity of emotions, range of different authors, sources Contestation Minority voices Share of content that is purposefully biased 6
  7. 7. Research question  Which algorithmic setting in news recommender system produces the largest and smallest amount of diversity on the different dimensions of diversity? 7
  8. 8. Methods Data: 1000 Simulated recommendation sets in different algorithmic settings based of data on Volkskrant.nl published between 19.9.2016 and 26.9 2016, N=21,973 articles Benchmark: Recommendation by the human editor 8
  9. 9. 9
  10. 10. Methods Operationalization (Preliminary): Diversity for democratic participation: Overlap in topic; Diversity for autonomy: Overlap in section of the newspaper, Diversity for delibartion: Overlap in tone and emotions 10
  11. 11. How to measure overlap  Naïve approach: (1) map: same feature (topic)  1; different feature (topic)  0 (2) sum In a recommendation set of three articles, we can get the scores {0; 1; 2; 3} Example: An article about sport leads to the recommendation set {politics; sport; sport}  score 2 11
  12. 12. Results Benchmark Popularity Col. filter Semantic Participation 1.237 1.094 1.286 1.251 Autonomy 0.38 0.29 0.38 0.43 Deliberation 0.603 0.512 0.585 0.654 12
  13. 13. How to measure overlap  But this naïve approach oversimplifies.  Better: Instead of binary [0/1] have [0;1] interval  Distance between topics as feature; not presence as feature 13
  14. 14. 14 Spo rtDome stic politic s Internat ional Relation s
  15. 15. 15 Spo rtDome stic politic s Internat ional Relation s d
  16. 16. (1) Use MDS to represent each topic t by coordinates (x,y) in two-dimensional space (2) Each document D is represented by vector of topic weights 3) Calculate topic dissimilarity matrix M (50x50) 16 Preparing the matrices
  17. 17. (1) We multiply M with a and , for document 1 and 2 , resulting in two matrices: These represent the topic dissimilarity matrices weighed by the topic occurrence in the document in question. (2) We can then calculate the Euclidian distance between the two matrices using the Frobenius norm: 17 Comparing two documents
  18. 18. Apply to recommendation sets Given that each document in the dataset generated three recommendations, we propose to calculate several measures:  the mean of the distance of each article in the reommendation set with the original article.  the mean of the distances within the recommendation set 18
  19. 19. Results
  20. 20. 20
  21. 21. Future work  Evaluate topic distance measure: crowdcoding?  More diverse corpora  Include minority voice diversity (Wikipedia)  DART 21
  22. 22. Work in progress… optimize diversity in recommender systems.
  23. 23. Thanks Want to know more? @judith_moeller, @damian0604 Personalised-communication.net

×