Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms

813 views

Published on

Onur Kucuktunc (presenter), Erik Saule, Kamer Kaya, Umit V. Catalyurek
WWW'13 presentation, Research Track - Recommender Systems and OSN

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
813
On SlideShare
0
From Embeds
0
Number of Embeds
434
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms

  1. 1. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 1/25Diversified Recommendation on Graphs:Pitfalls, Measures, and AlgorithmsOnur Küçüktunç1,2Erik Saule1Kamer Kaya1Ümit V. Çatalyürek1,3WWW 2013, May 13–17, 2013, Rio de Janeiro, Brazil.1Dept. Biomedical Informatics2Dept. of Computer Science and Engineering3Dept. of Electrical and Computer EngineeringThe Ohio State University
  2. 2. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 2/25Outline•  Problem definition–  Motivation–  Result diversification algorithms•  How to measure diversity–  Classical relevance and diversity measures–  Bicriteria optimization?!–  Combined measures•  Best Coverage method–  Complexity, submodularity–  A greedy solution, relaxation•  Experiments
  3. 3. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 3/25Problem definitionG = (V, E)Q ✓ VOnline shoppingproductco-purchasing•  one product•  previous purchases•  page visit historyAcademicpaper-to-papercitations•  paper/field of interest•  set of references•  researcher himself/herselfproduct recommendations“you might also like…”R ⇢ V references for related worknew collaboratorscollaborationnetworkSocialfriendshipnetwork•  user himself/herself•  set of peoplefriend recommendations“you might also know…”Let G = (V, E) be an undirected graph.Given a set of m seed nodesQ = {q1, . . . , qm} s.t. Q ✓ V ,and a parameter k, return top-k itemswhich are relevant to the ones in Q,but diverse among themselves, coveringdi↵erent aspects of the query.
  4. 4. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 4/25Problem definitionLet G = (V, E) be an undirected graph.Given a set of m seed nodesQ = {q1, . . . , qm} s.t. Q ✓ V ,and a parameter k, return top-k itemswhich are relevant to the ones in Q,but diverse among themselves, coveringdi↵erent aspects of the query.•  We assume that the graph itself is the only information we have, andno categories or intents are available•  no comparisons to intent-aware algorithms [Agrawal09,Welch11,etc.]•  but we will compare against intent-aware measures•  Relevance scores are obtained with Personalized PageRank (PPR)[Haveliwala02]p⇤(v) =(1/m, if v 2 Q0, otherwise.
  5. 5. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 5/25Result diversification algorithms•  GrassHopper [Zhu07]–  ranks the graph k times•  turns the highest-ranked vertex into a sink node at each iteration0 5 10024680 5 10051000.0050.010.015g10 5 1005100246g20051000.511.5(a) (b) (c) (d)Figure 1: (a) A toy data set. (b) The stationary distribution π reflects centrality. The item withprobability is selected as the first item g1. (c) The expected number of visits v to each node after gan absorbing state. (d) After both g1 and g2 become absorbing states. Note the diversity in g1, g2,0 5 10024680 5 10051000.0050.010.015g10 5 1005100246g20051000.511.5(a) (b) (c) (dFigure 1: (a) A toy data set. (b) The stationary distribution π reflects centrality. The item withprobability is selected as the first item g1. (c) The expected number of visits v to each node afteran absorbing state. (d) After both g1 and g2 become absorbing states. Note the diversity in g1, g2come from different groups.Items at group centers have higher probabilities, and 2.3 Ranking the Remaining Items5 10 0 5 10051000.0050.010.015g10 5 1005100246g20 5 10051000.511.5g3(a) (b) (c) (d)highest-rankedvertexR = {g1}R = {g1,g2}R = {g1,g2,g3}g1 turned intoa sink nodehighest-rankedin the next step
  6. 6. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 6/25Result diversification algorithms•  GrassHopper [Zhu07]–  ranks the graph k times•  turns the highest-ranked vertex into a sink node at each iteration•  DivRank [Mei10]–  based on vertex-reinforced random walks (VRRW)•  adjusts the transition matrix based on the number of visits to thevertices (rich-gets-richer mechanism)(a) An illustrative network. (b) Weighting with PageRank. (c) A diverse weighting.(a) An illustrative network. (b) Weighting with PageRank. (c) A diverse weighting.(a) An illustrative network. (b) Weighting with PageRank. (c) A diverse weighting.sample graph weighting with PPR diverse weighting
  7. 7. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 7/25Result diversification algorithms•  GrassHopper [Zhu07]–  ranks the graph k times•  turns the highest-ranked vertex into a sink node at each iteration•  DivRank [Mei10]–  based on vertex-reinforced random walks (VRRW)•  adjusts the transition matrix based on the number of visits to thevertices (rich-gets-richer mechanism)•  Dragon [Tong11]–  based on optimizing the goodness measure•  punishes the score when two neighbors are included in theresults
  8. 8. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 8/25Measuring diversityRelevance measures•  Normalized relevance•  Difference ratio•  nDCGDiversity measures•  l-step graph density•  l-expansion ratiorel(S) =Pv2S ⇡vPki=1 ˆ⇡idi↵(S, ˆS) = 1|S ˆS||S|nDCGk =⇡s1 +Pki=2⇡silog2 iˆ⇡1 +Pki=2ˆ⇡ilog2 idens`(S) =Pu,v2S,u6=v d`(u, v)|S| ⇥ (|S| 1)`(S) =|N`(S)|nN`(S) = S [ {v 2 (V S) : 9u 2 S, d(u, v)  `}where
  9. 9. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 9/25Bicriteria optimization measures•  aggregate a relevance and a diversity measure•  [Carbonell98]•  [Li11]•  [Vieira11]•  max-sum diversification, max-min diversification,k-similar diversification set, etc. [Gollapudi09]fMMR(S) = (1 )Xv2S⇡vXu2Smaxv2Su6=vsim(u, v)fL(S) =Xv2S⇡v +|N(S)|nfMSD(S) = (k 1)(1 )Xv2S⇡v + 2Xu2SXv2Su6=vdiv(u, v)
  10. 10. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 10/25Bicriteria optimization is not the answer•  Objective: diversify top-10 results•  Two query-oblivious algorithms:–  top-% + random–  top-% + greedy-σ2
  11. 11. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 11/25Bicriteria optimization is not the answer•  normalized relevance and 2-step graph density•  evaluating result diversification as a bicriteria optimization problem with–  a relevance measure that ignores diversity, and–  a diversity measure that ignores relevancy.00.20.40.60.810 0.2 0.4 0.6 0.8 1dens2rel10-RLMBC1BC1VBC2BC2VBC1100BC11000BC150BC2100BC21000BC250CDivRankDRAGONGRASSHOPPERGSPARSEIL1IL2LMPDivRankPRk-RLMtop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll Random00.20.40.60.810 0.2 0.4 0.6 0.8 1dens2rel10-RLMBC1BC1VBC2BC2VBC1100BC11000BC150BC2100BC21000BC250CDivRankDRAGONGRASSHOPPERGSPARSEIL1IL2LMPDivRankPRk-RLMtop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll Randombetter00.20.40.60.810 0.2 0.4 0.6 0.8 1dens2rel10-RLMBC1BC1VBC2BC2VBC1100BC11000BC150BC2100BC21000BC250CDivRankDRAGONGRASSHOPPERGSPARSEIL1IL2LMPDivRankPRk-RLMtop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll Random00.20.40.60.810 0.2 0.4 0.6 0.8 1dens2rel10-RLMBC1BC1VBC2BC2VBC1100BC11000BC150BC2100BC21000BC250CDivRankDRAGONGRASSHOPPERGSPARSEIL1IL2LMPDivRankPRk-RLMtop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll Randombetter00.050.10.150.20.250.30 0.2 0.4 0.6 0.8 1σ2rel00.050.10.150.20.250.30 0.2 0.4 0.6 0.8 1σ2relbetter2 0.4 0.6 0.8 1othersothersothersothersotherstop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll randomothersothersotherstop-90%+greedy-σ2top-75%+greedy-σ2top-50%+greedy-σ2top-25%+greedy-σ2All greedy-σ2othersothersothersother algorithms00.20.40.60.810 0.2 0.4 0.6 0.8othersothersothersothersotherstop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll randomothersothersotherstop-90%+greedy-σ2top-75%+greedy-σ2top-50%+greedy-σ2top-25%+greedy-σ2All greedy-σ2othersothersothersother algorithms
  12. 12. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 12/25A better measure? Combine both•  We need a combined measure that tightlyintegrates both relevance and diversity aspectsof the result set•  goodness [Tong11]–  downside: highly dominated by relevancefG(S) = 2Xi2S⇡i dXi,j2SA(j, i)⇡j(1 d)Xj2S⇡jXi2Sp⇤(i)max-sum relevancepenalize the score when two results sharean edge
  13. 13. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 13/25Proposed measure: l-step expanded relevance•  a combined measure of–  l-step expansion ratio (σ2)–  relevance scores (π)•  quantifies: relevance ofthe covered regionof the graph•  do some sanity checkwith this new measure`-step expanded relevance:exprel`(S) =Xv2N`(S)⇡vwhere N`(S) is the `-step expansionset of the result set S, and ⇡ is thePPR scores of the items in the graph.00.20.40.60.81510 20 50 100exprel2k00.20.40.60.81510 20 50 100exprel2k0.20.40.60.81othersothersothersothersotherstop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll randomothersothersotherstop-90%+greedy-σ2top-75%+greedy-σ2top-50%+greedy-σ2top-25%+greedy-σ2All greedy-σ2othersothersothersother algorithms0.20.40.60.81othersothersothersothersotherstop-90%+randomtop-75%+randomtop-50%+randomtop-25%+randomAll randomothersothersotherstop-90%+greedy-σ2top-75%+greedy-σ2top-50%+greedy-σ2top-25%+greedy-σ2All greedy-σ2othersothersothersother algorithms
  14. 14. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 14/25Correlations of the measuresrelevancediversitygoodness is dominated bythe relevancy measuresexprel has no high correlations withother relevance or diversity measures
  15. 15. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 15/25Proposed algorithm: Best Coverage•  Can we use -step expandedrelevance as an objective function?•  Define:•  Complexity: generalization of weighted maximum coverage problem–  NP-hard!–  but exprell is a submodular function (Lemma 4.2)–  a greedy solution (Algorithm 1) that selects the itemwith the highest marginal utilityat each step is the best possible polynomial timeapproximation (proof based on [Nemhauser78])•  Relaxation: computes BestCoverage onhighest ranked vertices to improve runtimeexprel`-diversified top-k ranking (DTR`)S = argmaxS0✓V|S0|=kexprel`(S0)g(v, S) =Pv02N`({v}) N`(S) ⇡v0ALGORITHM 1: BestCoverageInput: k, G, ⇡, `Output: a list of recommendations SS = ;while |S| < k dov⇤argmaxv g(v, S)S S [ {v⇤}return SALGORITHM 1: BestCoverage (relaxed)ALGORITHM 2: BestCoverage (relaxed)Input: k, G, ⇡, `Output: a list of recommendations SS = ;Sort(V ) w.r.t ⇡i non-increasingS1 V [1..k0], i.e., top-k0vertices where k0= k¯`8v 2 S1, g(v) g(v, ;)8v 2 S1, c(v) Uncoveredwhile |S| < k dov⇤argmaxv2S1 g(v)S S [ {v⇤}S2 N`({v⇤})for each v02 S2 doif c(v0) = Uncovered thenS3 N`({v0})8u 2 S3, g(u) g(u) ⇡v0c(v0) Coveredreturn S`
  16. 16. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 16/25Experiments•  5 target application areas, 5 graphs from SNAP•  Queries generated based on 3 scenario types–  one random vertex–  random vertices from one area of interest–  multiple vertices from multiple areas of interestDataset |V | |E| ¯ D D90% CCamazon0601 403.3K 3.3M 16.8 21 7.6 0.42ca-AstroPh 18.7K 396.1K 42.2 14 5.1 0.63cit-Patents 3.7M 16.5M 8.7 22 9.4 0.09soc-LiveJournal1 4.8M 68.9M 28.4 18 6.5 0.31web-Google 875.7K 5.1M 11.6 22 8.1 0.60
  17. 17. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 17/25Results – relevance•  Methods should trade-off relevance for better diversity•  Normalized relevance of top-k set is always 1•  DRAGON always return results having 70% similar itemsto top-k, with more than 80% rel scoreamazon0601, combined00.20.40.60.815 10 20 50 100relkPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)00.20.40.60.815 10 20 50 100relkPPR (top-kGrassHopDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxsoc-LiveJournal1, combined101001000100005 10 20 50 100time(sec)PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)101001000100005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)amazon0601, combined0.60.81diffPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)0.60.81diffPPR (top-GrassHopDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxsoc-LiveJournal1, combined00.050.10.150.20.250.30.350.40.455 10 2σ200.050.10.150.20.250.30.350.40.455 10 20 50 100σ2kPGDPCkGBBBB00.050.10.150.20.250.30.350.40.455 10 20 50σ2k00.10.20.30.40.50.60.70.80.95 10 2σ2Figureing k.highestand k-R0.8top-kDRAGON
  18. 18. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 18/25Results – coverage•  l-step expansion ratio (σ2) gives the graph coverage ofthe result set: better coverage = better diversity•  BestCoverage and DivRank variants, especiallyBC2 and PDivRank, have the highest coverage100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)nedPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)inedamazon0601, combined00.050.10.150.20.250.30.350.40.455 10 20 50 100σ2kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)00.10.20.30.40.50.60.70.80.95 10 20 50 100σ2kPPR (top-kGrassHopDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxBC2 (relaxca-AstroPh, combined00.10.20.30.40.50.60.70.80.9σ2PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)00.10.20.30.40.50.60.70.80.95 10 20 50 100σ2kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)Figure 3: Coverage ( 2) of the algorithms with vary-ing k. BestCoverage and DivRank variants have thehighest coverage on the graphs while Dragon, GSparse,and k-RLM have similar coverages to top-k results.amazon0601, combined0.8PPR (top-k)0.8PPR (top-k)soc-LiveJournal1, combined
  19. 19. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 19/25Results – expanded relevance•  combined measure for relevance and diversity•  BestCoverage variants and GrassHopper perform better•  Although PDivRank gives the highest coverage onamazon graph, it fails to cover the relevant parts!100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)entrseele-the2.3,nedores00.10.20.30.45 10 20 50 100σkBC2 (relaxed)00.10.25 10 20 50 100king k. BestCoverage and DivRank variants have thehighest coverage on the graphs while Dragon, GSparse,and k-RLM have similar coverages to top-k results.amazon0601, c o m b i n e d0.350.40.450.50.550.60.650.70.750.85 10 20 50 100exprel2kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)0.30.350.40.450.50.550.60.650.70.750.85 10 20 50 100exprel2kPPR (top-kGrassHoppDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxesoc-LiveJournal1, c o m b i n e d0.30.350.40.450.50.550.60.650.70.750.85 10 20 50 100exprel2kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)Figure 4: Expanded relevance (exprel2 ) with vary-ing k. BC1 and BC2 variants mostly score the best,GrassHopper performs high in soc-LiveJournal1. Al-though PDivRank gives the highest coverage on ama-zon0601 (Fig. 3), it fails to cover the relevant parts.BC2BC1PDivRankGrassHopper
  20. 20. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 20/25Results – efficiency•  BC1 always performsbetter, with a runningtime less than, DivRankand GrassHopper•  BC1 (relaxed) offersreasonable diversity,with a very littleoverhead on top of thePPR computation100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)3))100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)30.010.11105 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)20 50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)ca-AstroPh, combined101001000100005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)0.11101005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)101001000100005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC1 (relaxed)soc-LiveJournal1, combined101001000100005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)100PPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)50 100kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)cit-Patents, scenario 111010010005 10 20 50 100time(sec)kPPR (top-k)GrassHopperDragonPDivRankCDivRankk-RLMGSparseBC1BC2BC1 (relaxed)BC2 (relaxed)web-Google, scenario 1Figure 7: Running times of the algorithms withvarying k. BC1 method always perform better with arunning time less than GrassHopper and DivRank vari-ants, while the relaxed versions score similarly witha slight overhead on top of the PPR computation.BC1BC1BC1BC1DivRankDivRankDivRank DivRankBC1 (relaxed)BC1 (relaxed)BC1 (relaxed)BC1 (relaxed)
  21. 21. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 21/25Results – intent aware experiments•  evaluation of intent-oblivious algorithms againstintent-aware measures•  two measures–  group coverage [Li11]–  S-recall [Zhai03]•  cit-Patent dataset has the categoricalinformation–  426 class labels, belong to 36 subtopics
  22. 22. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 22/25Results – intent aware experiments•  group coverage [Li11]–  How many different groups are covered by the results?–  omits the actual intent of the query•  top-k results are not diverse enough•  AllRandom results cover the most number of groups•  PDivRank and BC2 follows0102030405060705 10 20 50 100ClasscoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(a) Class coverage0510152025305 10 20 50 100SubtopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(b) Subtopic coverage01234565 10 20Topiccoverage(c) Topic0510152025305 10 20 50Subtopiccoveragek50 100kPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom) Topic coverage100PPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom100PPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandomge0510152025305 10 20 50 100SubtopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(b) Subtopic coverage01234565 10 20 50 100TopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(c) Topic coverage0510152025305 10 20 50 100SubtopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandomses (e) S-recall on subtopics (f) S-recall on topicsIntent-aware results on cit-Patents dataset with scenario-3 queries.top-ktop-krandomBC2BC2
  23. 23. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 23/2501020304050605 10 20 50 100ClasscoveragekPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(a) Class coverage05101520255 10 20 50 100SubtopiccoveragekPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(b) Subtopic coverage0123455 10 20Topiccoverage(c) Topic05101520255 10 20 50Subtopiccoveragek00.10.20.30.40.50.65 10 20S-recall(class)k(d) S-recall on classes00.10.20.30.40.50.60.75 10 20S-recall(subtopic) k(e) S-recall on subtopics (f) S-recalFigure 8: Intent-aware results on cit-Patents dataset with sclevel topics7. Here we present an evaluation of the intent-oblivious algorithms against intent-aware measures. Thisevaluation provides a validation of the diversification tech-niques with an external measure such as group coverage [14]and S-recall [23].sults of all of the divethose two extremes, wand Dragon covers theHowever, S-recall indwas actually useful orResults – intent aware experiments•  S-recall [Zhai03], Intent-coverage [Zhu11]–  percentage of relevant subtopics covered by the result set–  the intent is given with the classes of the seed nodes•  AllRandom brings irrelevant items from the search space•  top-k results do not have the necessary diversity•  BC2 variants and BC1 perform better than DivRank•  BC1 (relaxed) and DivRank scores similar, but BC1r much fastertop-krandomDivRankBC0102030405060705 10 20 50 100kPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(a) Class coverage0510152025305 10 20 50 100SubtopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(b) Subtopic coverage01234565 10 20 50 100TopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(c) Topic coverage0510152025305 10 20 50 100SubtopiccoveragekPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandom(d) S-recall on classes (e) S-recall on subtopics (f) S-recall on topicsPPR (top-k)DragonPDivRankCDivRankk-RLMBC1BC2BC1 (relaxed)BC2 (relaxed)AllRandomFigure 8: Intent-aware results on cit-Patents dataset with scenario-3 queries.pics7. Here we present an evaluation of the intent-s algorithms against intent-aware measures. Thison provides a validation of the diversification tech-with an external measure such as group coverage [14]ecall [23].ts of a query set Q is extracted by collecting thesubtopics, and topics of each seed node. Since ouro evaluate the results based on the coverage of dif-sults of all of the diversification algorithms reside betweenthose two extremes, where the PDivRank covers the mostand Dragon covers the least number of groups.However, S-recall index measures whether a covered groupwas actually useful or not. Obviously, AllRandom scores thelowest as it dismisses the actual query (you may omit the S-recall on topics since there are only 6 groups in this granularity level). Among the algorithms, BC2 variants and BC1 scorerandomrandomrandomrandomrandomtop-ktop-ktop-ktop-ktop-kDivRankDivRankDivRankDivRankDRBCBCBCBCBC
  24. 24. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 24/25Conclusions•  Result diversification should not be evaluated as abicriteria optimization problem with–  a relevance measure that ignores diversity, and–  a diversity measure that ignores relevancy•  l-step expanded relevance is a simple measure thatcombines both relevance and diversity•  BestCoverage, a greedy solution that maximizesexprell is a (1-1/e)-approximation of the optimal solution•  BestCoverage variants perform better than others, itsrelaxation is extremely efficient•  goodness in DRAGON is dominated by relevancy•  DivRank variants implicitly optimize expansion ratio
  25. 25. Kucuktunc et al. “Diversified Recommendation on Graphs: Pitfalls, Measures, and Algorithms”, WWW’13 25/25Thank you•  For more information visit•  http://bmi.osu.edu/hpc•  Research at the HPC Lab is funded by

×