Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                     The Choice of a Da...
Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
                  1   Notation
   C. Castillo

Notatio...
•›››››››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••››››››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation        ...
••››››››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation        ...
•••›››››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••››››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
•••••›››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
•••••›››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
•••••›››››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
••••••››››››››››››
   Damping
 Functions for
 Link Ranking                                 Distribution of shortest paths
...
•••••••›››››››››››
   Damping
 Functions for

                                0.30
 Link Ranking

R. Baeza-Yates,         ...
••••••••››››››››››
   Damping
 Functions for
                                 0.30
 Link Ranking
                         ...
••••••••››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
••••••••››››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
•••••••••›››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
•••••••••›››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
•••••••••›››››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                 ...
••••••••••››››››››
   Damping
 Functions for
 Link Ranking     1: for i : 1 . . . N do {Initialization}
                  ...
••••••••••››››››››
   Damping
 Functions for
 Link Ranking     1:   for i : 1 . . . N do {Initialization}
                ...
••••••••••››››››››
   Damping
 Functions for
 Link Ranking      1:   for i : 1 . . . N do {Initialization}
               ...
••••••••••››››››››
   Damping
 Functions for
 Link Ranking      1:   for i : 1 . . . N do {Initialization}
               ...
••••••••••››››››››
   Damping
 Functions for
 Link Ranking      1:   for i : 1 . . . N do {Initialization}
               ...
•••••••••••›››››››
   Damping
 Functions for
 Link Ranking     Other functions studied in the paper:
R. Baeza-Yates,
 P. B...
•••••••••••›››››››
   Damping
 Functions for
 Link Ranking     Other functions studied in the paper:
R. Baeza-Yates,
 P. B...
••••••••••••››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••••••••••››››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
•••••••••••••›››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation
       ...
•••••••••••••›››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation
       ...
•••••••••••••›››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation
       ...
•••••••••••••›››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation
       ...
••••••••••••••››››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
                  Experimental C...
•••••••••••••••›››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
                  Prediction of ...
••••••••••••••••››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••••••••••••••››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••••••••••••••››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••••••••••••••››
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
•••••••••••••••••›
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
•••••••••••••••••›
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
•••••••••••••••••›
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
••••••••••••••••••
   Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo

Notation

Rewrit...
Damping
 Functions for
 Link Ranking

R. Baeza-Yates,
 P. Boldi and
   C. Castillo
                  Baeza-Yates, R., Bold...
Upcoming SlideShare
Loading in …5
×

Generalizing PageRank (Pisa)

946 views

Published on

Published in: Technology, Design
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
946
On SlideShare
0
From Embeds
0
Number of Embeds
28
Actions
Shares
0
Downloads
27
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Generalizing PageRank (Pisa)

  1. 1. Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo The Choice of a Damping Function for Notation Propagating Page Importance Rewriting PageRank in Link-Based Ranking Functional Rankings Algorithms Comparison Ricardo Baeza-Yates1 , Paolo Boldi2 and Carlos Castillo3 Conclusions 1. Yahoo Research – Barcelona, Spain 2. Universit` di Milano – Italy a 3. Universit` di Roma “La Sapienza” – Italy a February 6th, 2005
  2. 2. Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and 1 Notation C. Castillo Notation 2 Rewriting PageRank Rewriting PageRank Functional Rankings 3 Functional Rankings Algorithms Comparison 4 Algorithms Conclusions 5 Comparison 6 Conclusions
  3. 3. •››››››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting PageRank Let PN×N be the normalized link matrix of a graph Functional Rankings Row-normalized Algorithms No “sinks” Comparison Conclusions
  4. 4. ••›››››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Definition (PageRank) Rewriting PageRank Stationary state of: Functional Rankings (1 − α) αP + 1N×N Algorithms N Comparison Conclusions
  5. 5. ••›››››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Definition (PageRank) Rewriting PageRank Stationary state of: Functional Rankings (1 − α) αP + 1N×N Algorithms N Comparison Conclusions Follow links with probability α Random jump with probability 1 − α
  6. 6. •••››››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting Rewriting PageRank [Boldi et al., 2005] PageRank Functional ∞ Rankings (1 − α) r(α) = (αP)t . Algorithms N t=0 Comparison Conclusions
  7. 7. ••••›››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting Definition (Branching contribution of a path) PageRank Given a path p = x1 , x2 , . . . , xt of length t = |p| Functional Rankings 1 Algorithms branching(p) = Comparison d1 d2 · · · dt−1 Conclusions where di are the out-degrees of the members of the path
  8. 8. •••••››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Explicit formula for PageRank [Newman et al., 2001] Notation (1 − α)α|p| Rewriting ri (α) = branching(p) PageRank N p∈Path(−,i) Functional Rankings Algorithms Comparison Conclusions
  9. 9. •••••››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Explicit formula for PageRank [Newman et al., 2001] Notation (1 − α)α|p| Rewriting ri (α) = branching(p) PageRank N p∈Path(−,i) Functional Rankings Algorithms Path(−, i) are incoming paths in node i Comparison Conclusions
  10. 10. •••••››››››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Explicit formula for PageRank [Newman et al., 2001] Notation (1 − α)α|p| Rewriting ri (α) = branching(p) PageRank N p∈Path(−,i) Functional Rankings Algorithms Path(−, i) are incoming paths in node i Comparison Conclusions General functional ranking damping(|p|) ri (α) = branching(p) N p∈Path(−,i)
  11. 11. ••••••›››››››››››› Damping Functions for Link Ranking Distribution of shortest paths R. Baeza-Yates, P. Boldi and .it (40M pages) .uk (18M pages) C. Castillo 0.3 0.3 Notation 0.2 0.2 Frequency Frequency Rewriting PageRank 0.1 0.1 Functional Rankings 0.0 0.0 5 10 15 20 25 30 5 10 15 20 25 30 Algorithms Distance Distance Comparison .eu.int (800K pages) Synthetic graph (100K pages) Conclusions 0.3 0.3 0.2 0.2 Frequency Frequency 0.1 0.1 0.0 0.0 5 10 15 20 25 30 5 10 15 20 25 30 Distance Distance
  12. 12. •••••••››››››››››› Damping Functions for 0.30 Link Ranking R. Baeza-Yates, damping(t) with α=0.8 P. Boldi and C. Castillo damping(t) with α=0.7 Notation 0.20 Rewriting Weight PageRank Functional Rankings 0.10 Algorithms Comparison Conclusions 0.00 1 2 3 4 5 6 7 8 9 10 Length of the path (t) Exponential damping = PageRank damping(t) = α(1 − α)t
  13. 13. ••••••••›››››››››› Damping Functions for 0.30 Link Ranking damping(t) with L=15 R. Baeza-Yates, damping(t) with L=10 P. Boldi and C. Castillo 0.20 Weight Notation Rewriting PageRank Functional 0.10 Rankings Algorithms Comparison 0.00 Conclusions 1 2 3 4 5 6 7 8 9 10 Length of the path (t) Linear damping 2(L−t) L(L+1) t<L damping(t) = 0 t≥L
  14. 14. ••••••••›››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo For calculating LinearRank we use: Notation ∞ Rewriting 1 PageRank LinearRank = damping(t)Pt Functional N t=0 Rankings L−1 Algorithms 1 2(L − t) t Comparison = P N L(L + 1) Conclusions t=0
  15. 15. ••••••••›››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo For calculating LinearRank we use: Notation ∞ Rewriting 1 PageRank LinearRank = damping(t)Pt Functional N t=0 Rankings L−1 Algorithms 1 2(L − t) t Comparison = P N L(L + 1) Conclusions t=0 However, we cannot hold the temporary Pt in memory!
  16. 16. •••••••••››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo We have to rewrite to be able to calculate: Notation 2 Rewriting R(0) = PageRank L+1 Functional (L − k − 1) (k) Rankings R(k+1) = R P Algorithms (L − k) Comparison Conclusions
  17. 17. •••••••••››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo We have to rewrite to be able to calculate: Notation 2 Rewriting R(0) = PageRank L+1 Functional (L − k − 1) (k) Rankings R(k+1) = R P Algorithms (L − k) Comparison L−1 Conclusions LinearRank = R(k) k=0
  18. 18. •••••••••››››››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo We have to rewrite to be able to calculate: Notation 2 Rewriting R(0) = PageRank L+1 Functional (L − k − 1) (k) Rankings R(k+1) = R P Algorithms (L − k) Comparison L−1 Conclusions LinearRank = R(k) k=0 Now we can give the algorithm . . .
  19. 19. ••••••••••›››››››› Damping Functions for Link Ranking 1: for i : 1 . . . N do {Initialization} 2 R. Baeza-Yates, 2: Score[i] ← R[i] ← L+1 P. Boldi and C. Castillo 3: end for Notation Rewriting PageRank Functional Rankings Algorithms Comparison Conclusions
  20. 20. ••••••••••›››››››› Damping Functions for Link Ranking 1: for i : 1 . . . N do {Initialization} 2 R. Baeza-Yates, 2: Score[i] ← R[i] ← L+1 P. Boldi and C. Castillo 3: end for Notation 4: for k : 1 . . . L − 1 do {Iteration step} Rewriting 5: Aux ← 0 PageRank Functional Rankings Algorithms Comparison Conclusions
  21. 21. ••••••••••›››››››› Damping Functions for Link Ranking 1: for i : 1 . . . N do {Initialization} 2 R. Baeza-Yates, 2: Score[i] ← R[i] ← L+1 P. Boldi and C. Castillo 3: end for Notation 4: for k : 1 . . . L − 1 do {Iteration step} Rewriting 5: Aux ← 0 PageRank 6: for i : 1 . . . N do {Follow links in the graph} Functional Rankings 7: for all j such that there is a link from i to j do Algorithms 8: Aux[j] ← Aux[j] + R[i]/outdegree(i) Comparison 9: end for Conclusions 10: end for
  22. 22. ••••••••••›››››››› Damping Functions for Link Ranking 1: for i : 1 . . . N do {Initialization} 2 R. Baeza-Yates, 2: Score[i] ← R[i] ← L+1 P. Boldi and C. Castillo 3: end for Notation 4: for k : 1 . . . L − 1 do {Iteration step} Rewriting 5: Aux ← 0 PageRank 6: for i : 1 . . . N do {Follow links in the graph} Functional Rankings 7: for all j such that there is a link from i to j do Algorithms 8: Aux[j] ← Aux[j] + R[i]/outdegree(i) Comparison 9: end for Conclusions 10: end for 11: for i : 1 . . . N do {Add to ranking value} 12: R[i] ← Aux[i] × (L−k−1) (L−k) 13: Score[i] ← Score[i] + R[i] 14: end for 15: end for 16: return Score
  23. 23. ••••••••••›››››››› Damping Functions for Link Ranking 1: for i : 1 . . . N do {Initialization} 2 R. Baeza-Yates, 2: Score[i] ← R[i] ← L+1 P. Boldi and C. Castillo 3: end for Notation 4: for k : 1 . . . L − 1 do {Iteration step} Rewriting 5: Aux ← 0 PageRank 6: for i : 1 . . . N do {Follow links in the graph} Functional Rankings 7: for all j such that there is a link from i to j do Algorithms 8: Aux[j] ← Aux[j] + R[i]/outdegree(i) Comparison 9: end for Conclusions 10: end for 11: for i : 1 . . . N do {Add to ranking value} 12: R[i] ← Aux[i] × (L−k−1) (L−k) 13: Score[i] ← Score[i] + R[i] 14: end for 15: end for 16: return Score
  24. 24. •••••••••••››››››› Damping Functions for Link Ranking Other functions studied in the paper: R. Baeza-Yates, P. Boldi and Hyperbolic damping C. Castillo Notation Rewriting PageRank Functional Rankings Algorithms Comparison Conclusions
  25. 25. •••••••••••››››››› Damping Functions for Link Ranking Other functions studied in the paper: R. Baeza-Yates, P. Boldi and Hyperbolic damping C. Castillo Empirical damping Notation Rewriting 0.7 PageRank Average text similarity Functional Rankings 0.6 Algorithms 0.5 Comparison Conclusions 0.4 0.3 0.2 1 2 3 4 5 Link distance
  26. 26. ••••••••••••›››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting PageRank How to approximate one functional ranking with another? Functional Rankings Algorithms Comparison Conclusions
  27. 27. ••••••••••••›››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting PageRank How to approximate one functional ranking with another? Functional Analysis (in the paper): match the first few levels of their Rankings damping functions Algorithms Comparison In practice the orderings can be very similar . . . Conclusions
  28. 28. •••••••••••••››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Experimental comparison: 18-million nodes in the U.K. Web Rewriting PageRank Graph Functional Rankings Algorithms Comparison Conclusions
  29. 29. •••••••••••••››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Experimental comparison: 18-million nodes in the U.K. Web Rewriting PageRank Graph Functional Rankings Calculated PageRank with α = 0.1, 0.2, . . . , 0.9 Algorithms Comparison Conclusions
  30. 30. •••••••••••••››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Experimental comparison: 18-million nodes in the U.K. Web Rewriting PageRank Graph Functional Rankings Calculated PageRank with α = 0.1, 0.2, . . . , 0.9 Algorithms Calculated LinearRank with L = 5, 10, . . . , 25 Comparison Conclusions
  31. 31. •••••••••••••››››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Experimental comparison: 18-million nodes in the U.K. Web Rewriting PageRank Graph Functional Rankings Calculated PageRank with α = 0.1, 0.2, . . . , 0.9 Algorithms Calculated LinearRank with L = 5, 10, . . . , 25 Comparison For certain combinations of parameters, the rankings are Conclusions almost equal!
  32. 32. ••••••••••••••›››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and Experimental Comparison in the U.K. Web Graph C. Castillo Notation Rewriting 1.00 0.95 τ PageRank Functional 0.90 Rankings 0.85 τ ≥ 0.95 Algorithms 0.80 Comparison Conclusions 25 20 15 0.9 L 10 0.7 0.8 5 0.5 0.6 α
  33. 33. •••••••••••••••››› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and Prediction of Best Parameter Combinations (Analysis) C. Castillo 25 Actual optimum Notation Predicted optimum with length=5 Rewriting L that maximizes Kendall’s τ 20 PageRank Functional Rankings 15 Algorithms Comparison 10 Conclusions 5 0.5 0.6 0.7 0.8 0.9 Exponent α
  34. 34. ••••••••••••••••›› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What have we done? PageRank Functional Rankings Algorithms Comparison Conclusions
  35. 35. ••••••••••••••••›› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What have we done? PageRank Separate the damping from the calculation Functional Rankings Algorithms Comparison Conclusions
  36. 36. ••••••••••••••••›› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What have we done? PageRank Separate the damping from the calculation Functional Rankings Show that different damping functions can provide the Algorithms same ranking Comparison Conclusions
  37. 37. ••••••••••••••••›› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What have we done? PageRank Separate the damping from the calculation Functional Rankings Show that different damping functions can provide the Algorithms same ranking Comparison Conclusions Analysis and experiments in the paper
  38. 38. •••••••••••••••••› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What can we do with this? PageRank Functional Fast approximation of PageRank using linear damping Rankings Algorithms Comparison Conclusions
  39. 39. •••••••••••••••••› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What can we do with this? PageRank Functional Fast approximation of PageRank using linear damping Rankings Algorithms Fast calculation of other link-based rankings (e.g. HITS) Comparison Conclusions
  40. 40. •••••••••••••••••› Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting What can we do with this? PageRank Functional Fast approximation of PageRank using linear damping Rankings Algorithms Fast calculation of other link-based rankings (e.g. HITS) Comparison Spam detection (e.g.: cut the first levels of links) Conclusions
  41. 41. •••••••••••••••••• Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Notation Rewriting PageRank Functional Rankings Thank you! Algorithms Comparison Conclusions
  42. 42. Damping Functions for Link Ranking R. Baeza-Yates, P. Boldi and C. Castillo Baeza-Yates, R., Boldi, P., and Castillo, C. (2005). Notation The choice of a damping function for propagating importance in link-based ranking. Rewriting PageRank Technical report, Dipartimento di Scienze dell’Informazione, Universit degli Studi di Milano. Functional Rankings Boldi, P., Santini, M., and Vigna, S. (2005). Algorithms Pagerank as a function of the damping factor. In Proceedings of the 14th international conference on World Wide Web, Comparison pages 557–566, Chiba, Japan. ACM Press. Conclusions Newman, M. E., Strogatz, S. H., and Watts, D. J. (2001). Random graphs with arbitrary degree distributions and their applications. Phys Rev E Stat Nonlin Soft Matter Phys, 64(2 Pt 2).

×