Extrapolation
Upcoming SlideShare
Loading in...5
×
 

Extrapolation

on

  • 462 views

 

Statistics

Views

Total Views
462
Views on SlideShare
462
Embed Views
0

Actions

Likes
0
Downloads
1
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Why does the Power Method Work?
  • Assume that lambda 1 is less than 1 and all other eigenvalues are strictly less than 1.
  • Here, talk about in the past, how lambda 2 is often close to 1, so the power method is not useful. However, in our case,
  • Note : derivation given here is slightly different from what’s in the paper the one here is perhaps more intuitive the one in the paper is more compact

Extrapolation Extrapolation Presentation Transcript

  • Extrapolation Methods for Accelerating PageRank Computations Sepandar D. Kamvar Taher H. Haveliwala Christopher D. Manning Gene H. Golub Stanford University
  • Motivation
    • Problem:
      • Speed up PageRank
    • Motivation:
      • Personalization
      • “ Freshness”
    Note: PageRank Computations don’t get faster as computers do. Results: 1. The Official Site of the San Francisco Giants Search: Giants Results: 1. The Official Site of the New York Giants
  • Outline
    • Definition of PageRank
    • Computation of PageRank
    • Convergence Properties
    • Outline of Our Approach
    • Empirical Results
    0.4 0.2 0.4 Repeat: u 1 u 2 u 3 u 4 u 5 u 1 u 2 u 3 u 4 u 5
  • Link Counts Linked by 2 Important Pages Linked by 2 Unimportant pages Sep’s Home Page Taher’s Home Page Yahoo! CNN DB Pub Server CS361
  • Definition of PageRank
    • The importance of a page is given by the importance of the pages that link to it.
    importance of page i pages j that link to page i number of outlinks from page j importance of page j
  • Definition of PageRank Yahoo! CNN DB Pub Server Taher Sep 1/2 1/2 1 1 0.1 0.1 0.1 0.05 0.25
  • PageRank Diagram Initialize all nodes to rank 0.333 0.333 0.333
  • PageRank Diagram Propagate ranks across links (multiplying by link weights) 0.167 0.167 0.333 0.333
  • PageRank Diagram 0.333 0.5 0.167
  • PageRank Diagram 0.167 0.167 0.5 0.167
  • PageRank Diagram 0.5 0.333 0.167
  • PageRank Diagram After a while… 0.4 0.4 0.2
  • Computing PageRank
    • Initialize:
    • Repeat until convergence:
    importance of page i pages j that link to page i number of outlinks from page j importance of page j
  • Matrix Notation 0 .2 0 .3 0 0 .1 .4 0 .1 = .1 .3 .2 .3 .1 .1 .2 . 1 .3 .2 .3 .1 .1
  • Matrix Notation Find x that satisfies: . 1 .3 .2 .3 .1 .1 0 .2 0 .3 0 0 .1 .4 0 .1 = .1 .3 .2 .3 .1 .1 .2
  • Power Method
    • Initialize:
    • Repeat until convergence:
    • PageRank doesn’t actually use P T . Instead, it uses A=cP T + (1-c)E T .
    • So the PageRank problem is really:
    • not:
    A side note Find x that satisfies: Find x that satisfies:
  • Power Method
    • And the algorithm is really . . .
    • Initialize:
    • Repeat until convergence:
  • Outline
    • Definition of PageRank
    • Computation of PageRank
    • Convergence Properties
    • Outline of Our Approach
    • Empirical Results
    0.4 0.2 0.4 Repeat: u 1 u 2 u 3 u 4 u 5 u 1 u 2 u 3 u 4 u 5
  • Power Method u 1 1 u 2  2 u 3  3 u 4  4 u 5  5 Express x (0) in terms of eigenvectors of A
  • Power Method u 1 1 u 2  2  2 u 3  3  3 u 4  4  4 u 5  5  5
  • Power Method u 1 1 u 2  2  2 2 u 3  3  3 2 u 4  4  4 2 u 5  5  5 2
  • Power Method u 1 1 u 2  2  2 k u 3  3  3 k u 4  4  4 k u 5  5  5 k
  • Power Method u 1 1 u 2  u 3  u 4  u 5 
  • Why does it work?
    • Imagine our n x n matrix A has n distinct eigenvectors u i .
    u 1 1 u 2  2 u 3  3 u 4  4 u 5  5
    • Then, you can write any n -dimensional vector as a linear combination of the eigenvectors of A .
  • Why does it work?
    • From the last slide:
    • To get the first iterate, multiply x (0) by A .
    • First eigenvalue is 1.
    • Therefore:
    All less than 1
  • Power Method u 1 1 u 2  2 u 3  3 u 4  4 u 5  5 u 1 1 u 2  2  2 u 3  3  3 u 4  4  4 u 5  5  5 u 1 1 u 2  2  2 2 u 3  3  3 2 u 4  4  4 2 u 5  5  5 2
    • The smaller  2 , the faster the convergence of the Power Method.
    Convergence u 1 1 u 2  2  2 k u 3  3  3 k u 4  4  4 k u 5  5  5 k
  • Our Approach u 1 u 2 u 3 u 4 u 5 Estimate components of current iterate in the directions of second two eigenvectors, and eliminate them.
  • Why this approach?
    • For traditional problems:
      • A is smaller, often dense.
      •  2 often close to   , making the power method slow.
    • In our problem,
      • A is huge and sparse
      • More importantly,  2 is small 1 .
    • Therefore, Power method is actually much faster than other methods.
    1 (“The Second Eigenvalue of the Google Matrix” dbpubs.stanford.edu/pub/2003-20.)
  • Using Successive Iterates u 1 x (0) u 1 u 2 u 3 u 4 u 5
  • Using Successive Iterates u 1 x (1) x (0) u 1 u 2 u 3 u 4 u 5
  • Using Successive Iterates u 1 x (1) x (0) x (2) u 1 u 2 u 3 u 4 u 5
  • Using Successive Iterates x (0) u 1 x (1) x (2) u 1 u 2 u 3 u 4 u 5
  • Using Successive Iterates x (0) x’ = u 1 x (1) u 1 u 2 u 3 u 4 u 5
  • How do we do this?
    • Assume x (k) can be written as a linear combination of the first three eigenvectors ( u 1 , u 2 , u 3 ) of A.
    • Compute approximation to { u 2 , u 3 }, and subtract it from x (k) to get x (k) ’
  • Assume
    • Assume the x (k) can be represented by first 3 eigenvectors of A
  • Linear Combination
    • Let’s take some linear combination of these 3 iterates.
  • Rearranging Terms
    • We can rearrange the terms to get:
    Goal: Find  1 ,  2 ,  3 so that coefficients of u 2 and u 3 are 0, and coefficient of u 1 is 1.
  • Summary
    • We make an assumption about the current iterate.
    • Solve for dominant eigenvector as a linear combination of the next three iterates.
    • We use a few iterations of the Power Method to “clean it up”.
  • Outline
    • Definition of PageRank
    • Computation of PageRank
    • Convergence Properties
    • Outline of Our Approach
    • Empirical Results
    u 1 u 2 u 3 u 4 u 5 u 1 u 2 u 3 u 4 u 5 0.4 0.2 0.4 Repeat:
  • Results Quadratic Extrapolation speeds up convergence. Extrapolation was only used 5 times!
  • Results Extrapolation dramatically speeds up convergence, for high values of c (c=.99)
  • Take-home message
    • Speeds up PageRank by a fair amount, but not by enough for true Personalized PageRank.
    • Ideas are useful for further speedup algorithms.
    • Quadratic Extrapolation can be used for a whole class of problems.
  • The End
    • Paper available at http://dbpubs.stanford.edu/pub/2003-16