Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Link Analysis in a Penguin and Disa... by Christoph C. Cemper 3565 views
- Guide for Link Analysis by Tripp H... by Anton Shulke 879 views
- Using Link Analysis to Setup Ergono... by MathesonWorkSafe 2088 views
- Link analysis by Rajendra Akerkar 1508 views
- Link Analysis of Life Sciences Link... by Michel Dumontier 757 views
- Link Analysis by Carlos Castillo (... 2198 views

781 views

Published on

Part of the Search Engine course given in the Technion (2011)

No Downloads

Total views

781

On SlideShare

0

From Embeds

0

Number of Embeds

2

Shares

0

Downloads

21

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Link Structure Analysis Kira Radinsky All of the following slides are courtesy of Ronny Lempel (Yahoo!)
- 2. 29 November 2010 236620 Search Engine Technology 2 Link Analysis In the Lecture • HITS: topic-specific algorithm – Assigns each page two scores – a hub score and an authority score – with respect to a topic • PageRank: query independent algorithm – Assigns each page a single, global importance score • Both algorithms reduced to the computation of principal eigenvectors of certain matrices Today’s Tutorial: 1. Graph modifications in link analysis algorithms 2. SALSA – HITS with a random-walk twist 3. Topic-Sensitive PageRank
- 3. 30 November 2010 236620 Search Engine Technology 3 Graph Modifications in Link-Analysis Algorithms 1. Delete irrelevant elements (pages, links) from the collection. Non-informative links Pages that are deemed irrelevant (mostly by similarity of content to the query), and their incident links [Bharat and Henzinger, 1998] 2. Assign varying (positive) link weights to the non-deleted links. – Similarity of anchor text to the query [CLEVER] – Links incident to pre-defined relevant pages [CLEVER] – Multiple links from pages of site A to pages of site B [Bharat and Henzinger, 1998] • Note that some of the above modifications are only applicable to topic distillation algorithms
- 4. 29 November 2010 236620 Search Engine Technology 4 SALSA – Stochastic Approach to Link Structure Analysis • SALSA, like HITS, is a topic-distillation algorithm that aims to assign pages both hub and authority scores – SALSA analyzes the same topic-centric graph as HITS, but splits each node into two – a “hub personality” without in-links and an “authority personality” without out-links – Examines the resulting bipartite graph • Innovation: incorporate stochastic analysis with the authority-hub paradigm – Examine two separate random walk Markov chains: an authority chain A, and a hub chain H. – A single step in each chain is composed of two link traversals on the Web - one link forward, and one link backwards. – The principal community of each type: the most frequently visited pages in the corresponding Markov Chain
- 5. Forming bi-pirate graph in Salsa
- 6. 29 November 2010 236620 Search Engine Technology 6 Pr (23) = 2/5*1/3 • Formally, The transition probability matrix: SALSA – Authority Chain Example [PA]i,j = {k| ki, kj} (iin)-1(kout)-1
- 7. 29 November 2010 236620 Search Engine Technology 7 SALSA: Analysis • The transition probabilities induce a probability distribution on the authorities (hubs) in the authority (hub) Markov chain – If the chains are not irreducible, the probability depends on the initial distribution (chosen to be uniform) • The principal community of authorities (hubs) is defined as the k most probable pages in the authority (hub) chain • While one can compute the scores by calculating the principal eigenvector of the stochastic transition matrices, a more efficient way exists
- 8. 29 November 2010 236620 Search Engine Technology 8 Mathematical Analysis of SALSA leads to the following theorem: SALSA’s authority weights reflect The normalized in-degree of each page, multiplied by the relative size of the page’s component in the authority side of the graph x 3 4 a(x) = ----- x ----- = 0.25 3 +5 4 +2 SALSA: Analysis (cont.)
- 9. 29 November 2010 236620 Search Engine Technology 9 SALSA: Proof for Irreducible Authority Chains • The proof assumes a weighted graph, in which the link kj has weight w(kj) – The examples shown so far assumed that all links have a weight of 1 • Define W as the sum of all links weights • Define a distribution vector π by πj = din(j)/W, where din(j) is the sum of weights of j’s incoming links – Similarly, dout(k) is the sum of weights of k’s outgoing links • It is enough to prove that πPA=π, since PA has a single stationary eigenvector (Ergodic Theorem) – Recall that PA is the transition matrix of the authority chain – PA is always aperiodic
- 10. 29 November 2010 236620 Search Engine Technology 10 SALSA: Proof for Irreducible Authority Chains
- 11. 29 November 2010 236620 Search Engine Technology 11 Topic Sensitive PageRank [T. Haveliwala, 2002] • A topic T is defined by a set of on-topic pages ST. • A T-biased PageRank is PageRank where the random jumps (teleportations) land u.a.r. on ST rather than on any arbitrary Web page • Recall the alternative interpretation of PageRank, as walking random paths of geometrically distributed lengths between resets – Here, a reset returns to some on-topic page • If we assume that pages tend to link to pages with topical affinity, short paths starting at ST will not stray too far away from on-topic pages, hence the PageRanks will be T-biased – Note that pages unreachable from ST will receive a T-biased PageRank of 0 • Where would be a good place to find sets ST for certain topics? – The pages classified under the 16 top-level topics of the Open Directory Project (see next slide)
- 12. 29 November 2010 236620 Search Engine Technology 12
- 13. 29 November 2010 236620 Search Engine Technology 13 Topic-Sensitive PageRank (cont.) • 16 PageRank vectors are computed, PR1,…,PR16 • Given a query q, its affinity to the 16 topics T1,…,T16 is computed – Based on the probability of generating the query by the language model induced by the set of pages ST – A distribution vector [α1,…,α16] is computed, where αj ~ Prob(q | language model of STj) • The PageRank vector that will be used to serve q is PRq = αjPRj • The idea of biasing PageRank’s random jump destinations is also used for personalized PageRank flavors [e.g. Jeh and Widom 2003]
- 14. 29 November 2010 236620 Search Engine Technology 14 Link Analysis Algorithms - Summary • Many variants and refinements of both HITS and PageRank have been proposed. • Other approaches include: – Max-flow techniques [Flake et al., SIGKDD 2000] – Machine learning and Bayesian techniques • Examples of applications: – Ranking pages (topic specific/global importance/ personalized rankings) – Categorization, clustering, finding related pages – Identifying virtual communities • Computational issues: – Distributed computations of eigenvectors of massive, sparse matrices – Convergence acceleration, approximations • A wealth of literature

No public clipboards found for this slide

Be the first to comment