CSE509 Lecture 3

754
-1

Published on

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
754
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Existing work classify ranking algorithms into two classes as follows. Content-based method: exploiting the vertex information (that is, contents of web pages).Link-based method: exploiting the edges information (that is, a link structure of web).
  • Now, I explain the original PageRank. The main idea of PageRank is that a web page is more important if it is pointed by many other important web pages.The importance of a web page (called PageRank value) represents the probability that a user visits the web pageI will show how user visit a web page by using this figure. (In this figure, circles represent web pages and arrows represent directed links.) Assume a user is on the web page F. Then, the user can visit the web page C by following the outlink FC of F, and visit other web pages by following the outlinks of C, and so on. (If he gets bored with clicking on the outlinks to visit another web page) The user can also type the address of a random web page and jump into it. Here, user on F randomly jumps to web page B. Obviously, as there are many links to web page C, the user may frequently visit C. Thus, C may be an important web page.Since user has two ways to visit the web page, the probability that user visit a web page or the PageRank value of that web page consists of 2 part. The first part is the probability that user visit web page p by following the outlinks of web pages that have links to p. The second part is the probability that user visit web page p by randomly jump from any web page.
  • Highly successful, all major businesses use an RDB system
  • Spam domainThe domain contributing in web spam.Non-spam domainUniverse of domains – {spam domain}
  • Trusted domain is a subset a non-spam domains set and already known to human as well.
  • Anti-Trusted domains is a subset of spam domainsSome sex websites
  • The rank coming from non-spam domain is estimated by TrustRank and the rank coming from non-spam is estimated by TrustRank value from PageRank valueSpam Mass is spam detection algorithm
  • Detect spam domains by expanding the seed set of spam domains by counting outlinks to the spam domains
  • Here are the contents of my presentationFirst I introduce the background and motivation of my researchThen, I present related work that contains two approaches for improving the ranking qualityAfter that, I present the algorithms that combine these two approachesIn the main part, I present the performance evaluation Finally, I present the conclusions
  • Detect spam domains on basis of many bidirectional linksDetect more spam domains by counting outlinks to the spam domains
  • Detect spam domains on basis of many bidirectional linksDetect more spam domains by counting outlinks to the spam domains
  • CSE509 Lecture 3

    1. 1. CSE509: Introduction to Web Science and Technology<br />Lecture 3: The Structure of the Web, Link Analysis and Web Search<br />Muhammad AtifQureshi and ArjumandYounus<br />Web Science Research Group<br />Institute of Business Administration (IBA)<br />
    2. 2. Last Time…<br />Basic Information Retrieval<br />Approaches<br />Bag of Words Assumption<br />Information Retrieval Models<br />Boolean model<br />Vector-space model<br />Topic/Language models<br />July 23, 2011<br />
    3. 3. Today<br />Search Engine Architecture<br />Overview of Web Crawling<br />Web Link Structure<br />Ranking Problem<br />SEO and Web Spam<br />Web Spam Research<br />July 23, 2011<br />
    4. 4. Introduction<br />World Wide Web has evolved from a handful of pages to billions of pages<br />In January 2008, Google reported indexing 30 billion pages and Yahoo 37 billion.<br />In this huge amount of data, search engines play a significant role in finding the needed information<br />Search engines consist of the following basic operations<br />Web crawling<br />Ranking<br />Keyword extraction<br />Query processing<br />July 23, 2011<br />
    5. 5. General Architecture of a Web Search Engine<br />Web<br />User<br />Query<br />Crawler<br />Indexing<br />Visual <br />Interface<br />Index<br />Ranking<br />Query<br />Operations<br />July 23, 2011<br />
    6. 6. CRAWLING MODULE<br />July 23, 2011<br />
    7. 7. Web Crawler<br />Definition<br />Program that collects Web pages by recursively fetching links (i.e., URLs) starting from a set of seed pages [HN99]<br />Objective<br />Acquisition of large collections of Web pages to be indexed by the search engine for efficient execution of user queries<br />July 23, 2011<br />Introduction<br />
    8. 8. Basic Crawler Operation<br />Place known seed URLs in the URL queue<br />Repeat following steps until a threshold number of pages downloaded<br />Fetch a URL on the URL queue and download the corresponding Web page<br />For each downloaded Web page<br />Extract URLs from the Web page<br /> For each extracted URL, check validity and availability of URL using checking modules<br /> Place the URLs that pass the checks on the URL queue<br />July 23, 2011<br />New URLs<br />Seed URLs<br />NOTATIONS USED<br />: queue<br />: module<br />: data flow <br />URLs to<br />crawl<br />Web pages<br />Checking module<br />Extracted URLs<br />URL<br />duplication check<br />Web page <br />downloader<br />URL queue<br />Link<br />extractor<br />DNS<br />resolver<br />Crawled <br />Web pages<br />URLs to<br />crawl<br />Robots<br />check<br />Web<br />
    9. 9. Crawling Issues<br />Load at visited Web sites<br />Load at crawler<br />Scope of crawl<br />Incremental crawling<br />July 23, 2011<br />
    10. 10. RANKING MODULE<br />July 23, 2011<br />
    11. 11. Problems of TFIDF Vector<br />Works well on small controlled corpus, but not on the Web<br />Top result for “American Airlines” query: accident report of American Airline flights<br />Do users really care how many times American Airlines mentioned?<br />Easy to spam<br />Ranking purely based on page content<br />Authors can manipulate page content to get high ranking<br />Any idea?<br />July 23, 2011<br />
    12. 12. Web Page Ranking<br />Motivation<br /> User queries return huge amount of relevant web pages, but the users want to browse the most important ones<br />Note: Relevancerepresents that a web page matches the user’s query<br />Concept<br /> Ordering the relevant web pages according to their importance<br />Note: Importance represents the interest of a user on the relevant web pages<br />Methods<br />Link-based method: exploiting the link structure of web for ordering the search results<br />Content-based method: exploiting the contents of web pages for ordering the search results<br />July 23, 2011<br />
    13. 13. Link Structure of Web<br />Concept<br />Web can be modeled as a graph G(V, E) where V is a set of vertices representing web nodes, and E is a set of edges representing directed links between the nodes.<br />Note: Web node represents either a web page or a web domain. Links are classifed into two classes as follows:<br />The link structure is called web graph.<br />Example<br /><ul><li>Inlink: the incoming link to a web node.
    14. 14. Outlink: the outgoing link from a web node.</li></ul>V = {A, B, C}<br />E = {AB, BC}<br />AB is an outlink of the web node A.<br />BC is an outlink of the web node B.<br />AB is an inlink of the web node B.<br />BC is an inlink of the web node C.<br />B<br />C<br />A<br />Fig. 1: An example of a web graph.<br />July 23, 2011<br />
    15. 15. PageRank: Basic Idea<br />Think of ….<br />People as pages<br />Recommendations as links<br />Therefore, <br />“Pages are popular, if popular pages link them”<br /> “PageRank is a global ranking of all Web pages regardless of their content, based solely on their location in the Web’s graph structure” [Page et al 1998] <br />July 23, 2011<br />
    16. 16. PageRank<br />Overview<br />A web page is more important if it is pointed by many other important web pages<br />The importance of a web page (called PageRank value) represents the probability that a user visits the web page<br />Function<br />July 23, 2011<br />web page<br />important web page<br />D<br />link<br />C<br />E<br />random jump from F to B<br />A<br />user<br />F<br />B<br />jump to a random page<br />< User’s behavior on the web graph ><br />PR[p]: PageRank value of web page p <br />Nolink(q): number of outlinks of web page q<br />d: damping factor (probability of following a link)<br />v[p]: probability that a user randomly jumps to web page p <br />(random jump value over web page p) <br />
    17. 17. PageRank Example<br />July 23, 2011<br />1<br />2<br />3<br />4<br />
    18. 18. PageRank: Problems on the Real Web<br />Dangling nodes<br />A page with no links to send importance<br />All importance “leak out of” the Web<br />Solution: Random surfer model<br />Crawler trap<br />A group of one or more pages that have no links out of the group<br />Accumulate all the importance of the Web<br />Solution: Damping factor<br />July 23, 2011<br />
    19. 19. Link Analysis in Modern Web Search<br />PageRank like ideas play basic role in the ranking functions of Google, Yahoo! And Bing<br />Current ranking functions far from pure PageRank<br />Far more complex<br />Evolve all the time<br />Kept in secret!<br />July 23, 2011<br />
    20. 20. Search Engine Optimization<br />Important game-theoretic principle: the world reacts and adapts to the rules<br />Web page authors create their Web pages with the search engine’s ranking formula in mind<br />July 23, 2011<br />
    21. 21. A Huge Challenge for Today’s Search Engines<br />SEO gives birth to nuisance of Web spam<br />July 23, 2011<br />
    22. 22. Web Spam<br />Concept<br /> Any deliberate action in order to boost a web node’s rank, without improving its real merit.<br />Link spam: web spam against link-based methods<br />An action that changes the link structure of web in order to boost web node's ranking.<br />Example<br />N1<br />N2<br />I want to boost the rank of the web node N3<br />The web nodes N1and N2 are not involved in link spam, so they care called non-spam nodes<br />N4<br />Actor creates the web node N3 to Nx<br />N3<br />N5<br />Web nodes N3-Nx are involved in link spam, so they are called spam nodes<br />…<br />Nx<br />Node Link Actor<br />Fig. 2: An example of link spam.<br />July 23, 2011<br />
    23. 23. TrustRank<br />Overview [GGP04]<br />Trusted domains(e.g., well-known non-spam domains such as .gov and .edu) usually point to non-spam domains by using outlinks.<br />Trust scores are propagated through the outlinks of trusted domains.<br />Domains having high trust scores(≥threshold) at the end of propagation are declared as non-spam domains.<br />Example<br />Observation<br /> Trust scores can propagate to spam domains if trusted domain outlinks to the spam domains.<br />1/2<br />A domain being considered<br />5/12<br />1<br />1/2<br />3<br />5/12<br />t(1)=1<br />A seed non-spam domain<br />1/3<br />t(3)=5/6<br />1/3<br />t(i): The trust score of domain i<br />2<br />4<br />t(2)=1<br />The domain 3 gets trust scores from the domains 1 and 2.<br />1/3<br />t(4)=1/3<br />Fig. 3: An example for explaining TrustRank.<br />July 23, 2011<br />
    24. 24. Anti-TrustRank<br />Overview [KR06]<br />Anti-trusted domains (e.g., well-known spam domains) are usually pointed by spam domains by using inlinks.<br />Anti-trust scores are propagated by the inlinks of anti-trusted domains.<br />Domains having high anti-trust scores(≥threshold) at the end of propagation are declared as spam domains.<br />Example<br />Observation<br /> Anti-trust score can propagate to non-spam domains if a non-spam domain outlinks to spam domain.<br />1/2<br />A domain being considered<br />5/12<br />1<br />1/2<br />A seed spam domain<br />3<br />5/12<br />at(1)=1<br />1/3<br />at(3)=5/6<br />at(i): The anti-trust score of domain i<br />2<br />1/3<br />4<br />The domain 3 gets anti-trust scores from the domains 1 and 2.<br />at(2)=1<br />at(4)=1/3<br />1/3<br />Fig. 4: An example for explaining Anti-TrustRank.<br />July 23, 2011<br />
    25. 25. Spam Mass<br />Overview [GBG06]<br />A domain is spam if it has excessively high spam score.<br />Spam score is estimated as subtraction from a PageRank score to a non-spam score.<br />Non-spam score is estimated as a trust score computed by TrustRank.<br />Example<br />Observation<br />Since the Spam Mass has use TrustRank, it has inherently the same problem as TrustRank does.<br />1<br />A domain being considered<br />2<br />7<br />5<br />6<br />A seed non-spam domain<br />3<br />4<br />Fig. 5: An example for explaining Spam Mass.<br />The domain 5 receives many inlinks but only one indirect inlink from a non-spam domain.<br />July 23, 2011<br />
    26. 26. Link Farm Spam<br />Overview[WD05]<br />A domain is spam if it has many bidirectional links with domains.<br />A domain is spam if it has many outlinks pointing to spam domains.<br />Example<br />Observation<br />Link Farm Spam does not take any input seed set.<br />A domain can have many bidirectional links with trusted domains as well.<br />2<br />1<br />3<br />A domain being considered<br />4<br />5<br />The domains 1, 3, and 4 have two directional links.<br />Fig. 6: An example for explaining Link Farm Spam.<br />July 23, 2011<br />
    27. 27. RESEARCH SECTION<br />July 23, 2011<br />
    28. 28. Web Spam Filtering Algorithm<br />Overview<br />The web spam filtering algorithms output spam nodes to be filtered out [GBG06].<br />In order to identify spam nodes, a web spam filtering algorithm needs spam or non-spam nodes (called input seed sets) as an input [GGP04, KR06, GBG06, WD05].<br />Spam input seed set: the input seed set containing spam nodes.<br />Non-spam input seed set: the input seed set containing non-spam nodes.<br />The input seed set can be used as the basis for grading the degree of whether web nodes are spam or non-spam nodes [GGP04, KR06, GBG06].<br />Observation<br />The output quality of web spam filtering algorithms is dependent on that of the input seed sets.<br />The output of the one web spam filtering algorithm can be used as the input of the other web spam filtering algorithm.<br />  The algorithms may support one another if placed in appropriate succession.<br />July 23, 2011<br />
    29. 29. Motivation and Goal<br />Motivation<br />There is no well-known study which addresses the refinement of the input seed sets for web spam filtering algorithms.<br />There is no well-known study on successions among web spam filtering algorithms.<br />Goal <br />Improving the quality of web spam filtering by using seed refinement.<br />Improving the quality of web spam filtering by finding the appropriate succession among web spam filtering algorithms.<br />July 23, 2011<br />
    30. 30. Contributions<br />We propose modified algorithms that apply seed refinement techniques using both spam and non-spam input seed sets to well-known web spam filtering algorithms.<br />We propose a strategy that makes the best succession of the modified algorithms.<br />We conduct extensive experiments in order to show quality improvement for our work.<br />We compare the original(i.e., well-known) algorithms with the respective modified algorithms.<br />We evaluate the best succession among our modified algorithms.<br />July 23, 2011<br />
    31. 31. Web Spam Filtering Using Seed Refinement<br />Objectives<br />Decrease the number of domains incorrectly detected as belonging to the class of non-spam domains (called False Positives).<br />Increase the number of domains correctly detected as belonging to the class of spam domains (called True Positives).<br />Our approaches<br />We modify the spam filtering algorithms by using both spam and non-spam domains in order to decrease False Positives.<br />We use non-spam domains so that their goodness should not propagate to spam domains.<br />We use spam domains so that their badness should not propagate to non-spam domains.<br />We make the succession of these algorithms in order to increase True Positives.<br />We make the succession of the seed refinement algorithm followed by the spam detection algorithm so that the spam detection algorithm uses the refined input seed sets, which is produced by the seed refinement algorithm.<br />July 23, 2011<br />
    32. 32. Modified TrustRank<br />Modification<br /> Trust score should not propagate to spam domains.<br />Example<br />5/12<br />1/2<br />A seed spam domain<br />5/12<br />5<br />6<br />1<br />1/2<br />A domain being considered<br />3<br />t(6)=5/12 + …<br />t(5)=5/12 + …<br />t(1)=1<br />5/12<br />t(3)=5/6<br />1/3<br />A seed non-spam domain<br />1/3<br />2<br />5/12<br />t(i): The trust score of domain i<br />4<br />t(2)=1<br />The domains 5 and 6 are involved in Web spam.<br />1/3<br />t(4)=1/3<br />Fig. 7: An example explaining Modified TrustRank.<br />July 23, 2011<br />
    33. 33. Modified Anti-TrustRank<br />Modification<br /> Anti-Trust score should not propagate to non-spam domains.<br />Example<br />5/12<br />A seed spam domain<br />at(5)=5/12<br />1/2<br />3<br />1<br />5<br />A domain being considered<br />7<br />5/12<br />1/2<br />5/12<br />5/12<br />at(1)=1<br />at(3)=5/6<br />5/12<br />6<br />at(7)=5/12 + …<br />A seed non-spam domain<br />1/3<br />4<br />at(6)=5/12 + …<br />2<br />1/3<br /> at(i): The anti-trust score of domain i<br />at(2)=1<br />at(4)=1/3<br />1/3<br />The domains 5 ,6 and 7 are non- spam domains.<br />Fig. 8: An example explaining Modified Anti-TrustRank.<br />July 23, 2011<br />
    34. 34. Modified Spam Mass<br />Modification<br /> Use modified TrustRank in place of TrustRank.<br />Example<br />A seed spam domain<br />1<br />A domain being considered<br />2<br />5<br />7<br />6<br />A seed non-spam domain<br />3<br />4<br />The domain 5 receives many inlinks<br />but only one indirect inlink from a non-spam domain.<br />Fig. 9: An example explaining Modified Spam Mass.<br />July 23, 2011<br />
    35. 35. Modified Link Farm Spam<br />Modification<br />Use two types (i.e., spam and non-spam domain) of input seed sets.<br />A domain having many bidirectional links with only trusted domains is not detected as a spam domain.<br />Example<br />6<br />8<br />2<br />7<br />A seed non-spam domain<br />1<br />3<br />A domain being considered<br />4<br />5<br />The domains 1, 3, and 4 have two directional links.<br />Fig. 10: An example explaining Modified Link Farm Spam.<br />July 23, 2011<br />
    36. 36. Modified Link Farm Spam<br />Overview<br />We make the succession of the seed refinement algorithms (simply, Seed Refiner) followed by the spam detection algorithms (simply, Spam Detector).<br />We also consider the execution order of algorithms belonging to Seed Refiner and Spam Detector, respectively.<br /><ul><li>Strategy
    37. 37. Consideration of the execution order in Seed Refiner.
    38. 38. Modified TrustRank followed by Modified Anti-TrustRank.
    39. 39. Modified Anti-TrustRank followed by Modified TrustRank.
    40. 40. Consideration of the execution order in Spam Detector.
    41. 41. Modified Spam Mass followed by Modified Link Farm Spam.
    42. 42. Modified Link Farm Spam followed by Modified Spam Mass.</li></ul>Manually labeled spam and non-spam domains<br />Seed Refiner<br />Refined <br />spam and non-spam <br />domains<br />Spam Detector<br />Detected <br />spam domains<br />Class<br />Data flow<br />Fig. 11: The strategy of succession.<br />July 23, 2011<br />
    43. 43. Performance Evaluation<br />Purpose<br />Show the effect of seed refinement on the quality of web spam filtering.<br />Show the effect of succession on the quality of web spam filtering.<br />Experiments <br />We conduct two sets of the experiments according to the two purposes as mentioned above.<br />Table. 1: Summary of the experiments.<br />July 23, 2011<br />
    44. 44. Experimental Parameters<br />Table. 2: Parameters used in experiments.<br />July 23, 2011<br />
    45. 45. <ul><li>Experimental data [BCD08] [CDB06] [CDG07]</li></ul>Experimental Data<br />Table. 3: Characteristics of the data set in terms of domains and web pages.<br />Table. 4: Classification of the data set as Seed Set and Test Set.<br />July 23, 2011<br />
    46. 46. Experimental Measure<br />Table. 5: Description of the measures.<br />1False negatives are the number of domains incorrectly labeled as not belonging to the class (i.e., spam or non-spam).<br />July 23, 2011<br />
    47. 47. Comparison between Originaland Modified Algorithms (1/3)<br /><ul><li>Experiment 1: Comparison Between TR and MTR
    48. 48. MTR performs either comparable to or slightly better than TR in terms of both true positives and false positives.
    49. 49. We find cutoffTreffective till 100% mark indicating that after 100% detection becomes unstable in terms of false positives.</li></ul>  For later experiments, we fix the cutoffTrrange till 100%.<br /><ul><li>Experiment 2: Comparison Between ATR and MATR
    50. 50. MATR generally performs better than ATR in terms of true positives
    51. 51. We find cutoffATreffective till 180% mark indicating that after 100% detection becomes unstable in terms of false positives.</li></ul>  For later experiments, we fix the cutoffATr at 100% to ensure high precision.<br />July 23, 2011<br />
    52. 52. Comparison between Originaland Modified Algorithms (2/3)<br />Experiment 3: Comparison Between SM and MSM<br />MSM performs slightly better than SM in terms of true positives and comparable in terms of false positives<br />We find relativeMasseffective between the range of 0.95 to 0.99 in terms of maximizing true positives and minimizing false positives.<br />  For later experiments, we keep the range from 0.8 to 0.99 of relativeMass as effective range.<br />Experiment 4: Comparison Between LFS and MLFS<br />MLFS performs better than LFS in terms of false positives while at some expense of true positives.<br />We find limitBL and limitOL highly effective at 7 and 7 respectively in terms of minimizing many false positives.<br />  For later experiments, we keep limitBL = 7 and limitOL = 7.<br />July 23, 2011<br />
    53. 53. Comparison between Originaland Modified Algorithms (3/3)<br />Summary<br />We have found all modified algorithms providing better quality than the respective original algorithms.<br />We found SM as the best original web spam detection algorithms among ATR, SM, and LFS algorithms due to high true positives and relatively less false positives.<br />We also found MSM as the best modified web spam detection algorithms among MATR, MSM, and MLFS algorithms due to high true positives and relatively less false positives.<br />July 23, 2011<br />
    54. 54. The Best Succession for the Seed Refiner<br />Identical performance for both successions<br />Identical performance for both successions<br />Identical performance for both successions<br />Better performance for MATR-MTR compared toMTR-MATR<br />Table. 6: Comparison for the seed refiner.<br />Therefore, MATR-MTR is found to be the winner, and hence we select it as the seed refiner.<br />July 23, 2011<br />
    55. 55. The Best Successionfor the Spam Detector<br />Comparison<br />We pick 0.99 of relativeMass since false positives are minimum at this value compared to other values of relativeMass while true positives are almost comparable for all values of relativeMass.<br />We observe MLFS fails to detect considerable number of spam domains.<br />We obtain the precisions 0.86, 0.86, 0.93, and 0.87 for MLFS-MSM, MSM-MLFS, MLFS, and MSM respectively.<br />We obtain the recalls 0.80, 0.80, 0.33, and 0.76 for MLFS-MSM, MSM-MLFS, MLFS, and MSM respectively.<br />MLFS-MSM and MSM-MLFS are best and identical in performance, we choose MLFS-MSM as the best spam detector without loss of generality.<br />Fig. 12: Comparison for the spam detector.<br />July 23, 2011<br />
    56. 56. Comparison<br />We pick 0.99 of relativeMass since false positives are minimum at this value compared to other values of relativeMass while true positives are almost comparable for all values of relativeMass.<br />We observe MATR-MTR-MLFS-MSM finds more true positives and some more false positives.<br />We obtain the precisions 0.85, 0.86, and 0.86 for SM, MSM, and MATR-MTR-LFS-MSM respectively.<br />We obtain the recalls 0.64, 0.70, and 0.80 for SM, MSM, and MATR-MTR-LFS-MSM respectively.<br />Comparison among the Best Succession, theBest Known Algorithm and the Best Modified Algorithm<br />Fig. 13:Comparison among MATR-MTR-MLFS-MSM, SM, and MSM.<br />Therefore, MATR-MTR-MLFS-MSM is more effective.<br />July 23, 2011<br />
    57. 57. Conclusions<br />We have improved the quality of web spam filtering by using seed refinement<br />We have proposed modifications in four well-known web spam filtering algorithms.<br />We have proposed a strategy of succession of modified algorithms<br />Seed Refiner contains order of executions for seed refinement algorithms.<br />Spam Detector contains order of executions for spam detection algorithms.<br />We have conducted extensive experiments in order to show the effect of seed refinement on the quality of web spam filtering<br />We find that every modified algorithm performs better than the respective original algorithm.<br />We find the best performance among the successions by MATR followed by MTR, MLFS, and MSM (i.e., MATR-MTR-MSM). This succession outperforms the best original algorithm i.e., SM, by up to 1.25 times in recall and is comparable in terms of precision.<br />July 23, 2011<br />
    58. 58. QUESTIONS?<br />July 23, 2011<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×