Survey: Adversarial IR on the Web

Carlos Castillo <chato@yahoo-inc.com> Yahoo! Research · Barcelona Adversarial IR on the Web Thanks to: Brian Davison, Lehigh University

What's on the Web?

And on the Web 2.0?

What else?

milliondollarhomepage.com

Two webs

Closed web
Open web

Terrence Brooks http://Informationr.net/ir/8-3/paper154.html

What is web spam Web spam detection Content analysis Links and trust Usage analysis Social media sites Conclusions

Everything for a click

Origin of “spam” http://www.nowpublic.com/culture/monty-python-spam-skit http://www.flickr.com/photos/42162585@N00/262091025

How much spam? Ntoulas et al. WWW 2006

Search engine spam “web pages that hold no informational value, but are created to lure web searchers to sites they would otherwise not visit” Fetterly et al. WebDB 2004

Spam results are still present

3.3M results for “porn mortgage” in S1
1.4M results for “free mp3 hilton viagra” in S2
2.5M results for “credit vicodin loan” in S3

May 2009

Who?

Activists
Marketers
Optimizers
Spammers

Why?

Activists
Marketers – for money
Optimizers – for money
Spammers – for money

Costs

Users
Search engines
Publishers

Features Web Pages Training Labels Learning Machine Learning System 0.3 0.9 1.7 4.5 3.2 0.0 0.3 0.9 1.7 4.5 3.2 0.0 0.3 0.9 1.7 4.5 3.2 0.0

Supervised learning

Obtain training data
Extract features

Index-time features: contents, links
Crawl-time, rank-time features
Usage features

Choose a learning scheme

Non-graphical / graphical

Evaluate

Training data: do editors agree? Sometimes ( k =0.56) Castillo et al. http://www.acm.org/sigs/sigir/forum/2006D-TOC.html

Features Training set Learning process 0.3 0.9 1.7 4.5 3.2 0.0 0.3 0.9 1.7 4.5 3.2 0.0 0.3 0.9 1.7 4.5 3.2 0.0

(e.g.) decision trees x <= a x > a 60 pages= 50 spam + 10 normal 100 pages 40 pages 35 pages= 34 normal + 1 spam 5 pages= 4 spam + 1 normal y <= b y > b

Graphical learning

Nodes are not isolated
N->S is unlikely

Castillo et al. SIGIR 2007

Average spam fraction of in-links Castillo et al. SIGIR 2007

Average spam fraction of out-links Castillo et al. SIGIR 2007

Weight out-links a lot, in-links a little Abernethy, Chapelle and Castillo, AIRWeb 2008

Evaluation

Point-wise measures: TPR, FPR, F1
Most schemes output a family of classifiers
Area under ROC

ROC curve http://wwwiti.cs.uni-magdeburg.de/~sschimke/sose05/15-platasign/Evaluation_of_Biometric_Systems.html False negative rate False positive rate

Content spam

Why? query-document matching
How? repeat / weave /stitch / copy
Which features are useful?

URL/title length, document size, compression rate, fraction of popular terms

Methods

Document classification
Language model disagreement
Coding style similarity
Near-duplicate detection (for plagiarized content)

Cloaking different contents at the same URL User Search engine results page Click on the result Search Engine Normal document Buy viagra now!

Redirection redirect hidden from search engine User Search engine results page Click on the result redirect Search Engine Normal document Buy viagra now!

Link farms

Why? Pagerank/HITS and other link analysis methods
Sybil attacks/mutual admiration societies

Detection methods

Finding dense sub-graphs
Finding anomalous linking patterns
Finding abnormal link change rates

Finding dense sub-graphs is hard http://www.flickr.com/photos/docco/771439289/

Demotion methods

De-duping Sybils
Down-weighting “unqualified” links

Propagation of trust and distrust “ Trusted” Nodes Suspicious Gyongyi, Garcia-Molina and Pedersen, VLDB 2004

Advertising on the Web http://www.flickr.com/photos/66606673@N00/1308955915

Click fraud

Why? For (many) pennies
Two types

Affiliate click fraud (publisher clicks on his/her ads)
Competitive click fraud (click on ads of competition)

http://www.jameshyman.com/blog/archives/2006_01.html

Search log spam

Why? Spammers will try anything

Manipulate search suggestion, expected click rates, etc.

Automated search traffic Buehrer, Stokes and Chellapilla, AIRWeb 2008

Usage data can help fight spam

Many spammers can be easily spotted
Using search logs

Abnormal fraction of popular terms

Using toolbar logs

Abnormal dwell time or bounce rate
Abnormal fraction of search-engine visits

http://www.maxpower.ca/wp-content/uploads/2006/10/bad_splog.png

Splog update regularities Normal blog Power blog Splog Lin et al. AIRWeb 2007

Publicly-writeable pages

Wikis

Spam/vandalism

Blog comments

Comment spam http://hojtsy.hu/blog/2008-apr-01/thanks-mollom-protecting-blog-spam

Social network spam

Fake content

“suicide videos in Youtube”
Opinion/review spam
Self promotion (where is the limit?)

Tagging/voting spam

Fake reviews http://www.dilbert.com/

SEO's perspective “[I]f search engines do not flag [some spam] pages today, some day they will. Beyond search engine smarts, overoptimized pages also leave you vulnerable to being reported by your competitors to search engines for spamming – causing a human editor to look check the page and possibly ban your sites” Moran and Hunt “Search Engine Marketing, Inc” 2006

Search engine's perspective “Victory does not require perfection, just a rate of detection that alters the economic balance for the would-be spammer” Ntoulas et al., WWW 2006

Main research topic

How to provide the maximum freedom possible to users, while keeping the systems open and generative?
Systems that allow cooperative users to defeat non-cooperative users easily

Carlos Castillo · Yahoo! Research · Barcelona buy vicodin xanax viagra online generic cheap tramadol prevacid triamcinolone mesothiloma asbestos home insurance weight loss valium mortgage mp3 ripper music free Thank you!