Crawling the web for fun and profit

Crawling the Web (for fun and profit) Federico Feroldi

“A Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner.” Wikipedia Picture greetings to photoholic1 --LennyB

Search engines only show you what their crawlers can catch Picture greetings to jimbrickett

The deep web contains a lot of valuable information e-commerce finance transportation yellow pages medicine government opinions real estate personal intranets social Picture greetings to tricky ™

Dig deeper with your own crawler Picture greetings to Super*Junk

Information = Competitive Advantage Picture greetings to mastrobiggo

B a cku p h i s t o r i c a l data: web sites, blogs

Social network analysis: find influencers and interests based on “social circles”

Find what people like

Sentiment analysis: find what people say about your brand or product

http://www.cnblogs.com	182
http://cloudify.me	67
http://www.slideshare.net	15
http://www.lmodules.com	7
http://cnblogs.com	2

Crawling the web for fun and profit

by Federico Feroldi on May 07, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 273

Statistics

Crawling the web for fun and profit — Presentation Transcript