Roll your own web crawler. RubyDay 2013
- 4,553 views
It is all about data. …
It is all about data.
Having the right data at the right time might make the difference between you and your competitor. Google can show you just what it can catch. If you know where to find the data of your interest, let's go deeper and roll your own web crawler framework.
Taking the advantage of the latest cool technologies I will show you how to build your distributed web crawler based on Redis and Mongo