A Company and Technology Overview Read What Matters

A Company and Technology Overview

What is RSS ? History First standard published in 1997, very quickly picked up by Netscape In 1999, BBC introduces RSS to millions in a form of a ‘news ticker toolbar’ Since then, hundreds of millions of blogs, newspapers, magazines, and other content distributors have published their own RSS feeds: RSS news feeds allow you to see when websites have added new content. You can get the latest headlines in one place , as soon as its published , without having to visit the websites you have taken the feed from. 2003 2007

History

First standard published in 1997, very quickly picked up by Netscape

In 1999, BBC introduces RSS to millions in a form of a ‘news ticker toolbar’

Since then, hundreds of millions of blogs, newspapers, magazines, and other content distributors have published their own RSS feeds:

The Problem - Information Overload

Read what matters

PostRank™ Filtering

PostRank™ Filtering

Read How You Want

Launch day @ 00:00 - 10 EC2 instances - 5,000 most popular RSS feeds - ~20,000 new stories a day Launch day @ 12:00 - 30 EC2 instances - 12,000 RSS feeds - 85,000 new stories Launch day @ 24:00 - 100 EC2 instances - 35,000 RSS feeds - 440,000 new stories Today - 20-30 EC2 instances - 70,000 RSS feeds - 200,000-400,000 new stories / day Aide RSS Launch Stats

Startup math Amazon AWS Platform Development: Testing cluster (on-demand) + 5 CPUs / 5 months + 250mb/s uplink (Free!) + S3 for messaging ------------------------------- $180 total (1,400% savings!) Production: Dynamic cluster for updates + 20-100 CPUs + 250mb/s uplink (Free!) + SQS for messaging ------------------------------- less than $5,000/ month (> 300% savings / month!) Dedicated Route Development: Testing cluster (on-demand) + 5 CPUs / 5 months + 250mb/s uplink ( Scary! ) + Own cloud service ----------------------------- (5 servers x $100) x 5 Bandwidth + Cloud ------------------------------ at least $2,500 / month Production: Dynamic cluster for updates + 20-100 CPUs + 250mb/s uplink (Free!) + SQS-alike for messaging ------------------------------- (20-100) x $150 / month Bandwidth + Messaging ------------------------------- greater than $15,000 / month

Our Architecture (1) Aide RSS Database S3 EC2 Scheduler

Our Architecture (2) Aide RSS Database SQS EC2 SQS Migrating to SQS (Distributed Queue) - 256kb per message (max) - Serialize & compress - August: 500,000 messages - September: 1,000,000+ messages Budget: $150 Yesterday - 69,856 RSS feeds - 245,422 new stories - 20 EC2 instances (avg.) - Dynamic cluster scaling - 2 SQS queues Next - More optimizations! - Clustering - Recommendations - Keyword filtering - and much more… Scheduler

Cluster monitoring

Ilya Grigorik Francis Lau Kevin Thomason [email_address] [email_address] [email_address] “ Read What Matters” Questions? We are looking for partnerships. http://www.aiderss.com