AideRSS - AWS The Start-Up Project
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

AideRSS - AWS The Start-Up Project

on

  • 7,321 views

AideRSS presents at Amazon Web Services' The Start-Up Project - Boston

AideRSS presents at Amazon Web Services' The Start-Up Project - Boston

Statistics

Views

Total Views
7,321
Views on SlideShare
7,200
Embed Views
121

Actions

Likes
4
Downloads
136
Comments
0

5 Embeds 121

http://aws.typepad.com 69
http://s3.amazonaws.com 43
http://www.slideshare.net 6
http://www.typepad.com 2
http://feeds.feedburner.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

AideRSS - AWS The Start-Up Project Presentation Transcript

  • 1.
    • A Company and Technology Overview
    Read What Matters
  • 2. What is RSS ?
    • History
    • First standard published in 1997, very quickly picked up by Netscape
    • In 1999, BBC introduces RSS to millions in a form of a ‘news ticker toolbar’
    • Since then, hundreds of millions of blogs, newspapers, magazines, and other content distributors have published their own RSS feeds:
    RSS news feeds allow you to see when websites have added new content. You can get the latest headlines in one place , as soon as its published , without having to visit the websites you have taken the feed from. 2003 2007
  • 3. The Problem - Information Overload
  • 4. Read what matters
  • 5. PostRank™ Filtering
  • 6. PostRank™ Filtering
  • 7. Read How You Want
  • 8. Launch day @ 00:00 - 10 EC2 instances - 5,000 most popular RSS feeds - ~20,000 new stories a day Launch day @ 12:00 - 30 EC2 instances - 12,000 RSS feeds - 85,000 new stories Launch day @ 24:00 - 100 EC2 instances - 35,000 RSS feeds - 440,000 new stories Today - 20-30 EC2 instances - 70,000 RSS feeds - 200,000-400,000 new stories / day Aide RSS Launch Stats
  • 9. Startup math Amazon AWS Platform Development: Testing cluster (on-demand) + 5 CPUs / 5 months + 250mb/s uplink (Free!) + S3 for messaging ------------------------------- $180 total (1,400% savings!) Production: Dynamic cluster for updates + 20-100 CPUs + 250mb/s uplink (Free!) + SQS for messaging ------------------------------- less than $5,000/ month (> 300% savings / month!) Dedicated Route Development: Testing cluster (on-demand) + 5 CPUs / 5 months + 250mb/s uplink ( Scary! ) + Own cloud service ----------------------------- (5 servers x $100) x 5 Bandwidth + Cloud ------------------------------ at least $2,500 / month Production: Dynamic cluster for updates + 20-100 CPUs + 250mb/s uplink (Free!) + SQS-alike for messaging ------------------------------- (20-100) x $150 / month Bandwidth + Messaging ------------------------------- greater than $15,000 / month
  • 10. Our Architecture (1) Aide RSS Database S3 EC2 Scheduler
  • 11. Our Architecture (2) Aide RSS Database SQS EC2 SQS Migrating to SQS (Distributed Queue) - 256kb per message (max) - Serialize & compress - August: 500,000 messages - September: 1,000,000+ messages Budget: $150 Yesterday - 69,856 RSS feeds - 245,422 new stories - 20 EC2 instances (avg.) - Dynamic cluster scaling - 2 SQS queues Next - More optimizations! - Clustering - Recommendations - Keyword filtering - and much more… Scheduler
  • 12. Cluster monitoring
  • 13. Ilya Grigorik Francis Lau Kevin Thomason [email_address] [email_address] [email_address] “ Read What Matters” Questions? We are looking for partnerships. http://www.aiderss.com