Transcript of "Building a Website to Scale to 100 Million Page Views Per Day and Beyond "
Building a Website To Scale Target: 200 Million page views per day and beyond! By Eric Pickup email@example.com Twitter: EricPickupYPwww.ManwinJobs.com
Contents 1. The Context 2. The Requirements 3. The Architecture 4. The Good and the Bad www.ManwinJobs.com
What are we talking about? Aug Apr Dec Feb Apr 2006 2007 2007 2008 2011 YP First 1 Million 100,000 100 million Acquired Launched daily Uploads daily by Manwin visitors page views www.ManwinJobs.com
Traffic In Perspective Source: Alexa.com Alexa global rank 95 100 Gb/s – 3 full DVDs streamed every single second www.ManwinJobs.com
The Context Written in PERL with a very complex architecture First few months dedicated to learning the site, maintain it, and plan the re-write. Re-write started in August 2011 and was originally planned for a delivery in mid-November. Actually launched at the end of January. www.ManwinJobs.com
The requirements 1 Support 200 million+ daily requests 2 100% transparent to users 3 Six years of legacy data 4 Even faster site www.ManwinJobs.com
The Architecture Ability to manage pools of servers with health checks. We maintain 2 pools: Write pool with fail-over to backup-Master. Read pool with all servers except Master. www.ManwinJobs.com
The Architecture Very normalized database since not used directly for site. Some tables have over 100 million rows. Used to populate Redis lists for new features www.ManwinJobs.com
The good and the bad Main reasons for the delays: Decisions concerning some of the technologies to use. Learning curve for new technologies longer than expected. Data transfer and restructuring in MySQL and Redis Staffing issues. www.ManwinJobs.com
The good and the bad Was it a success? Launch without any downtime New site about 10% faster Valuable expertise gained A GOOD SUCCESS STORY WITH LESSONS LEARNED www.ManwinJobs.com
Eric Pickup firstname.lastname@example.org Twitter: EricPickupYPwww.ManwinJobs.com
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.