Your SlideShare is downloading. ×
0
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Scaling the Britain's Got Talent Buzzer
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Scaling the Britain's Got Talent Buzzer

2,187

Published on

How Live Talkback scaled the Britain's Got Talent buzzer to support 50,000 requests/second

How Live Talkback scaled the Britain's Got Talent buzzer to support 50,000 requests/second

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,187
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
15
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. 1 Powering the Britain’s Got Talent buzzer* *And Big Data Big Data Meetup, London 25/5/2011Thursday, 26 May 2011 1
  • 2. 2 What we doThursday, 26 May 2011 2
  • 3. 3 Me Malcolm Box, Co-founder & CTO boxm@livetalkback.com @malcolmboxThursday, 26 May 2011 3
  • 4. 4 The Buzzer BIG DATAThursday, 26 May 2011 4
  • 5. 5 The challenge 10 Million+ viewers Design goal of 50,000 requests/s, 10,000 buzzes/second Equivalent to 130 Billion requests/month But just on Saturday night And four weeks to buildThursday, 26 May 2011 5
  • 6. 6 The challenge Where does 130 Billion requests fit? Source: http://www.google.com/adplanner/static/top1000/#Thursday, 26 May 2011 6
  • 7. 7 Where we started.... app.livetalkback.com cdn.livetalkback.com Control plane ELB CloudFront Zabbix Webserver Webserver Django Django Ubuntu Ubuntu MySQL S3Thursday, 26 May 2011 7
  • 8. 8 Step 1: Testing Started with a platform with a previous peak of 100 requests/s No idea where it would break Tsung! http://tsung.erlang-projects.org/Thursday, 26 May 2011 8
  • 9. 9 Step 2: ELB Amazon Elastic Load Balancer “Infinite capacity” BUT very long impulse response and NO controls :( HAProxy to the rescue 5K requests/s per nodeThursday, 26 May 2011 9
  • 10. 10 Step 3: Avoid the DB MySQL was never going to be able to handle 10,000 writes/s, nor 50,000 reads “Hey, Django does memcached. Problem solved” Help, our memcached server I/O is maxed out :( Two-layer cache: https://gist.github.com/953524 Write-behind dataThursday, 26 May 2011 10
  • 11. 11 But we want analytics! Now 10K things to write to disk every second Logging? Database? This is starting to look like BIG DATAThursday, 26 May 2011 11
  • 12. 12 Step 4: BabyThursday, 26 May 2011 12
  • 13. 13 Step 5: Cassandra Deployed Cassandra cluster on EC2 to handle buzz records Tested to > 10K writes/s All good! “So how many users did we have last night?”Thursday, 26 May 2011 13
  • 14. 14 Where we ended... app.livetalkback.com cdn.livetalkback.com 10 Control plane HAProxy HAProxy CloudFront nodes Chef Webserver Webserver 100+ nodes Django Django Ubuntu Ubuntu Zabbix Memcached Cassandra Memcached Cassandra RDS Master S3Thursday, 26 May 2011 14
  • 15. 15 Scaling up - and down Configuring 100+ servers by hand each week would have been a pain Used to Chef to automate Also builds the test swarm http://wiki.opscode.com/display/ chef/HomeThursday, 26 May 2011 15
  • 16. 16 Now what? Still challenges with analytics & ad-hoc queries Looking at Brisk and Hadoop We’re sucking the Twitter firehose for Tellybug MySQL is coping so far, but only justThursday, 26 May 2011 16
  • 17. 17 Questions? boxm@livetalkback.com @malcolmboxThursday, 26 May 2011 17

×