Your SlideShare is downloading. ×
Scaling small apps
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Scaling small apps

462
views

Published on

In March 2010, the UK had an election with a televised leadership debate. We built an application that scaled to 2.7 million hits in 90 minutes on top of AppEngine which we built in about 3 days. …

In March 2010, the UK had an election with a televised leadership debate. We built an application that scaled to 2.7 million hits in 90 minutes on top of AppEngine which we built in about 3 days. See the mistakes we made and what went well.

Published in: Technology, Business

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
462
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • Scalability on journalistic deadlines.\nOn Thursday 15th April 2010, there was an historic occasion, the UK’s first televised party leader debate for an election.\nAs a news organisation we wanted to not only report the news and information about the debate, but also get feedback from our readers as to what they thought of the debate.\n
  • Our coverage on the website looked something like this. This was the picture that we put on the main page to encourage people to view our live blog.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. Michael Brunton-Spall Developer Advocate guardian.co.uk @bruntonspall
    • 2. Scaling small apps on AppEngine
    • 3. Our coverage  
    • 4. Poll Charts• "Let people viewing the page vote at anytime whether they like or dislike what the party leader is saying.  Oh, and lets show it with a real time graph"• Bad words here • anytime • real-time
    • 5. The Design
    • 6. The reality
    • 7. Timeline• Thursday 8th April 2010 - Project Start• Thursday 15th April 2010 - Debate 1 - Go Live• Thursday 22th April 2010 - Debate 2• Thursday 29th April 2010 - Final Debate
    • 8. Countersget(self):    return self.val
    • 9. Countersincr(self, val):    self.val += val
    • 10. Locks are bad, Mkay!• transaction write locks• all threads are serialised
    • 11. Sharded Countersincr(self, val=1):    c = randomcounter c.val += val
    • 12. Sharded Countersget(self):    return sum(counters)
    • 13. Shards are better, Mkay! • writes lock only 1 shard at a time • n simultaneous writes
    • 14. On the first night
    • 15. The poll itself
    • 16. Requests Per Second
    • 17. In Detail
    • 18. Numbers• Average: 100-120 req/s• Max: 400 req/s• Total requests: 1,000,000 in 90 mins• Little cheating - 2000 requests
    • 19. Request Duration
    • 20. Request Duration• 1 to 8 seconds• Thread contention• Not enough shards?
    • 21. Not enough shards?• Increase number of shards? • by factor of 10? • No transaction failures • Each request still takes 200ms
    • 22. 200ms per incr?• Datastore write• requires disk access
    • 23. Memcached?• Change architecture • atomic increment/decrement • get from memcache • cronjob 1/min • memcache => datastore • votes now take 20 ms each
    • 24. The Flu
    • 25. The second night
    • 26. The Results?
    • 27. The Results?
    • 28. Some notes• Total of around 2,727,000 requests• Average of around 454 req/s• Peaked at 750 req/s 
    • 29. Requests per Second
    • 30. Request Duration
    • 31. The problems
    • 32. Errors / Second
    • 33. Google AppEngine• Request limit: • x requests per day • But also... • y requests per minute • approx 1/3 reqs refused
    • 34. Guardian• Serving up json via guardian domain • Heavier process than we thought • cronjob to upload to cdn 1/min
    • 35. Request Duration
    • 36. The solution!
    • 37. The third night
    • 38. The results
    • 39. Requests per second
    • 40. Some notes• Total of around 750,000 requests• Average of around 138 req/s• Peaked at around 230 req/s 
    • 41. £££ (or $$$)• Appengine Cost: $Negligable• plus cdn bandwidth• plus dev costs• etc...
    • 42. Any Questions? Michael Brunton-Spall @bruntonspallmichael.brunton-spall@guardian.co.uk