Ivo jansch - the art of scalability

698 views

Published on

Tweakers.net Developer Summit - presentatie Ivo Jansch - The art of scalability.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
698
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ivo jansch - the art of scalability

  1. 1. http://www.egeniq.com info@egeniq.com @egeniq The Art of Scalability Ivo Jansch / @ijanschTweakers.net Developer Summit, March 24 2011
  2. 2. What is Scalability? “[...] Scalability is the ability of a system, network, or process, to handle growing amounts of work in a graceful manner or its ability to be enlarged to accommodate that growth.” - Wikipedia 2
  3. 3. Different levels of scale Pageviews/ Site # Servers Views / server month Facebook 200B 30K 6.6M Netlog 4B 600 6.6M Hyves 6.5B 3000 2.1M Tweakers 60M 15 4M Nu.nl 340M 14 24M (See end of slide deck for sources) 3
  4. 4. Some of the tools at Netlog‣ Content Delivery Network • http://www.akamai.com/‣ Search Engine • http://sphinxsearch.com/ • (Also have a look at http://lucene.apache.org/solr/)‣ Database Replication 4
  5. 5. Some of the tools at Hyves‣ Caching • http://memcached.org/‣ Lightweight webserver • http://nginx.org/‣ Automated deployment • http://puppetlabs.com/ 5
  6. 6. Some of the tools at Nu.nl‣ Content pre-generation‣ Varnish Edge Side Includes • http://www.varnish-software.com/‣ Distributed Job Queues • http://gearman.org/ ** nu.nl uses a custom job queue but this is one I can recommend 6
  7. 7. Some of the tools at Facebook‣ PHP -> C++ compiler • https://github.com/facebook/hiphop-php‣ Distributed Logging • https://github.com/facebook/scribe‣ NoSQL Storage • http://cassandra.apache.org/ 7
  8. 8. How can you scale? Start with the basics 8
  9. 9. Scaling with Hardware If you run your app on multiple servers today, would it work?Typical problem areas:‣ Sessions‣ Files 9
  10. 10. Elasticity 10
  11. 11. A scalable funnel Load balancer Web Web Web Web Web Server Server Server Server Server Load balancer App App App Server Server Server Load balancer Database Services 11
  12. 12. Logical/Functional partitioning 12
  13. 13. Scalable Architectures‣ Service Oriented Architecture‣ Component Based Design‣ Abstraction 13
  14. 14. Scalability in Software‣ Caching‣ Graceful Degradation 14
  15. 15. Food for Thought Scaling Wisely 15
  16. 16. Food for thought Remember, this talk is called: “The Art of Scalability” 16
  17. 17. This is Art “House of Stairs” M.C. Escher 17
  18. 18. But this is Art, too “Composition with Blue” Piet Mondriaan 18
  19. 19. Moral of this story Scalability is about the ability to scale. Don’t scale prematurely. Keep it simple. 19
  20. 20. “You’re doing it wrong!”“Scalability is the ability of a system, network, or process, to handle growing large amountsof work in a graceful manner or its ability to be enlarged to accommodate that growth.” 20
  21. 21. http://www.egeniq.com info@egeniq.com @egeniqThank you! Questions? http://www.egeniq.com ivo@egeniq.com @ijansch
  22. 22. SourcesThe slide with servers and pageviews was based on:‣ http://cns.ucsd.edu/lecturearchive09.shtml#Roth (Facebook)‣ http://technologie.hyves.nl/ (Hyves)‣ http://www.slideshare.net/folke/netlog-what-we-learned-about-scalability-high- availability-430211 (Netlog)‣ http://www.slideshare.net/peter_ibuildings/surviving-a-plane-crash (Nu.nl)‣ http://tweakers.net/reviews/331/5/tweakers-punt-net-faq-plans-stats-servers-en-site- software.html (Tweakers)
  23. 23. CreditsThe following Creative Commons pictures were used in this presentation: ‣ ‘Datacenter Work’ by Leonardo Rizzi - http://www.flickr.com/photos/stars6/4381851322/ ‣ ‘Rubber Band Man’ by Abe Novy - http://flickr.com/photos/thenovys/3791884189/ ‣ ‘You’re doing it wrong’ by Adam Swank - http://www.flickr.com/photos/adeepbreath/3952587062/

×