Building Scalable Web Apps - LVL.UP KL


Published on

This is my talk from the July LVL.UP KL meeting (formerly WebCamp KL) held on August 6th at Mindvalley, Bangsar.

The talk covers a basic introduction to scalability, 5 things to consider/think about and 5 things you can do build at scale.

WebCampKL Group is here -

The video of this talk is available here: (also added as the 19th slide).

Published in: Internet
  • Be the first to comment

Building Scalable Web Apps - LVL.UP KL

  1. 1. Building Scalable Web Apps   @ShaolinTiger on Twitter
  2. 2. So who am I? Mindvalley CTO etc. - Also Sys Admin - DevOps Guy - Technical Architect
  3. 3. Pulse Check - Familiar with at least 1 of these technologies? - Setup/used one of these technologies? - Could debate the pros of nginx over Apache?
  4. 4. This Talk - Intro to scaling - 5 Points to consider - 5 Things you can do
  5. 5. Scalability vs Performance - Scalability can bring performance, but not vice-versa “Performance measures the speed with which a single request can be executed, while scalability measures the ability of a request to maintain its performance under increasing load.” – Steven Haines
  6. 6. Understand Scaling Scaling up (Vertical) Scaling out (Horizontal)
  7. 7. Scaling up vs Scaling up Scaling up - Can become very costly - Single point of failure (SPOF) - Less complex to manage - Has a Finite limit Scaling out - Cheaper now with cloud/vms/commodity h/w - Increases availability - Management can become very complex - Deployment can become slow
  8. 8. Points to consider 1) Logging - Logging can get OUT OF CONTROL - Use a centralized log management system - Check out Loggly or Papertrail (we use this)
  9. 9. Points to consider 2) User Sessions - Centralize your user sessions - Use a fast in memory store - Remember to invalidate/clear cache - Check out Redis (or just use Memcache if you already have it as part of your architecture)
  10. 10. Points to consider 3) Monitoring - Monitor EVERYTHING - Make data driven decisions - Check out Munin (my favourite)
  11. 11. Points to consider 4) Bottlenecks - Usually the database/datastore - Can also possibly be the filesystem (caused by logging, heavy static file delivery etc) - Also possible to saturate network bandwidth
  12. 12. Points to consider 5) Traffic Profile - Spiky traffic is a lot harder to deal with than linear growth (you have to build differently) - Even tools that ‘autoscale’ take time
  13. 13. Things you can do 1) CACHE! - You should really by trying to cache everything you can: - - Objects - - User Sessions - - API Requests - - Page Cache - - Partials Caching (In Rails)
  14. 14. Things you can do 2) Understand - Know your tools/stack (or hire someone who does) - AKA MySQL query_cache? What’s that? - Optimize but don’t overdo it (and not too early) - Build for scale, but don’t get complex without reason
  15. 15. Things you can do 3) Use Queues - There are great tools out there for this (RabbitMQ etc) - Makes it much less necessary to scale your database - Takes the burden of spiky loads without downtime
  16. 16. Things you can do 4) Ditch Search - This is what happens when you do a full text search on a database not designed for it. - Use a specialised data store for search: - Solr - Lucene - Elasticsearch
  17. 17. Things you can do 5) Leverage - Use a CDN (Like OnApp, CloudFront, Edgecast etc) - If you can use a specialist host (WPengine/ - Utilise a specialist DNS host (Route 53/UltraDNS) - Profile your app with NewRelic
  18. 18. The END! No Questions! Stalk me @ShaolinTiger This preso will be on