Dotscale2013 : How to scale ?

1,960
-1

Published on

Quentin ADAM slides on dotScale, about How to scale ?

Published in: Technology

Dotscale2013 : How to scale ?

  1. 1. SCALING,WHAT YOUNEED TOKNOWQUENTIN ADAM @WAXZCECLEVER CLOUD CEO2013
  2. 2. WHEN YOUNEED TOSCALETHERE ARE 2 WAYS
  3. 3. GROWING AND GROWINGUNTIL YOU EXPLODE ORBECOME WEIRD
  4. 4. OR SPLIT THE WORK ANDMAKE YOUR SOFTWAREWORK AS A TEAM
  5. 5. Build an army of fat appYOU CAN DO BOTH
  6. 6. SO WE NEED TO BE ABLETO DISPATCH THE WORKSCALE OUT•  Many workersdoing the samething•  No SPOF•  Growing is moreeasy•  Introduce bestpracticeSCALE UP•  1 Fat instance•  1 Fat application•  SPOF (single pointof failure)•  Hard to maintain•  Always has a limit•  Short termmeaning
  7. 7. SO, HOW TOSCALEOUT ?JUST SOME FACTS
  8. 8. SPLIT PROCESS ANDSTORAGEStorage• Databases• Files• Sessions• Events• …Code• Can be replicated• Stateless• Process
  9. 9. Picking one instance or another doesn’t matterSTATELESSNESS IS THE KEY
  10. 10. CONSIDER MORETHINGS AS DATA•  User account•  Users data•  Files•  Sessions•  Events
  11. 11. CHOOSEDATASTOREWISELYYOU CAN SHOULD USE MANY DATASTORES
  12. 12. DATASTORE CHOICESARE DRIVEN BY USAGEMakedecisionsbased onneedsDo I needatomicity ofrequests ?Do I needconcurrentaccess ?Do I mostlyread orwrite ?Do I needrelational ?Do I needbig storagecapacity ?Do I needhighavailability ?
  13. 13. •  Not a big volume•  DB have to managedata TTL•  Data model : K/V•  Multiple writes at thesame time•  High availabilityI need to store sessionsQUICK EXAMPLE
  14. 14. •  Not a big volumeIt’s OK, PG can handlesmall quantity of data•  DB have to managedata TTLNo, I have to do itmanually•  Data model : K/VNo, PG is relational(mainly)•  Multiple writes at thesame timeNo, PG is Atomic•  High availabilityPG is awesome ;-) Useof PG bouncer orsimilar allow goodclusteringI need to store sessionsQUICK EXAMPLE
  15. 15. •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes Redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeNo, redis is pseudoAtomic (master/slave)•  High availabilityRedis is great, butcauterization is rude…I need to store sessionsQUICK EXAMPLE
  16. 16. •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeOK, this is possiblewith memcachedprotocol•  High availabilityClustering is built in, nodowntime JI need to store sessionsQUICK EXAMPLE
  17. 17. USE ONLINEDATABASE / BEREADY TO TESTIN JUST A FEWMINUTESNO NEED TO TRASH YOUR COMPUTER
  18. 18. DON’T BE THAT GUY
  19. 19. DO NOT USE ATECHNOLOGY BECAUSEYOU <3 IT OR BECAUSEIT’S HYPE : USE ITBECAUSE IT FITS YOURNEEDSBALANCE YOUR LEARNING CURVE WITHTHE TIME SAVED
  20. 20. DO NOT CREATE MONSTERS
  21. 21. COMMON MISTAKES
  22. 22. DO NOT USEMEMORY ASDATABASELIKE : SHARED / GLOBAL VARIABLE,CACHE “IN THE CODE”, INTENSIVESESSION USAGE…
  23. 23. DO NOT USE A VARIABLEFOR MORE THAN ONEREQUEST
  24. 24. 2 + 2 = 4FOR SAME INPUT, SAME OUTPUT
  25. 25. And data will be lostCODE WILL FAIL
  26. 26. DO NOT USE FILESYSTEM AS DATASTOREFile system are POSIX compliant•  POSIX is ACID•  POSIX is powerful but is bottleneck•  File System is nightmare of ops•  File System is create coupling (host provider/OS/language)•  Free SPOF multi tenant File System is a unicornSTORE IN DATABASE, OR DATASTORE LIKE S3 (AWS)DEDICATED TO FILE MANAGEMENT
  27. 27. CAREFUL USE OFDARK MAGIC
  28. 28. SPLIT THE CODE :MODULES•  Smallest codebase•  Deploy asservice foreach other•  Focus on besttechnology fora problem
  29. 29. USE EVENT BROKER TOMODULARIZE YOUR APP•  AMQP•  Celery•  0MQ•  Redis•  JMS•  Some case : hadoop, akka…•  …CRON is not an event queue
  30. 30. MAKE HARDCOMPUTATION ASYNC
  31. 31. ALWAYS USE AREVERSE PROXYY U NOT USE ONE ?
  32. 32. USE PROCESSDEPLOYMENT
  33. 33. KEEP CALM UNDERFIRE
  34. 34. TRACK BUG & GETMETRICS
  35. 35. Quentin ADAMTwitter : @waxzceTHX FOR LISTENING& QUESTIONS TIMEcleverc l o u d

×