Dotscale2013 : How to scale ?

  • 1,560 views
Uploaded on

Quentin ADAM slides on dotScale, about How to scale ?

Quentin ADAM slides on dotScale, about How to scale ?

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,560
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
0
Comments
0
Likes
6

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. SCALING,WHAT YOUNEED TOKNOWQUENTIN ADAM @WAXZCECLEVER CLOUD CEO2013
  • 2. WHEN YOUNEED TOSCALETHERE ARE 2 WAYS
  • 3. GROWING AND GROWINGUNTIL YOU EXPLODE ORBECOME WEIRD
  • 4. OR SPLIT THE WORK ANDMAKE YOUR SOFTWAREWORK AS A TEAM
  • 5. Build an army of fat appYOU CAN DO BOTH
  • 6. SO WE NEED TO BE ABLETO DISPATCH THE WORKSCALE OUT•  Many workersdoing the samething•  No SPOF•  Growing is moreeasy•  Introduce bestpracticeSCALE UP•  1 Fat instance•  1 Fat application•  SPOF (single pointof failure)•  Hard to maintain•  Always has a limit•  Short termmeaning
  • 7. SO, HOW TOSCALEOUT ?JUST SOME FACTS
  • 8. SPLIT PROCESS ANDSTORAGEStorage• Databases• Files• Sessions• Events• …Code• Can be replicated• Stateless• Process
  • 9. Picking one instance or another doesn’t matterSTATELESSNESS IS THE KEY
  • 10. CONSIDER MORETHINGS AS DATA•  User account•  Users data•  Files•  Sessions•  Events
  • 11. CHOOSEDATASTOREWISELYYOU CAN SHOULD USE MANY DATASTORES
  • 12. DATASTORE CHOICESARE DRIVEN BY USAGEMakedecisionsbased onneedsDo I needatomicity ofrequests ?Do I needconcurrentaccess ?Do I mostlyread orwrite ?Do I needrelational ?Do I needbig storagecapacity ?Do I needhighavailability ?
  • 13. •  Not a big volume•  DB have to managedata TTL•  Data model : K/V•  Multiple writes at thesame time•  High availabilityI need to store sessionsQUICK EXAMPLE
  • 14. •  Not a big volumeIt’s OK, PG can handlesmall quantity of data•  DB have to managedata TTLNo, I have to do itmanually•  Data model : K/VNo, PG is relational(mainly)•  Multiple writes at thesame timeNo, PG is Atomic•  High availabilityPG is awesome ;-) Useof PG bouncer orsimilar allow goodclusteringI need to store sessionsQUICK EXAMPLE
  • 15. •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes Redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeNo, redis is pseudoAtomic (master/slave)•  High availabilityRedis is great, butcauterization is rude…I need to store sessionsQUICK EXAMPLE
  • 16. •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeOK, this is possiblewith memcachedprotocol•  High availabilityClustering is built in, nodowntime JI need to store sessionsQUICK EXAMPLE
  • 17. USE ONLINEDATABASE / BEREADY TO TESTIN JUST A FEWMINUTESNO NEED TO TRASH YOUR COMPUTER
  • 18. DON’T BE THAT GUY
  • 19. DO NOT USE ATECHNOLOGY BECAUSEYOU <3 IT OR BECAUSEIT’S HYPE : USE ITBECAUSE IT FITS YOURNEEDSBALANCE YOUR LEARNING CURVE WITHTHE TIME SAVED
  • 20. DO NOT CREATE MONSTERS
  • 21. COMMON MISTAKES
  • 22. DO NOT USEMEMORY ASDATABASELIKE : SHARED / GLOBAL VARIABLE,CACHE “IN THE CODE”, INTENSIVESESSION USAGE…
  • 23. DO NOT USE A VARIABLEFOR MORE THAN ONEREQUEST
  • 24. 2 + 2 = 4FOR SAME INPUT, SAME OUTPUT
  • 25. And data will be lostCODE WILL FAIL
  • 26. DO NOT USE FILESYSTEM AS DATASTOREFile system are POSIX compliant•  POSIX is ACID•  POSIX is powerful but is bottleneck•  File System is nightmare of ops•  File System is create coupling (host provider/OS/language)•  Free SPOF multi tenant File System is a unicornSTORE IN DATABASE, OR DATASTORE LIKE S3 (AWS)DEDICATED TO FILE MANAGEMENT
  • 27. CAREFUL USE OFDARK MAGIC
  • 28. SPLIT THE CODE :MODULES•  Smallest codebase•  Deploy asservice foreach other•  Focus on besttechnology fora problem
  • 29. USE EVENT BROKER TOMODULARIZE YOUR APP•  AMQP•  Celery•  0MQ•  Redis•  JMS•  Some case : hadoop, akka…•  …CRON is not an event queue
  • 30. MAKE HARDCOMPUTATION ASYNC
  • 31. ALWAYS USE AREVERSE PROXYY U NOT USE ONE ?
  • 32. USE PROCESSDEPLOYMENT
  • 33. KEEP CALM UNDERFIRE
  • 34. TRACK BUG & GETMETRICS
  • 35. Quentin ADAMTwitter : @waxzceTHX FOR LISTENING& QUESTIONS TIMEcleverc l o u d