• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Dotscale2013 : How to scale ?
 

Dotscale2013 : How to scale ?

on

  • 1,660 views

Quentin ADAM slides on dotScale, about How to scale ?

Quentin ADAM slides on dotScale, about How to scale ?

Statistics

Views

Total Views
1,660
Views on SlideShare
1,596
Embed Views
64

Actions

Likes
6
Downloads
0
Comments
0

4 Embeds 64

http://www.waxzce.org 50
https://twitter.com 9
http://localhost 4
https://web.tweetdeck.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Dotscale2013 : How to scale ? Dotscale2013 : How to scale ? Presentation Transcript

    • SCALING,WHAT YOUNEED TOKNOWQUENTIN ADAM @WAXZCECLEVER CLOUD CEO2013
    • WHEN YOUNEED TOSCALETHERE ARE 2 WAYS
    • GROWING AND GROWINGUNTIL YOU EXPLODE ORBECOME WEIRD
    • OR SPLIT THE WORK ANDMAKE YOUR SOFTWAREWORK AS A TEAM
    • Build an army of fat appYOU CAN DO BOTH
    • SO WE NEED TO BE ABLETO DISPATCH THE WORKSCALE OUT•  Many workersdoing the samething•  No SPOF•  Growing is moreeasy•  Introduce bestpracticeSCALE UP•  1 Fat instance•  1 Fat application•  SPOF (single pointof failure)•  Hard to maintain•  Always has a limit•  Short termmeaning
    • SO, HOW TOSCALEOUT ?JUST SOME FACTS
    • SPLIT PROCESS ANDSTORAGEStorage• Databases• Files• Sessions• Events• …Code• Can be replicated• Stateless• Process
    • Picking one instance or another doesn’t matterSTATELESSNESS IS THE KEY
    • CONSIDER MORETHINGS AS DATA•  User account•  Users data•  Files•  Sessions•  Events
    • CHOOSEDATASTOREWISELYYOU CAN SHOULD USE MANY DATASTORES
    • DATASTORE CHOICESARE DRIVEN BY USAGEMakedecisionsbased onneedsDo I needatomicity ofrequests ?Do I needconcurrentaccess ?Do I mostlyread orwrite ?Do I needrelational ?Do I needbig storagecapacity ?Do I needhighavailability ?
    • •  Not a big volume•  DB have to managedata TTL•  Data model : K/V•  Multiple writes at thesame time•  High availabilityI need to store sessionsQUICK EXAMPLE
    • •  Not a big volumeIt’s OK, PG can handlesmall quantity of data•  DB have to managedata TTLNo, I have to do itmanually•  Data model : K/VNo, PG is relational(mainly)•  Multiple writes at thesame timeNo, PG is Atomic•  High availabilityPG is awesome ;-) Useof PG bouncer orsimilar allow goodclusteringI need to store sessionsQUICK EXAMPLE
    • •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes Redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeNo, redis is pseudoAtomic (master/slave)•  High availabilityRedis is great, butcauterization is rude…I need to store sessionsQUICK EXAMPLE
    • •  Not a big volumeIt’s OK, redis canhandle small quantityof data•  DB have to managedata TTLYes redis can do it•  Data model : K/V Yes•  Multiple writes at thesame timeOK, this is possiblewith memcachedprotocol•  High availabilityClustering is built in, nodowntime JI need to store sessionsQUICK EXAMPLE
    • USE ONLINEDATABASE / BEREADY TO TESTIN JUST A FEWMINUTESNO NEED TO TRASH YOUR COMPUTER
    • DON’T BE THAT GUY
    • DO NOT USE ATECHNOLOGY BECAUSEYOU <3 IT OR BECAUSEIT’S HYPE : USE ITBECAUSE IT FITS YOURNEEDSBALANCE YOUR LEARNING CURVE WITHTHE TIME SAVED
    • DO NOT CREATE MONSTERS
    • COMMON MISTAKES
    • DO NOT USEMEMORY ASDATABASELIKE : SHARED / GLOBAL VARIABLE,CACHE “IN THE CODE”, INTENSIVESESSION USAGE…
    • DO NOT USE A VARIABLEFOR MORE THAN ONEREQUEST
    • 2 + 2 = 4FOR SAME INPUT, SAME OUTPUT
    • And data will be lostCODE WILL FAIL
    • DO NOT USE FILESYSTEM AS DATASTOREFile system are POSIX compliant•  POSIX is ACID•  POSIX is powerful but is bottleneck•  File System is nightmare of ops•  File System is create coupling (host provider/OS/language)•  Free SPOF multi tenant File System is a unicornSTORE IN DATABASE, OR DATASTORE LIKE S3 (AWS)DEDICATED TO FILE MANAGEMENT
    • CAREFUL USE OFDARK MAGIC
    • SPLIT THE CODE :MODULES•  Smallest codebase•  Deploy asservice foreach other•  Focus on besttechnology fora problem
    • USE EVENT BROKER TOMODULARIZE YOUR APP•  AMQP•  Celery•  0MQ•  Redis•  JMS•  Some case : hadoop, akka…•  …CRON is not an event queue
    • MAKE HARDCOMPUTATION ASYNC
    • ALWAYS USE AREVERSE PROXYY U NOT USE ONE ?
    • USE PROCESSDEPLOYMENT
    • KEEP CALM UNDERFIRE
    • TRACK BUG & GETMETRICS
    • Quentin ADAMTwitter : @waxzceTHX FOR LISTENING& QUESTIONS TIMEcleverc l o u d