The 5 Stages of     Scale   Christopher Smith
Who am I?Two decades experienceHalf of that in online advertisingInternet systems engineeringScaling web serving, data col...
Scalabilityscale: v.tr.  1. To clear or strip of scale or scales.  2. Weigh a specified weight.  3. Climb up or over (some...
Your App                                    Your App                                        ResponsivenessThroughput      ...
Your App       Trend                            Your App       Trend                                            Responsive...
Your App                                           Your App       Trend             Undeniable Extrapolation              ...
Your App                                           Your App       Trend             Undeniable Extrapolation              ...
Scalability       EnvelopesThere is always a “next” bottleneck.In case of scalability problem...6 envelopes
Envelope 0Session partitioningCommodity: load balancer, multi-*Linear scale for CPULimit: C10K?
Envelope 1Read Caching  Reverse-proxy  memcached  CDNlog(n) scale: thank you ZipfLimit: ~200 w/sec
Envelope 2Get a real persistence framework  Data structures FTW!  DB: concurrent read/write  MOM: queuing/event IO/TP moni...
Tipping over
Scaling Catamaran’sRAM caching I/ORAIDThreads (sometimes)Packet loss (UR DUING IT WRONG)SSD’s?
Jeff Dean’s NumbersLatency Comparison Numbers--------------------------L1 cache reference                            0.5 n...
Problem: IO latencyThroughput: 2x every 18 monthsLatency:  CPU: <2x every 18 months  LAN network: 2x every 2-3 years  Memo...
Problem IO LatencyTraditional indexes on the wrong side  Turns a scan in to a seek  Index lookup: scan 0.1% of records + 1...
Envelope 3Real partitioning of IOMove code, not dataCommodities: Map/Reduce (Hadoop), DHT(Cassandra, HBase, Riak)CAP Theor...
Envelope 4Route new data through datapartitions  Using MOM/EventIO “the right way”  ESP/CEP: Eigen, Storm, Esper,  StreamB...
Envelope 5Cheat more on reliability. UDP w/o reliability > TCP Measure loss vs. prevent loss Horseshoes, hand grenades, fe...
Integrated SystemsCombined IO management solutions:  real-time memory key/value lookup  LSM + bitmap indexes + etc.  event...
Efficient LoggingEvents in efficient machine parseableform: (protobuf, thrift, etc.)Event source writes only to NICUDP Mul...
message LogEvent {    required uint64 pid = 1;    optional uint64 tid = 2;    optional uint64 sid = 4;    required uint64 ...
AnnouncementsDedicated channel.Payload: channel IP, channel port,last seq, pid, tid, sid + statsAll announcers listen and ...
ConsolidationRedundant journalers (RAID)ESP: detect loss in real time windowIf necessary, Map/Reduce processingto try to r...
EfficiencyHundreds of nodes>50MB/sec>50,000 pps3-4 “journalers” resolving data>5TB reconciled data a day<0.1% data loss
Envelope 6Take out 6 envelopes...
Upcoming SlideShare
Loading in...5
×

The 5 Stages of Scale

3,265

Published on

Slides from my SCALE 11x talk on scaling applications, from the smallest size to the largest.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
3,265
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The 5 Stages of Scale

  1. 1. The 5 Stages of Scale Christopher Smith
  2. 2. Who am I?Two decades experienceHalf of that in online advertisingInternet systems engineeringScaling web serving, data collection& analysisPlaces big & small.
  3. 3. Scalabilityscale: v.tr. 1. To clear or strip of scale or scales. 2. Weigh a specified weight. 3. Climb up or over (something steep)
  4. 4. Your App Your App ResponsivenessThroughput 0 2 4 6 8 0 2 4 6 8 Usage Usage
  5. 5. Your App Trend Your App Trend ResponsivenessThroughput 0 2 4 6 8 0 2 4 6 8 Usage Usage
  6. 6. Your App Your App Trend Undeniable Extrapolation ResponsivenessThroughput 0 500000 1000000 0 500000 1000000 Usage Usage
  7. 7. Your App Your App Trend Undeniable Extrapolation Scalability: ResponsivenessThroughput Saving This Guy’s Job 0 500000 1000000 0 500000 1000000 Usage Usage
  8. 8. Scalability EnvelopesThere is always a “next” bottleneck.In case of scalability problem...6 envelopes
  9. 9. Envelope 0Session partitioningCommodity: load balancer, multi-*Linear scale for CPULimit: C10K?
  10. 10. Envelope 1Read Caching Reverse-proxy memcached CDNlog(n) scale: thank you ZipfLimit: ~200 w/sec
  11. 11. Envelope 2Get a real persistence framework Data structures FTW! DB: concurrent read/write MOM: queuing/event IO/TP monitors Cheat on ACID (particularly C & D)log(n) scale?1000-10000 w/sec
  12. 12. Tipping over
  13. 13. Scaling Catamaran’sRAM caching I/ORAIDThreads (sometimes)Packet loss (UR DUING IT WRONG)SSD’s?
  14. 14. Jeff Dean’s NumbersLatency Comparison Numbers--------------------------L1 cache reference 0.5 nsBranch mispredict 5 nsL2 cache reference 7 ns 14x L1 cacheMutex lock/unlock 25 nsMain memory reference 100 ns 20x L2 cache, 200x L1 cacheCompress 1K bytes with Zippy 3,000 nsSend 1K bytes over 1 Gbps network 10,000 ns 0.01 msRead 4K randomly from SSD* 150,000 ns 0.15 msRead 1 MB sequentially from memory 250,000 ns 0.25 msRound trip within same datacenter 500,000 ns 0.5 msRead 1 MB sequentially from SSD* 1,000,000 ns 1 ms 4X memoryDisk seek 10,000,000 ns 10 ms 20x datacenter roundtrip
  15. 15. Problem: IO latencyThroughput: 2x every 18 monthsLatency: CPU: <2x every 18 months LAN network: 2x every 2-3 years Memory: 2x every 3-5 years Disk: 2x every decade? (SSD?) WAN Network: 1x every...
  16. 16. Problem IO LatencyTraditional indexes on the wrong side Turns a scan in to a seek Index lookup: scan 0.1% of records + 1 random seek Scan: scan 100% of records, 0 random seek Seek is 10ms & Scan is 100Hz -> 10x win Seek is 1ms & Scan is 1GHz -> 1000x loss
  17. 17. Envelope 3Real partitioning of IOMove code, not dataCommodities: Map/Reduce (Hadoop), DHT(Cassandra, HBase, Riak)CAP Theory limiting sync’ing
  18. 18. Envelope 4Route new data through datapartitions Using MOM/EventIO “the right way” ESP/CEP: Eigen, Storm, Esper, StreamBase, 0mq, etc.
  19. 19. Envelope 5Cheat more on reliability. UDP w/o reliability > TCP Measure loss vs. prevent loss Horseshoes, hand grenades, features...?
  20. 20. Integrated SystemsCombined IO management solutions: real-time memory key/value lookup LSM + bitmap indexes + etc. eventual consistency mobile code for batch processingCassandra, HBase, etc.
  21. 21. Efficient LoggingEvents in efficient machine parseableform: (protobuf, thrift, etc.)Event source writes only to NICUDP MulticastRedundant listeners
  22. 22. message LogEvent { required uint64 pid = 1; optional uint64 tid = 2; optional uint64 sid = 4; required uint64 sequence = 5; required uint64 timestamp = 6; enum Level { PANIC = 0, ERROR = 1..} required Level level = 7; required bytes payload = 8;}
  23. 23. AnnouncementsDedicated channel.Payload: channel IP, channel port,last seq, pid, tid, sid + statsAll announcers listen and self-throttle.Directory service accumulates
  24. 24. ConsolidationRedundant journalers (RAID)ESP: detect loss in real time windowIf necessary, Map/Reduce processingto try to resolve partial loss.
  25. 25. EfficiencyHundreds of nodes>50MB/sec>50,000 pps3-4 “journalers” resolving data>5TB reconciled data a day<0.1% data loss
  26. 26. Envelope 6Take out 6 envelopes...
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×