Scalable Web
   Architectures
Common Patterns & Approaches

          Cal Henderson
Hello




Web 2.0 Expo Berlin, 5th November 2007           2
Scalable Web Architectures?


                   What does scalable mean?

                        What’s an architecture?...
1.

                                 Scaling

Web 2.0 Expo Berlin, 5th November 2007        4
Scalability – myths and lies


      • What is scalability?




Web 2.0 Expo Berlin, 5th November 2007   5
Scalability – myths and lies


      • What is scalability not ?




Web 2.0 Expo Berlin, 5th November 2007   6
Scalability – myths and lies


      • What is scalability not ?
            – Raw Speed / Performance
            – HA / ...
Scalability – myths and lies


      • So what is scalability?




Web 2.0 Expo Berlin, 5th November 2007   8
Scalability – myths and lies


      • So what is scalability?
            – Traffic growth
            – Dataset growth
 ...
Today
      • Two goals of application architecture:


                                         Scale
                    ...
Today
      • Three goals of application architecture:


                                         Scale
                  ...
Scalability

      • Two kinds:
            – Vertical (get bigger)
            – Horizontal (get more)




Web 2.0 Expo B...
Big Irons




                  Sunfire E20k             PowerEdge SC1435
             36x 1.8GHz processors       Dualcor...
Cost vs Cost




Web 2.0 Expo Berlin, 5th November 2007   14
That’s OK
      • Sometimes vertical scaling is right

      • Buying a bigger box is quick (ish)
      • Redesigning soft...
The H & the V


      • But we’ll mostly talk horizontal
            – Else this is going to be boring




Web 2.0 Expo Be...
2.

                Architecture

Web 2.0 Expo Berlin, 5th November 2007        17
Architectures then?


      • The way the bits fit together
      • What grows where
      • The trade-offs between good/f...
LAMP
      • We’re mostly talking about LAMP
            –   Linux
            –   Apache (or LightHTTPd)
            –   ...
Simple web apps

      • A Web Application
                        – Or “Web Site” in Web 1.0 terminology




            ...
Simple web apps

      • A Web Application
                        – Or “Web Site” in Web 1.0 terminology
                ...
App servers

      • App servers scale in two ways:




Web 2.0 Expo Berlin, 5th November 2007   22
App servers

      • App servers scale in two ways:

            – Really well




Web 2.0 Expo Berlin, 5th November 2007 ...
App servers

      • App servers scale in two ways:

            – Really well

            – Quite badly




Web 2.0 Expo...
App servers
      • Sessions!
            – (State)

            – Local sessions == bad
                  • When they mov...
Local sessions
      • Stored on disk
            – PHP sessions


      • Stored in memory
            – Shared memory bl...
Mobile local sessions
      • Custom built
            – Store last session location in cookie
            – If we hit a d...
Remote centralized sessions
      • Store in a central database
            – Or an in-memory cache

      • No porting ar...
No sessions
      • Stash it all in a cookie!

      • Sign it for safety
            – $data = $user_id . ‘-’ . $user_nam...
Super slim sessions
      • If you need more than the cookie (login status,
        user id, username), then pull their ac...
App servers

      • The Rasmus way
            – App server has ‘shared nothing’
            – Responsibility pushed down...
Trifle




Web 2.0 Expo Berlin, 5th November 2007   32
Trifle
                                          Fruit / Presentation



                                           Cream ...
Trifle
                                          Fruit / Presentation



                                           Cream ...
App servers




Web 2.0 Expo Berlin, 5th November 2007   35
App servers




Web 2.0 Expo Berlin, 5th November 2007   36
App servers




Web 2.0 Expo Berlin, 5th November 2007   37
Well, that was easy

      • Scaling the web app server part is easy

      • The rest is the trickier part
            – ...
The others
      • Other services scale similarly to web apps
            – That is, horizontally


      • The canonical ...
Amazon
      • Let’s talk about Amazon
            – S3 - Storage
            – EC2 – Compute! (XEN based)
            – S...
3.

        Load Balancing

Web 2.0 Expo Berlin, 5th November 2007        41
Load balancing

      • If we have multiple nodes in a class, we
        need to balance between them

      • Hardware or...
Hardware LB
      • A hardware appliance
            – Often a pair with heartbeats for HA


      • Expensive!
          ...
Software LB
      • Just some software
            – Still needs hardware to run on
            – But can run on existing ...
Software LB
      • Lots of options
            – Pound
            – Perlbal
            – Apache with mod_proxy


      ...
Wackamole




Web 2.0 Expo Berlin, 5th November 2007   46
Wackamole




Web 2.0 Expo Berlin, 5th November 2007   47
The layers
      • Layer 4
            – A ‘dumb’ balance


      • Layer 7
            – A ‘smart’ balance


      • OSI ...
4.

                             Queuing

Web 2.0 Expo Berlin, 5th November 2007        49
Parallelizable == easy!

      • If we can transcode/crawl in parallel, it’s
        easy
            – But think about qu...
Synchronous systems




Web 2.0 Expo Berlin, 5th November 2007   51
Asynchronous systems




Web 2.0 Expo Berlin, 5th November 2007   52
Helps with peak periods




Web 2.0 Expo Berlin, 5th November 2007   53
Synchronous systems




Web 2.0 Expo Berlin, 5th November 2007   54
Asynchronous systems




Web 2.0 Expo Berlin, 5th November 2007   55
Asynchronous systems




Web 2.0 Expo Berlin, 5th November 2007   56
5.

      Relational Data

Web 2.0 Expo Berlin, 5th November 2007        57
Databases

      • Unless we’re doing a lot of file serving, the
        database is the toughest part to scale

      • I...
More read power

      • Web apps typically have a read/write ratio
        of somewhere between 80/20 and 90/10
      • I...
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   60
Master-Slave Replication


                                         Reads and Writes




                                 ...
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   62
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   63
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   64
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   65
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   66
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   67
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   68
Master-Slave Replication




Web 2.0 Expo Berlin, 5th November 2007   69
6.

                               Caching

Web 2.0 Expo Berlin, 5th November 2007        70
Caching
      • Caching avoids needing to scale!
            – Or makes it cheaper


      • Simple stuff
            – mo...
Caching

      • Getting more complicated…
            – Write-through cache
            – Write-back cache
            – ...
Write-through cache




Web 2.0 Expo Berlin, 5th November 2007   73
Write-back cache




Web 2.0 Expo Berlin, 5th November 2007   74
Sideline cache




Web 2.0 Expo Berlin, 5th November 2007   75
Sideline cache
      • Easy to implement
            – Just add app logic


      • Need to manually invalidate cache
    ...
Memcache schemes

      • Layer 4
            – Good: Cache can be local on a machine
            – Bad: Invalidation gets...
Memcache schemes

      • Layer 7
            – Good: No wasted space
            – Good: linearly scaling invalidation
  ...
7.

                              HA Data

Web 2.0 Expo Berlin, 5th November 2007        79
But what about HA?




Web 2.0 Expo Berlin, 5th November 2007   80
But what about HA?




Web 2.0 Expo Berlin, 5th November 2007   81
SPOF!

      • The key to HA is avoiding SPOFs
            – Identify
            – Eliminate


      • Some stuff is hard...
Master-Master




Web 2.0 Expo Berlin, 5th November 2007   83
Master-Master
      • Either hot/warm or hot/hot

      • Writes can go to either
            – But avoid collisions
     ...
Rings
      • Master-master is just a small ring
            – With 2 nodes


      • Bigger rings are possible
          ...
Rings




Web 2.0 Expo Berlin, 5th November 2007   86
Rings




Web 2.0 Expo Berlin, 5th November 2007   87
Dual trees

      • Master-master is good for HA
            – But we can’t scale out the reads (or writes!)


      • We ...
Dual trees




Web 2.0 Expo Berlin, 5th November 2007   89
Cost models
      • There’s a problem here
            – We need to always have 200% capacity to
              avoid a SPO...
N+M
      • N+M
            – N = nodes needed to run the system
            – M = nodes we can afford to lose


      • H...
1+1 = 200% hardware




Web 2.0 Expo Berlin, 5th November 2007   92
3+1 = 133% hardware




Web 2.0 Expo Berlin, 5th November 2007   93
Meshed masters
      • Not possible with regular MySQL out-of-
        the-box today

      • But there is hope!
         ...
8.

                      Federation

Web 2.0 Expo Berlin, 5th November 2007        95
Data federation

      • At some point, you need more writes
            – This is tough
            – Each cluster of ser...
Simple things first
      • Vertical partitioning
            – Divide tables into sets that never get joined
            ...
Data federation
      • Split up large tables, organized by some
        primary object
            – Usually users


    ...
Data federation




Web 2.0 Expo Berlin, 5th November 2007   99
Data federation
      • Need more capacity?
            – Just add shards!
            – Don’t assign to shards based on u...
The wordpress.com approach
      • Hash users into one of n buckets
            – Where n is a power of 2


      • Put al...
Data federation
      • Heterogeneous hardware is fine
            – Just give a larger/smaller proportion of objects
    ...
Downsides
      • Need to keep stuff in the right place
      • App logic gets more complicated
      • More clusters to m...
Bottom line


              Data federation is how
               large applications are
                      scaled


We...
Bottom line
      • It’s hard, but not impossible

      • Good software design makes it easier
            – Abstraction!...
9.

               Multi-site HA

Web 2.0 Expo Berlin, 5th November 2007        106
Multiple Datacenters
      • Having multiple datacenters is hard
            – Not just with MySQL

      • Hot/warm with ...
Multiple Datacenters




Web 2.0 Expo Berlin, 5th November 2007   108
GSLB

      • Multiple sites need to be balanced
            – Global Server Load Balancing


      • Easiest are AkaDNS-l...
10.

              Serving Files

Web 2.0 Expo Berlin, 5th November 2007         110
Serving lots of files
      • Serving lots of files is not too tough
            – Just buy lots of machines and load bala...
Reverse proxy




Web 2.0 Expo Berlin, 5th November 2007   112
Reverse proxy
      • Serving out of memory is fast!
            – And our caching proxies can have disks too
            ...
Invalidation

      • Dealing with invalidation is tricky

      • We can prod the cache servers directly to
        clear...
Invalidation
      • We can change the URLs of modified
        resources
            – And let the old ones drop out cach...
Reverse proxy

      • Choices
            – L7 load balancer & Squid
                  • http://www.squid-cache.org/

   ...
High overhead serving
      • What if you need to authenticate your
        asset serving?
            – Private photos
  ...
Perlbal backhanding

      • Perlbal can do redirection magic
            – Client sends request to Perbal
            – P...
Perlbal backhanding




Web 2.0 Expo Berlin, 5th November 2007   119
Perlbal backhanding

      • Doesn’t keep database around while
        serving
      • Doesn’t keep app server around whi...
Permission URLs
      • But why bother!?
      • If we bake the auth into the URL then it
        saves the auth step
    ...
Permission URLs




Web 2.0 Expo Berlin, 5th November 2007   122
Permission URLs




                (or mod_perl)




Web 2.0 Expo Berlin, 5th November 2007   123
Permission URLs
      • Downsides
            – URL gives permission for life
            – Unless you bake in tokens
    ...
11.

                Storing Files

Web 2.0 Expo Berlin, 5th November 2007         125
Storing lots of files

      • Storing files is easy!
            – Get a big disk
            – Get a bigger disk
       ...
Connecting to storage
      • NFS
            – Stateful == Sucks
            – Hard mounts vs Soft mounts, INTR


      •...
Multiple volumes
      • Volumes are limited in total size
            – Except (in theory) under ZFS & others


      • S...
Multiple volumes




Web 2.0 Expo Berlin, 5th November 2007   129
Multiple hosts

      • Further down the road, a single host will
        be too small
      • Total throughput of machine...
Multiple hosts




Web 2.0 Expo Berlin, 5th November 2007   131
HA Storage

      • HA is important for assets too
            – We can back stuff up
            – But we tend to want ho...
HA Storage
      • But whole machines can fail
      • So we stick assets on multiple machines

      • In this case, we c...
HA Storage




Web 2.0 Expo Berlin, 5th November 2007   134
Self repairing systems
      • When something fails, repairing can be a
        pain
            – RAID rebuilds by itself...
12.

                      Field Work

Web 2.0 Expo Berlin, 5th November 2007         136
Real world examples

      • Flickr
            – Because I know it


      • LiveJournal
            – Because everyone c...
Flickr
  Architecture




Web 2.0 Expo Berlin, 5th November 2007   138
Flickr
  Architecture




Web 2.0 Expo Berlin, 5th November 2007   139
LiveJournal
  Architecture




Web 2.0 Expo Berlin, 5th November 2007   140
LiveJournal
  Architecture




Web 2.0 Expo Berlin, 5th November 2007   141
Buy my book!




Web 2.0 Expo Berlin, 5th November 2007   142
Or buy Theo’s




Web 2.0 Expo Berlin, 5th November 2007   143
The end!




Web 2.0 Expo Berlin, 5th November 2007   144
Awesome!


                    These slides are available online:
                           iamcal.com/talks/




Web 2.0...
Upcoming SlideShare
Loading in...5
×

Scalable Web Architectures: Common Patterns and Approaches

18,963

Published on

Speaker: Cal Henderson

Published in: Technology, Travel
0 Comments
53 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
18,963
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
1,084
Comments
0
Likes
53
Embeds 0
No embeds

No notes for slide

Scalable Web Architectures: Common Patterns and Approaches

  1. 1. Scalable Web Architectures Common Patterns & Approaches Cal Henderson
  2. 2. Hello Web 2.0 Expo Berlin, 5th November 2007 2
  3. 3. Scalable Web Architectures? What does scalable mean? What’s an architecture? An answer in 12 parts Web 2.0 Expo Berlin, 5th November 2007 3
  4. 4. 1. Scaling Web 2.0 Expo Berlin, 5th November 2007 4
  5. 5. Scalability – myths and lies • What is scalability? Web 2.0 Expo Berlin, 5th November 2007 5
  6. 6. Scalability – myths and lies • What is scalability not ? Web 2.0 Expo Berlin, 5th November 2007 6
  7. 7. Scalability – myths and lies • What is scalability not ? – Raw Speed / Performance – HA / BCP – Technology X – Protocol Y Web 2.0 Expo Berlin, 5th November 2007 7
  8. 8. Scalability – myths and lies • So what is scalability? Web 2.0 Expo Berlin, 5th November 2007 8
  9. 9. Scalability – myths and lies • So what is scalability? – Traffic growth – Dataset growth – Maintainability Web 2.0 Expo Berlin, 5th November 2007 9
  10. 10. Today • Two goals of application architecture: Scale HA Web 2.0 Expo Berlin, 5th November 2007 10
  11. 11. Today • Three goals of application architecture: Scale HA Performance Web 2.0 Expo Berlin, 5th November 2007 11
  12. 12. Scalability • Two kinds: – Vertical (get bigger) – Horizontal (get more) Web 2.0 Expo Berlin, 5th November 2007 12
  13. 13. Big Irons Sunfire E20k PowerEdge SC1435 36x 1.8GHz processors Dualcore 1.8 GHz processor Around $1,500 $450,000 - $2,500,000 Web 2.0 Expo Berlin, 5th November 2007 13
  14. 14. Cost vs Cost Web 2.0 Expo Berlin, 5th November 2007 14
  15. 15. That’s OK • Sometimes vertical scaling is right • Buying a bigger box is quick (ish) • Redesigning software is not • Running out of MySQL performance? – Spend months on data federation – Or, Just buy a ton more RAM Web 2.0 Expo Berlin, 5th November 2007 15
  16. 16. The H & the V • But we’ll mostly talk horizontal – Else this is going to be boring Web 2.0 Expo Berlin, 5th November 2007 16
  17. 17. 2. Architecture Web 2.0 Expo Berlin, 5th November 2007 17
  18. 18. Architectures then? • The way the bits fit together • What grows where • The trade-offs between good/fast/cheap Web 2.0 Expo Berlin, 5th November 2007 18
  19. 19. LAMP • We’re mostly talking about LAMP – Linux – Apache (or LightHTTPd) – MySQL (or Postgres) – PHP (or Perl, Python, Ruby) • All open source • All well supported • All used in large operations • (Same rules apply elsewhere) Web 2.0 Expo Berlin, 5th November 2007 19
  20. 20. Simple web apps • A Web Application – Or “Web Site” in Web 1.0 terminology Interwebnet App server Database Web 2.0 Expo Berlin, 5th November 2007 20
  21. 21. Simple web apps • A Web Application – Or “Web Site” in Web 1.0 terminology Storage array Cache Interwobnet App server Database AJAX!!!1 Web 2.0 Expo Berlin, 5th November 2007 21
  22. 22. App servers • App servers scale in two ways: Web 2.0 Expo Berlin, 5th November 2007 22
  23. 23. App servers • App servers scale in two ways: – Really well Web 2.0 Expo Berlin, 5th November 2007 23
  24. 24. App servers • App servers scale in two ways: – Really well – Quite badly Web 2.0 Expo Berlin, 5th November 2007 24
  25. 25. App servers • Sessions! – (State) – Local sessions == bad • When they move == quite bad – Centralized sessions == good – No sessions at all == awesome! Web 2.0 Expo Berlin, 5th November 2007 25
  26. 26. Local sessions • Stored on disk – PHP sessions • Stored in memory – Shared memory block (APC) • Bad! – Can’t move users – Can’t avoid hotspots – Not fault tolerant Web 2.0 Expo Berlin, 5th November 2007 26
  27. 27. Mobile local sessions • Custom built – Store last session location in cookie – If we hit a different server, pull our session information across • If your load balancer has sticky sessions, you can still get hotspots – Depends on volume – fewer heavier users hurt more Web 2.0 Expo Berlin, 5th November 2007 27
  28. 28. Remote centralized sessions • Store in a central database – Or an in-memory cache • No porting around of session data • No need for sticky sessions • No hot spots • Need to be able to scale the data store – But we’ve pushed the issue down the stack Web 2.0 Expo Berlin, 5th November 2007 28
  29. 29. No sessions • Stash it all in a cookie! • Sign it for safety – $data = $user_id . ‘-’ . $user_name; – $time = time(); – $sig = sha1($secret . $time . $data); – $cookie = base64(“$sig-$time-$data”); – Timestamp means it’s simple to expire it Web 2.0 Expo Berlin, 5th November 2007 29
  30. 30. Super slim sessions • If you need more than the cookie (login status, user id, username), then pull their account row from the DB – Or from the account cache • None of the drawbacks of sessions • Avoids the overhead of a query per page – Great for high-volume pages which need little personalization – Turns out you can stick quite a lot in a cookie too – Pack with base64 and it’s easy to delimit fields Web 2.0 Expo Berlin, 5th November 2007 30
  31. 31. App servers • The Rasmus way – App server has ‘shared nothing’ – Responsibility pushed down the stack • Ooh, the stack Web 2.0 Expo Berlin, 5th November 2007 31
  32. 32. Trifle Web 2.0 Expo Berlin, 5th November 2007 32
  33. 33. Trifle Fruit / Presentation Cream / Markup Custard / Page Logic Jelly / Business Logic Sponge / Database Web 2.0 Expo Berlin, 5th November 2007 33
  34. 34. Trifle Fruit / Presentation Cream / Markup Custard / Page Logic Jelly / Business Logic Sponge / Database Web 2.0 Expo Berlin, 5th November 2007 34
  35. 35. App servers Web 2.0 Expo Berlin, 5th November 2007 35
  36. 36. App servers Web 2.0 Expo Berlin, 5th November 2007 36
  37. 37. App servers Web 2.0 Expo Berlin, 5th November 2007 37
  38. 38. Well, that was easy • Scaling the web app server part is easy • The rest is the trickier part – Database – Serving static content – Storing static content Web 2.0 Expo Berlin, 5th November 2007 38
  39. 39. The others • Other services scale similarly to web apps – That is, horizontally • The canonical examples: – Image conversion – Audio transcoding – Video transcoding – Web crawling – Compute! Web 2.0 Expo Berlin, 5th November 2007 39
  40. 40. Amazon • Let’s talk about Amazon – S3 - Storage – EC2 – Compute! (XEN based) – SQS – Queueing • All horizontal • Cheap when small – Not cheap at scale Web 2.0 Expo Berlin, 5th November 2007 40
  41. 41. 3. Load Balancing Web 2.0 Expo Berlin, 5th November 2007 41
  42. 42. Load balancing • If we have multiple nodes in a class, we need to balance between them • Hardware or software • Layer 4 or 7 Web 2.0 Expo Berlin, 5th November 2007 42
  43. 43. Hardware LB • A hardware appliance – Often a pair with heartbeats for HA • Expensive! – But offers high performance • Many brands – Alteon, Cisco, Netscalar, Foundry, etc – L7 - web switches, content switches, etc Web 2.0 Expo Berlin, 5th November 2007 43
  44. 44. Software LB • Just some software – Still needs hardware to run on – But can run on existing servers • Harder to have HA – Often people stick hardware LB’s in front – But Wackamole helps here Web 2.0 Expo Berlin, 5th November 2007 44
  45. 45. Software LB • Lots of options – Pound – Perlbal – Apache with mod_proxy • Wackamole with mod_backhand – http://backhand.org/wackamole/ – http://backhand.org/mod_backhand/ Web 2.0 Expo Berlin, 5th November 2007 45
  46. 46. Wackamole Web 2.0 Expo Berlin, 5th November 2007 46
  47. 47. Wackamole Web 2.0 Expo Berlin, 5th November 2007 47
  48. 48. The layers • Layer 4 – A ‘dumb’ balance • Layer 7 – A ‘smart’ balance • OSI stack, routers, etc Web 2.0 Expo Berlin, 5th November 2007 48
  49. 49. 4. Queuing Web 2.0 Expo Berlin, 5th November 2007 49
  50. 50. Parallelizable == easy! • If we can transcode/crawl in parallel, it’s easy – But think about queuing – And asynchronous systems – The web ain’t built for slow things – But still, a simple problem Web 2.0 Expo Berlin, 5th November 2007 50
  51. 51. Synchronous systems Web 2.0 Expo Berlin, 5th November 2007 51
  52. 52. Asynchronous systems Web 2.0 Expo Berlin, 5th November 2007 52
  53. 53. Helps with peak periods Web 2.0 Expo Berlin, 5th November 2007 53
  54. 54. Synchronous systems Web 2.0 Expo Berlin, 5th November 2007 54
  55. 55. Asynchronous systems Web 2.0 Expo Berlin, 5th November 2007 55
  56. 56. Asynchronous systems Web 2.0 Expo Berlin, 5th November 2007 56
  57. 57. 5. Relational Data Web 2.0 Expo Berlin, 5th November 2007 57
  58. 58. Databases • Unless we’re doing a lot of file serving, the database is the toughest part to scale • If we can, best to avoid the issue altogether and just buy bigger hardware • Dual Opteron/Intel64 systems with 16+GB of RAM can get you a long way Web 2.0 Expo Berlin, 5th November 2007 58
  59. 59. More read power • Web apps typically have a read/write ratio of somewhere between 80/20 and 90/10 • If we can scale read capacity, we can solve a lot of situations • MySQL replication! Web 2.0 Expo Berlin, 5th November 2007 59
  60. 60. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 60
  61. 61. Master-Slave Replication Reads and Writes Reads Web 2.0 Expo Berlin, 5th November 2007 61
  62. 62. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 62
  63. 63. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 63
  64. 64. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 64
  65. 65. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 65
  66. 66. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 66
  67. 67. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 67
  68. 68. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 68
  69. 69. Master-Slave Replication Web 2.0 Expo Berlin, 5th November 2007 69
  70. 70. 6. Caching Web 2.0 Expo Berlin, 5th November 2007 70
  71. 71. Caching • Caching avoids needing to scale! – Or makes it cheaper • Simple stuff – mod_perl / shared memory • Invalidation is hard – MySQL query cache • Bad performance (in most cases) Web 2.0 Expo Berlin, 5th November 2007 71
  72. 72. Caching • Getting more complicated… – Write-through cache – Write-back cache – Sideline cache Web 2.0 Expo Berlin, 5th November 2007 72
  73. 73. Write-through cache Web 2.0 Expo Berlin, 5th November 2007 73
  74. 74. Write-back cache Web 2.0 Expo Berlin, 5th November 2007 74
  75. 75. Sideline cache Web 2.0 Expo Berlin, 5th November 2007 75
  76. 76. Sideline cache • Easy to implement – Just add app logic • Need to manually invalidate cache – Well designed code makes it easy • Memcached – From Danga (LiveJournal) – http://www.danga.com/memcached/ Web 2.0 Expo Berlin, 5th November 2007 76
  77. 77. Memcache schemes • Layer 4 – Good: Cache can be local on a machine – Bad: Invalidation gets more expensive with node count – Bad: Cache space wasted by duplicate objects Web 2.0 Expo Berlin, 5th November 2007 77
  78. 78. Memcache schemes • Layer 7 – Good: No wasted space – Good: linearly scaling invalidation – Bad: Multiple, remote connections • Can be avoided with a proxy layer – Gets more complicated » Last indentation level! Web 2.0 Expo Berlin, 5th November 2007 78
  79. 79. 7. HA Data Web 2.0 Expo Berlin, 5th November 2007 79
  80. 80. But what about HA? Web 2.0 Expo Berlin, 5th November 2007 80
  81. 81. But what about HA? Web 2.0 Expo Berlin, 5th November 2007 81
  82. 82. SPOF! • The key to HA is avoiding SPOFs – Identify – Eliminate • Some stuff is hard to solve – Fix it further up the tree • Dual DCs solves Router/Switch SPOF Web 2.0 Expo Berlin, 5th November 2007 82
  83. 83. Master-Master Web 2.0 Expo Berlin, 5th November 2007 83
  84. 84. Master-Master • Either hot/warm or hot/hot • Writes can go to either – But avoid collisions – No auto-inc columns for hot/hot • Bad for hot/warm too • Unless you have MySQL 5 – But you can’t rely on the ordering! – Design schema/access to avoid collisions • Hashing users to servers Web 2.0 Expo Berlin, 5th November 2007 84
  85. 85. Rings • Master-master is just a small ring – With 2 nodes • Bigger rings are possible – But not a mesh! – Each slave may only have a single master – Unless you build some kind of manual replication Web 2.0 Expo Berlin, 5th November 2007 85
  86. 86. Rings Web 2.0 Expo Berlin, 5th November 2007 86
  87. 87. Rings Web 2.0 Expo Berlin, 5th November 2007 87
  88. 88. Dual trees • Master-master is good for HA – But we can’t scale out the reads (or writes!) • We often need to combine the read scaling with HA • We can simply combine the two models Web 2.0 Expo Berlin, 5th November 2007 88
  89. 89. Dual trees Web 2.0 Expo Berlin, 5th November 2007 89
  90. 90. Cost models • There’s a problem here – We need to always have 200% capacity to avoid a SPOF • 400% for dual sites! – This costs too much • Solution is straight forward – Make sure clusters are bigger than 2 Web 2.0 Expo Berlin, 5th November 2007 90
  91. 91. N+M • N+M – N = nodes needed to run the system – M = nodes we can afford to lose • Having M as big as N starts to suck – If we could make each node smaller, we can increase N while M stays constant – (We assume smaller nodes are cheaper) Web 2.0 Expo Berlin, 5th November 2007 91
  92. 92. 1+1 = 200% hardware Web 2.0 Expo Berlin, 5th November 2007 92
  93. 93. 3+1 = 133% hardware Web 2.0 Expo Berlin, 5th November 2007 93
  94. 94. Meshed masters • Not possible with regular MySQL out-of- the-box today • But there is hope! – NBD (MySQL Cluster) allows a mesh – Support for replication out to slaves in a coming version • RSN! Web 2.0 Expo Berlin, 5th November 2007 94
  95. 95. 8. Federation Web 2.0 Expo Berlin, 5th November 2007 95
  96. 96. Data federation • At some point, you need more writes – This is tough – Each cluster of servers has limited write capacity • Just add more clusters! Web 2.0 Expo Berlin, 5th November 2007 96
  97. 97. Simple things first • Vertical partitioning – Divide tables into sets that never get joined – Split these sets onto different server clusters – Voila! • Logical limits – When you run out of non-joining groups – When a single table grows too large Web 2.0 Expo Berlin, 5th November 2007 97
  98. 98. Data federation • Split up large tables, organized by some primary object – Usually users • Put all of a user’s data on one ‘cluster’ – Or shard, or cell • Have one central cluster for lookups Web 2.0 Expo Berlin, 5th November 2007 98
  99. 99. Data federation Web 2.0 Expo Berlin, 5th November 2007 99
  100. 100. Data federation • Need more capacity? – Just add shards! – Don’t assign to shards based on user_id! • For resource leveling as time goes on, we want to be able to move objects between shards – Maybe – not everyone does this – ‘Lockable’ objects Web 2.0 Expo Berlin, 5th November 2007 100
  101. 101. The wordpress.com approach • Hash users into one of n buckets – Where n is a power of 2 • Put all the buckets on one server • When you run out of capacity, split the buckets across two servers • Then you run out of capacity, split the buckets across four servers • Etc Web 2.0 Expo Berlin, 5th November 2007 101
  102. 102. Data federation • Heterogeneous hardware is fine – Just give a larger/smaller proportion of objects depending on hardware • Bigger/faster hardware for paying users – A common approach – Can also allocate faster app servers via magic cookies at the LB Web 2.0 Expo Berlin, 5th November 2007 102
  103. 103. Downsides • Need to keep stuff in the right place • App logic gets more complicated • More clusters to manage – Backups, etc • More database connections needed per page – Proxy can solve this, but complicated • The dual table issue – Avoid walking the shards! Web 2.0 Expo Berlin, 5th November 2007 103
  104. 104. Bottom line Data federation is how large applications are scaled Web 2.0 Expo Berlin, 5th November 2007 104
  105. 105. Bottom line • It’s hard, but not impossible • Good software design makes it easier – Abstraction! • Master-master pairs for shards give us HA • Master-master trees work for central cluster (many reads, few writes) Web 2.0 Expo Berlin, 5th November 2007 105
  106. 106. 9. Multi-site HA Web 2.0 Expo Berlin, 5th November 2007 106
  107. 107. Multiple Datacenters • Having multiple datacenters is hard – Not just with MySQL • Hot/warm with MySQL slaved setup – But manual (reconfig on failure) • Hot/hot with master-master – But dangerous (each site has a SPOF) • Hot/hot with sync/async manual replication – But tough (big engineering task) Web 2.0 Expo Berlin, 5th November 2007 107
  108. 108. Multiple Datacenters Web 2.0 Expo Berlin, 5th November 2007 108
  109. 109. GSLB • Multiple sites need to be balanced – Global Server Load Balancing • Easiest are AkaDNS-like services – Performance rotations – Balance rotations Web 2.0 Expo Berlin, 5th November 2007 109
  110. 110. 10. Serving Files Web 2.0 Expo Berlin, 5th November 2007 110
  111. 111. Serving lots of files • Serving lots of files is not too tough – Just buy lots of machines and load balance! • We’re IO bound – need more spindles! – But keeping many copies of data in sync is hard – And sometimes we have other per-request overhead (like auth) Web 2.0 Expo Berlin, 5th November 2007 111
  112. 112. Reverse proxy Web 2.0 Expo Berlin, 5th November 2007 112
  113. 113. Reverse proxy • Serving out of memory is fast! – And our caching proxies can have disks too – Fast or otherwise • More spindles is better • We stay in sync automatically • We can parallelize it! – 50 cache servers gives us 50 times the serving rate of the origin server – Assuming the working set is small enough to fit in memory in the cache cluster Web 2.0 Expo Berlin, 5th November 2007 113
  114. 114. Invalidation • Dealing with invalidation is tricky • We can prod the cache servers directly to clear stuff out – Scales badly – need to clear asset from every server – doesn’t work well for 100 caches Web 2.0 Expo Berlin, 5th November 2007 114
  115. 115. Invalidation • We can change the URLs of modified resources – And let the old ones drop out cache naturally – Or prod them out, for sensitive data • Good approach! – Avoids browser cache staleness – Hello Akamai (and other CDNs) – Read more: • http://www.thinkvitamin.com/features/webapps/serving-javascript-fast Web 2.0 Expo Berlin, 5th November 2007 115
  116. 116. Reverse proxy • Choices – L7 load balancer & Squid • http://www.squid-cache.org/ – mod_proxy & mod_cache • http://www.apache.org/ – Perlbal and Memcache? • http://www.danga.com/ Web 2.0 Expo Berlin, 5th November 2007 116
  117. 117. High overhead serving • What if you need to authenticate your asset serving? – Private photos – Private data – Subscriber-only files • Two main approaches – Proxies w/ tokens – Path translation Web 2.0 Expo Berlin, 5th November 2007 117
  118. 118. Perlbal backhanding • Perlbal can do redirection magic – Client sends request to Perbal – Perlbl plugin verifies user credentials • token, cookies, whatever • tokens avoid data-store access – Perlbal goes to pick up the file from elsewhere – Transparent to user Web 2.0 Expo Berlin, 5th November 2007 118
  119. 119. Perlbal backhanding Web 2.0 Expo Berlin, 5th November 2007 119
  120. 120. Perlbal backhanding • Doesn’t keep database around while serving • Doesn’t keep app server around while serving • User doesn’t find out how to access asset directly Web 2.0 Expo Berlin, 5th November 2007 120
  121. 121. Permission URLs • But why bother!? • If we bake the auth into the URL then it saves the auth step • We can do the auth on the web app servers when creating HTML • Just need some magic to translate to paths • We don’t want paths to be guessable Web 2.0 Expo Berlin, 5th November 2007 121
  122. 122. Permission URLs Web 2.0 Expo Berlin, 5th November 2007 122
  123. 123. Permission URLs (or mod_perl) Web 2.0 Expo Berlin, 5th November 2007 123
  124. 124. Permission URLs • Downsides – URL gives permission for life – Unless you bake in tokens • Tokens tend to be non-expirable – We don’t want to track every token » Too much overhead • But can still expire • Upsides – It works – Scales nicely Web 2.0 Expo Berlin, 5th November 2007 124
  125. 125. 11. Storing Files Web 2.0 Expo Berlin, 5th November 2007 125
  126. 126. Storing lots of files • Storing files is easy! – Get a big disk – Get a bigger disk – Uh oh! • Horizontal scaling is the key – Again Web 2.0 Expo Berlin, 5th November 2007 126
  127. 127. Connecting to storage • NFS – Stateful == Sucks – Hard mounts vs Soft mounts, INTR • SMB / CIFS / Samba – Turn off MSRPC & WINS (NetBOIS NS) – Stateful but degrades gracefully • HTTP – Stateless == Yay! – Just use Apache Web 2.0 Expo Berlin, 5th November 2007 127
  128. 128. Multiple volumes • Volumes are limited in total size – Except (in theory) under ZFS & others • Sometimes we need multiple volumes for performance reasons – When using RAID with single/dual parity • At some point, we need multiple volumes Web 2.0 Expo Berlin, 5th November 2007 128
  129. 129. Multiple volumes Web 2.0 Expo Berlin, 5th November 2007 129
  130. 130. Multiple hosts • Further down the road, a single host will be too small • Total throughput of machine becomes an issue • Even physical space can start to matter • So we need to be able to use multiple hosts Web 2.0 Expo Berlin, 5th November 2007 130
  131. 131. Multiple hosts Web 2.0 Expo Berlin, 5th November 2007 131
  132. 132. HA Storage • HA is important for assets too – We can back stuff up – But we tend to want hot redundancy • RAID is good – RAID 5 is cheap, RAID 10 is fast Web 2.0 Expo Berlin, 5th November 2007 132
  133. 133. HA Storage • But whole machines can fail • So we stick assets on multiple machines • In this case, we can ignore RAID – In failure case, we serve from alternative source – But need to weigh up the rebuild time and effort against the risk – Store more than 2 copies? Web 2.0 Expo Berlin, 5th November 2007 133
  134. 134. HA Storage Web 2.0 Expo Berlin, 5th November 2007 134
  135. 135. Self repairing systems • When something fails, repairing can be a pain – RAID rebuilds by itself, but machine replication doesn’t • The big appliances self heal – NetApp, StorEdge, etc • So does MogileFS (reaper) Web 2.0 Expo Berlin, 5th November 2007 135
  136. 136. 12. Field Work Web 2.0 Expo Berlin, 5th November 2007 136
  137. 137. Real world examples • Flickr – Because I know it • LiveJournal – Because everyone copies it Web 2.0 Expo Berlin, 5th November 2007 137
  138. 138. Flickr Architecture Web 2.0 Expo Berlin, 5th November 2007 138
  139. 139. Flickr Architecture Web 2.0 Expo Berlin, 5th November 2007 139
  140. 140. LiveJournal Architecture Web 2.0 Expo Berlin, 5th November 2007 140
  141. 141. LiveJournal Architecture Web 2.0 Expo Berlin, 5th November 2007 141
  142. 142. Buy my book! Web 2.0 Expo Berlin, 5th November 2007 142
  143. 143. Or buy Theo’s Web 2.0 Expo Berlin, 5th November 2007 143
  144. 144. The end! Web 2.0 Expo Berlin, 5th November 2007 144
  145. 145. Awesome! These slides are available online: iamcal.com/talks/ Web 2.0 Expo Berlin, 5th November 2007 145
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×