High-performance                   Robust                   HTTP                   Front-ends                           / ...
Who am I? @postwait on twitter                           Author of “Scalable Internet Architectures”                      ...
Agenda                      •    Why only HTTP?                      •    HTTP-like protocols                      •    Pe...
HTTP                      •    Why only HTTP... it’s what we do.                      •    User-based, immediate, short-li...
Performance                      •    ATS (Apache Traffic Server)                           •   supports SSL              ...
Performance Expectations                      •    from a single server, you should be able to:                           ...
Performance Achievements                      •    Good load balancers achieve this performance                      •    ...
The Basic Rules: Content                      •    You must serve content from cache                      •    Your cache ...
The Basic Rules: CPU                      •    You must cache SSL sessions                           •   SSL key negotiati...
The Basic Rules: Network                      •    You must not run a stateful firewall in front                          ...
Availability                      •    We learned in the performance section:                           •   1 machine / 10...
Availability: Constraints                      •    Client TCP sessions are relatively short lived.                      •...
Availability: Setup                      •    You are behind a capable router (it was a rule)                      •    Us...
Working Stacks       •       Linux       (OS/TCP stack)   •   Illumos (OS/TCP stack)       •       Varnish (HTTP)         ...
Future!                      •    This stuff is fast.                      •    In the end, we’re not looking for faster s...
Future: my thoughts                      •    SPDY is relatively simple to implement on the server                      • ...
Thank you.                      •    Thank you Олег Бунин                      •    Thanks to the Varnish and ATS develope...
Upcoming SlideShare
Loading in...5
×

Http front-ends

3,147

Published on

The HTTP caching talk I gave at RIT++

http://ritconf.ru/

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,147
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
28
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Http front-ends

  1. 1. High-performance Robust HTTP Front-ends / tips, tricks and expectationsSaturday, April 23, 2011
  2. 2. Who am I? @postwait on twitter Author of “Scalable Internet Architectures” Pearson, ISBN: 067232699X Contributor to “Web Operations” O’Reilly, ISBN: Founder of OmniTI, Message Systems, Fontdeck, & Circonus I like to tackle problems that are “always on” and “always growing.” I am an Engineer A practitioner of academic computing. IEEE member and Senior ACM member. On the Editorial Board of ACM’s Queue magazine. 2Saturday, April 23, 2011
  3. 3. Agenda • Why only HTTP? • HTTP-like protocols • Performance • AvailabilitySaturday, April 23, 2011
  4. 4. HTTP • Why only HTTP... it’s what we do. • User-based, immediate, short-lived transactions occupy my life. • So, not just HTTP. • HTTPS • SPDY (... we’ll get to this)Saturday, April 23, 2011
  5. 5. Performance • ATS (Apache Traffic Server) • supports SSL • battle-hardened codebase • very multi-code capable • Varnish • VCL adds unparalleled flexibility • no SSL! • nginx • I don’t see much of this out on the edgeSaturday, April 23, 2011
  6. 6. Performance Expectations • from a single server, you should be able to: • support 500k concurrent users • this is only 40k sockets/core • push in excess of 100k requests/second • this is only 9k requests/core*second • push close to 10 gigabits • this is why 10G was inventedSaturday, April 23, 2011
  7. 7. Performance Achievements • Good load balancers achieve this performance • with dual socket Westmere processors, we’re able to achieve in software on general purpose hardware what was only possible in hardware ASICs. • ATS and Varnish can do this today.Saturday, April 23, 2011
  8. 8. The Basic Rules: Content • You must serve content from cache • Your cache should fit in memory • If it does not, it should spill to SSD, not spinning media.Saturday, April 23, 2011
  9. 9. The Basic Rules: CPU • You must cache SSL sessions • SSL key negotiation is expensive. • SSL encryption is not* • Common cases must not cause state on the firewall. • It’s hard enough to serve 150k requests/second. • You will spend too much time in kernel in iptables, ipf, or pf. • allow port 80 and port 443. • enable SYN flood prevention * crypto obviously costs CPU; symmetric crypto is relatively cheapSaturday, April 23, 2011
  10. 10. The Basic Rules: Network • You must not run a stateful firewall in front • too expensive • too little value • You must be directly behind capable router(s) • expect anywhere from 1MM to 20MM packets per second • we need to run BGP for availabilitySaturday, April 23, 2011
  11. 11. Availability • We learned in the performance section: • 1 machine / 10Gbps uplink performs well enough • We need redundancy: • Linux HA? • VRRP/HSRP? • CARP? • No...Saturday, April 23, 2011
  12. 12. Availability: Constraints • Client TCP sessions are relatively short lived. • The web is a largely idempotent place. • Clients are capable of retrying on failure. • This means: • forget stateful failover. • focus on availability for new connections.Saturday, April 23, 2011
  13. 13. Availability: Setup • You are behind a capable router (it was a rule) • Use routing protocols (BGP) to maintain availability. BGP 10.1.0.0/24 10.1.1.0/24 10.1.0.0/23 10.1.0.0/23Saturday, April 23, 2011
  14. 14. Working Stacks • Linux (OS/TCP stack) • Illumos (OS/TCP stack) • Varnish (HTTP) • ATS (HTTP/HTTPS) • Quagga (BGP) • Quagga (BGP)Saturday, April 23, 2011
  15. 15. Future! • This stuff is fast. • In the end, we’re not looking for faster servers, we’re looking for improved user experience. • Enter SPDY • Google’s multi-channel HTTP super-protocol • Allows multiplexing of concurrent HTTP(like) request/response on a single TCP session. • Defeats slow startup • Allows for content prioritization on serverSaturday, April 23, 2011
  16. 16. Future: my thoughts • SPDY is relatively simple to implement on the server • SPDY is very very hard to leverage on the server • If ATS implemented SPDY in and out • and provided a robust configuration language to leverage it ... the future would be today.Saturday, April 23, 2011
  17. 17. Thank you. • Thank you Олег Бунин • Thanks to the Varnish and ATS developers. • Спасибо.Saturday, April 23, 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×