Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Choosing A Proxy Server
ApacheCon 2014
Bryan Call
ATS Committer / Yahoo
About Me
• Yahoo! Employee
– WebRing, GeoCities, Personals, Tiger Team, Platform
Architect, Edge Team, Research, ATS and H...
Overview
• Types of Proxies
• Features
• Architecture
• Cache Architecture
• Performance
• Pros and Cons
How are you going to
use a proxy server?
Reverse Proxy
Reverse Proxy
• Proxy in front of your own web servers
• Caching?
• Geographic location?
• Connection handling?
• SSL term...
Forward Proxy
Intercepting Proxy
Forward / Intercepting Proxy
• Proxy in front of the Internet
• Configure clients to use proxy?
• Caching?
• SSL - CONNECT...
Choices
Plenty of Proxy Servers
PerlBal
Plenty of Proxy Servers
Features And Options
Features
ATS NGiNX Squid Varnish Apache httpd
mod_proxy
Reverse Proxy Y Y Y Y Y
Forward Proxy Y N Y N Y
Transp. Proxy Y N ...
SSL Features
Source: https://istlsfastyet.com/ - Ilya Grigorik
What type of proxy do you need?
• Of our candidates, only three fully supports all
proxy modes
HTTP/1.1 Compliance
HTTP/1.1 Compliance
• Accept-Encoding - gzip
• Vary
• Age
• If-None-Match
How things can go wrong: Vary
$ curl -D - -o /dev/null -s --compress http://10.118.73.168/
HTTP/1.1 200 OK
Server: nginx/1...
How things can go wrong: Vary
$ curl -D - -o /dev/null -s http://10.118.73.168/
HTTP/1.1 200 OK
Server: nginx/1.3.9
Date: ...
CoAdvisor HTTP protocol quality tests
for reverse proxies
0 100 200 300 400 500 600
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varn...
CoAdvisor HTTP protocol quality tests
for reverse proxies
0 100 200 300 400 500 600
ATS 3.3.1
Nginx 1.3.9
Squid 3.2.5
Varn...
Architecture
Architecture And Process Models
• Multithreading
• Events
• Process
• Fibers
– Co-operative multitasking, getcontext/setco...
Threads
Thread 1
Thread 2
Thread 3
Thread 1
Thread 3
Time
Single CPU
Thread 1 Thread 2
Thread 3
Thread 1
Thread 3
Time
Dua...
Threads
• Pros
– Easy to share memory
– Lightweight context switching
• Cons
– Easy to (accidently) share memory
• Overwri...
Event Processing
Event
Loop
Scheduled
events
Network
events
Disk I/O
events
Disk
handler
HTTP state
machine
Accept
handler...
Problems with Event Processing
• Doesn’t work well with
blocking APIs
– open(), locking
• It doesn’t scale on SMP by
itself
Process Model And Architecture
ATS NGiNX Squid Varnish Apache httpd
mod_proxy
Threads X X X
Events X X X partial X
Process...
Caching Architecture
Cache
• Mainly two types
– File system
– Database like
• In memory index
– Bytes per object
• Minimize disk seeks and syst...
Cache
ATS NGiNX Squid Varnish Apache httpd
mod_cache
File system X X X
mmap X
Raw disk/direct IO X X
Ram cache X X
Memory ...
Performance Testing
ATS Configuration
etc/trafficserver/remap.config:
map / http://origin.example.com
etc/trafficserver/records.config:
CONFIG...
NGiNX Configuration
worker_processes 24;
access_log logs/access.log main;
proxy_cache_path /mnt/nginx_cache levels=1:2 key...
Squid Configuration
http_access allow all
http_port 80 accel
workers 24
cache_mem 4096 MB
memory_cache_shared on
cache_dir...
Varnish Configuration
backend default {
.host = ”origin.example.com”;
.port = "80";
}
Varnish Configuration (Cont)
sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p
thread_pool_max=4000
s...
Apache httpd Configuration
LoadModule cache_module modules/mod_cache.so
LoadModule cache_disk_module modules/mod_cache_dis...
Benchmark 1
• 1,000 clients
• 8KB response
• 100% cache hit
• Keep-alive on
• 100K rps rate limited
• Squid used the most CPU
and the worst median
latency
• 95th percentile latency
with NiGNX, Squid and
httpd 0
500
1000
15...
Benchmark 2
• 1,000 clients
• 8KB response
• 100% cache hit
• Keep-alive off
• Squid used the most
CPU again
• NGiNX had latency
issues
• ATS most throughput 0
500
1000
1500
2000
2500
ATS NGiNX Squid...
ATS
• Pros
– Scales well automatically, little config needed
– Best cache implementation
• Cons
– Too many config files
– ...
NGiNX
• Pros
– Lots of plugins
– FastCGI support
• Cons
– HTTP/1.1 compliance
– Latency issues around accepting new connec...
Squid
• Pros
– Best HTTP/1.1 compliance
• Cons
– Memory index for cache using 10x that of ATS
– Least efficient with CPU
–...
Varnish
• Pros
– VCL (Varnish Configuration Language)
• Can do a lot without writing plugins
• Cons
– Thread per connectio...
Apache httpd
• Pros
– Lots of plugins
– Most used http server
– Best 95th percentile latency for non-keep-alive
• Cons
– S...
Why ATS?
• Scales well
– CPU Usage, auto config
• Cache scales well
– Efficient memory index, minimizes seeks
• Apache Com...
References
• ATS - http://trafficserver.apache.org/
• NGiNX - http://nginx.org/
• Squid - http://www.squid-cache.org/
• Va...
Choosing A Proxy Server - Apachecon 2014
Upcoming SlideShare
Loading in …5
×

Choosing A Proxy Server - Apachecon 2014

44,849 views

Published on

Published in: Internet, Technology
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Choosing A Proxy Server - Apachecon 2014

  1. 1. Choosing A Proxy Server ApacheCon 2014 Bryan Call ATS Committer / Yahoo
  2. 2. About Me • Yahoo! Employee – WebRing, GeoCities, Personals, Tiger Team, Platform Architect, Edge Team, Research, ATS and HTTP (HTTP/2 and TLS at IETF) • Working on Traffic Server for 7 years – Since 2007 • Part of the team that open sourced it in 2009 • ATS Committer
  3. 3. Overview • Types of Proxies • Features • Architecture • Cache Architecture • Performance • Pros and Cons
  4. 4. How are you going to use a proxy server?
  5. 5. Reverse Proxy
  6. 6. Reverse Proxy • Proxy in front of your own web servers • Caching? • Geographic location? • Connection handling? • SSL termination? • SPDY support? • Adding business logic?
  7. 7. Forward Proxy
  8. 8. Intercepting Proxy
  9. 9. Forward / Intercepting Proxy • Proxy in front of the Internet • Configure clients to use proxy? • Caching? • SSL - CONNECT? • SSL - termination?
  10. 10. Choices
  11. 11. Plenty of Proxy Servers PerlBal
  12. 12. Plenty of Proxy Servers
  13. 13. Features And Options
  14. 14. Features ATS NGiNX Squid Varnish Apache httpd mod_proxy Reverse Proxy Y Y Y Y Y Forward Proxy Y N Y N Y Transp. Proxy Y N Y N Y Plugin APIs Y Y partial Y Y Cache Y Y Y Y Y ESI Y N Y partial N ICP Y N Y N N SSL Y Y Y N Y SPDY Y* Y N N partial * 5.0.0 (May 2014)
  15. 15. SSL Features Source: https://istlsfastyet.com/ - Ilya Grigorik
  16. 16. What type of proxy do you need? • Of our candidates, only three fully supports all proxy modes
  17. 17. HTTP/1.1 Compliance
  18. 18. HTTP/1.1 Compliance • Accept-Encoding - gzip • Vary • Age • If-None-Match
  19. 19. How things can go wrong: Vary $ curl -D - -o /dev/null -s --compress http://10.118.73.168/ HTTP/1.1 200 OK Server: nginx/1.3.9 Date: Wed, 12 Dec 2012 18:00:48 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000 Expires: Sun, 19 Nov 1978 05:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip
  20. 20. How things can go wrong: Vary $ curl -D - -o /dev/null -s http://10.118.73.168/ HTTP/1.1 200 OK Server: nginx/1.3.9 Date: Wed, 12 Dec 2012 18:00:57 GMT Content-Type: text/html; charset=utf-8 Content-Length: 8051 Connection: keep-alive Cache-Control: public, max-age=900 Last-Modified: Wed, 12 Dec 2012 17:52:42 +0000 Expires: Sun, 19 Nov 1978 05:00:00 GMT Vary: Cookie,Accept-Encoding Content-Encoding: gzip EPIC FAIL! Note: no gzip request
  21. 21. CoAdvisor HTTP protocol quality tests for reverse proxies 0 100 200 300 400 500 600 ATS 3.3.1 Nginx 1.3.9 Squid 3.2.5 Varnish 3.0.3 Failures Violations Success 49% 81% 51% 68%
  22. 22. CoAdvisor HTTP protocol quality tests for reverse proxies 0 100 200 300 400 500 600 ATS 3.3.1 Nginx 1.3.9 Squid 3.2.5 Varnish 3.0.3 Failures Violations Success 25% 6% 27% 15%
  23. 23. Architecture
  24. 24. Architecture And Process Models • Multithreading • Events • Process • Fibers – Co-operative multitasking, getcontext/setcontext
  25. 25. Threads Thread 1 Thread 2 Thread 3 Thread 1 Thread 3 Time Single CPU Thread 1 Thread 2 Thread 3 Thread 1 Thread 3 Time Dual CPU
  26. 26. Threads • Pros – Easy to share memory – Lightweight context switching • Cons – Easy to (accidently) share memory • Overwriting another threads memory – Locking • Deadlocks, race conditions, starvation
  27. 27. Event Processing Event Loop Scheduled events Network events Disk I/O events Disk handler HTTP state machine Accept handler Queue Can generate new events
  28. 28. Problems with Event Processing • Doesn’t work well with blocking APIs – open(), locking • It doesn’t scale on SMP by itself
  29. 29. Process Model And Architecture ATS NGiNX Squid Varnish Apache httpd mod_proxy Threads X X X Events X X X partial X Processes X X X
  30. 30. Caching Architecture
  31. 31. Cache • Mainly two types – File system – Database like • In memory index – Bytes per object • Minimize disk seeks and system calls
  32. 32. Cache ATS NGiNX Squid Varnish Apache httpd mod_cache File system X X X mmap X Raw disk/direct IO X X Ram cache X X Memory index X X X* Persistent cache X X X X
  33. 33. Performance Testing
  34. 34. ATS Configuration etc/trafficserver/remap.config: map / http://origin.example.com etc/trafficserver/records.config: CONFIG proxy.config.http.server_ports STRING 80 CONFIG proxy.config.accept_threads INT 3
  35. 35. NGiNX Configuration worker_processes 24; access_log logs/access.log main; proxy_cache_path /mnt/nginx_cache levels=1:2 keys_zone=my-cache:8m max_size=16384m inactive=600m; proxy_temp_path /mnt/nginx_temp; server { set $ae ""; if ($http_accept_encoding ~* gzip) { set $ae "gzip"; } location / { proxy_pass http://origin.example.com; proxy_cache my-cache; proxy_set_header If-None-Match ""; proxy_set_header If-Modified-Since ""; proxy_set_header Accept-Encoding $ae; proxy_cache_key $uri$is_args$args$ae; } location ~ /purge_it(/.*) { proxy_cache_purge example.com $1$is_args$args$myae }
  36. 36. Squid Configuration http_access allow all http_port 80 accel workers 24 cache_mem 4096 MB memory_cache_shared on cache_dir rock /usr/local/squid/cache 1000 max-size=32768 cache_peer origin.example.com parent 80 0 no-query originserver
  37. 37. Varnish Configuration backend default { .host = ”origin.example.com”; .port = "80"; }
  38. 38. Varnish Configuration (Cont) sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=4000 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=200 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=0 sudo /usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl -p thread_pool_max=2000 -p thread_pool_add_delay=2 -p thread_pool_min=1000 -p session_linger=10
  39. 39. Apache httpd Configuration LoadModule cache_module modules/mod_cache.so LoadModule cache_disk_module modules/mod_cache_disk.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_http_module modules/mod_proxy_http.so Include conf/extra/httpd-mpm.conf ProxyPass / http://origin.example.com/ <IfModule mod_cache_disk.c> CacheRoot /usr/local/apache2/cache CacheEnable disk / CacheDirLevels 5 CacheDirLength 3 </IfModule> MaxKeepAliveRequests 10000
  40. 40. Benchmark 1 • 1,000 clients • 8KB response • 100% cache hit • Keep-alive on • 100K rps rate limited
  41. 41. • Squid used the most CPU and the worst median latency • 95th percentile latency with NiGNX, Squid and httpd 0 500 1000 1500 2000 2500 ATS NGiNX Squid Varnish httpd RPS / CPU Usage 0 20000 40000 60000 80000 100000 120000 ATS NGiNX Squid Varnish httpd Requests Per Second 0 2 4 6 8 10 12 14 16 18 ATS NGiNX Squid Varnish httpd Latency Median 95th
  42. 42. Benchmark 2 • 1,000 clients • 8KB response • 100% cache hit • Keep-alive off
  43. 43. • Squid used the most CPU again • NGiNX had latency issues • ATS most throughput 0 500 1000 1500 2000 2500 ATS NGiNX Squid Varnish httpd RPS / CPU Usage 0 5000 10000 15000 20000 25000 30000 ATS NGiNX Squid Varnish httpd Requests Per Second 0 5 10 15 20 25 30 35 40 ATS NGiNX Squid Varnish httpd Latency Median 95th
  44. 44. ATS • Pros – Scales well automatically, little config needed – Best cache implementation • Cons – Too many config files – Too many options in the default config files
  45. 45. NGiNX • Pros – Lots of plugins – FastCGI support • Cons – HTTP/1.1 compliance – Latency issues around accepting new connections – Rebuild server for new plugins
  46. 46. Squid • Pros – Best HTTP/1.1 compliance • Cons – Memory index for cache using 10x that of ATS – Least efficient with CPU – Worst median latency for keep-alive benchmarks
  47. 47. Varnish • Pros – VCL (Varnish Configuration Language) • Can do a lot without writing plugins • Cons – Thread per connection – mmap for cache • Persistence is experimental – No SSL or SPDY support
  48. 48. Apache httpd • Pros – Lots of plugins – Most used http server – Best 95th percentile latency for non-keep-alive • Cons – SPDY Support
  49. 49. Why ATS? • Scales well – CPU Usage, auto config • Cache scales well – Efficient memory index, minimizes seeks • Apache Community • Plugin support – Easy to port existing plugins over
  50. 50. References • ATS - http://trafficserver.apache.org/ • NGiNX - http://nginx.org/ • Squid - http://www.squid-cache.org/ • Varnish - https://www.varnish-cache.org/ • Apache httpd - http://httpd.apache.org/

×