Surge 2012 fred_moyer_lightning

0 views
324 views

Published on

Lightning talk I gave at SurgeCon 2012

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
0
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Surge 2012 fred_moyer_lightning

  1. 1. How netfilter saved my bacon Fred Moyer @phredmoyer Silver Lining NetworksThursday, September 27, 12
  2. 2. Free WiFi! Ad bar inserted at the top of page pays the WiFi bill (Silver Lining Ad Bar shown here)Thursday, September 27, 12
  3. 3. Others have built it, you may have used theirs (notice this page didn’t load fully)Thursday, September 27, 12
  4. 4. How does theirs work? • Tinyproxy runs on the network gateway • Inserts Javascript into the HTTP response which splits the page into two frames, one for the ad bar, one for the web page content • Proxying HTTP responses through userspace on network devices is *slow* • Users get angry; this solution sucksThursday, September 27, 12
  5. 5. Ignorance is bliss; how I built it from scratch • iptables rules on the gateway device NAT forwarded HTTP requests to co-located mod_perl web proxy • Better performance than tinyproxy • Running all web traffic through colocation doesn’t scale though (and is really expensive)Thursday, September 27, 12
  6. 6. Making it scale • Avoid sending static content requests (images, videos, etc) through the colocated proxy • HTTP proxy rewrites static content links: • http://foo.com/image.jpg => • http://foo.com:8135/image.jpg • Redirect port 8135 to port 80 via router iptables rule: • iptables -t NAT -A PREROUTING -i $LAN -p tcp --dport 8135 -j DNAT --to :80Thursday, September 27, 12
  7. 7. Scalability achieved • 95% of traffic offloaded from the co- located proxy and fetched directly from the destination • Hillbilly architecture driven by desperation and experimentation rather than elegant planning • Performance was much better than the tinyproxy approach used by competitorsThursday, September 27, 12
  8. 8. Yo dawg, I heard you like 400s • Whoops, it doesn’t completely work • Apache handles http://foo.com:8135 requests to port 80 just fine • lighttpd throws a 400 Bad Request! • ~20% of static content requests returning 400s makes users (and network operators) angryThursday, September 27, 12
  9. 9. Linux based routers use the sk_buff socket buffer struct in kernel space. Maybe a netfilter module can remove the :8135 from the hostname...Thursday, September 27, 12
  10. 10. 3 months of Netfilter codingThursday, September 27, 12
  11. 11. Architectural Overview gateway iptables redirect to proxy GET http://foo.com/ HTTP proxy iptables -t NAT -A PREROUTING -i $LAN -p tcp --dport 80 --dst ! 192.168.0.0/16 -j DNAT --to 69.36.240.29:80Thursday, September 27, 12
  12. 12. Architectural Overview gateway iptables redirect to proxy GET http://foo.com/ HTTP proxy html response with ad, subrequest hrefs on port 8135 index.html proxied request foo.comThursday, September 27, 12
  13. 13. Architectural Overview gateway iptables redirect to proxy GET http://foo.com/ HTTP proxy html response with ad, subrequest hrefs on port 8135 index.html browser parses page, proxied request makes image subrequest GET http://foo.com:8135/bar.jpg foo.comThursday, September 27, 12
  14. 14. Architectural Overview gateway iptables redirect to proxy GET http://foo.com/ HTTP proxy html response with ad, subrequest hrefs on port 8135 index.html browser parses page, proxied request makes image subrequest netfilter module removes :8135 hostport GET http://foo.com:8135/bar.jpg foo.com subrequest bypasses proxy, fetches image directlyThursday, September 27, 12
  15. 15. It works (finally) • ‘Host: foo.com:8135’ => ‘Host: foo.com’ in kernel space. No user space copying. • < 500 ms additional latency for main page requests through the co-located HTTP proxy • Blows tinyproxy out of the water • Product didn’t survive business needs though, the ad revenue wasn’t there :(Thursday, September 27, 12
  16. 16. Thank you Surge 2012 • Stuck in an architectural dead end? Maybe this approach can help you. • github.com/redhotpenguin/SL-Kernel • www.skbuff.net/skbuff.html • banu.com/tinyproxyThursday, September 27, 12

×