Tips for going fast in a slow world: Michael May at OSCON 2015

CACHE MONEY:
Tips for going FAST in a
slow world
Michael May
API Engineer
@ohaimmay 2015-07-22

Real-time Content Delivery Network
Leverage fork of Varnish Cache
sub-millisecond TTFB
Enable, not constrain

ALL communication is
bounded by
the speed of light

C(vac) = 186,000 miles/s
C(fiber) = 124,000 miles/s
c(air) ~ c(vac)

We’re really impatient
(when it comes to cat gifs, at least)

The NASA Problem
Get pics from Deep Space to Earth
1 kb/sec bandwidth
Single (compressed) 2.5Mb image takes ~1 hour
Complete sensor dataset won’t arrive until 2016!
*source:
http://www.planetary.org/blogs/emily-‐lakdawalla/2015/01300800-‐talking-‐to-‐pluto-‐is-‐hard.html
New Horizons

©
Ryan
Wick
/
Wikimedia
Commons
/
CC-‐BY-‐SA-‐2.0
/
GFDL

Credit:
NASA,
Expedition
24
Crew

We have the same problem on Earth!
Credit:
NASA,
Expedition
24
Crew

mmay:~ $ ping fast.mmay.net
PING fast.mmay.net: 56 data bytes
64 bytes received: icmp_seq=0 ttl=44 time=273.722 ms
--- fast.mmay.net ping statistics —
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 269.116/293.231/503.991/77.754ms
ATL to SYD RTT (9k miles)

©
User:Timwether
/
Wikimedia
Commons
/

Moving bits around the world
takes measurable time
©
User:Timwether
/
Wikimedia
Commons
/

Moving bits around the ^world
takes measurable time
©
User:Timwether
/
Wikimedia
Commons
/
universe

ATL to SYD RTT (what we want)
mmay:~ $ ping fast.mmay.net
PING fallback.global-ssl.fastly.net (23.235.39.249): 56 data bytes
64 bytes from 23.235.39.249: icmp_seq=0 ttl=57 time=22.411 ms
--- fallback.global-ssl.fastly.net ping statistics ---
6 packets transmitted, 6 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 22.411/27.042/38.025/5.510 ms

We’re not going to beat physics
Credit:
US
Navy
/
CC-‐BY-‐2.0

We’re not going to beat physics
Credit:
US
Navy
/
CC-‐BY-‐2.0
But we will be clever engineers

> “Middle Mile” Optimizations

> “Middle Mile” Optimizations
> Global Cache

Will it cache?
Image
Credit:
©
Ryan
Notch
/
www.areographers.com
/
CC-‐SA-‐3.0
STATIC

“Event Driven” Content
> Valid for > 0 seconds (pseudo-static?)
> Changes unpredictably
> That change requires immediate update

mmay:~ $ ping new-horizons.nasa.space
PING new-horizons.nasa.space: 56 data bytes
1 byte received: time=16200s (4.5 hours)
--- ping new-horizons.nasa.space ping statistics —
1 packets transmitted, 1 packets received
SFO to PLUTO RTT (3 billion mi)

Web Perf 101
Critical path optimization / use Google PageSpeed
Rendering optimizations
Compress/Minify text (html, js, css), images, fonts
Optimize HTTP Caching Strategies!!!!

*Optimize HTTP Caching Strategies

Varnish
HTTP Reverse Proxy Cache

Is my Varnish Working?
mmay:~ $ curl -svo /dev/null https://app.fastly.com
* Connected to app.fastly.com
> GET / HTTP/1.1
> User-Agent: curl
> Host: app.fastly.com
>
< HTTP/1.1 200 OK
< Connection: keep-alive
< Via: 1.1 varnish
< X-Served-By: cache-atl6230-ATL
< X-Cache: MISS
< X-Cache-Hits: 0
< Vary: Accept-Encoding
[data not shown]
* Connection #0 to host app.fastly.com left intact

> GET / HTTP/1.1
> User-Agent: curl
>
< HTTP/1.1 200 OK
< Via: 1.1 varnish
< X-Cache: MISS
< X-Cache-Hits: 0
[data not shown]
I’ll be talking to ya

> GET / HTTP/1.1
> User-Agent: curl
>
< HTTP/1.1 200 OK
< Via: 1.1 varnish
< X-Cache: MISS
< X-Cache-Hits: 0
[data not shown]
Hey,
I was proxied

> GET / HTTP/1.1
> User-Agent: curl
>
< HTTP/1.1 200 OK
< Via: 1.1 varnish
< X-Served-By: cache-pdx999-PDX
< X-Cache: HIT
< X-Cache-Hits: 10
[data not shown]

> GET / HTTP/1.1
> User-Agent: curl
>
< HTTP/1.1 200 OK
< Via: 1.1 varnish
< X-Served-By: cache-pdx999-PDX
< X-Cache: HIT
< X-Cache-Hits: 10
[data not shown]
Cool cache stuff

*Optimize HTTP Caching Strategies
Cache-Control: public, max-age=10

Is the object in the cache valid?

Is the object in the cache valid?
aka Revalidation

HTTP Request with If-None-Match

stale-if-error / stale-while-revalidate
Cache-Control:
max-age=3600,
s-maxage=864000,
stale-while-revalidate=3600,
stale-if-error=3600

Back in the day,
the server was responsible
for rendering pages

AJAX
Extract server-side rendering of private
content into APIs
Fetch & render private data client-side
Cache API for increased performance

API Caching with event-driven purge
Set Cache-Control with a reasonably long TTL

Set Surrogate-Key(s) with unique cache key(s)

Set Surrogate-Key(s) with unique cache key(s)
Invalidate cached responses by Surrogate-Key purge

Example response handlers with API Caching
GET /content.json {
- Validation
- DB Lookup
- Set Cache-Control Header
- Set ‘Surrogate-Key: my_cache_key’ Header
- Render response
}
PUT /content.json {
- Validation
- Update DB
- Purge ‘my_cache_key’
(e.g. curl -XPOST https://api.fastly.com/purge/my_cache_key)
- Render response

AJAX “like” request flow w/ Varnish
Varnish

Request flow with Varnish Synthetic
Varnish

Synthetic Response VCL
synthetic {"OK"};

Synthetic Response VCL for “like”
hello
world
sub vcl_recv {
if (req.url ~ "^/like" && req.request == "POST") {
error 702 "OK";
}
}

hello
world
sub vcl_error {
if (obj.status == 702) {
set obj.status = 200;
set obj.http.Content-Type = “text/plain;”
synthetic {"OK"};
}
return (deliver);
}

hello
world
sub vcl_log {
if (req.url ~ "^/like" && req.method == "POST") {
log {"syslog “like :: “} req.url {" "} resp.status;
}
}

Set-Cookie
Sent by server
“Cookie” sent by client
If cookies in response, not cached by default
Vary’ing on Cookie is unwise

Vary
Specifies which headers the cache will
take into account on object lookup.
(outside of Host and Path)

Out of a sample of 100,000 requests,
there were 8,000 different User-Agents!

Caching when cookies present
1. Remove cookie and save in temp variable
2. Do normal cache lookup
3. Set cookie from temp var right before delivery
hello
world
hello
world

Caching when cookies present (VCL)
// Request received
sub vcl_recv {
// If request includes Cookie, save temporarily
// & remove for cache lookup
if (req.http.Cookie ~ "mycookie=") {
set req.http.Tmp-Set-Cookie = req.http.Cookie;
unset req.http.Cookie;
} else {
set req.hash_always_miss = true;
}
}

Caching when cookies present (VCL)
// Cache miss, fetch from origin
sub vcl_fetch {
// if origin returns Set-Cookie, save temporarily
// & remove so response can be cached
if (beresp.http.Set-Cookie) {
set req.http.Tmp-Set-Cookie = beresp.http.Set-Cookie;
unset beresp.http.Set-Cookie;
}
}
// One last thing
sub vcl_deliver {
// Include Set-Cookie in response to client
if (req.http.Tmp-Set-Cookie) {
set resp.http.Set-Cookie = req.http.Tmp-Set-Cookie;
}
}

Wrapping Up
c is our speed limit, moving data around takes time

Wrapping Up
CDNs are more than a dumb static content cache

Wrapping Up
CDNs can be more than a dumb static content cache
Cache “event driven” content and script logic at the
edge using VCL

Wrapping Up
CDNs can do more than be a static content cache
Cache “event driven” content and script logic at the
edge using VCL
Increase that CACHE HIT RATIO!

Tips for going fast in a slow world: Michael May at OSCON 2015

Tips for going fast in a slow world: Michael May at OSCON 2015

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Tips for going fast in a slow world: Michael May at OSCON 2015

Similar to Tips for going fast in a slow world: Michael May at OSCON 2015 (20)

More from Fastly

More from Fastly (20)

Recently uploaded

Recently uploaded (20)

Tips for going fast in a slow world: Michael May at OSCON 2015