Edge Caching Dynamic Apps

Edge Caching Dynamic Apps
GORUCO | NYC | June 21, 2014
Michael May | @ehm_may

GoRuCo | 6/21/2014 | @ehm_may
Overview
• Caching
• Content Delivery Networks (CDNs)
• Modern Innovations in CDNs (dynamic edge caching)
• Dynamic edge caching with Ruby/Rails

Caching Foundations
• I have data stored somewhere (HDD, DB, Cloud)
• I access that data often
• Move data closer to where I am
• Move data to faster storage !
• Side effect: Reduce load on original storage location

Terminology

Terminology
CACHE HIT!
requested data is contained in
the cache

Terminology
CACHE MISS!
requested data is NOT in the
cache
!

Terminology
HIT RATIO!
percentage of accesses that
result in cache hits

Terminology
PURGE!
data is removed from the cache
!

Terminology
Origin!
HTTP application server
(e.g. Rails server)
!

Terminology
Edge!
HTTP caching server
!

Caches operate with a limited
amount of storage

Need a strategy for what to
keep in cache

Least Recently Used is one type
of strategy

LRU Interface
• set(object) - O(1) - stores object, calls prune( ) (if cache is full)

LRU Interface
• get(key) - O(1) - returns object, moves it to the “head” of the cache

LRU Interface
• get(key) - O(1) - returns object, moves it to the “head” of the cache
• prune( ) - removes object from “tail” of the cache

LRU Implementation
• HashTable to store data - get( ) and set( ) functions
• Doubly Linked List for tracking “time since last use”

Cache Hierarchy
CPU/HARDWARE CACHE (L1-L4)
MAIN MEMORY
MEMCACHED / VARNISH CACHE
CONTENT DELIVERY NETWORK
SOFTWARE/APPLICATION CACHE (e.g. Rails Cache)

Content Delivery Network is a
globally distributed network of
cache servers

Content Delivery Networks

Traditionally CDNs only cache
static content

Static Content
•Images
•Javascripts
•Stylesheets

Edge caching static content is easy!
The Asset Pipeline builds links with the asset_host
!
Rails:
# config/environments/production.rb

config.action_controller.asset_host = “mycdn.coolcdn.com”

!
Sinatra: (with something like sinatra_asset_pipeline)
# in app.rb
set :assets_host, ‘mycdn.coolcdn.com’

Cache invalidation is hard.
Especially when your cache is
globally distributed

Content Delivery Networks
• How data/requests get to the edge
• Controlling content with HTTP Headers
• Lesser known CDN features

How data gets to “The Edge”

Push | Pull

PUSH CDN
• You “push” content to the CDN
• Usually happens when an asset is updated
• Harder to keep assets in sync
• Can strain origin server when pushing

PULL CDN
• CDN “pulls” content from your origin server
• Seamless updates when assets change
• Small latency cost on ﬁrst request
• Acts as a reverse proxy

Reverse Proxy

1st request
2nd, …, n requests

How requests get to “The Edge”
* The edge closest to you…usually

Request Routing
• Request made by client

Request Routing
• Request triggers DNS lookup

Request Routing
• DNS resolves request to a geographical region

Request Routing
• DNS resolves request to a geographical region
• Request forwarded to CDN Edge nearest to that region

Controlling content on your
Content Delivery Network
Using HTTP Headers

Cache-Control HTTP Headers

Cache-Control
• Specify behavior intended to prevent caches from interfering with
the request or response
• Override the default caching algorithm
• Specify what can be cached
• Specify how long to cache

Cache-Control: max-age=3600

Cache-Control: max-age!
!
Indicates how long to keep response
fresh. Respected by all shared caches
(CDN) and private caches (browser).

Use Cache-Control: max-age as an
umbrella strategy to cache
responses

Cache-Control: private, max-age=3600

Cache-Control: private!
!
Indicates response MUST NOT be
cached by shared caches. However,
private caches may cache.

Use Cache-Control: private when
responses contain sensitive user-
speciﬁc data

Cache-Control: s-maxage=3600

Cache-Control: s-maxage!
!
Indicates how long to keep
response fresh. Respected ONLY
by shared caches.

Use Cache-Control: s-maxage to
specify different times to cache
between shared and private caches

Cache-Control:no-cache
Cache-Control:max-age=0
DON’T CACHE ME!

Surrogate-Control

“Surrogate” refers to reverse
proxies

Surrogate-Control: max-age=3600

Surrogate-Control: max-age!
!
Indicates how long to keep
response fresh. Respected ONLY
by reverse proxies.

Sneaky Example
Cache-Control: max-age=3600

Sneaky Example
• Surrogate-Control takes priority over Cache-Control
• Surrogate will cache for 1 year
• Browser will cache for 30 minutes
• Sometimes Surrogate-Control is stripped by the Surrogate

Interesting features in modern
CDNs

Transmission Control Protocol

Transmission Control Protocol
What web browsers use to connect to servers

How TCP establishes
connections

TCP Handshake
aka “3-Way” Handshake

HTTP Keep-Alive

HTTP Keep-Alive Header
• Connection: Keep-Alive
• TCP Session is kept open for an extended period
• Eliminate costly TCP Handshake
• In CDN Land edge nodes may keep connections alive with origin =>
faster cache misses (Dynamic Site Acceleration)

Instant Purging

Instant Purging
• PURGE request comes in to edge node
• Edge node purges and passes to on to nearest neighbors
• Recurse until everyone has purged
brucespang.com/bimodal

Instant Purging
• Purges happen globally in < 300ms on 95th percentile
• Enables dynamic content caching at the edge
brucespang.com/bimodal
www.fastly.com/blog/building-fast-and-reliable-purging-system

Dynamic Caching
• Explain process
• Example using Rails Fragment caching
• Map fragment caching to edge caching

Dynamic Content

Dynamic Content
• Data that changes frequently

Dynamic Content
• Frequently does not mean continuously

Dynamic Content
• Frequently does not mean continuously
• Periods of time where data is not being changed

Dynamic Caching is when you
cache data in-between regeneration

Beneﬁcial when dynamic data is
requested multiple times

Dynamic Caching
• API requests
• Comments
• News article headlines
• E-commerce inventory
• Search results

Dynamic Caching - How To
1. Create unique cache keys

2. Bind data to cache keys

3. Purge with cache key on data updates

Rails Fragment Caching

# products/index.html.erb
<% Product.all.each do |p| %>
<% cache(p.cache_key) do %>
<%= link_to p.name, product_url(p) %>
<% end %>
<% end %>
!
# in products_controller.rb
expire_fragment(cache_key)

If you can Rails Fragment Cache,
you can Dynamic Edge Cache

How dynamic caching is done
on the edge

In CDN land, Surrogate-Keys are
your cache keys

Surrogate-Key is a HTTP
Header that tells the cache to
associate key(s) with response
data

Use Surrogate-Key on HTTP GETs
GET /product/123

!
HTTP/1.1 200 OK

Content-Type: text/json


Surrogate-Key: products product/123
{id: 123, name: “Kewl Keyboard”, price: “$124.99”}

3. Purge with cache key on data updates
PURGE Surrogate-Key on !
HTTP POST, PUT, DELETE
PUT /product/123

{price: “$1.00”}

!
# In PUT request handler

PURGE /product/123

Implementing edge caching in
Ruby on Rails

fastly-rails!
github.com/fastly/fastly-rails
!
Contains helpers for integrating with
dynamic edge caching

Dynamic Edge Caching on Rails
1. Extend models with surrogate key instance methods

class Product < ActiveRecord::Base
!
def resource_key
“#{table_key}/#{id}" # e.g. “products/1234”
end
!
def table_key
self.class.table_name # e.g. “products”
end
end

2. Add ActionController helpers to set up headers

class ProductsController < ApplicationController
before_ﬁlter :set_cache_control_headers, only: [:index, :show]
!
def index
@products = Product.all
set_surrogate_key_header @product.table_key
respond_with @products
end
!
def show
@product = Products.ﬁnd(params[:id])
set_surrogate_key_header @product.record_key
respond_with @product
end
end

before_ﬁlter :set_cache_control_headers, only: [:index, :show]!
!
def index
set_surrogate_key_header @product.table_key
end
!
def show
set_surrogate_key_header @product.record_key
end
end

def set_cache_control_headers(max_age = 3600)
response.headers['Cache-Control'] = "public, no-cache"
response.headers['Surrogate-Control'] = "max-age=#{max_age}"
request.session_options[:skip] = true # don’t cache cookies
end

before_ﬁlter :set_cache_control_headers, only: [:index, :show]
!
def index
set_surrogate_key_header @product.table_key!
end
!
def show
set_surrogate_key_header @product.record_key!
end
end

def set_surrogate_key_header(key)
response.headers['Surrogate-Key'] = key
end

2. Add ActionController helpers to set up headers
3. Add purge/purge_all instance methods to models

!
def create
@product = Product.new(params)
if @product.save
@product.purge_all!
render @product
end
end

def update
@product = Product.ﬁnd(params[:id])
if @product.update(params)
@product.purge!
render @product
end
end

def delete
@product = Product.ﬁnd(params[:id])
if @product.destroy
@product.purge_all!
end
end
end

fastly-rails!

Use fastly-rails to iteratively set
up dynamic edge caching

In Summary
• Ofﬂoad static assets onto a content delivery network

In Summary
• CDN Instant purging enables dynamic edge caching

In Summary
• CDN Instant purging enables dynamic edge caching
• Use fastly-rails to integrate dynamic edge caching into Rails

Thank You
michael@fastly.com | @ehm_may

Edge Caching Dynamic Apps

Recommended

Recommended

More Related Content

Similar to Edge Caching Dynamic Apps

Similar to Edge Caching Dynamic Apps (20)

Recently uploaded

Recently uploaded (20)

Edge Caching Dynamic Apps