Leverage HTTP to deliver cacheable websites - Thijs Feryn - Codemotion Rome 2018

Leverage HTTP to deliver
cacheable websites
Thijs Feryn

Don’t
recompute
if the data
hasn’t
changed

With ReCaPro *
User ReCaPro Server
* Reverse Caching Proxy

HTTP caching mechanisms
Expires: Sat, 09 Sep 2017 14:30:00 GMT
Cache-control: public, max-age=3600,
s-maxage=86400
Cache-control: private, no-cache, no-store

✓Stateless
✓Well-defined TTL
✓Cache / no-cache per resource
✓Cache variations
✓Conditional requests
✓HTTP fragments for non-
cacheable content
In an ideal world

Whatifwe
coulddesign
oursoftware
withHTTP
cachingin
mind?

✓Portability
✓Developer empowerment
✓Control
✓Consistent caching behavior
Caching state of mind

composer require symfony/flex
composer require annotations twig translation

<?php
namespace AppController;
use SymfonyComponentHttpFoundationRequest;
use SensioBundleFrameworkExtraBundleConfigurationRoute;
use SymfonyBundleFrameworkBundleControllerController;
class DefaultController extends Controller
{
/**
* @Route("/", name="home")
*/
public function index()
{
return $this->render('index.twig');
}
}
src/Controller/DefaultController.php

{% extends "base.twig" %}
{% block title %}Home{% endblock %}
{% block content %}
<div class="page-header">
<h1>{{ 'example' | trans }}</h1>
</div>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consequat orci
eget libero sollicitudin, non ultrices turpis mollis. Aliquam sit amet tempus elit.
Ut viverra risus enim, ut venenatis justo accumsan nec. Praesent a dolor tellus.
Maecenas non mauris leo. Pellentesque lobortis turpis at dapibus laoreet. Mauris
rhoncus nulla et urna mollis, et lobortis magna ornare. Etiam et sapien consequat,
egestas felis sit amet, dignissim enim.</p>
<p>Quisque quis mollis justo, imperdiet fermentum velit. Aliquam nulla justo,
consectetur et diam non, luctus commodo metus. Vestibulum fermentum efficitur nulla
non luctus. Nunc lorem nunc, mollis id efficitur et, aliquet sit amet ante. Sed
ipsum turpis, vehicula eu semper eu, malesuada eget leo. Vestibulum venenatis dui
id pulvinar suscipit. Etiam nec massa pharetra justo pharetra dignissim quis non
magna. Integer id convallis lectus. Nam non ullamcorper metus. Ut vestibulum ex ut
massa posuere tincidunt. Vestibulum hendrerit neque id lorem rhoncus aliquam. Duis
a facilisis metus, a faucibus nulla.</p>
{% endblock %}
templates/index.twig

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/html" xmlns:hx="http://purl.org/NET/hinclude">
<head>
<title>{% block title %}{% endblock %}</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css"
integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u"
crossorigin="anonymous">
</head>
<body>
<div class="container-fluid">
{{ include('header.twig') }}
<div class="row">
<div class="col-sm-3 col-lg-2">
{{ include('nav.twig') }}
</div>
{% block content %}{% endblock %}
</div>
</div>
{{ include('footer.twig') }}
</div>
</body>
</html>
templates/base.twig

<nav class="navbar navbar-default navbar-fixed-side">
<ul class="nav">
<li><a href="{{ url('home') }}">Home</a></li>
<li><a href="{{ url('login') }}">{{ 'log_in' | trans }}</a></li>
<li><a href="{{ url('private') }}">Private</a></li>
</ul>
</nav>
templates/nav.twig

<footer>
<hr />
<small>Footer</small>
</footer>
templates/footer.twig

home: Home
welcome : Welcome to the site
rendered : Rendered at %date%
example : An example page
log_in : Log in
login : Login
log_out : Log out
username : Username
password : Password
private : Private
privatetext : Looks like some very private data
translations/messages.en.yml

home: Start
welcome : Welkom op de site
rendered : Samengesteld op %date%
example : Een voorbeeldpagina
log_in : Inloggen
login : Login
log_out : Uitloggen
username : Gebruikersnaam
password : Wachtwoord
private : Privé
privatetext : Deze tekst ziet er vrij privé uit
translations/messages.nl.yml

<?php
namespace AppEventListener;
use SymfonyComponentHttpKernelEventGetResponseEvent;
class LocaleListener
{
public function onKernelRequest(GetResponseEvent $event)
{
$request = $event->getRequest();
$preferredLanguage = $request->getPreferredLanguage();
if(null !== $preferredLanguage) {
$request->setLocale($preferredLanguage);
}
}
}
src/EventListener/LocaleListener.php

services:
_defaults:
autowire: true
autoconfigure: true
public: false
App:
resource: '../src/*'
exclude: '../src/{Entity,Migrations,Tests,Kernel.php}'
AppController:
resource: '../src/Controller'
tags: ['controller.service_arguments']
AppEventListenerLocaleListener:
tags:
- { name: kernel.event_listener, event: kernel.request, priority: 100}
config/services.yml

Accept-Language: en
Accept-Language: nl
VS

The mission
Maximum
Cacheability

Cache-Control: public, s-maxage=500

<?php
namespace AppController;
use SensioBundleFrameworkExtraBundleConfigurationRoute;
use SymfonyBundleFrameworkBundleControllerController;
class DefaultController extends Controller
{
/**
*/
{
return $this
->render('index.twig')
->setSharedMaxAge(500)
->setPublic();
}
}

Only fetch
payload that has
changed

Otherwise:
HTTP/1.1 304 Not Modified

Conditional requests
HTTP/1.1 200 OK
Host: localhost
Etag: 7c9d70604c6061da9bb9377d3f00eb27
Content-type: text/html; charset=UTF-8
Hello world output
GET / HTTP/1.1
Host: localhost
User-Agent: curl/7.48.0

Host: localhost
Etag: 7c9d70604c6061da9bb9377d3f00eb27
GET / HTTP/1.1
Host: localhost
If-None-Match:
7c9d70604c6061da9bb9377d3f00eb27

HTTP/1.1 200 OK
Host: localhost
Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
Content-type: text/html; charset=UTF-8
Hello world output
GET / HTTP/1.1
Host: localhost

Host: localhost
Last-Modified: Fri, 22 Jul 2016 10:11:16 GMT
GET / HTTP/1.1
Host: localhost
If-Last-Modified: Fri, 22 Jul 2016 10:11:16
GMT

composer require symfony-bundles/redis-bundle

<?php
use SymfonyBridgeMonologLogger;
use SymfonyComponentHttpFoundationResponse;
use SymfonyComponentHttpKernelEventGetResponseEvent;
use SymfonyComponentHttpKernelEventFilterResponseEvent;
use SymfonyBundlesRedisBundleRedisClient as RedisClient;
class ConditionalRequestListener
{
protected $redis;
protected $logger;
public function __construct(RedisClient $redis)
{
$this->redis = $redis;
}
protected function isModified(Request $request, $etag)
{
if ($etags = $request->getETags()) {
return in_array($etag, $etags) || in_array('*', $etags);
}
return true;
}
...
src/EventListener/ConditionalRequestListener.php

{
$this->redis = $redis;
$this->logger = $logger;
}
protected function isModified(Request $request, $etag)
{
if ($etags = $request->getETags()) {
return in_array($etag, $etags) || in_array('*', $etags);
}
return true;
}
public function onKernelRequest(GetResponseEvent $event)
{
$etag = $this->redis->get('etag:'.md5($request->getUri()));
if(!$this->isModified($request,$etag)) {
$event->setResponse(Response::create('Not Modified',Response::HTTP_NOT_MODIFIED));
}
}
public function onKernelResponse(FilterResponseEvent $event)
{
$response = $event->getResponse();
$etag = md5($response->getContent());
$response->setEtag($etag);
if($this->isModified($request,$etag)) {
$this->redis->set('etag:'.md5($request->getUri()),$etag);
}
}
}
src/EventListener/ConditionalRequestListener.php

Cache-Control: private, no-store

/**
* @Route("/private", name="private")
*/
public function private()
{
$response = $this
->render('private.twig')
->setPrivate();
$response->headers->addCacheControlDirective('no-store');
return $response;
}
Dot not
cache private
page

Code
renders
singleHTTP
response

Lowest
common
denominator:
nocache

<esi:include src="/header" />
Edge Side Includes
✓Placeholder
✓Parsed by Varnish
✓Output is a composition of blocks
✓State per block
✓TTL per block

Surrogate-Capability: key="ESI/1.0"
Surrogate-Control: content="ESI/1.0"
Varnish
Backend
Parse ESI placeholdersVarnish

✓ Server-side
✓ Standardized
✓ Processed on the
“edge”, no in the
browser
✓ Generally faster
Edge-Side Includes
- Sequential
- One fails, all fail
- Limited
implementation in
Varnish

✓ Client-side
✓ Common knowledge
✓ Parallel processing
✓ Graceful
degradation
AJAX
- Processed by the
browser
- Extra roundtrips
- Somewhat slower

<div class="container-fluid"> 
{{ include('header.twig') }} 
<div class="row"> 
<div class="col-sm-3 col-lg-2"> 
{{ include('nav.twig') }} 
</div> 
{% block content %}{% endblock %} 
</div> 
</div> 
{{ include('footer.twig') }} 
</div>
<div class="container-fluid"> 
{{ render_esi(url('header')) }} 
<div class="row"> 
{{ render_esi(url('nav')) }} 
</div> 
{% block content %}{% endblock %} 
</div> 
</div> 
{{ render_esi(url('footer')) }} 
</div>

<div class="container-fluid">
<div class="row">
<esi:include src="/nav" />
</div>
<h1>An example page <small>Rendered at 2017-05-17 16:57:14</small></h1>
</div>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consequat orci eget libero
sollicitudin,…</p>
</div>
</div>
<esi:include src="/footer" />
</div>

/**
*/
{
return $this
->render('index.twig')
->setPublic()
->setSharedMaxAge(500);
}
/**
* @Route("/header", name="header")
*/
public function header()
{
$response = $this
->render('header.twig')
->setPrivate();
$response->headers->addCacheControlDirective('no-store');
return $response;
}
/**
* @Route("/footer", name="footer")
*/
public function footer()
{
$response = $this->render('footer.twig');
$response
->setPublic();
return $response;
}
/**
* @Route("/nav", name="nav")
*/
public function nav()
{
$response = $this->render('nav.twig');
$response
->setVary('X-Login',false)
->setPublic();
return $response;
}
Controller
action per
fragment

Problem:
no language
cache
variation

<?php
use SymfonyComponentHttpKernelEventFilterResponseEvent;
class VaryListener
{
public function onKernelResponse(FilterResponseEvent $event)
{
$response = $event->getResponse();
$response->setVary('Accept-Language',false);
}
}
src/EventListener/VaryListener.php

✓Navigation page
✓Private page
Weak spots
Not cached
because of
stateful content

Replace PHP session with
JSON Web Tokens

JWT
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1pb
iIsImV4cCI6MTQ5NTUyODc1NiwibG9naW4iOnRydWV9.u4Idy-
SYnrFdnH1h9_sNc4OasORBJcrh2fPo1EOTre8
✓3 parts
✓Dot separated
✓Base64 encoded JSON
✓Header
✓Payload
✓Signature (HMAC with secret)

eyJzdWIiOiJhZG1pbiIsIm
V4cCI6MTQ5NTUyODc1Niwi
bG9naW4iOnRydWV9
{
"alg": "HS256",
"typ": "JWT"
}
{
"sub": "admin",
"exp": 1495528756,
"login": true
}
HMACSHA256(
base64UrlEncode(header) + "." +
base64UrlEncode(payload),
secret
)
eyJhbGciOiJIUzI1NiIsI
nR5cCI6IkpXVCJ9
u4Idy-
SYnrFdnH1h9_sNc4OasOR
BJcrh2fPo1EOTre8

JWT
Cookie:token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJz
dWIiOiJhZG1pbiIsImV4cCI6MTQ5NTUyODc1NiwibG9naW4iOnRydW
V9.u4Idy-SYnrFdnH1h9_sNc4OasORBJcrh2fPo1EOTre8
✓Stored in a cookie
✓Can be validated by Varnish
✓Payload can be processed by any
language (e.g. Javascript)

sub jwt {
std.log("Ready to perform some JWT magic");
if(cookie.isset("jwt_cookie")) {
#Extract header data from JWT
var.set("token", cookie.get("jwt_cookie"));
var.set("header", regsub(var.get("token"),"([^.]+).[^.]+.[^.]+","1"));
var.set("type", regsub(digest.base64url_decode(var.get("header")),{"^.*?"typ"s*:s*"(w+)".*?$"},"1"));
var.set("algorithm", regsub(digest.base64url_decode(var.get("header")),{"^.*?"alg"s*:s*"(w+)".*?$"},"1"));
#Don't allow invalid JWT header
if(var.get("type") == "JWT" && var.get("algorithm") == "HS256") {
#Extract signature & payload data from JWT
var.set("rawPayload",regsub(var.get("token"),"[^.]+.([^.]+).[^.]+$","1"));
var.set("signature",regsub(var.get("token"),"^[^.]+.[^.]+.([^.]+)$","1"));
var.set("currentSignature",digest.base64url_nopad_hex(digest.hmac_sha256(var.get("key"),var.get("header") + "." + var.get("rawPayload"))));
var.set("payload", digest.base64url_decode(var.get("rawPayload")));
var.set("exp",regsub(var.get("payload"),{"^.*?"exp"s*:s*([0-9]+).*?$"},"1"));
var.set("jti",regsub(var.get("payload"),{"^.*?"jti"s*:s*"([a-z0-9A-Z_-]+)".*?$"},"1"));
var.set("userId",regsub(var.get("payload"),{"^.*?"uid"s*:s*"([0-9]+)".*?$"},"1"));
var.set("roles",regsub(var.get("payload"),{"^.*?"roles"s*:s*"([a-z0-9A-Z_-, ]+)".*?$"},"1"));
#Only allow valid userId
if(var.get("userId") ~ "^d+$") {
#Don't allow expired JWT
if(std.time(var.get("exp"),now) >= now) {
#SessionId should match JTI value from JWT
if(cookie.get(var.get("sessionCookie")) == var.get("jti")) {
#Don't allow invalid JWT signature
if(var.get("signature") == var.get("currentSignature")) {
#The sweet spot
set req.http.X-login="true";
} else {
std.log("JWT: signature doesn't match. Received: " + var.get("signature") + ", expected: " + var.get("currentSignature"));
}
} else {
std.log("JWT: session cookie doesn't match JTI." + var.get("sessionCookie") + ": " + cookie.get(var.get("sessionCookie")) + ", JTI:" + var.get("jti"));
}
} else {
std.log("JWT: token has expired");
}
} else {
std.log("UserId '"+ var.get("userId") +"', is not numeric");
}
} else {
std.log("JWT: type is not JWT or algorithm is not HS256");
}
std.log("JWT processing finished. UserId: " + var.get("userId") + ". X-Login: " + req.http.X-login);
}
#Look for full private content
if(req.url ~ "/node/2" && req.url !~ "^/user/login") {
if(req.http.X-login != "true") {
return(synth(302,"/user/login?destination=" + req.url));
}
}
}
Insert incomprehensible
Varnish VCL code here …

X-Login: true
End result:
X-Login: false
Custom request
header set by
Varnish

Extra cache
variation
required

Vary: Accept-Language, X-Login
Content for logged-in
& anonymous differs

Does not require
backend access

composer require firebase/php-jwt

<?php
namespace AppService;
use FirebaseJWTJWT;
use SymfonyComponentHttpFoundationCookie;
class JwtAuthentication
{
protected $key;
protected $username;
protected $password;
public function __construct($key,$username,$password)
{
$this->key = $key;
$this->username = $username;
$this->password = $password;
}
public function jwt($username)
{
return JWT::encode([
'sub'=>$username,
'exp'=>time() + (4 * 24 * 60 * 60),
'login'=>true,
],$this->key);
}
public function createCookie($username)
{
return new Cookie("token",$this->jwt($username), time() + (3600 * 48), '/', null,
false, false);
}
public function validate($token)
{
src/Service/JwtAuthentication.php

$this->password = $password;
}
public function jwt($username)
{
return JWT::encode([
'sub'=>$username,
'exp'=>time() + (4 * 24 * 60 * 60),
//'exp'=>time() + 60,
'login'=>true,
],$this->key);
}
public function createCookie($username)
{
return new Cookie("token",$this->jwt($username), time() + (3600 * 48), '/', null,
false, false);
}
public function validate($token)
{
try {
$data = JWT::decode($token,$this->key,['HS256']);
$data = (array)$data;
if($data['sub'] !== $this->username) {
return false;
}
return true;
} catch(UnexpectedValueException $e) {
return false;
}
}
}

services:
AppServiceJwtAuthentication:
arguments:
$key: '%env(JWT_KEY)%'
$username: '%env(JWT_USERNAME)%'
$password: '%env(JWT_PASSWORD)%'

###> JWT authentication ###
JWT_KEY=SlowWebSitesSuck
JWT_USERNAME=admin
JWT_PASSWORD=$2y$10$431rvq1qS9ewNFP0Gti/o.kBbuMK4zs8IDTLlxm5uzV7cbv8wKt0K
###< JWT authentication ###
.env

/**
* @Route("/login", name="login", methods="GET")
*/
public function login(Request $request, JwtAuthentication $jwt)
{
if($jwt->validate($request->cookies->get('token'))) {
return new RedirectResponse($this->generateUrl('home'));
}
$response = $this->render('login.twig',['loginLogoutUrl'=>$this-
>generateUrl('login'),'loginLogoutLabel'=>'log_in']);
$response
->setPublic();
return $response;
}
/**
* @Route("/login", name="loginpost", methods="POST")
*/
public function loginpost(Request $request, JwtAuthentication $jwt)
{
$username = $request->get('username');
$password = $request->get('password');
if(!$username || !$password || getenv('JWT_USERNAME') != $username || !
password_verify($password,getenv('JWT_PASSWORD'))) {
return new RedirectResponse($this->generateUrl('login'));
}
$response = new RedirectResponse($this->generateUrl('home'));
$response->headers->setCookie($jwt->createCookie($username));
return $response;
}

/**
* @Route("/logout", name="logout")
*/
public function logout()
{
$response = new RedirectResponse($this->generateUrl('login'));
$response->headers->clearCookie('token');
return $response;
}

/**
* @Route("/nav", name="nav")
*/
public function nav(Request $request, JwtAuthentication $jwt)
{
if($jwt->validate($request->cookies->get('token'))) {
$loginLogoutUrl = $loginLogoutUrl = $this->generateUrl('logout');
$loginLogoutLabel = 'log_out';
} else {
$loginLogoutUrl = $this->generateUrl('login');
$loginLogoutLabel = 'log_in';
}
$response = $this->render('nav.twig',
['loginLogoutUrl'=>$loginLogoutUrl,'loginLogoutLabel'=>$loginLogoutLabel]);
$response
->setPublic();
return $response;
}

{% extends "base.twig" %}
{% block title %}Login{% endblock %}
{% block content %}
<h1>{{ 'log_in' | trans }}</h1>
</div>
<form class="form-horizontal" method="post" action="{{ url('loginpost') }}">
<div class="form-group">
<label for="usernameInput" class="col-sm-2 control-label">{{ 'username' | trans }}</label>
<div class="col-sm-6">
<input type="text" name="username" class="form-control" id="usernameInput"
placeholder="{{ 'username' | trans }}">
</div>
</div>
<label for="passwordInput" class="col-sm-2 control-label">{{ 'password' | trans }}</label>
<div class="col-sm-6">
<input type="password" name="password" class="form-control" id="passwordInput"
placeholder="{{ 'password' | trans }}">
</div>
</div>
<div class="col-sm-offset-2 col-sm-10">
<button type="submit" class="btn btn-default">{{ 'log_in' | trans }}</button>
</div>
</div>
</form>
{% endblock %}
templates/login.twig

Cookie:
token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1Ni
J9.eyJzdWIiOiJhZG1pbiIsImV4cCI6MTUyMDk0M
jExNSwibG9naW4iOnRydWV9._161Lf8BKkbzjqVv
5d62O5aMdCotKvCqd7F8qFqZC2Y

With the proper VCL, Varnish
can process the JWT and
make cache variations for
authenticated & anonymous
content

https://github.com/ThijsFeryn/
cacheable-sites-symfony4

https://feryn.eu
https://twitter.com/ThijsFeryn
https://instagram.com/ThijsFeryn

Leverage HTTP to deliver cacheable websites - Thijs Feryn - Codemotion Rome 2018

Leverage HTTP to deliver cacheable websites - Thijs Feryn - Codemotion Rome 2018

More Related Content

What's hot

Similar to Leverage HTTP to deliver cacheable websites - Thijs Feryn - Codemotion Rome 2018

More from Codemotion

Recently uploaded

Leverage HTTP to deliver cacheable websites - Thijs Feryn - Codemotion Rome 2018