Honing headers for highly hardened highspeed hypertext

presents
Honing HTTP
headers
The good and the bad of the Web’s hidden metadata
Andrew Betts, Fastly

$ telnet bank.example.com 80
GET /statement HTTP/1.1
Host: bank.example.com
HTTP/1.1 200 OK
Date: Tue, 27 Feb 2018 13:28:47 GMT
Content-Type: text/html
Content-Length: 34882
<html>
...

HTTP/1.1 200 OK
Cache-Control: private, max-age=3600
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Last-Modified: Fri, 02 Feb 2018 07:21:05 GMT
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>

• Alt-Svc
• Feature-Policy
• Origin-Policy
• Clear-Site-Data
New HTTP response headers
• Integrity
• Signature

• Alexa top 1,000,000 websites
• Around 500,000 pages analysed
• Over 50 million requests per run
• Captures full request and
response data, timing metrics
etc.
• Runs using WebPageTest
• Makes raw result data available
in BigQuery

https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

P3P
Machine readable privacy policy
https://www.w3.org/TR/P3P11/
Domains sending
9.8%
Commonly set to
cp="this is not a p3p policy"
Standardised in
2002

• Intended as a declaration of privacy policy
• Too hard for users to understand/use
• Only ever implemented by Internet Explorer, to gate access to third
party cookies in IFRAMEs.
– ... but not validated
• Commonly set to “this is not a P3P policy” which satisfies the check
Platform for Privacy Preferences Project?
https://bigquery.cloud.google.com/savedquery/598614557294:9c69db8c47f84c4d9a4b57668ac8ba58

Expires
Sets expiry time for local caching
https://tools.ietf.org/html/rfc7234#section-5.3
Domains sending
78%
Standardised in
1997

Expires: Thu, 01 Dec 1994 16:00:00 GMT

“Note: if a response includes a Cache-Control
field with the max-age directive, that directive
overrides the Expires field.”

Expires: Thu, 01 Dec 1994 16:00:00 GMT
Cache-Control: private, no-store, no-cache, no-transform, must-revalidate, max-age=0, post-check=0, pre-
check=0
Pragma: no-cache

Domains sending Expires
78%
Domains sending Expires and
Cache-Control with max-age:
64%
https://bigquery.cloud.google.com/savedquery/598614557294:98e14323d29740678fe1b3012c9186db

Cache-Control: private, no-store

X-Cache
Records whether the page came from cache
upstream (probably)
Domains sending
13%
Standardised in
Never
Commonly set to
HIT

x-cache x-aspnet-version x-varnish x-request-
id x-cache-hits x-cacheable x-aspnetmvc-
version x-runtime x-generator x-drupal-cache
host referer x-served-by x-proxy-cache server
x-type x-cache-group x-cache-status x-accel-
version
Meaningless to the browser
All headers shown above are returned by at least 5,000 domains in the HTTP Archive dataset
https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

Outputting debug headers conditionally
if (!req.http.Reveal-Debug && !req.http.Cookie:RevealDebug) {
unset resp.http.Server;
unset resp.http.X-Powered-By;
unset resp.http.X-Cache;
// ... etc
}
Works in Fastly and Varnish cache

X-Frame-Options
Prohibits third party framing of your site
https://tools.ietf.org/html/rfc7034
Domains sending
22.7%
Standardised in
2013
Almost always set to
sameorigin

Stop anyone from framing your site:
Content-Security-Policy: frame-ancestors 'self'
But... equivalent to...
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

Via
Lists proxies through which the request passes
https://httpwg.org/specs/rfc7230.html#header.via
Domains sending
8.7%
Standardised in
1997

Proxy behaviour
User Fastly Server
Via: 1.1 varnish
Via: 1.1 varnish

Proxy forwarding loops
User Fastly Another CDN™

Content-Security-
Policy
Restricts network access from a page
https://w3c.github.io/webappsec-csp/
Domains sending
2.9%
Standardised in
2016
Average length
593 bytes

Content-Security-Policy: default-src 'self'; img-src *; media-src media1.com media2.com; script-src
userscripts.example.com
Simple CSP example:
By default, only allow access to the same origin as the page. But allow images to come from
anywhere. Media and scripts from a whitelist of specific origins.

Content-Security-Policy: default-src 'self'; font-src 'self' data: http://*.gstatic.com https://*.gstatic.com https://*.symantec.com https://*.criteo.com http://*.resultspage.com https://*.resultspage.com; child-src 'self' https://www.google.com/ads/ https://*.listrak.com
https://www.google.pl/ads/ https://www.google.ua/ads/ http://*.google.com.ua/ads/ https://www.youtube.com/ https://www.googleadservices.com/ https://googleads.g.doubleclick.net/pagead/ https://*.symantec.com https://*.criteo.com http://*.rfksrv.com
https://*.rfksrv.com http://*.resultspage.com https://*.resultspage.com; object-src 'self' http://*.verisign.com http://*.iesnare.com https://*.verisign.com https://*.iesnare.com https://*.symantec.com https://*.zmags.com; img-src 'self' data: https://ssl.emailcli.com *.sli-
spark.com https://*.veinteractive.com http://*.bazaarvoice.com/ http://blog.natchezss.com/ https://blog.natchezss.com/ http://*.avmws.com http://*.websecurity.norton.com http://*.google.com.ua http://*.natchezss.com https://www.pepperjamnetwork.com/
http://*.gstatic.com http://*.ywxi.net https://www.google.com/ads/ http://s3.amazonaws.com/product.reflektion.com/ http://*.reflektion.com/ https://stats.g.doubleclick.net/ https://d26opx5dl8t69i.cloudfront.net/ http://*.google-analytics.com/
https://seal.networksolutions.com/ http://*.listrakbi.com/ http://*.chartbeat.net/ http://*.scanalert.com/ http://*.avantlink.com https://*.bazaarvoice.com/ https://*.listrakbi.com/ https://*.chartbeat.net/ https://*.google-analytics.com/ https://*.reflektion.com/
https://*.bazaarvoice.com/ https://*.google.com.ua https://*.listrakbi.com/ https://*.chartbeat.net/ https://*.scanalert.com/ https://*.avantlink.com https://*.amazonaws.com/ https://*.scanalert.com/ https://*.norton.com https://*.ywxi.net/ https://*.cloudfront.net/
https://*.zmags.com/ https://*.adnxs.com http://*.adnxs.com https://*.symantec.com https://*.r1cdn.com https://ad.doubleclick.net https://go.flx1.com https://*.g.doubleclick.net https://*.optimizely.com https://*.yahoo.com https://*.sitescout.com https://*.1rx.io
https://*.tubemogul.com https://*.simpli.fi https://*.ipredictive.com https://*.wtp101.com https://*.pubmatic.com https://*.media.net https://*.demdex.net https://*.smartclip.net https://*.bit.ly https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com
http://*.resultspage.com https://*.resultspage.com; script-src 'self' 'unsafe-inline' 'unsafe-eval' *.sli-r.com *.sli-spark.com *.resultspage.com *.resultsstage.com tagmanager.google.com *.googletagmanager.com https://*.veinteractive.com http://*.avmws.com
http://*.optimizely.com http://*.iesnare.com https://*.listrak.com https://www.googleadservices.com/ http://*.verisign.com http://*.googleapis.com http://*.nr-data.net http://*.newrelic.com http://*.ywxi.net http://*.bazaarvoice.com/ http://*.cloudfront.net/
http://*.listrakbi.com/ https://seal.networksolutions.com/ http://*.google-analytics.com/ https://ping.chartbeat.net/ http://*.reflektion.com/ http://*.chartbeat.com/ https://*.listrak.com/ https://product.reflektion.com/ https://display.ugc.bazaarvoice.com/
https://www.google-analytics.com/ https://cdn.listrakbi.com/ https://*.verisign.com https://*.googleapis.com https://*.nr-data.net https://*.newrelic.com https://*.ywxi.net https://*.bazaarvoice.com/ https://*.cloudfront.net/ https://*.listrakbi.com/ https://*.google-
analytics.com/ https://*.reflektion.com/ https://*.chartbeat.com/ https://*.optimizely.com https://*.avmws.com/ http://*.zmags.com https://*.zmags.com https://*.dpmsrv.com http://*.dpmsrv.com https://*.adnxs.com/ http://*.adnxs.com/ https://*.doubleclick.net
http://*.doubleclick.net https://*.google.com https://*.gstatic.com/ https://*.symantec.com https://*.statsstory.com https://*.hotjar.com https://*.cloudfront.net https://*.jsdelivr.net https://c.vepxl1.net https://c.flx1.com https://go.flx1.com https://*.youtube.com
https://s3.amazonaws.com https://*.ytimg.com http://*.criteo.com https://*.criteo.com http://*.criteo.net https://*.criteo.net http://*.resultspage.com https://*.resultspage.com https://*.sli-r.com; style-src 'self' 'unsafe-inline' tagmanager.google.com
http://*.amazonaws.com/ http://*.reflektion.com/ http://*.googleapis.com http://*.bazaarvoice.com/ https://*.bazaarvoice.com/ https://*.amazonaws.com/ https://*.listrakbi.com http://*.listrakbi.com https://*.symantec.com https://*.cloudfront.net https://*.criteo.com
http://*.rfksrv.com https://*.rfksrv.com http://*.resultspage.com https://*.resultspage.com; frame-src 'self' http://*.bazaarvoice.com http://*.listrak.com http://*.youtube.com/ http://*.zmags.com/ woobox.com https://*.bazaarvoice.com https://*.listrak.com
https://*.youtube.com/ https://*.zmags.com/ https://*.listrakbi.com https://*.googleadservices.com http://*.googleadservices.com https://*.doubleclick.net http://*.doubleclick.net https://*.google.com https://*.google.ad https://*.google.ae https://*.google.com.af
https://*.google.com.ag https://*.google.com.ai https://*.google.al https://*.google.am https://*.google.co.ao https://*.google.com.ar https://*.google.as https://*.google.at https://*.google.com.au https://*.google.az https://*.google.ba https://*.google.com.bd
https://*.google.be https://*.google.bf https://*.google.bg https://*.google.com.bh https://*.google.bi https://*.google.bj https://*.google.com.bn https://*.google.com.bo https://*.google.com.br https://*.google.bs https://*.google.bt https://*.google.co.bw
https://*.google.by https://*.google.com.bz https://*.google.ca https://*.google.cd https://*.google.cf https://*.google.cg https://*.google.ch https://*.google.ci https://*.google.co.ck https://*.google.cl https://*.google.cm https://*.google.cn https://*.google.com.co
https://*.google.co.cr https://*.google.com.cu https://*.google.cv https://*.google.com.cy https://*.google.cz https://*.google.de https://*.google.dj https://*.google.dk https://*.google.dm https://*.google.com.do https://*.google.dz https://*.google.com.ec
https://*.google.ee https://*.google.com.eg https://*.google.es https://*.google.com.et https://*.google.fi https://*.google.com.fj https://*.google.fm https://*.google.fr https://*.google.ga https://*.google.ge https://*.google.gg https://*.google.com.gh
https://*.google.com.gi https://*.google.gl https://*.google.gm https://*.google.gp https://*.google.gr https://*.google.com.gt https://*.google.gy https://*.google.com.hk https://*.google.hn https://*.google.hr https://*.google.ht https://*.google.hu
https://*.google.co.id https://*.google.ie https://*.google.co.il https://*.google.im https://*.google.co.in https://*.google.iq https://*.google.is https://*.google.it https://*.google.je https://*.google.com.jm https://*.google.jo https://*.google.co.jp
https://*.google.co.ke https://*.google.com.kh https://*.google.ki https://*.google.kg https://*.google.co.kr https://*.google.com.kw https://*.google.kz https://*.google.la https://*.google.com.lb https://*.google.li https://*.google.lk https://*.google.co.ls
https://*.google.lt https://*.google.lu https://*.google.lv https://*.google.com.ly https://*.google.co.ma https://*.google.md https://*.google.me https://*.google.mg https://*.google.mk https://*.google.ml https://*.google.com.mm https://*.google.mn
https://*.google.ms https://*.google.com.mt https://*.google.mu https://*.google.mv https://*.google.mw https://*.google.com.mx https://*.google.com.my https://*.google.co.mz https://*.google.com.na https://*.google.com.nf https://*.google.com.ng
https://*.google.com.ni https://*.google.ne https://*.google.nl https://*.google.no https://*.google.com.np https://*.google.nr https://*.google.nu https://*.google.co.nz https://*.google.com.om https://*.google.com.pa https://*.google.com.pe
https://*.google.com.pg https://*.google.com.ph https://*.google.com.pk https://*.google.pl https://*.google.pn https://*.google.com.pr https://*.google.ps https://*.google.pt https://*.google.com.py https://*.google.com.qa https://*.google.ro https://*.google.ru
https://*.google.rw https://*.google.com.sa https://*.google.com.sb https://*.google.sc https://*.google.se https://*.google.vu https://*.google.ws https://*.google.rs https://*.google.co.za https://*.google.co.zm https://*.google.co.zw https://*.google.cat
https://*.symantec.com https://*.mcafeesecure.com https://*.veinteractive.com https://*.hotjar.com https://*.youtube.com https://*.hotjar.com https://*.criteo.com *.dotomi.com http://*.rfksrv.com https://*.rfksrv.com; connect-src 'self' https://*.statsstory.com
https://*.hotjar.com wss://*.hotjar.com https://*.veinteractive.com https://*.servicebus.windows.net https://*.optimizely.com https://*.youtube.com https://*.criteo.com;
9643 bytes!

Strict-Transport-
Security
Always connect over TLS, even for first request
https://tools.ietf.org/html/rfc6797
Domains sending
11.0%
Standardised in
2012

Strict-Transport-Security: max-age=10886400;
Kill off insecure HTTP once and for all

What to send in referrals from this page
https://w3c.github.io/webappsec-referrer-policy/
Referrer-Policy
Domains sending
2.4%
Standardised in
2017

Referrer policy options
Policy Referrer (same origin) Referrer (to foreign origin)
no-referrer omitted omitted
no-referrer-when-downgrade https://example.com/page.html?query https://example.com/page.html?query
origin https://example.com https://example.com
origin-when-cross-origin https://example.com/page.html?query https://example.com
same-origin https://example.com/page.html?query omitted

Link (preload)
Declare a resource that’s important early on
https://w3c.github.io/preload/#x2.link-type-preload
Domains sending
19.1%
Standardised in
2016

Link: <some-font-face.woff2>; rel="preload"; as="font"; crossorigin
Link: <main-styles.css>; rel="preload"; as="style"
Get your fonts and styles down sharpish:

https://calendar.perfplanet.com/2017/a-tinder-progressive-web-app-performance-case-study/

https://speakerdeck.com/
patrickhamann/
css-and-the-first-meaningful-
paint-css-conf-eu-may-2017

Status code must
be sent before headers

The status code problem
DNS
Lookup
TLS TTFB
Status code +
LINK headers
received
Database
Auth
Templating
API queries

Sends headers before status code
https://tools.ietf.org/html/draft-ietf-httpbis-early-hints-05
103 Early Hints
Domains sending
Unknown
Status
Proposed

HTTP/1.1 103 Early Hints
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 10:02:11 GMT
Content-Type: text/html; charset=utf-8
<!doctype html>
Get your fonts and styles down even sharper:

Preloaded resources start preloading earlier
DNS
Lookup
TLS
Early hints +
Link headers
received
Real status
received

Selectively adjusts browser capabilities
https://wicg.github.io/feature-policy/
Feature-Policy
Domains sending
0.001%
Status
Proposed

Feature-Policy: autoplay 'none'; speaker 'self'; unsized-media some3rdparty.com
Just turn off the bad stuff
No-one can
autoplay video on
this page
Only I can use the
speakers. No noisy
ads, thanks.
Images only take size
from their contents when
loaded from
some3rdparty.com

Feature-Policy: doc-write 'none'; sync-xhr 'none'; sync-resources 'none'
The future-future:
Bye bye
document.write.
No more
synchronous
XMLHTTPRequest
All JS and CSS is
deferrable

Moves origin-wide headers into a file
https://wicg.github.io/origin-policy/
Sec-Origin-
Policy
Domains sending
0
Status
Proposed

HTTP/1.1 200 OK
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>
...

{
"headers": [
{
"name": "Content-Security-Policy",
"value": "script-src 'self' https://cdn.example.com",
"type": "fallback"
}, {
"name": "Referrer-Policy",
"value": "origin-when-cross-origin",
"type": "fallback"
}, {
"name": "Content-Security-Policy",
"value": "object-src 'none'; frame-ancestors 'none'",
"type": "baseline"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=10886400; includeSubDomains; preload",
"type": "baseline"
}, {
"name": "X-Content-Type-Options",
"value": "nosniff",
"type": "baseline"
}
],
"cors-preflight": {
"origins": "*"
}
}
/.well-known/origin-policy/policy-1

HTTP/1.1 200 OK
Sec-Origin-Policy: "policy-1"
Vary: sec-origin-policy
<html>
...

Integrity
Signature
Accept-CH
Early Hints
Sec-Origin-Policy
P3P
Expires
X-Cache
X-Frame-Options
Via
Recap
Feature-Policy
Content-Security-Policy
Strict-Transport-Security
Referrer-Policy
Link
Alt-Svc
Clear-Site-Data
Expect-CT
X-Content-Type-Options

Thanks for listening
I am Get the slides:
Andrew Betts
@triblondon
abetts@fastly.com
fastly.us/headers

Honing headers for highly hardened highspeed hypertext

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Honing headers for highly hardened highspeed hypertext

Similar to Honing headers for highly hardened highspeed hypertext (20)

More from Fastly

More from Fastly (20)

Recently uploaded

Recently uploaded (20)

Honing headers for highly hardened highspeed hypertext

Editor's Notes