JS Fest 2019. Andrew Betts. Headers for hackers

Headers for hackers
Uncovering the Web’s superpowers
Andrew Betts, Fastly

$ telnet bank.example.com 80
GET /statement HTTP/1.1
Host: bank.example.com
HTTP/1.1 200 OK
Date: Tue, 27 Feb 2018 13:28:47 GMT
Content-Type: text/html
Content-Length: 34882
<html>
...

HTTP/1.1 200 OK
Cache-Control: private, max-age=3600
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Last-Modified: Fri, 02 Feb 2018 07:21:05 GMT
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>

• Alt-Svc
• Feature-Policy
• Origin-Policy
• Clear-Site-Data
New HTTP response headers
• Integrity
• Signature
• Accept-CH
• Server-Timing

ClearClear-Site-Data
Clears the browser cache
https://www.w3.org/TR/clear-site-data/
Domains sending
0.01%
Standardised in
2017

Clear-Site-Data: "cache", "cookies", "storage", "executionContexts"
Delete an entire origin’s storage

https://calendar.perfplanet.com/2017/clearing-cache-in-the-browser/

• Alexa top 1,000,000 websites
• Around 500,000 pages analysed
• Over 50 million requests per run
• Captures full request and
response data, timing metrics etc.
• Runs using WebPageTest
• Makes raw result data available in
BigQuery

https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

P3PP3P
Machine readable privacy policy
https://www.w3.org/TR/P3P11/
Domains sending
9.8%
Commonly set to
cp="this is not a p3p policy"
Standardised in
2002

• Intended as a declaration of privacy policy
• Too hard for users to understand/use
• Only ever implemented by Internet Explorer, to gate access to third party
cookies in IFRAMEs.
– ... but not validated
• Commonly set to “this is not a P3P policy” which satisfies the check
Platform for Privacy Preferences Project?
https://bigquery.cloud.google.com/savedquery/598614557294:9c69db8c47f84c4d9a4b57668ac8ba58

ExpiExpires
Sets expiry time for local caching
https://tools.ietf.org/html/rfc7234#section-5.3
Domains sending
78%
Standardised in
1997

Expires: Thu, 01 Dec 1994 16:00:00 GMT

“Note: if a response includes a Cache-Control
field with the max-age directive, a recipient
MUST ignore the Expires field.”

Domains sending Expires
78%
Domains sending Expires and
Cache-Control with max-age:
64%
https://bigquery.cloud.google.com/savedquery/598614557294:98e14323d29740678fe1b3012c9186db

Expires: Thu, 01 Dec 1994 16:00:00 GMT
Cache-Control: private, no-store, no-cache, no-transform, must-revalidate,
max-age=0, post-check=0, pre-check=0
Pragma: no-cache

Cache-Control: private, no-store

X-CacX-Cache
Records whether the page came from cache
upstream (probably)
Domains sending
13%
Standardised in
Never
Commonly set to
HIT

x-cache x-aspnet-version x-varnish x-request-id
x-cache-hits x-cacheable x-aspnetmvc-version
x-runtime x-generator x-drupal-cache host
referer x-served-by x-proxy-cache server
x-type x-cache-group x-cache-status
x-accel-version
Meaningless to the browser
All headers shown above are returned by at least 5,000 domains in the HTTP Archive dataset
https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

ExpressJS vs Varnish/edge cache
if (!req.http.Reveal-Debug && !req.http.Cookie:RevealDebug) {
unset resp.http.Server;
unset resp.http.X-Powered-By;
unset resp.http.X-Cache;
// ... etc
}
Works in Fastly and Varnish cache
app.disable('x-powered-by');

X-FraX-Frame-Options
Prohibits third party framing of your site
https://tools.ietf.org/html/rfc7034
Domains sending
22.7%
Standardised in
2013
Almost always set to
sameorigin

Stop anyone from framing your site:
Content-Security-Policy: frame-ancestors 'self'
But... equivalent to...
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

ViaVia
Lists proxies through which the request passes
https://httpwg.org/specs/rfc7230.html#header.via
Domains sending
8.7%
Standardised in
1997

Proxy behaviour
User Fastly Server
Via: 1.1 varnish
Via: 1.1 varnish

Proxy forwarding loops
User Fastly Another CDN™

ConteContent-Security-
Policy
Restricts network access from a page
https://w3c.github.io/webappsec-csp/
Domains sending
2.9%
Standardised in
2016
Average length
593 bytes

Content-Security-Policy: default-src 'self'; img-src *; media-src media1.com
media2.com; script-src userscripts.example.com
Simple CSP example:
By default, only allow access to the same origin as the page. But allow images to come from
anywhere. Media and scripts from a whitelist of specific origins.

Content-Security-Policy: default-src 'self'; font-src 'self' data: http://*.gstatic.com https://*.gstatic.com https://*.symantec.com https://*.criteo.com http://*.resultspage.com
https://*.resultspage.com; child-src 'self' https://www.google.com/ads/ https://*.listrak.com https://www.google.pl/ads/ https://www.google.ua/ads/ http://*.google.com.ua/ads/
https://www.youtube.com/ https://www.googleadservices.com/ https://googleads.g.doubleclick.net/pagead/ https://*.symantec.com https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com
http://*.resultspage.com https://*.resultspage.com; object-src 'self' http://*.verisign.com http://*.iesnare.com https://*.verisign.com https://*.iesnare.com https://*.symantec.com
https://*.zmags.com; img-src 'self' data: https://ssl.emailcli.com *.sli-spark.com https://*.veinteractive.com http://*.bazaarvoice.com/ http://blog.natchezss.com/ https://blog.natchezss.com/
http://*.avmws.com http://*.websecurity.norton.com http://*.google.com.ua http://*.natchezss.com https://www.pepperjamnetwork.com/ http://*.gstatic.com http://*.ywxi.net
https://www.google.com/ads/ http://s3.amazonaws.com/product.reflektion.com/ http://*.reflektion.com/ https://stats.g.doubleclick.net/ https://d26opx5dl8t69i.cloudfront.net/
http://*.google-analytics.com/ https://seal.networksolutions.com/ http://*.listrakbi.com/ http://*.chartbeat.net/ http://*.scanalert.com/ http://*.avantlink.com https://*.bazaarvoice.com/
https://*.listrakbi.com/ https://*.chartbeat.net/ https://*.google-analytics.com/ https://*.reflektion.com/ https://*.bazaarvoice.com/ https://*.google.com.ua https://*.listrakbi.com/
https://*.chartbeat.net/ https://*.scanalert.com/ https://*.avantlink.com https://*.amazonaws.com/ https://*.scanalert.com/ https://*.norton.com https://*.ywxi.net/ https://*.cloudfront.net/
https://*.zmags.com/ https://*.adnxs.com http://*.adnxs.com https://*.symantec.com https://*.r1cdn.com https://ad.doubleclick.net https://go.flx1.com https://*.g.doubleclick.net
https://*.optimizely.com https://*.yahoo.com https://*.sitescout.com https://*.1rx.io https://*.tubemogul.com https://*.simpli.fi https://*.ipredictive.com https://*.wtp101.com
https://*.pubmatic.com https://*.media.net https://*.demdex.net https://*.smartclip.net https://*.bit.ly https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com http://*.resultspage.com
https://*.resultspage.com; script-src 'self' 'unsafe-inline' 'unsafe-eval' *.sli-r.com *.sli-spark.com *.resultspage.com *.resultsstage.com tagmanager.google.com *.googletagmanager.com
https://*.veinteractive.com http://*.avmws.com http://*.optimizely.com http://*.iesnare.com https://*.listrak.com https://www.googleadservices.com/ http://*.verisign.com http://*.googleapis.com
http://*.nr-data.net http://*.newrelic.com http://*.ywxi.net http://*.bazaarvoice.com/ http://*.cloudfront.net/ http://*.listrakbi.com/ https://seal.networksolutions.com/
http://*.google-analytics.com/ https://ping.chartbeat.net/ http://*.reflektion.com/ http://*.chartbeat.com/ https://*.listrak.com/ https://product.reflektion.com/
https://display.ugc.bazaarvoice.com/ https://www.google-analytics.com/ https://cdn.listrakbi.com/ https://*.verisign.com https://*.googleapis.com https://*.nr-data.net https://*.newrelic.com
https://*.ywxi.net https://*.bazaarvoice.com/ https://*.cloudfront.net/ https://*.listrakbi.com/ https://*.google-analytics.com/ https://*.reflektion.com/ https://*.chartbeat.com/
https://*.optimizely.com https://*.avmws.com/ http://*.zmags.com https://*.zmags.com https://*.dpmsrv.com http://*.dpmsrv.com https://*.adnxs.com/ http://*.adnxs.com/ https://*.doubleclick.net
http://*.doubleclick.net https://*.google.com https://*.gstatic.com/ https://*.symantec.com https://*.statsstory.com https://*.hotjar.com https://*.cloudfront.net https://*.jsdelivr.net
https://c.vepxl1.net https://c.flx1.com https://go.flx1.com https://*.youtube.com https://s3.amazonaws.com https://*.ytimg.com http://*.criteo.com https://*.criteo.com http://*.criteo.net
https://*.criteo.net http://*.resultspage.com https://*.resultspage.com https://*.sli-r.com; style-src 'self' 'unsafe-inline' tagmanager.google.com http://*.amazonaws.com/
http://*.reflektion.com/ http://*.googleapis.com http://*.bazaarvoice.com/ https://*.bazaarvoice.com/ https://*.amazonaws.com/ https://*.listrakbi.com http://*.listrakbi.com
https://*.symantec.com https://*.cloudfront.net https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com http://*.resultspage.com https://*.resultspage.com; frame-src 'self'
http://*.bazaarvoice.com http://*.listrak.com http://*.youtube.com/ http://*.zmags.com/ woobox.com https://*.bazaarvoice.com https://*.listrak.com https://*.youtube.com/ https://*.zmags.com/
https://*.listrakbi.com https://*.googleadservices.com http://*.googleadservices.com https://*.doubleclick.net http://*.doubleclick.net https://*.google.com https://*.google.ad
https://*.google.ae https://*.google.com.af https://*.google.com.ag https://*.google.com.ai https://*.google.al https://*.google.am https://*.google.co.ao https://*.google.com.ar
https://*.google.as https://*.google.at https://*.google.com.au https://*.google.az https://*.google.ba https://*.google.com.bd https://*.google.be https://*.google.bf
https://*.google.bg https://*.google.com.bh https://*.google.bi https://*.google.bj https://*.google.com.bn https://*.google.com.bo https://*.google.com.br https://*.google.bs
https://*.google.bt https://*.google.co.bw https://*.google.by https://*.google.com.bz https://*.google.ca https://*.google.cd https://*.google.cf https://*.google.cg
https://*.google.ch https://*.google.ci https://*.google.co.ck https://*.google.cl https://*.google.cm https://*.google.cn https://*.google.com.co https://*.google.co.cr
https://*.google.com.cu https://*.google.cv https://*.google.com.cy https://*.google.cz https://*.google.de https://*.google.dj https://*.google.dk https://*.google.dm
https://*.google.com.do https://*.google.dz https://*.google.com.ec https://*.google.ee https://*.google.com.eg https://*.google.es https://*.google.com.et https://*.google.fi
https://*.google.com.fj https://*.google.fm https://*.google.fr https://*.google.ga https://*.google.ge https://*.google.gg https://*.google.com.gh https://*.google.com.gi
https://*.google.gl https://*.google.gm https://*.google.gp https://*.google.gr https://*.google.com.gt https://*.google.gy https://*.google.com.hk https://*.google.hn
https://*.google.hr https://*.google.ht https://*.google.hu https://*.google.co.id https://*.google.ie https://*.google.co.il https://*.google.im https://*.google.co.in
https://*.google.iq https://*.google.is https://*.google.it https://*.google.je https://*.google.com.jm https://*.google.jo https://*.google.co.jp https://*.google.co.ke
https://*.google.com.kh https://*.google.ki https://*.google.kg https://*.google.co.kr https://*.google.com.kw https://*.google.kz https://*.google.la https://*.google.com.lb
https://*.google.li https://*.google.lk https://*.google.co.ls https://*.google.lt https://*.google.lu https://*.google.lv https://*.google.com.ly https://*.google.co.ma
https://*.google.md https://*.google.me https://*.google.mg https://*.google.mk https://*.google.ml https://*.google.com.mm https://*.google.mn https://*.google.ms
https://*.google.com.mt https://*.google.mu https://*.google.mv https://*.google.mw https://*.google.com.mx https://*.google.com.my https://*.google.co.mz https://*.google.com.na
https://*.google.com.nf https://*.google.com.ng https://*.google.com.ni https://*.google.ne https://*.google.nl https://*.google.no https://*.google.com.np https://*.google.nr
https://*.google.nu https://*.google.co.nz https://*.google.com.om https://*.google.com.pa https://*.google.com.pe https://*.google.com.pg https://*.google.com.ph https://*.google.com.pk
https://*.google.pl https://*.google.pn https://*.google.com.pr https://*.google.ps https://*.google.pt https://*.google.com.py https://*.google.com.qa https://*.google.ro
https://*.google.ru https://*.google.rw https://*.google.com.sa https://*.google.com.sb https://*.google.sc https://*.google.se https://*.google.vu https://*.google.ws
https://*.google.rs https://*.google.co.za https://*.google.co.zm https://*.google.co.zw https://*.google.cat https://*.symantec.com https://*.mcafeesecure.com https://*.veinteractive.com
https://*.hotjar.com https://*.youtube.com https://*.hotjar.com https://*.criteo.com *.dotomi.com http://*.rfksrv.com https://*.rfksrv.com; connect-src 'self' https://*.statsstory.com
https://*.hotjar.com wss://*.hotjar.com https://*.veinteractive.com https://*.servicebus.windows.net https://*.optimizely.com https://*.youtube.com https://*.criteo.com;
9643 bytes!
unsafe-inline
unsafe-eval
*.cloudfront.net
tagmanager.google.com
*.amazonaws.com

ExpecX-Content-Type-
Options
aka “Please actually believe my content type”
https://fetch.spec.whatwg.org/#x-content-type-options-header
Domains sending
20.0%
Introduced in
IE8
Almost always set to
nosniff

• You operate a site that accepts user uploads and provides a means of
rendering user content
• Attacker uploads a dangerous file, pretending to be something else
• Attacker somehow gets your site’s users to view that file
• Even though your server applies an image Content-Type, the browser
executes the file as script or renders a web page because it detects the
type.
"MIME confusion attack"

• Content-Type-Options only applies to script-like and style destinations
– Does not apply to navigations
– Browsers already refuse to sniff content on navigation
• Chrome and Firefox will refuse to sniff JavaScript if the advertised content
type starts with ‘image/’
• In practice, attack window is exceptionally small
However...
http://great-sausage.glitch.me/

Maybe prevent some very niche XSS vuln:

StricStrict-Transport-
Security
Always connect over TLS, even for first request
https://tools.ietf.org/html/rfc6797
Domains sending
11.0%
Standardised in
2012

Strict-Transport-Security: max-age=10886400;
Kill off insecure HTTP once and for all

AccepWhat to send in referrals from this page
https://w3c.github.io/webappsec-referrer-policy/
Referrer-Policy
Domains sending
2.4%
Standardised in
2017

Referrer policy options
Policy Referrer (same origin) Referrer (to foreign origin)
no-referrer omitted omitted
no-referrer-when-downgrade https://example.com/page.html?query https://example.com/page.html?query
origin https://example.com https://example.com
origin-when-cross-origin https://example.com/page.html?query https://example.com
same-origin https://example.com/page.html?query omitted

AccesAllow access to cross-origin resources
https://fetch.spec.whatwg.org/#cors-protocol
Access-Control-*
Domains sending
6.7%
Standardised in
2010

Access-Control-Allow-Origin: requesting-origin.example.com
Access-Control-Allow-Credentials: true
Access-Control-Max-Age: 86400
Access-Control-Expose-Headers: X-Debug
Access-Control-Allow-Headers: DNT, User-Agent
Allow authenticated cross-origin requests:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

AccepRequests client hints data be sent in future
http://httpwg.org/http-extensions/client-hints.html
Accept-CH
Domains sending
0.001%
Status
Proposed

Accept-CH: DPR, Width, Viewport-Width, Save-Data
Accept-CH-Lifetime: 86400
Request that client send all current CHs:
DPR: 2.0
Width: 320
Viewport-Width: 320
Save-Data: 1
Subsequent requests:

Proposed new client hints!
Sec-CH-UA: "Chrome 74"
Sec-CH-UA-Platform: "macOS 12"
Sec-CH-UA-Arch: "ARM64"
Sec-CH-UA-Engine: "Blink"
Sec-CH-Lang: "en-US", "en", "de"

LinkLink (preload)
Declare a resource that’s important early on
https://w3c.github.io/preload/#x2.link-type-preload
Domains sending
19.1%
Standardised in
2016

Link: <some-font-face.woff2>; rel="preload"; as="font"; crossorigin
Link: <main-styles.css>; rel="preload"; as="style"
Get your fonts and styles down sharpish:

https://calendar.perfplanet.com/2017/a-tinder-progressive-web-app-performance-case-study/

Status code must
be sent before headers

The status code problem
DNS
Lookup
TLS TTFB
Status code +
LINK headers
received
Database
Auth
Templating
API queries

EarlySends headers before status code
https://tools.ietf.org/html/draft-ietf-httpbis-early-hints-05
103 Early Hints
Domains sending
Unknown
Status
Proposed

HTTP/1.1 103 Early Hints
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 10:02:11 GMT
Content-Type: text/html; charset=utf-8
<!doctype html>
Get your fonts and styles down even sharper:

Preloaded resources start preloading earlier
DNS
Lookup
TLS
Early hints +
Link headers
received
Real status
received

ServeAllows exposure of granular server-timing data
https://www.w3.org/TR/server-timing
Server-Timing

Server-Timing: miss, db;dur=53, app;dur=47.2
Server-Timing: customView, dc;desc=atl
Server-Timing: cache;desc="Cache Read";dur=23.2
Tell the browser what happened on the server:

Server-timing metrics appear in developer tools

Wait, what...
A JavaScript API into a header
dictionary?

Server-Timing: customerID;desc=1234567, edition;desc="EMEA"
Server-Timing: paywallArticlesRemaining;desc=8
Tell the browser…. anything?:
performance.getEntriesByType('navigation')[0]
.serverTiming
.reduce((soFar, entry) => ({
...soFar,
[entry.name]: entry.description
}), {})
;
{customerID: "1234567", edition: "EMEA", paywallArticlesRemaining: "8"}

But server-timing might be unknown in headers
DNS
Lookup
TLS TTFB
Headers
sent
Only now we know:
Total bytes sent
Last byte timestamp
Client data rate

HTTP/1.1 200 OK
Date: Tue, 27 Feb 2018 13:28:47 GMT
Content-Length: 12
Server-Timing: miss, db;dur=53, app;dur=47.2
Trailers: Server-Timing
Hello world!
Server-Timing: sentduration;dur=234
Headers, but like, at the end?

FeatuSelectively adjusts browser capabilities
https://wicg.github.io/feature-policy/
Feature-Policy
Domains sending
0.001%
Status
Proposed

Feature-Policy: autoplay 'none'; speaker 'self'; unsized-media some3rdparty.com
Turn off the bad stuff
No-one can
autoplay video on
this page
Only I can use the
speakers. No noisy
ads, thanks.
Images only take size
from their contents when
loaded from
some3rdparty.com

What policies are available?
autoplay
camera
document-domain
encrypted-media
fullscreen
geolocation
microphone
midi
payment
vr

Feature-Policy: accelerometer 'none'; ambient-light-sensor 'none'; animations
'none'; autoplay 'none'; camera 'none'; document-write 'none'; encrypted-media
'none'; fullscreen 'none'; geolocation 'none'; gyroscope 'none';
image-compression 'none'; legacy-image-formats 'none'; magnetometer 'none';
max-downscaling-image 'none'; microphone 'none'; midi 'none'; payment 'none';
picture-in-picture 'none'; speaker 'none'; sync-script 'none'; sync-xhr 'none';
unsized-media 'none'; usb 'none'; vertical-scroll 'none'; vr 'none';
Today's tightest feature-policy (in Chrome):

OrigiMoves origin-wide headers into a file
https://wicg.github.io/origin-policy/
Sec-Origin-
Policy
Domains sending
0
Status
Proposed

HTTP/1.1 200 OK
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>
...

{
"headers": [
{
"name": "Content-Security-Policy",
"value": "script-src 'self' https://cdn.example.com",
"type": "fallback"
}, {
"name": "Referrer-Policy",
"value": "origin-when-cross-origin",
"type": "fallback"
}, {
"name": "Content-Security-Policy",
"value": "object-src 'none'; frame-ancestors 'none'",
"type": "baseline"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=10886400; includeSubDomains; preload",
"type": "baseline"
}, {
"name": "X-Content-Type-Options",
"value": "nosniff",
"type": "baseline"
}
],
"cors-preflight": {
"origins": "*"
}
}
/.well-known/origin-policy/policy-1

HTTP/1.1 200 OK
Sec-Origin-Policy: "policy-1"
Vary: sec-origin-policy
<html>
...

Integrity
Signature
Accept-CH
Early Hints
Sec-Origin-Policy
P3P
Expires
X-Cache
X-Frame-Options
Via
Recap
Feature-Policy
Content-Security-Policy
Strict-Transport-Security
Referrer-Policy
Link: rel=preload
Server-Timing
Clear-Site-Data
Expect-CT
X-Content-Type-Options

fastly.com/blog/
headers-we-dont-want

Thanks for listening
I am Get the slides:
Andrew Betts
@triblondon
abetts@fastly.com
fastly.us/headers

JS Fest 2019. Andrew Betts. Headers for hackers

Recommended

Recommended

More Related Content

What's hot

What's hot (11)

Similar to JS Fest 2019. Andrew Betts. Headers for hackers

Similar to JS Fest 2019. Andrew Betts. Headers for hackers (20)

More from JSFestUA

More from JSFestUA (20)

Recently uploaded

Recently uploaded (20)

JS Fest 2019. Andrew Betts. Headers for hackers