Varnish Summit 2016

2. Analysing your web traffic with
real-time statistics
Varnish Summit, Los Angeles

3. Federico G. Schwindt
Technical Presales & Engineer
London, UK
fgsch@varnish-software.com

4. Scenario
● Traffic flows through Varnish
● Gathering and organising traffic data is challenging
● Existing tools are fragile
● More visibility is desired

5. Varnish Custom Statistics
● Realtime
● Aggregated statistics
● Custom metrics (keys)
● Uses VCL

6. Varnish Custom Statistics (cont’d)
● Flexible query API
● Regex search
● JSON format output
● Web UI

7. Architecture
VCS
Varnish
Cache
VCS
Probe
shared mem
Varnish
Cache
VCS
Probe
shared mem
Varnish
Cache
VCS
Probe
shared mem
VCS
Probe
shared mem
Varnish
Cache
VCS
Probe
shared mem
Varnish
Cache
VCS UI
ZeroMQ
JSON
API

8. VCL
import std;
sub vcl_deliver {
std.log("vcs-key:HOST/" + req.http.host);
std.log("vcs-key:URL/" + req.http.host + req.url);
std.log("vcs-key:TOTAL");
if (obj.hits == 0) {
std.log("vcs-key:MISS");
}
}

9. Querying the metrics
● All keys:
○ /all
● Stats for a particular key:
○ /key/example.com
● Stats for all keys matching the criterion:
○ /match/example.com?verbose=1
● Top 10 slowest responses:
○ /all/top_ttfb

10. JSON output
{
"URL/example.com/path/to/file.ext": [
{
"timestamp": "2016-09-14T01:23:45",
"n_req": 1,
"n_req_uniq": "NaN",
"n_miss": 1,
"avg_restarts": 0.000000,
"n_bodybytes": 343,
"reqbytes": 0,
"respbytes": 0,
"berespbytes": 0,
"bereqbytes": 0,
"ttfb_miss": 0.000483,
"ttfb_hit": "NaN",
"resp_1xx": 0,
...

11. Web UI

12. Why VCS?
● Realistic realtime
● Debugging
● Granular Varnish statistics
● Answer to business’ questions

13. Other use cases
● A/B testing
● Tracking conversion rates
● Measuring click-through rate
● Counting users per video stream

14. Build your own
http://mestlest.vg.no/

15. Demo

16. Thank you!