This document discusses using varnishlog to debug VCL code that is not behaving as expected. It provides examples of how to use varnishlog to investigate 500 responses, misses, timing details, and requests over 5 seconds. It also discusses timestamps in varnishlog output and how they can help debug timing issues. Examples are given for reproducing issues using varnishtest. The document encourages reaching out for support and exploring panic dumps and core files if Varnish crashes.
9. Debugging with varnishlog
● man vsl
● printf debugging (std.log / std.syslog)
● Enable tracing (param.set vsl_mask +VCL_trace)
● Reproduce with varnishtest
10. Timestamps
Timestamp - Timing information
Contains timing information for the Varnish worker threads.
The format is:
%s: %f %f %f
| | | |
| | | +- Time since last timestamp
| | +---- Time since start of work unit
| +------- Absolute time of event
+----------- Event label
11. Timestamped events
Requests:
Start The start of request processing (first byte received or restart).
Req Complete client request received.
ReqBody Client request body processed (discarded, cached or passed to the backend).
Waitinglist Came off waitinglist.
Fetch Fetch processing finished (completely fetched or ready for streaming).
Process Processing finished, ready to deliver the client response.
Resp Delivery of response to the client finished.
Restart Client request is being restarted.
Backend requests:
Start Start of the backend fetch processing.
Bereq Backend request sent.
Beresp Backend response headers received.
BerespBody Backend response body received.
Retry Backend request is being retried.
Error Backend request failed to vcl_backend_error.
16. Stability
● Crash, panic or something else
● Check varnishadm panic.show and syslog
● File a bug or get your hands dirty
17. Getting your hands dirty
● Not for the faint of heart
● The panic dump
● Install debugging symbols and allow coredumps
● thread apply all bt full
● My approach: backtracking
○ Defensive programming