Thijs Feryn, profile picture
Uploaded byThijs Feryn
15 views

The Subtle Art Of Cache Invalidation (FOSSingapore 2024)

The document discusses strategies for optimizing web performance through caching techniques using Varnish software, highlighting the importance of efficient cache management. It covers various methods for cache invalidation, including expiration, LRU nuking, purging, and banning, along with practical implementation examples in the Varnish Configuration Language (VCL). Additionally, it addresses best practices for handling HTTP caching mechanisms and provides insights into managing cache during high user demand.

Embed presentation

1 / 106
2 / 106
3 / 106
4 / 106
5 / 106
6 / 106
7 / 106
8 / 106
9 / 106
10 / 106
11 / 106
12 / 106
13 / 106
14 / 106
15 / 106
16 / 106
17 / 106
18 / 106
19 / 106
20 / 106
21 / 106
22 / 106
23 / 106
24 / 106
25 / 106
26 / 106
27 / 106
28 / 106
29 / 106
30 / 106
31 / 106
32 / 106
33 / 106
34 / 106
35 / 106
36 / 106
37 / 106
38 / 106
39 / 106
40 / 106
41 / 106
42 / 106
43 / 106
44 / 106
45 / 106
46 / 106
47 / 106
48 / 106
49 / 106
50 / 106
51 / 106
52 / 106
53 / 106
54 / 106
55 / 106
56 / 106
57 / 106
58 / 106
59 / 106
60 / 106
61 / 106
62 / 106
63 / 106
64 / 106
65 / 106
66 / 106
67 / 106
68 / 106
69 / 106
70 / 106
71 / 106
72 / 106
73 / 106
74 / 106
75 / 106
76 / 106
77 / 106
78 / 106
79 / 106
80 / 106
81 / 106
82 / 106
83 / 106
84 / 106
85 / 106
86 / 106
87 / 106
88 / 106
89 / 106
90 / 106
91 / 106
92 / 106
93 / 106
94 / 106
95 / 106
96 / 106
97 / 106
98 / 106
99 / 106
100 / 106
101 / 106
102 / 106
103 / 106
104 / 106
105 / 106
106 / 106
BY THIJS FERYN
Slow websites SUCK
WEB PERFORMANCE IS AN ESSENTIAL PART OF THE USER EXPERIENCE
SLOW~DOWN
THROWING SERVERS ATTHEPROBLEM
MO' MONEY MO' SERVERS MO' PROBLEMS
IDENTIFY SLOWEST PARTS
OPTIMIZE
AFTER A WHILE YOU HIT THE LIMITS
CACHING
CACHE INVALIDATION
CACHE PURGING
WHY RECOMPUTE IF THE DATA HASN'T CHANGED?
HI, I'M THIJS
I'M THE TECH EVANGELIST AT VARNISH SOFTWARE
WE BUILD SOFTWARE-DEFINED WEB ACCELERATION & CONTENT DELIVERY SOLUTIONS
SERVER USER
UNDER PRESSURE SERVER
USER VARNISH SERVER
HTTP HAS BUILT-IN CACHING MECHANISMS
Cache-Control: public, max-age=500
Cache-Control: private, no-cache, no-store
AND VARNISH COMPLIES TO THOSE HTTP CACHING CONVENTIONS
VARNISH CONFIGURATION LANGUAGE
vcl 4.1; backend default { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if(req.url ~ "^/admin(/.*|$)") { return(pass); } unset req.http.Cookie; }
vcl 4.1; import cookie; backend default { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if (req.http.cookie) { cookie.parse(req.http.cookie); cookie.keep("PHPSESSID"); set req.http.cookie = cookie.get_string(); if (req.http.cookie ~ "^s*$") { unset req.http.cookie; } } }
PUTTING STUFF IN THE BOXES: ADDING PAGES & OTHER OBJECTS TO THE CACHE
EMPTYING THE BOXES
THERE'S ONLY ONE THING WORSE THAN NOT CACHING ENOUGH, AND THAT'S CACHING FOR TOO LONG " THIJS FERYN
WHEN ARE OBJECTS REMOVED? 1. EXPIRATION WHEN THE REMAINING TTL HITS ZERO 2. LRU NUKING WHEN THE CACHE IS FULL 3. PURGE REMOVE ONE OBJECT 4. BAN REMOVE MULTIPLE OBJECTS USING PATTERN MATCHING 5. SECONDARY KEYS REMOVE MULTIPLE OBJECTS USING TAGS
1. EXPIRATION
Cache-Control: public, max-age=500
sub vcl_backend_response { set beresp.ttl = 500s; }
$ varnishadm param.show default_ttl default_ttl Value is: 120.000 [seconds] (default) Minimum is: 0.000 The TTL assigned to objects if neither the backend nor the VCL code assigns one. NB: This parameter is evaluated only when objects are created. To change it for all objects, restart or ban everything.
varnishstat -f MAIN.n_object -f MAIN.n_expired MAIN.n_object 10 object structs made MAIN.n_expired 5 Number of expired objects
2. LRU NUKING
varnishstat -1 -f MAIN.n_object -f MAIN.n_expired -f MAIN.n_lru_nuked -f "SMA.*.g_bytes" -f "SMA.*.g_space" MAIN.n_object 938751 object structs made MAIN.n_expired 100 Number of expired objects MAIN.n_lru_nuked 20 Number of LRU nuked objects SMA.s0.g_bytes 104857600 Bytes outstanding SMA.s0.g_space 1048576 Bytes available SMA.Transient.g_bytes 0 Bytes outstanding SMA.Transient.g_space 0 Bytes available
3. PURGING
acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } sub vcl_recv { if(req.method == "PURGE") { if(!client.ip ~ purge) { return(synth(405)); } return (purge); } }
curl -XPURGE http://localhost
PURGE / HTTP/1.1 Host: localhost HTTP/1.1 200 Purged Server: Varnish X-Varnish: 2 Content-Type: text/html; charset=utf-8 Content-Length: 236 <!DOCTYPE html> <html> <head> <title>200 Purged</title> </head> <body> <h1>Error 200 Purged</h1> <p>Purged</p> <h3>Guru Meditation:</h3> <p>XID: 2</p> <hr> <p>Varnish cache server</p> </body> </html>
$ varnishstat -1 -f MAIN.n_purges MAIN.n_purges 1 . Number of purge operations executed
bin/magento setup:config:set --http-cache-hosts=localhost
varnishlog -g request -q "ReqMethod eq 'PURGE'"
* << Request >> 131079 - Begin req 131078 rxreq - Timestamp Start: 1733554258.896232 0.000000 0.000000 - Timestamp Req: 1733554258.896232 0.000000 0.000000 - ReqStart 172.19.0.1 58068 a0 - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: localhost - ReqHeader User-Agent: curl/8.7.1 - ReqHeader Accept: */* - ReqHeader X-Forwarded-For: 172.19.0.1 - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqURL / - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_acl MATCH purge "172.16.0.0/12" - VCL_return purge - VCL_call HASH - VCL_return lookup - VCL_call PURGE - VCL_return synth
- RespHeader Date: Sat, 07 Dec 2024 06:50:58 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 131079 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason Purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733554258.896488 0.000256 0.000256 - RespHeader Content-Length: 241 - Storage mse s0 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1733554258.897431 0.001198 0.000942 - ReqAcct 74 0 74 219 241 460
4. BANNING
<field> <operator> <arg> [&& <field> <oper> <arg> ...] BAN EXPRESSIONS MATCH MULTIPLE OBJECTS
req.url == / && req.http.host == example.com obj.status == 404 obj.http.Content-Type ~ ^image/ req.url ~ ^/products(/.+|$) && req.http.host == example.com
$ varnishadm ban "obj.http.Content-Type ~ ^image/"
acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } sub vcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return (synth(405)); } if (!req.http.x-invalidate-pattern) { return (purge); } ban("req.url ~ ^" + req.http.x-invalidate-pattern + " && req.http.host == " + req.http.host); return (synth(200,"Ban added")); } }
curl -XPURGE -H "x-invalidate-pattern:/products/" http://localhost
$ varnishadm ban.list Present bans: 1603272627.622051 0 - obj.http.Content-Type eq text/css 1603270370.244746 11 C $ varnishadm ban.list Present bans: 1603272627.622051 11 - obj.http.Content-Type eq text/css $ varnishadm ban.list Present bans: 1603272627.622051 11 C obj.http.Content-Type eq text/css
$ varnishstat -1 -f MAIN.bans MAIN.bans 1 . Count of bans
Purge-Cache-Tags: config:block_list config:block.block.olivero_account_menu config:block.block.olivero_search_form_wide config:block.block.olivero_breadcrumbs config:block.block.olivero_content config:block.block.olivero_help config:block.block.olivero_page_title config:block.block.olivero_main_menu config:block.block.olivero_search_form_narrow config:block.block.olivero_messages config:block.block.olivero_primary_admin_actions config:block.block.olivero_primary_local_tasks config:block.block.olivero_secondary_local_tasks config:block.block.olivero_powered config:block.block.olivero_site_branding config:block.block.olivero_syndicate block_view node_view node:1 config:filter.format.basic_html user:1 local_task config:system.menu.account config:search.settings CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form config:system.menu.main config:system.site rendered http_response config:user.role.anonymous
$ varnishadm ban.list Present bans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$)) 1727438962.414895 0 C obj.http.X-Magento-Tags ~ .* 1727273313.798067 1681 C X-Magento-Tags: store,cms_b,cms_b_1,cms_b_footer_links_block,cat_p_1,cat_p,cat_p_6,cat_p _3,cat_p_4,cat_p_5,cat_p_7,cat_p_12,cat_p_14,cat_p_13,review_block PURGE / HTTP/1.1 Host: localhost X-Magento-Tags-Pattern: ((^|,)cat_p_1(,|$)) PURGE / HTTP/1.1 Host: localhost X-Magento-Tags-Pattern: .*
$ varnishadm ban.list Present bans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$)) 1727438962.414895 0 C obj.http.X-Magento-Tags ~ .* 1727273313.798067 1681 C THE BAN LURKER THREAD PROCESSES THE BAN LIST
obj.http.Content-Type ~ ^image/ req.url ~ ^/products(/.+|$) && req.http.host == example.com NOT LURKER FRIENDLY LURKER FRIENDLY
sub vcl_backend_response { set beresp.http.url = bereq.url; set beresp.http.host = bereq.http.host; } sub vcl_deliver { unset resp.http.url; unset resp.http.host; } INJECT REQUEST INFORMATION
acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } sub vcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return (synth(405)); } if (!req.http.x-invalidate-pattern) { return (purge); } ban("obj.http.url ~ " + req.http.x-invalidate-pattern + " && obj.http.host == " + req.http.host); return (synth(200,"Ban added")); } }
ban("obj.http.url ~ " + req.http.x-invalidate-pattern + " && obj.http.host == " + req.http.host); varnishadm ban obj.http.url ~ /products && obj.http.host == example.com
varnishlog -g request -q "ReqMethod eq 'PURGE'"
* << Request >> 65538 - Begin req 65537 rxreq - Timestamp Start: 1733555192.617397 0.000000 0.000000 - Timestamp Req: 1733555192.617397 0.000000 0.000000 - VCL_use boot - ReqStart 172.19.0.3 60574 http - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: varnish - ReqHeader user-agent: purge_purger_http module for Drupal 8. - ReqHeader purge-cache-tags: node_list,node_list:page,node:1 - ReqHeader X-Forwarded-For: 172.19.0.3 - ReqHeader Via: 1.1 fddab3bb8c94 (Varnish/7.6) - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqUnset Host: varnish - ReqHeader Host: varnish - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_return synth - VCL_call HASH - VCL_return lookup
- RespProtocol HTTP/1.1 - RespStatus 200 - RespReason Ban added. - RespHeader Date: Sat, 07 Dec 2024 07:06:32 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 65538 - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733555192.620607 0.003209 0.003209 - RespHeader Content-Length: 252 - Storage malloc Transient - Filters - RespHeader Connection: keep-alive - Timestamp Resp: 1733555192.620675 0.003278 0.000068 - ReqAcct 136 0 136 200 252 452 - End
Σ BANS X Σ OBJECTS = ⏳
5. SECONDARY KEYS
X-Magento-Tags: store,cms_b,cms_b_1,cms_b_footer_links_block,cat_p_1,cat_p,cat_p_6,cat_p _3,cat_p_4,cat_p_5,cat_p_7,cat_p_12,cat_p_14,cat_p_13,review_block MAGENTO USES TAGS (AB)USES BANS DUE TO LACK OF TAG SUPPORT IN VARNISH $ varnishadm ban.list Present bans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$))
import ykey; sub vcl_backend_response { ykey.add_header(beresp.http.Purge-Cache-Tags, sep=" "); unset beresp.http.Purge-Cache-Tags; ykey.add_key("everything"); if(beresp.http.Content-Type ~ "^image/") { ykey.add_key("image"); } }
-- BerespHeader Purge-Cache-Tags: config:block_list config:block.block.olivero_account_menu config:block.block.olivero_search_form_wide config:block.block.olivero_breadcrumbs config:block.block.olivero_content config:block.block.olivero_help config:block.block.olivero_page_title config:block.block.olivero_main_menu config:block.block.olivero_search_form_narrow config:block.block.olivero_messages config:block.block.olivero_primary_admin_actions config:block.block.olivero_primary_local_tasks config:block.block.olivero_secondary_local_tasks config:block.block.olivero_powered config:block.block.olivero_site_branding config:block.block.olivero_syndicate block_view node_view node:1 config:filter.format.basic_html user:1 local_task config:system.menu.account config:search.settings CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form config:system.menu.main config:system.site rendered http_response config:user.role.anonymous -- YKEY key: ADD config:block_list -- YKEY key: ADD config:block.block.olivero_account_menu -- YKEY key: ADD config:block.block.olivero_search_form_wide -- YKEY key: ADD config:block.block.olivero_breadcrumbs -- YKEY key: ADD config:block.block.olivero_content -- YKEY key: ADD config:block.block.olivero_help -- YKEY key: ADD config:block.block.olivero_page_title -- YKEY key: ADD config:block.block.olivero_main_menu -- YKEY key: ADD config:block.block.olivero_search_form_narrow -- YKEY key: ADD config:block.block.olivero_messages -- YKEY key: ADD config:block.block.olivero_primary_admin_actions -- YKEY key: ADD config:block.block.olivero_primary_local_tasks -- YKEY key: ADD config:block.block.olivero_secondary_local_tasks -- YKEY key: ADD config:block.block.olivero_powered -- YKEY key: ADD config:block.block.olivero_site_branding -- YKEY key: ADD config:block.block.olivero_syndicate
-- YKEY key: ADD block_view -- YKEY key: ADD node_view -- YKEY key: ADD node:1 -- YKEY key: ADD config:filter.format.basic_html -- YKEY key: ADD user:1 -- YKEY key: ADD local_task -- YKEY key: ADD config:system.menu.account -- YKEY key: ADD config:search.settings -- YKEY key: ADD CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form -- YKEY key: ADD config:system.menu.main -- YKEY key: ADD config:system.site -- YKEY key: ADD rendered -- YKEY key: ADD http_response -- YKEY key: ADD config:user.role.anonymous -- BerespUnset Purge-Cache-Tags -- YKEY key: ADD everything
import ykey; acl purge { "localhost"; "127.0.0.1"; "::1"; } sub vcl_recv { if(req.method == "PURGE") { if(!client.ip ~ purge) { return(synth(405)); } if (req.http.Purge-Cache-Tags) { return (synth(200, ykey.purge_header(req.http.Purge-Cache-Tags, sep=",") + " objects purged")); } return(purge); } }
ykey.purge_header(req.http.Purge-Cache-Tags, sep=",")
* << Request >> 65538 - Begin req 65537 rxreq - Timestamp Start: 1733556901.729447 0.000000 0.000000 - Timestamp Req: 1733556901.729447 0.000000 0.000000 - ReqStart 172.19.0.3 35294 a0 - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: varnish - ReqHeader user-agent: purge_purger_http module for Drupal 8. - ReqHeader purge-cache-tags: node_list,node_list:page,node:1 - ReqHeader X-Forwarded-For: 172.19.0.3 - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqURL / - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_acl MATCH purge "172.16.0.0/12" - YKEY key: PURGE node_list - YKEY key: PURGE node_list:page - YKEY key: PURGE node:1 - VCL_return synth - VCL_call HASH - VCL_return lookup
- RespHeader Date: Sat, 07 Dec 2024 07:35:01 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 65538 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason 1 objects purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733556901.733476 0.004029 0.004029 - RespHeader Content-Length: 270 - Storage mse s0 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1733556901.733543 0.004096 0.000067 - ReqAcct 136 0 136 228 270 498 - End
PERFORMANCE IMPACT OF CACHE PURGING
CACHE INVALIDATION MAKE CACHED OBJECTS INVALID NO NEED TO REMOVE THEM
ykey.purge_header(req.http.Purge-Cache-Tags, sep=",",)
ykey.purge_header(req.http.Purge-Cache-Tags, sep=",", soft=true)
ASYNCHRONOUS REVALIDATION
USER VARNISH SERVER ASYNC FETCH SEND STALE RESPONSE WHILE FETCHING
OBJECT LIFETIME = TTL + GRACE + KEEP
OBJECT LIFETIME = TTL + GRACE + KEEP DEFAULT VALUE: 2 MINUTES
OBJECT LIFETIME = TTL + GRACE + KEEP SERVE STALE CONTENT, ASYNC REVALIDATION DEFAULT VALUE: 10 SECONDS CAN BE SET BY "STALE-WHILE- REVALIDATE"
OBJECT LIFETIME = TTL + GRACE + KEEP KEEP THE OBJECT AROUND FOR LATER USE DEFAULT VALUE: 0 SECONDS SYNCHRONOUS REVALIDATION
OBJECT LIFETIME = TTL + GRACE + KEEP SOFT PURGING SETS TTL TO ZERO
sub vcl_backend_response { set beresp.grace = 1d; }
import ykey; sub vcl_backend_response { ykey.add_header(beresp.http.Purge-Cache-Tags, sep=" "); unset beresp.http.Purge-Cache-Tags; ykey.add_key("everything"); if(beresp.http.Content-Type ~ "^image/") { ykey.add_key("image"); } set beresp.grace = 1d; }
Cache-Control: max-age=60, public, stale-while-revalidate=86400
- VCL_call HASH - VCL_return lookup - Hit 32775 -25.943176 7200.000000 0.000000 - VCL_call HIT - VCL_return deliver - Link bereq 34 bgfetch
$ varnishstat -1 -f MAIN.cache_hit_grace MAIN.cache_hit_grace 1 Cache grace hits
1. EXPIRATION WE'RE BACK AT EXPIRATION
FIND THE RIGHT FRAMEWORK PLUGIN OR INTEGRATE PURGING YOURSELF
The Subtle Art Of Cache Invalidation (FOSSingapore 2024)

More Related Content

PPTX
Varnish –Http Accelerator
bySmruti Ranjan Tripathy
 
PDF
Advanced cache invalidation
byPer Buer
 
PDF
Developing cacheable backend applications - Appdevcon 2019
byThijs Feryn
 
PPTX
Varnish
byShaopeng He
 
PDF
Supercharging Content Delivery with Varnish
bySamantha Quiñones
 
PDF
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
bygeorgepenkov
 
PDF
In-depth caching in Varnish - GOG Varnish Meetup, march 2019
byGOG.com dev team
 
PDF
Best practices for caching Plone with Varnish at large scale - PloneConf Bras...
byThijs Feryn
 
Varnish –Http Accelerator
bySmruti Ranjan Tripathy
 
Advanced cache invalidation
byPer Buer
 
Developing cacheable backend applications - Appdevcon 2019
byThijs Feryn
 
Varnish
byShaopeng He
 
Supercharging Content Delivery with Varnish
bySamantha Quiñones
 
Saving The World From Guaranteed APOCALYPSE* Using Varnish and Memcached
bygeorgepenkov
 
In-depth caching in Varnish - GOG Varnish Meetup, march 2019
byGOG.com dev team
 
Best practices for caching Plone with Varnish at large scale - PloneConf Bras...
byThijs Feryn
 

Similar to The Subtle Art Of Cache Invalidation (FOSSingapore 2024)

PDF
PLNOG 4: Leszek Urbański - A modern HTTP accelerator for content providers
byPROIDEA
 
PPTX
Starting with varnish cache
byRobert Richelieu
 
PDF
Varnish Enterprise - when you need the full power of caching
byThijs Feryn
 
PDF
HTTP logging with varnishlog (Brussels PHP 2022)
byThijs Feryn
 
PDF
Accelerate your Kubernetes clusters with Varnish Caching
byThijs Feryn
 
PDF
VUG5: Varnish at Opera Software
byCosimo Streppone
 
PPTX
cache concepts and varnish-cache
byMarc Cortinas Val
 
PDF
Varnish Cache Plus. Random notes for wise web developers
byCarlos Abalde
 
ODP
PHP London Dec 2013 - Varnish - The 9 circles of hell
byluis-ferro
 
PDF
Caching the uncacheable in Varnish - SREday London 2024.pdf
byThijs Feryn
 
ODP
Varnish qconsp 2011
byFlavio Torres
 
PDF
Performance
byVarnish Software
 
PDF
Optimizing Varnish for Magento: Advanced Techniques for Performance and Scala...
byThijs Feryn
 
PPTX
Varnish & Magento TechTalk @Lyracons
byEzequiel Kupelian
 
PDF
Caching with Varnish
byschoefmax
 
PDF
HTTP logging met Varnishlog - PHPWVL 2022
byThijs Feryn
 
PDF
Handling 10k requests per second with Symfony and Varnish - SymfonyCon Berlin...
byAlexander Lisachenko
 
PDF
From Web Acceleration to Content Delivery with Varnish - Howest Brugge 2024
byThijs Feryn
 
PDF
Varnish caching in Plone - PloneConf Brasilia 2024
byThijs Feryn
 
PDF
Caching the uncacheable with Varnish - DevDays 2021
byThijs Feryn
 
PLNOG 4: Leszek Urbański - A modern HTTP accelerator for content providers
byPROIDEA
 
Starting with varnish cache
byRobert Richelieu
 
Varnish Enterprise - when you need the full power of caching
byThijs Feryn
 
HTTP logging with varnishlog (Brussels PHP 2022)
byThijs Feryn
 
Accelerate your Kubernetes clusters with Varnish Caching
byThijs Feryn
 
VUG5: Varnish at Opera Software
byCosimo Streppone
 
cache concepts and varnish-cache
byMarc Cortinas Val
 
Varnish Cache Plus. Random notes for wise web developers
byCarlos Abalde
 
PHP London Dec 2013 - Varnish - The 9 circles of hell
byluis-ferro
 
Caching the uncacheable in Varnish - SREday London 2024.pdf
byThijs Feryn
 
Varnish qconsp 2011
byFlavio Torres
 
Performance
byVarnish Software
 
Optimizing Varnish for Magento: Advanced Techniques for Performance and Scala...
byThijs Feryn
 
Varnish & Magento TechTalk @Lyracons
byEzequiel Kupelian
 
Caching with Varnish
byschoefmax
 
HTTP logging met Varnishlog - PHPWVL 2022
byThijs Feryn
 
Handling 10k requests per second with Symfony and Varnish - SymfonyCon Berlin...
byAlexander Lisachenko
 
From Web Acceleration to Content Delivery with Varnish - Howest Brugge 2024
byThijs Feryn
 
Varnish caching in Plone - PloneConf Brasilia 2024
byThijs Feryn
 
Caching the uncacheable with Varnish - DevDays 2021
byThijs Feryn
 

More from Thijs Feryn

PDF
Developing Cacheable PHP Applications - PHP SP 2024
byThijs Feryn
 
PDF
10 things that helped me advance my career - PHP UK Conference 2024
byThijs Feryn
 
PDF
Distributed load testing with K6 - NDC London 2024
byThijs Feryn
 
PDF
HTTP headers that make your website go faster - devs.gent November 2023
byThijs Feryn
 
PDF
Living on the edge - EBU Horizons 2023
byThijs Feryn
 
PDF
Distributed Load Testing with k6 - DevOps Barcelona
byThijs Feryn
 
PDF
Core web vitals meten om je site sneller te maken - Combell Partner Day 2023
byThijs Feryn
 
PDF
HTTP headers that make your website go faster
byThijs Feryn
 
PDF
HTTP headers that will make your website go faster
byThijs Feryn
 
PDF
Distributed load testing with k6
byThijs Feryn
 
PDF
Taking Laravel to the edge with HTTP caching and Varnish
byThijs Feryn
 
PDF
Build your own CDN with Varnish - Confoo 2022
byThijs Feryn
 
PDF
How Cloud addresses the needs of todays internet - Korazon 2018
byThijs Feryn
 
PDF
Developing cacheable PHP applications - PHPLimburgBE 2018
byThijs Feryn
 
PDF
Leverage HTTP to deliver cacheable websites - Codemotion Rome 2018
byThijs Feryn
 
PDF
Developing cacheable PHP applications - Confoo 2018
byThijs Feryn
 
Developing Cacheable PHP Applications - PHP SP 2024
byThijs Feryn
 
10 things that helped me advance my career - PHP UK Conference 2024
byThijs Feryn
 
Distributed load testing with K6 - NDC London 2024
byThijs Feryn
 
HTTP headers that make your website go faster - devs.gent November 2023
byThijs Feryn
 
Living on the edge - EBU Horizons 2023
byThijs Feryn
 
Distributed Load Testing with k6 - DevOps Barcelona
byThijs Feryn
 
Core web vitals meten om je site sneller te maken - Combell Partner Day 2023
byThijs Feryn
 
HTTP headers that make your website go faster
byThijs Feryn
 
HTTP headers that will make your website go faster
byThijs Feryn
 
Distributed load testing with k6
byThijs Feryn
 
Taking Laravel to the edge with HTTP caching and Varnish
byThijs Feryn
 
Build your own CDN with Varnish - Confoo 2022
byThijs Feryn
 
How Cloud addresses the needs of todays internet - Korazon 2018
byThijs Feryn
 
Developing cacheable PHP applications - PHPLimburgBE 2018
byThijs Feryn
 
Leverage HTTP to deliver cacheable websites - Codemotion Rome 2018
byThijs Feryn
 
Developing cacheable PHP applications - Confoo 2018
byThijs Feryn
 

Recently uploaded

PPTX
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
PPTX
Building Agents in Microsoft Agent Framework.pptx
byUdaiappa Ramachandran
 
PDF
AI Powered Document Processing and Data Extraction
byRobert McDermott
 
PDF
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
PPTX
MGw_MRS Benfits seu beficios de redes 4g
byJoaquimBarros18
 
PDF
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
PDF
Unlocking the Power of Salesforce Architecture: Frameworks for Effective Solu...
byvarsha30tiwari
 
PDF
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
PPTX
Top _HR Technology Trends _for 2026_.pptx
byAutomationEdge Technologies
 
PPTX
Technology Consulting _ by Slidesgo.pptx
byfh82277
 
PPTX
UiPath Autonomous Agents | Building and Orchestrating Agents End-to-End
byUiPathCommunity
 
PDF
The State of the Gen AI economy - 2025 - The Meliora Company
byClive Dickens
 
PDF
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
PPTX
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
PDF
100 Insights After the 200th Issue of NewMind AI Journal
byNewMind AI
 
PPTX
Session 2 - Solving Unstructured & Complex Documents with UiPath IXP
byDianaGray10
 
PPTX
Software Analysis &Design ethiopia chap-2.pptx
byAbbas484771
 
PDF
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
PDF
Exam Prep Plan Overview: Amazon Web Services (AWS) Certified
byVICTOR MAESTRE RAMIREZ
 
PPTX
wob-report.pptxwob-report.pptxwob-report.pptx
byssuser0d171c
 
Building Cyber Resilience for 2026: Best Practices for a Secure, AI-Driven Bu...
byYasir Naveed Riaz
 
Building Agents in Microsoft Agent Framework.pptx
byUdaiappa Ramachandran
 
AI Powered Document Processing and Data Extraction
byRobert McDermott
 
Internet_of_Things_IoT_for_Next_Generation_Smart_Systems_Utilizing.pdf
byRoshanSyed12
 
MGw_MRS Benfits seu beficios de redes 4g
byJoaquimBarros18
 
Lab 1.5 - Sharing Secrets with GPG ~ 2nd Sight Lab Cloud Security Class
by2nd Sight Lab
 
Unlocking the Power of Salesforce Architecture: Frameworks for Effective Solu...
byvarsha30tiwari
 
DevFest El Jadida 2025 - Product Thinking
byElmehdi AMLOU
 
Top _HR Technology Trends _for 2026_.pptx
byAutomationEdge Technologies
 
Technology Consulting _ by Slidesgo.pptx
byfh82277
 
UiPath Autonomous Agents | Building and Orchestrating Agents End-to-End
byUiPathCommunity
 
The State of the Gen AI economy - 2025 - The Meliora Company
byClive Dickens
 
Access Control 2025: From Security Silo to Software-Defined Ecosystem
byMemoori
 
Gemini vs ChatGPT : Comparing Top AI Models
byDigicarrom
 
100 Insights After the 200th Issue of NewMind AI Journal
byNewMind AI
 
Session 2 - Solving Unstructured & Complex Documents with UiPath IXP
byDianaGray10
 
Software Analysis &Design ethiopia chap-2.pptx
byAbbas484771
 
Dev Dives: AI that builds with you - UiPath Autopilot for effortless RPA & AP...
byUiPathCommunity
 
Exam Prep Plan Overview: Amazon Web Services (AWS) Certified
byVICTOR MAESTRE RAMIREZ
 
wob-report.pptxwob-report.pptxwob-report.pptx
byssuser0d171c
 

The Subtle Art Of Cache Invalidation (FOSSingapore 2024)

  • 1.
  • 2.
  • 3.
    WEB PERFORMANCE ISAN ESSENTIAL PART OF THE USER EXPERIENCE
  • 4.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
    AFTER A WHILEYOU HIT THE LIMITS
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    I'M THE TECHEVANGELIST AT VARNISH SOFTWARE
  • 22.
    WE BUILD SOFTWARE-DEFINED WEBACCELERATION & CONTENT DELIVERY SOLUTIONS
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
    AND VARNISH COMPLIES TOTHOSE HTTP CACHING CONVENTIONS
  • 31.
  • 33.
    vcl 4.1; backend default{ .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if(req.url ~ "^/admin(/.*|$)") { return(pass); } unset req.http.Cookie; }
  • 34.
    vcl 4.1; import cookie; backenddefault { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if (req.http.cookie) { cookie.parse(req.http.cookie); cookie.keep("PHPSESSID"); set req.http.cookie = cookie.get_string(); if (req.http.cookie ~ "^s*$") { unset req.http.cookie; } } }
  • 35.
    PUTTING STUFF IN THEBOXES: ADDING PAGES & OTHER OBJECTS TO THE CACHE
  • 36.
  • 37.
    THERE'S ONLY ONETHING WORSE THAN NOT CACHING ENOUGH, AND THAT'S CACHING FOR TOO LONG " THIJS FERYN
  • 39.
    WHEN ARE OBJECTSREMOVED? 1. EXPIRATION WHEN THE REMAINING TTL HITS ZERO 2. LRU NUKING WHEN THE CACHE IS FULL 3. PURGE REMOVE ONE OBJECT 4. BAN REMOVE MULTIPLE OBJECTS USING PATTERN MATCHING 5. SECONDARY KEYS REMOVE MULTIPLE OBJECTS USING TAGS
  • 40.
  • 41.
  • 42.
    sub vcl_backend_response { setberesp.ttl = 500s; }
  • 43.
    $ varnishadm param.showdefault_ttl default_ttl Value is: 120.000 [seconds] (default) Minimum is: 0.000 The TTL assigned to objects if neither the backend nor the VCL code assigns one. NB: This parameter is evaluated only when objects are created. To change it for all objects, restart or ban everything.
  • 44.
    varnishstat -f MAIN.n_object-f MAIN.n_expired MAIN.n_object 10 object structs made MAIN.n_expired 5 Number of expired objects
  • 45.
  • 46.
    varnishstat -1 -fMAIN.n_object -f MAIN.n_expired -f MAIN.n_lru_nuked -f "SMA.*.g_bytes" -f "SMA.*.g_space" MAIN.n_object 938751 object structs made MAIN.n_expired 100 Number of expired objects MAIN.n_lru_nuked 20 Number of LRU nuked objects SMA.s0.g_bytes 104857600 Bytes outstanding SMA.s0.g_space 1048576 Bytes available SMA.Transient.g_bytes 0 Bytes outstanding SMA.Transient.g_space 0 Bytes available
  • 47.
  • 48.
    acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } subvcl_recv { if(req.method == "PURGE") { if(!client.ip ~ purge) { return(synth(405)); } return (purge); } }
  • 49.
  • 50.
    PURGE / HTTP/1.1 Host:localhost HTTP/1.1 200 Purged Server: Varnish X-Varnish: 2 Content-Type: text/html; charset=utf-8 Content-Length: 236 <!DOCTYPE html> <html> <head> <title>200 Purged</title> </head> <body> <h1>Error 200 Purged</h1> <p>Purged</p> <h3>Guru Meditation:</h3> <p>XID: 2</p> <hr> <p>Varnish cache server</p> </body> </html>
  • 51.
    $ varnishstat -1-f MAIN.n_purges MAIN.n_purges 1 . Number of purge operations executed
  • 54.
  • 55.
    varnishlog -g request-q "ReqMethod eq 'PURGE'"
  • 56.
    * << Request>> 131079 - Begin req 131078 rxreq - Timestamp Start: 1733554258.896232 0.000000 0.000000 - Timestamp Req: 1733554258.896232 0.000000 0.000000 - ReqStart 172.19.0.1 58068 a0 - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: localhost - ReqHeader User-Agent: curl/8.7.1 - ReqHeader Accept: */* - ReqHeader X-Forwarded-For: 172.19.0.1 - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqURL / - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_acl MATCH purge "172.16.0.0/12" - VCL_return purge - VCL_call HASH - VCL_return lookup - VCL_call PURGE - VCL_return synth
  • 57.
    - RespHeader Date:Sat, 07 Dec 2024 06:50:58 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 131079 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason Purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733554258.896488 0.000256 0.000256 - RespHeader Content-Length: 241 - Storage mse s0 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1733554258.897431 0.001198 0.000942 - ReqAcct 74 0 74 219 241 460
  • 58.
  • 59.
    <field> <operator> <arg>[&& <field> <oper> <arg> ...] BAN EXPRESSIONS MATCH MULTIPLE OBJECTS
  • 60.
    req.url == /&& req.http.host == example.com obj.status == 404 obj.http.Content-Type ~ ^image/ req.url ~ ^/products(/.+|$) && req.http.host == example.com
  • 61.
    $ varnishadm ban"obj.http.Content-Type ~ ^image/"
  • 62.
    acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } subvcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return (synth(405)); } if (!req.http.x-invalidate-pattern) { return (purge); } ban("req.url ~ ^" + req.http.x-invalidate-pattern + " && req.http.host == " + req.http.host); return (synth(200,"Ban added")); } }
  • 63.
    curl -XPURGE -H"x-invalidate-pattern:/products/" http://localhost
  • 64.
    $ varnishadm ban.list Presentbans: 1603272627.622051 0 - obj.http.Content-Type eq text/css 1603270370.244746 11 C $ varnishadm ban.list Present bans: 1603272627.622051 11 - obj.http.Content-Type eq text/css $ varnishadm ban.list Present bans: 1603272627.622051 11 C obj.http.Content-Type eq text/css
  • 65.
    $ varnishstat -1-f MAIN.bans MAIN.bans 1 . Count of bans
  • 68.
    Purge-Cache-Tags: config:block_list config:block.block.olivero_account_menu config:block.block.olivero_search_form_wide config:block.block.olivero_breadcrumbs config:block.block.olivero_content config:block.block.olivero_help config:block.block.olivero_page_title config:block.block.olivero_main_menu config:block.block.olivero_search_form_narrow config:block.block.olivero_messages config:block.block.olivero_primary_admin_actions config:block.block.olivero_primary_local_tasks config:block.block.olivero_secondary_local_tasks config:block.block.olivero_powered config:block.block.olivero_site_branding config:block.block.olivero_syndicateblock_view node_view node:1 config:filter.format.basic_html user:1 local_task config:system.menu.account config:search.settings CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form config:system.menu.main config:system.site rendered http_response config:user.role.anonymous
  • 69.
    $ varnishadm ban.list Presentbans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$)) 1727438962.414895 0 C obj.http.X-Magento-Tags ~ .* 1727273313.798067 1681 C X-Magento-Tags: store,cms_b,cms_b_1,cms_b_footer_links_block,cat_p_1,cat_p,cat_p_6,cat_p _3,cat_p_4,cat_p_5,cat_p_7,cat_p_12,cat_p_14,cat_p_13,review_block PURGE / HTTP/1.1 Host: localhost X-Magento-Tags-Pattern: ((^|,)cat_p_1(,|$)) PURGE / HTTP/1.1 Host: localhost X-Magento-Tags-Pattern: .*
  • 70.
    $ varnishadm ban.list Presentbans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$)) 1727438962.414895 0 C obj.http.X-Magento-Tags ~ .* 1727273313.798067 1681 C THE BAN LURKER THREAD PROCESSES THE BAN LIST
  • 71.
    obj.http.Content-Type ~ ^image/ req.url~ ^/products(/.+|$) && req.http.host == example.com NOT LURKER FRIENDLY LURKER FRIENDLY
  • 72.
    sub vcl_backend_response { setberesp.http.url = bereq.url; set beresp.http.host = bereq.http.host; } sub vcl_deliver { unset resp.http.url; unset resp.http.host; } INJECT REQUEST INFORMATION
  • 73.
    acl purge { "localhost"; "127.0.0.1"; "::1"; "172.16.0.0/12"; } subvcl_recv { if (req.method == "PURGE") { if (!client.ip ~ purge) { return (synth(405)); } if (!req.http.x-invalidate-pattern) { return (purge); } ban("obj.http.url ~ " + req.http.x-invalidate-pattern + " && obj.http.host == " + req.http.host); return (synth(200,"Ban added")); } }
  • 74.
    ban("obj.http.url ~ "+ req.http.x-invalidate-pattern + " && obj.http.host == " + req.http.host); varnishadm ban obj.http.url ~ /products && obj.http.host == example.com
  • 75.
    varnishlog -g request-q "ReqMethod eq 'PURGE'"
  • 76.
    * << Request>> 65538 - Begin req 65537 rxreq - Timestamp Start: 1733555192.617397 0.000000 0.000000 - Timestamp Req: 1733555192.617397 0.000000 0.000000 - VCL_use boot - ReqStart 172.19.0.3 60574 http - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: varnish - ReqHeader user-agent: purge_purger_http module for Drupal 8. - ReqHeader purge-cache-tags: node_list,node_list:page,node:1 - ReqHeader X-Forwarded-For: 172.19.0.3 - ReqHeader Via: 1.1 fddab3bb8c94 (Varnish/7.6) - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqUnset Host: varnish - ReqHeader Host: varnish - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_return synth - VCL_call HASH - VCL_return lookup
  • 77.
    - RespProtocol HTTP/1.1 -RespStatus 200 - RespReason Ban added. - RespHeader Date: Sat, 07 Dec 2024 07:06:32 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 65538 - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733555192.620607 0.003209 0.003209 - RespHeader Content-Length: 252 - Storage malloc Transient - Filters - RespHeader Connection: keep-alive - Timestamp Resp: 1733555192.620675 0.003278 0.000068 - ReqAcct 136 0 136 200 252 452 - End
  • 78.
    Σ BANS XΣ OBJECTS = ⏳
  • 79.
  • 80.
    X-Magento-Tags: store,cms_b,cms_b_1,cms_b_footer_links_block,cat_p_1,cat_p,cat_p_6,cat_p _3,cat_p_4,cat_p_5,cat_p_7,cat_p_12,cat_p_14,cat_p_13,review_block MAGENTO USES TAGS (AB)USES BANS DUETO LACK OF TAG SUPPORT IN VARNISH $ varnishadm ban.list Present bans: 1727438962.423744 2 - obj.http.X-Magento-Tags ~ ((^|,)cat_p_1(,|$))
  • 81.
    import ykey; sub vcl_backend_response{ ykey.add_header(beresp.http.Purge-Cache-Tags, sep=" "); unset beresp.http.Purge-Cache-Tags; ykey.add_key("everything"); if(beresp.http.Content-Type ~ "^image/") { ykey.add_key("image"); } }
  • 82.
    -- BerespHeader Purge-Cache-Tags:config:block_list config:block.block.olivero_account_menu config:block.block.olivero_search_form_wide config:block.block.olivero_breadcrumbs config:block.block.olivero_content config:block.block.olivero_help config:block.block.olivero_page_title config:block.block.olivero_main_menu config:block.block.olivero_search_form_narrow config:block.block.olivero_messages config:block.block.olivero_primary_admin_actions config:block.block.olivero_primary_local_tasks config:block.block.olivero_secondary_local_tasks config:block.block.olivero_powered config:block.block.olivero_site_branding config:block.block.olivero_syndicate block_view node_view node:1 config:filter.format.basic_html user:1 local_task config:system.menu.account config:search.settings CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form config:system.menu.main config:system.site rendered http_response config:user.role.anonymous -- YKEY key: ADD config:block_list -- YKEY key: ADD config:block.block.olivero_account_menu -- YKEY key: ADD config:block.block.olivero_search_form_wide -- YKEY key: ADD config:block.block.olivero_breadcrumbs -- YKEY key: ADD config:block.block.olivero_content -- YKEY key: ADD config:block.block.olivero_help -- YKEY key: ADD config:block.block.olivero_page_title -- YKEY key: ADD config:block.block.olivero_main_menu -- YKEY key: ADD config:block.block.olivero_search_form_narrow -- YKEY key: ADD config:block.block.olivero_messages -- YKEY key: ADD config:block.block.olivero_primary_admin_actions -- YKEY key: ADD config:block.block.olivero_primary_local_tasks -- YKEY key: ADD config:block.block.olivero_secondary_local_tasks -- YKEY key: ADD config:block.block.olivero_powered -- YKEY key: ADD config:block.block.olivero_site_branding -- YKEY key: ADD config:block.block.olivero_syndicate
  • 83.
    -- YKEY key:ADD block_view -- YKEY key: ADD node_view -- YKEY key: ADD node:1 -- YKEY key: ADD config:filter.format.basic_html -- YKEY key: ADD user:1 -- YKEY key: ADD local_task -- YKEY key: ADD config:system.menu.account -- YKEY key: ADD config:search.settings -- YKEY key: ADD CACHE_MISS_IF_UNCACHEABLE_HTTP_METHOD:form -- YKEY key: ADD config:system.menu.main -- YKEY key: ADD config:system.site -- YKEY key: ADD rendered -- YKEY key: ADD http_response -- YKEY key: ADD config:user.role.anonymous -- BerespUnset Purge-Cache-Tags -- YKEY key: ADD everything
  • 84.
    import ykey; acl purge{ "localhost"; "127.0.0.1"; "::1"; } sub vcl_recv { if(req.method == "PURGE") { if(!client.ip ~ purge) { return(synth(405)); } if (req.http.Purge-Cache-Tags) { return (synth(200, ykey.purge_header(req.http.Purge-Cache-Tags, sep=",") + " objects purged")); } return(purge); } }
  • 85.
  • 86.
    * << Request>> 65538 - Begin req 65537 rxreq - Timestamp Start: 1733556901.729447 0.000000 0.000000 - Timestamp Req: 1733556901.729447 0.000000 0.000000 - ReqStart 172.19.0.3 35294 a0 - ReqMethod PURGE - ReqURL / - ReqProtocol HTTP/1.1 - ReqHeader Host: varnish - ReqHeader user-agent: purge_purger_http module for Drupal 8. - ReqHeader purge-cache-tags: node_list,node_list:page,node:1 - ReqHeader X-Forwarded-For: 172.19.0.3 - VCL_call RECV - ReqHeader Surrogate-Capability: Varnish=ESI/1.0 - ReqURL / - ReqURL / - ReqHeader X-Forwarded-Proto: http - VCL_acl MATCH purge "172.16.0.0/12" - YKEY key: PURGE node_list - YKEY key: PURGE node_list:page - YKEY key: PURGE node:1 - VCL_return synth - VCL_call HASH - VCL_return lookup
  • 87.
    - RespHeader Date:Sat, 07 Dec 2024 07:35:01 GMT - RespHeader Server: Varnish - RespHeader X-Varnish: 65538 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespReason 1 objects purged - VCL_call SYNTH - RespHeader Content-Type: text/html; charset=utf-8 - RespHeader Retry-After: 5 - VCL_return deliver - Timestamp Process: 1733556901.733476 0.004029 0.004029 - RespHeader Content-Length: 270 - Storage mse s0 - RespHeader Accept-Ranges: bytes - RespHeader Connection: keep-alive - Timestamp Resp: 1733556901.733543 0.004096 0.000067 - ReqAcct 136 0 136 228 270 498 - End
  • 88.
  • 89.
  • 90.
  • 91.
  • 92.
  • 93.
    USER VARNISH SERVER ASYNCFETCH SEND STALE RESPONSE WHILE FETCHING
  • 94.
    OBJECT LIFETIME =TTL + GRACE + KEEP
  • 95.
    OBJECT LIFETIME =TTL + GRACE + KEEP DEFAULT VALUE: 2 MINUTES
  • 96.
    OBJECT LIFETIME =TTL + GRACE + KEEP SERVE STALE CONTENT, ASYNC REVALIDATION DEFAULT VALUE: 10 SECONDS CAN BE SET BY "STALE-WHILE- REVALIDATE"
  • 97.
    OBJECT LIFETIME =TTL + GRACE + KEEP KEEP THE OBJECT AROUND FOR LATER USE DEFAULT VALUE: 0 SECONDS SYNCHRONOUS REVALIDATION
  • 98.
    OBJECT LIFETIME =TTL + GRACE + KEEP SOFT PURGING SETS TTL TO ZERO
  • 99.
    sub vcl_backend_response { setberesp.grace = 1d; }
  • 100.
    import ykey; sub vcl_backend_response{ ykey.add_header(beresp.http.Purge-Cache-Tags, sep=" "); unset beresp.http.Purge-Cache-Tags; ykey.add_key("everything"); if(beresp.http.Content-Type ~ "^image/") { ykey.add_key("image"); } set beresp.grace = 1d; }
  • 101.
    Cache-Control: max-age=60, public,stale-while-revalidate=86400
  • 102.
    - VCL_call HASH -VCL_return lookup - Hit 32775 -25.943176 7200.000000 0.000000 - VCL_call HIT - VCL_return deliver - Link bereq 34 bgfetch
  • 103.
    $ varnishstat -1-f MAIN.cache_hit_grace MAIN.cache_hit_grace 1 Cache grace hits
  • 104.
  • 105.
    FIND THE RIGHT FRAMEWORKPLUGIN OR INTEGRATE PURGING YOURSELF