SlideShare a Scribd company logo
1 of 24
Download to read offline
Varnish kann Alles
8. Juli 2015
Nils Goroll
Varnish kann Alles
in 25 Minuten
8. Juli 2015
Nils Goroll
Varnish kann Alles
in 25 Minuten
8. Juli 2015
Nils Goroll
ein kurzer Blick aufein kurzer Blick auf
effizientes Web-Cachingeffizientes Web-Caching
Setup Reverse Proxy
● + Caching
● Keine Generierung von Inhalten
Effizienz!
● Bild home von Varnish – System Call Trace
1436329479.806468 getsockname(17, {sa_family=AF_INET6, sin6_port=htons(8080), inet_pton(AF_INET6, "::1",
&sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0
1436329479.806645 setsockopt(17, SOL_SOCKET, SO_LINGER, {onoff=0, linger=0}, 8) = 0
1436329479.806718 setsockopt(17, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
1436329479.806762 setsockopt(17, SOL_TCP, TCP_NODELAY, [1], 4) = 0
1436329479.806804 ioctl(17, FIONBIO, [0]) = 0
1436329479.806854 ioctl(17, FIONBIO, [0]) = 0
1436329479.806888 poll([{fd=17, events=POLLIN}], 1, 50) = 1 ([{fd=17, revents=POLLIN}])
1436329479.806938 read(17, "GET / HTTP/1.1rnUser-Agent: curl"..., 32767) = 75
1436329479.807612 writev(17, [{"HTTP/1.1", 8}, {" ", 1}, {"200", 3}, {" ", 1}, {"OK", 2}, {"rn", 2}, {"Content-
Type: text/html;charset="..., 37}, {"rn", 2}, {"Content-Language: de-DE", 23}, {"rn", 2}, {"Last-Modified:
Wed, 08 Jul 2015 "..., 44}, {"rn", 2}, {"X-UA-Compatible: IE=Edge,chrome="..., 33}, {"rn", 2}, {"X-Refresh:
h=23|g=103975.000|c=G"..., 98}, {"rn", 2}, {"Server: Apache", 14}, {"rn", 2}, {"Cache-Control: public, max-
age=3"..., 33}, {"rn", 2}, {"Expires: Wed, 08 Jul 2015 04:25:"..., 38}, {"rn", 2}, {"Date: Wed, 08 Jul 2015
04:24:38 "..., 35}, {"rn", 2}, {"Vary: Accept-Encoding", 21}, {"rn", 2}, {"X-Cache: MISS from non_disclosed",
32}, {"rn", 2}, {"X-Cache-Lookup: MISS from non_di"..., 44}, {"rn", 2}, {"Via: 1.0 non_disclosed:3128
(squ"..., 35}, {"rn", 2}, ...], 46) = 33415
1436329479.808307 writev(17, [{" href="/reise/startseite/reise/r"..., 32768}], 1) = 32768
1436329479.808803 writev(17, [{"n>Ruland 342200236Manta, Manta 2342200234?</"..., 32768}], 1) = 32768
1436329479.809250 writev(17, [{" "..., 32768}], 1) = 32768
1436329479.809745 writev(17, [{"ss="kicker">Land Rover</span>n "..., 32768}], 1) = 32768
1436329479.810218 writev(17, [{"d.gif" data-img-src="http://bild"..., 32768}], 1) = 32768
1436329479.810640 writev(17, [{"echenland-gabriel_46440882-14362"..., 32768}], 1) = 32768
1436329479.811115 writev(17, [{".de/fotos/ondemand-23661532/Bild"..., 32768}], 1) = 32768
1436329479.811643 writev(17, [{"e2=' + window.location.pathname"..., 32768}], 1) = 32768
1436329479.811873 writev(17, [{"/faq-artikel-web/bildplus/faq-we"..., 6500}], 1) = 6500
1436329479.811979 poll([{fd=17, events=POLLIN}], 1, 50) = 1 ([{fd=17, revents=POLLIN}])
1436329479.839042 read(17, "", 32767) = 0
1436329479.839113 close(17) = 0
● 21 syscalls, ~12 minimum, 32ms von Anfang
bis Ende (cache hit)
Laptop als bild.de
Server
ab -H 'Accept-Encoding: gzip' -H 'Host:
www.bild.de' -c 1000 -n 100000 localhost:8080/
...
Requests per second: 18175.82 [#/sec] (mean)
Time per request: 55.018 [ms] (mean)
Time per request: 0.055 [ms] (mean,
across all concurrent requests)
Transfer rate: 969279.78 [Kbytes/sec]
received
Varnish Performance
● Massiv multihreaded
● Moderne kernel-Interfaces
● mmap()
● event-interfaces Linux kqueues / Solaris ports
● 64bit
● Lock-freie Datenstrukturen
● Gzip-offload (seit 3.x)
Früher war alles
besser
Konfiguration
● Einige globale Parameter
● Kein klassisches Config-File
● Konfiguration ist CODE
● Kompilierung: VCL → C → Maschinencode
● Shared object („library“) → zur Laufzeit
ersetzen
● VCL subs = Hooks
VCL
sub recv_pool_www_special {
# redirect all .bto content to .bild
if (req.url ~ "^(.*).bto.([^.][^.]*)$") {
# redirect to .bild.suffix
set req.http.Location = "http://" req.http.Host
pmatch.1 ".bild." pmatch.2;
error 701 "Moved Permanently (bto)";
}
call recv_pool_www_xmli;
if (req.url ~ "^/no_url|^/gadget/version.xml") {
error 410;
}
}
In-line C
C{
time_t tnow, tcheck;
const time_t max_diff = 30;
time(&tnow);
tcheck = atol(VRT_r_pmatch(sp, &local_pmatch, 3));
if (labs(tnow - tcheck) > max_diff) {
VRT_error(sp, 400, "G2O time delta exceeded");
VRT_done(sp, VCL_RET_ERROR);
}
}C
VMODs
● VCL:
set req.http.X-DeviceClass =
dcs.type_class(dcs.classify());
● VMOD:
VCL_STRING vmod_type_class(dcs_ctx *ctx, VCL_INT e)
{
const VCL_INT t = dcs_match_type_id(e);
(void) ctx;
return dcs_type_class(t > 0 ? t : 0);
}
// ~
VCL_INT vmod_classify(dcs_ctx *ctx) {
return dcs_varnish_classify(ctx);
}
Custom-Entwicklung für bild.de
● BildPlus: Paywall / Vauth
● Video Token Service
● Addefend Integration
● Bilder URL-Normalisierung / Parameter Check
● Akamai G2O
● URL-Kanonifizierung
Custom-Entwicklung für bild.de
● VSLP Director
● Clustering
● Backend Conditional Requests
VSLP Director
● Varnish Stateless Persistence Director
● Sharding über Backends
● Bei 404 Objekt vom „frischen“ Backend
holen
cobildweb212
cobildweb218
cobildweb221
cobildcache101
Server-ids beispielhaft
Clustering
cobildcache101
cobildcache102
cobildcache103
cobildcache104
cobildcache105
cobildcache106
cobildcache107
cobildcache108
cobildweb212
cobildweb218
cobildweb221
Server-ids beispielhaft
Backend Conditional
Requests
●
If­Modified­Since und/oder If­None­Match
→ Backend
● Abgelaufene Objekte für eine begrenzte Zeit
im Cache behalten (keep).
● Konditionaler Request ans Backend, falls
Last­Modified und/oder Etag vorhanden
● UPLEX Entwicklung war experimenteller
Branch
● Default seit 4.0
Varnish bei bild.de
● Konfig-Generierung
Konfig-Generierung
● Warum?
● 11 verschiedene Server
– 8 prod
– X dev (war: 7, jetzt 1, In Zukunft ?)
– 1 stage
● Features an- und abschalten
● Varnish4 Migration aus einer Basis
Konfig-Generierung
● VCL
if (($ROLE == ROLE_STAGE) &&
$FEATURE[FEAT_STAGE_PROTECT]) {
VCL <<EOF;
sub vcl_recv {
unset req.http.X-Auth-Saved;
if (! (client.ip ~ relaunch_ip )) {
...
Konfig-Generierung
● Perl generiert VCL
● my %vservernames_bypool = (
"sport" => [
"www.ein-herz-fuer-kinder.de",
"sportbild.bild.de",
"tv.device.bild.de",
],
"ssl_channelizer" => [
"ssl-channelizer.bild.de",
]);
● VCL gen_pseudonyms_code(%vservernames_bypool,
sub { ("set req.http.X-V-Pool = "". $_[0]."";n".
"tcall vcl_recv_pool_". $_[0].";");
});
Konfig-Generierung
...
else if ((req.http.host == "www.ein-herz-fuer-kinder.de") ||
(req.http.host == "sportbild.bild.de") ||
(req.http.host == "tv.device.bild.de")) {
set req.http.X-V-Pool = "sport";
call vcl_recv_pool_sport;
}
else if ((req.http.host == "ssl-channelizer.bild.de")) {
set req.http.X-V-Pool = "ssl_channelizer";
call vcl_recv_pool_ssl_channelizer;
}
...
ABER
Bild.de läuft noch auf unserem Varnish 2.0.3
fork
:-(
● Migration auf 4.1 / master RSN
Danke!
Fragen?
● nils.goroll@uplex.de, +49-170-2723133

More Related Content

What's hot

PerfUG - Disruptor at ABC Arbitrage - March 2018
PerfUG - Disruptor at ABC Arbitrage - March 2018PerfUG - Disruptor at ABC Arbitrage - March 2018
PerfUG - Disruptor at ABC Arbitrage - March 2018Olivier Coanet
 
The Ring programming language version 1.5.2 book - Part 53 of 181
The Ring programming language version 1.5.2 book - Part 53 of 181The Ring programming language version 1.5.2 book - Part 53 of 181
The Ring programming language version 1.5.2 book - Part 53 of 181Mahmoud Samir Fayed
 
Richard Salter: Using the Titanium OpenGL Module
Richard Salter: Using the Titanium OpenGL ModuleRichard Salter: Using the Titanium OpenGL Module
Richard Salter: Using the Titanium OpenGL ModuleAxway Appcelerator
 
Understanding the nodejs event loop
Understanding the nodejs event loopUnderstanding the nodejs event loop
Understanding the nodejs event loopSaurabh Kumar
 
The Ring programming language version 1.5.4 book - Part 8 of 185
The Ring programming language version 1.5.4 book - Part 8 of 185The Ring programming language version 1.5.4 book - Part 8 of 185
The Ring programming language version 1.5.4 book - Part 8 of 185Mahmoud Samir Fayed
 
Box2D with SIMD in JavaScript
Box2D with SIMD in JavaScriptBox2D with SIMD in JavaScript
Box2D with SIMD in JavaScriptIntel® Software
 
Hunting Performance Problems in Node.js and beyond
Hunting Performance Problems in Node.js and beyondHunting Performance Problems in Node.js and beyond
Hunting Performance Problems in Node.js and beyondDaniel Khan
 
The Ring programming language version 1.5.1 book - Part 7 of 180
The Ring programming language version 1.5.1 book - Part 7 of 180The Ring programming language version 1.5.1 book - Part 7 of 180
The Ring programming language version 1.5.1 book - Part 7 of 180Mahmoud Samir Fayed
 
Flashback, el primer malware masivo de sistemas Mac
Flashback, el primer malware masivo de sistemas MacFlashback, el primer malware masivo de sistemas Mac
Flashback, el primer malware masivo de sistemas MacESET Latinoamérica
 
The Ring programming language version 1.5.3 book - Part 8 of 184
The Ring programming language version 1.5.3 book - Part 8 of 184The Ring programming language version 1.5.3 book - Part 8 of 184
The Ring programming language version 1.5.3 book - Part 8 of 184Mahmoud Samir Fayed
 
The Ring programming language version 1.2 book - Part 42 of 84
The Ring programming language version 1.2 book - Part 42 of 84The Ring programming language version 1.2 book - Part 42 of 84
The Ring programming language version 1.2 book - Part 42 of 84Mahmoud Samir Fayed
 
Atm machine using c++
Atm machine using c++Atm machine using c++
Atm machine using c++Aqib Memon
 

What's hot (20)

Multi qubit entanglement
Multi qubit entanglementMulti qubit entanglement
Multi qubit entanglement
 
PerfUG - Disruptor at ABC Arbitrage - March 2018
PerfUG - Disruptor at ABC Arbitrage - March 2018PerfUG - Disruptor at ABC Arbitrage - March 2018
PerfUG - Disruptor at ABC Arbitrage - March 2018
 
The Ring programming language version 1.5.2 book - Part 53 of 181
The Ring programming language version 1.5.2 book - Part 53 of 181The Ring programming language version 1.5.2 book - Part 53 of 181
The Ring programming language version 1.5.2 book - Part 53 of 181
 
Richard Salter: Using the Titanium OpenGL Module
Richard Salter: Using the Titanium OpenGL ModuleRichard Salter: Using the Titanium OpenGL Module
Richard Salter: Using the Titanium OpenGL Module
 
Understanding the nodejs event loop
Understanding the nodejs event loopUnderstanding the nodejs event loop
Understanding the nodejs event loop
 
HTML5 video filters
HTML5 video filtersHTML5 video filters
HTML5 video filters
 
The Ring programming language version 1.5.4 book - Part 8 of 185
The Ring programming language version 1.5.4 book - Part 8 of 185The Ring programming language version 1.5.4 book - Part 8 of 185
The Ring programming language version 1.5.4 book - Part 8 of 185
 
Box2D with SIMD in JavaScript
Box2D with SIMD in JavaScriptBox2D with SIMD in JavaScript
Box2D with SIMD in JavaScript
 
OpenGL L06-Performance
OpenGL L06-PerformanceOpenGL L06-Performance
OpenGL L06-Performance
 
Single qubit-gates operations
Single qubit-gates operationsSingle qubit-gates operations
Single qubit-gates operations
 
Hunting Performance Problems in Node.js and beyond
Hunting Performance Problems in Node.js and beyondHunting Performance Problems in Node.js and beyond
Hunting Performance Problems in Node.js and beyond
 
The Ring programming language version 1.5.1 book - Part 7 of 180
The Ring programming language version 1.5.1 book - Part 7 of 180The Ring programming language version 1.5.1 book - Part 7 of 180
The Ring programming language version 1.5.1 book - Part 7 of 180
 
Flashback, el primer malware masivo de sistemas Mac
Flashback, el primer malware masivo de sistemas MacFlashback, el primer malware masivo de sistemas Mac
Flashback, el primer malware masivo de sistemas Mac
 
The Ring programming language version 1.5.3 book - Part 8 of 184
The Ring programming language version 1.5.3 book - Part 8 of 184The Ring programming language version 1.5.3 book - Part 8 of 184
The Ring programming language version 1.5.3 book - Part 8 of 184
 
Google V8 engine
Google V8 engineGoogle V8 engine
Google V8 engine
 
The Ring programming language version 1.2 book - Part 42 of 84
The Ring programming language version 1.2 book - Part 42 of 84The Ring programming language version 1.2 book - Part 42 of 84
The Ring programming language version 1.2 book - Part 42 of 84
 
C programs
C programsC programs
C programs
 
ملخص البرمجة المرئية - الوحدة الثالثة
ملخص البرمجة المرئية - الوحدة الثالثةملخص البرمجة المرئية - الوحدة الثالثة
ملخص البرمجة المرئية - الوحدة الثالثة
 
Atm machine using c++
Atm machine using c++Atm machine using c++
Atm machine using c++
 
Ee
EeEe
Ee
 

Viewers also liked

SplunkLive! Munich 2015 - Graphmasters
SplunkLive! Munich 2015 - GraphmastersSplunkLive! Munich 2015 - Graphmasters
SplunkLive! Munich 2015 - GraphmastersSplunk
 
SplunkLive! München 2016 - Splunk für IT Operations
SplunkLive! München 2016 - Splunk für IT OperationsSplunkLive! München 2016 - Splunk für IT Operations
SplunkLive! München 2016 - Splunk für IT OperationsSplunk
 
Splunk Webinar: Verwandeln Sie Datensilos in Operational Intelligence
Splunk Webinar: Verwandeln Sie Datensilos in Operational IntelligenceSplunk Webinar: Verwandeln Sie Datensilos in Operational Intelligence
Splunk Webinar: Verwandeln Sie Datensilos in Operational IntelligenceGeorg Knon
 
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data Onboarding
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data OnboardingSplunkLive! München 2016 - Splunk Enterprise 6.3 - Data Onboarding
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data OnboardingSplunk
 
SplunkLive! München 2016 - Getting started with Splunk
SplunkLive! München 2016 - Getting started with SplunkSplunkLive! München 2016 - Getting started with Splunk
SplunkLive! München 2016 - Getting started with SplunkSplunk
 
SplunkLive! Customer Presentation – Directv
SplunkLive! Customer Presentation – DirectvSplunkLive! Customer Presentation – Directv
SplunkLive! Customer Presentation – DirectvSplunk
 
SplunkLive! München 2016 - Splunk für Security
SplunkLive! München 2016 - Splunk für SecuritySplunkLive! München 2016 - Splunk für Security
SplunkLive! München 2016 - Splunk für SecuritySplunk
 
SplunkLive! München 2016 - Splunk @ Datev
SplunkLive! München 2016 - Splunk @ DatevSplunkLive! München 2016 - Splunk @ Datev
SplunkLive! München 2016 - Splunk @ DatevSplunk
 
SplunkLive! München 2016 - Splunk @ UniCredit
SplunkLive! München 2016 - Splunk @ UniCreditSplunkLive! München 2016 - Splunk @ UniCredit
SplunkLive! München 2016 - Splunk @ UniCreditSplunk
 

Viewers also liked (9)

SplunkLive! Munich 2015 - Graphmasters
SplunkLive! Munich 2015 - GraphmastersSplunkLive! Munich 2015 - Graphmasters
SplunkLive! Munich 2015 - Graphmasters
 
SplunkLive! München 2016 - Splunk für IT Operations
SplunkLive! München 2016 - Splunk für IT OperationsSplunkLive! München 2016 - Splunk für IT Operations
SplunkLive! München 2016 - Splunk für IT Operations
 
Splunk Webinar: Verwandeln Sie Datensilos in Operational Intelligence
Splunk Webinar: Verwandeln Sie Datensilos in Operational IntelligenceSplunk Webinar: Verwandeln Sie Datensilos in Operational Intelligence
Splunk Webinar: Verwandeln Sie Datensilos in Operational Intelligence
 
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data Onboarding
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data OnboardingSplunkLive! München 2016 - Splunk Enterprise 6.3 - Data Onboarding
SplunkLive! München 2016 - Splunk Enterprise 6.3 - Data Onboarding
 
SplunkLive! München 2016 - Getting started with Splunk
SplunkLive! München 2016 - Getting started with SplunkSplunkLive! München 2016 - Getting started with Splunk
SplunkLive! München 2016 - Getting started with Splunk
 
SplunkLive! Customer Presentation – Directv
SplunkLive! Customer Presentation – DirectvSplunkLive! Customer Presentation – Directv
SplunkLive! Customer Presentation – Directv
 
SplunkLive! München 2016 - Splunk für Security
SplunkLive! München 2016 - Splunk für SecuritySplunkLive! München 2016 - Splunk für Security
SplunkLive! München 2016 - Splunk für Security
 
SplunkLive! München 2016 - Splunk @ Datev
SplunkLive! München 2016 - Splunk @ DatevSplunkLive! München 2016 - Splunk @ Datev
SplunkLive! München 2016 - Splunk @ Datev
 
SplunkLive! München 2016 - Splunk @ UniCredit
SplunkLive! München 2016 - Splunk @ UniCreditSplunkLive! München 2016 - Splunk @ UniCredit
SplunkLive! München 2016 - Splunk @ UniCredit
 

Similar to Varnish kann alles

4 Node.js Gotchas: What your ops team needs to know
4 Node.js Gotchas: What your ops team needs to know4 Node.js Gotchas: What your ops team needs to know
4 Node.js Gotchas: What your ops team needs to knowDynatrace
 
Static analysis of C++ source code
Static analysis of C++ source codeStatic analysis of C++ source code
Static analysis of C++ source codePVS-Studio
 
Static analysis of C++ source code
Static analysis of C++ source codeStatic analysis of C++ source code
Static analysis of C++ source codeAndrey Karpov
 
Nsd, il tuo compagno di viaggio quando Domino va in crash
Nsd, il tuo compagno di viaggio quando Domino va in crashNsd, il tuo compagno di viaggio quando Domino va in crash
Nsd, il tuo compagno di viaggio quando Domino va in crashFabio Pignatti
 
Vectorization on x86: all you need to know
Vectorization on x86: all you need to knowVectorization on x86: all you need to know
Vectorization on x86: all you need to knowRoberto Agostino Vitillo
 
Eta lang Beauty And The Beast
Eta lang Beauty And The Beast Eta lang Beauty And The Beast
Eta lang Beauty And The Beast Jarek Ratajski
 
Top 10 bugs in C++ open source projects, checked in 2016
Top 10 bugs in C++ open source projects, checked in 2016Top 10 bugs in C++ open source projects, checked in 2016
Top 10 bugs in C++ open source projects, checked in 2016PVS-Studio
 
Analyzing the Performance of Mobile Web
Analyzing the Performance of Mobile WebAnalyzing the Performance of Mobile Web
Analyzing the Performance of Mobile WebAriya Hidayat
 
Performance measurement and tuning
Performance measurement and tuningPerformance measurement and tuning
Performance measurement and tuningAOE
 
Snake Game on FPGA in Verilog
Snake Game on FPGA in VerilogSnake Game on FPGA in Verilog
Snake Game on FPGA in VerilogKrishnajith S S
 
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeks
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeksBeginning direct3d gameprogramming05_thebasics_20160421_jintaeks
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeksJinTaek Seo
 
The Ring programming language version 1.8 book - Part 12 of 202
The Ring programming language version 1.8 book - Part 12 of 202The Ring programming language version 1.8 book - Part 12 of 202
The Ring programming language version 1.8 book - Part 12 of 202Mahmoud Samir Fayed
 
All I know about rsc.io/c2go
All I know about rsc.io/c2goAll I know about rsc.io/c2go
All I know about rsc.io/c2goMoriyoshi Koizumi
 
如何透過 Go-kit 快速搭建微服務架構應用程式實戰
如何透過 Go-kit 快速搭建微服務架構應用程式實戰如何透過 Go-kit 快速搭建微服務架構應用程式實戰
如何透過 Go-kit 快速搭建微服務架構應用程式實戰KAI CHU CHUNG
 
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...Andrey Karpov
 
HTML5 - Daha Flash bir web?
HTML5 - Daha Flash bir web?HTML5 - Daha Flash bir web?
HTML5 - Daha Flash bir web?Ankara JUG
 
LSFMM 2019 BPF Observability
LSFMM 2019 BPF ObservabilityLSFMM 2019 BPF Observability
LSFMM 2019 BPF ObservabilityBrendan Gregg
 
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015Windows Developer
 

Similar to Varnish kann alles (20)

4 Node.js Gotchas: What your ops team needs to know
4 Node.js Gotchas: What your ops team needs to know4 Node.js Gotchas: What your ops team needs to know
4 Node.js Gotchas: What your ops team needs to know
 
Static analysis of C++ source code
Static analysis of C++ source codeStatic analysis of C++ source code
Static analysis of C++ source code
 
Static analysis of C++ source code
Static analysis of C++ source codeStatic analysis of C++ source code
Static analysis of C++ source code
 
Nsd, il tuo compagno di viaggio quando Domino va in crash
Nsd, il tuo compagno di viaggio quando Domino va in crashNsd, il tuo compagno di viaggio quando Domino va in crash
Nsd, il tuo compagno di viaggio quando Domino va in crash
 
Vectorization on x86: all you need to know
Vectorization on x86: all you need to knowVectorization on x86: all you need to know
Vectorization on x86: all you need to know
 
Osol Pgsql
Osol PgsqlOsol Pgsql
Osol Pgsql
 
Eta lang Beauty And The Beast
Eta lang Beauty And The Beast Eta lang Beauty And The Beast
Eta lang Beauty And The Beast
 
Top 10 bugs in C++ open source projects, checked in 2016
Top 10 bugs in C++ open source projects, checked in 2016Top 10 bugs in C++ open source projects, checked in 2016
Top 10 bugs in C++ open source projects, checked in 2016
 
Analyzing the Performance of Mobile Web
Analyzing the Performance of Mobile WebAnalyzing the Performance of Mobile Web
Analyzing the Performance of Mobile Web
 
Performance measurement and tuning
Performance measurement and tuningPerformance measurement and tuning
Performance measurement and tuning
 
Snake Game on FPGA in Verilog
Snake Game on FPGA in VerilogSnake Game on FPGA in Verilog
Snake Game on FPGA in Verilog
 
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeks
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeksBeginning direct3d gameprogramming05_thebasics_20160421_jintaeks
Beginning direct3d gameprogramming05_thebasics_20160421_jintaeks
 
The Ring programming language version 1.8 book - Part 12 of 202
The Ring programming language version 1.8 book - Part 12 of 202The Ring programming language version 1.8 book - Part 12 of 202
The Ring programming language version 1.8 book - Part 12 of 202
 
20151224-games
20151224-games20151224-games
20151224-games
 
All I know about rsc.io/c2go
All I know about rsc.io/c2goAll I know about rsc.io/c2go
All I know about rsc.io/c2go
 
如何透過 Go-kit 快速搭建微服務架構應用程式實戰
如何透過 Go-kit 快速搭建微服務架構應用程式實戰如何透過 Go-kit 快速搭建微服務架構應用程式實戰
如何透過 Go-kit 快速搭建微服務架構應用程式實戰
 
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
PVS-Studio and Continuous Integration: TeamCity. Analysis of the Open RollerC...
 
HTML5 - Daha Flash bir web?
HTML5 - Daha Flash bir web?HTML5 - Daha Flash bir web?
HTML5 - Daha Flash bir web?
 
LSFMM 2019 BPF Observability
LSFMM 2019 BPF ObservabilityLSFMM 2019 BPF Observability
LSFMM 2019 BPF Observability
 
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015
Build 2016 - B880 - Top 6 Reasons to Move Your C++ Code to Visual Studio 2015
 

Recently uploaded

GraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphGraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphNeo4j
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Maxim Salnikov
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMarkus Moeller
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationElement34
 
Novo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMsNovo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMsNeo4j
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14VMware Tanzu
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)Roberto Bettazzoni
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIInflectra
 
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Flutter Agency
 
Your Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | EvmuxYour Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | Evmuxevmux96
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insightsrapidoform
 
Transformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksTransformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksJinanKordab
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AIAGATSoftware
 
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaNeo4j
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...drm1699
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfSrushith Repakula
 
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
Workshop -  Architecting Innovative Graph Applications- GraphSummit MilanWorkshop -  Architecting Innovative Graph Applications- GraphSummit Milan
Workshop - Architecting Innovative Graph Applications- GraphSummit MilanNeo4j
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024MulesoftMunichMeetup
 

Recently uploaded (20)

GraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with GraphGraphSummit Milan - Neo4j: The Art of the Possible with Graph
GraphSummit Milan - Neo4j: The Art of the Possible with Graph
 
Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?Prompt Engineering - an Art, a Science, or your next Job Title?
Prompt Engineering - an Art, a Science, or your next Job Title?
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdf
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test Automation
 
Novo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMsNovo Nordisk: When Knowledge Graphs meet LLMs
Novo Nordisk: When Knowledge Graphs meet LLMs
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14
 
Abortion Pill Prices Turfloop ](+27832195400*)[ 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Turfloop ](+27832195400*)[ 🏥 Women's Abortion Clinic in ...Abortion Pill Prices Turfloop ](+27832195400*)[ 🏥 Women's Abortion Clinic in ...
Abortion Pill Prices Turfloop ](+27832195400*)[ 🏥 Women's Abortion Clinic in ...
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST API
 
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
 
Your Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | EvmuxYour Ultimate Web Studio for Streaming Anywhere | Evmux
Your Ultimate Web Studio for Streaming Anywhere | Evmux
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insights
 
Transformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with LinksTransformer Neural Network Use Cases with Links
Transformer Neural Network Use Cases with Links
 
BusinessGPT - Security and Governance for Generative AI
BusinessGPT  - Security and Governance for Generative AIBusinessGPT  - Security and Governance for Generative AI
BusinessGPT - Security and Governance for Generative AI
 
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
 
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
Abortion Pills For Sale WhatsApp[[+27737758557]] In Birch Acres, Abortion Pil...
 
Lessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdfLessons Learned from Building a Serverless Notifications System.pdf
Lessons Learned from Building a Serverless Notifications System.pdf
 
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
Workshop -  Architecting Innovative Graph Applications- GraphSummit MilanWorkshop -  Architecting Innovative Graph Applications- GraphSummit Milan
Workshop - Architecting Innovative Graph Applications- GraphSummit Milan
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 

Varnish kann alles

  • 1. Varnish kann Alles 8. Juli 2015 Nils Goroll
  • 2. Varnish kann Alles in 25 Minuten 8. Juli 2015 Nils Goroll
  • 3. Varnish kann Alles in 25 Minuten 8. Juli 2015 Nils Goroll ein kurzer Blick aufein kurzer Blick auf effizientes Web-Cachingeffizientes Web-Caching
  • 4. Setup Reverse Proxy ● + Caching ● Keine Generierung von Inhalten
  • 5. Effizienz! ● Bild home von Varnish – System Call Trace 1436329479.806468 getsockname(17, {sa_family=AF_INET6, sin6_port=htons(8080), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [28]) = 0 1436329479.806645 setsockopt(17, SOL_SOCKET, SO_LINGER, {onoff=0, linger=0}, 8) = 0 1436329479.806718 setsockopt(17, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 1436329479.806762 setsockopt(17, SOL_TCP, TCP_NODELAY, [1], 4) = 0 1436329479.806804 ioctl(17, FIONBIO, [0]) = 0 1436329479.806854 ioctl(17, FIONBIO, [0]) = 0 1436329479.806888 poll([{fd=17, events=POLLIN}], 1, 50) = 1 ([{fd=17, revents=POLLIN}]) 1436329479.806938 read(17, "GET / HTTP/1.1rnUser-Agent: curl"..., 32767) = 75 1436329479.807612 writev(17, [{"HTTP/1.1", 8}, {" ", 1}, {"200", 3}, {" ", 1}, {"OK", 2}, {"rn", 2}, {"Content- Type: text/html;charset="..., 37}, {"rn", 2}, {"Content-Language: de-DE", 23}, {"rn", 2}, {"Last-Modified: Wed, 08 Jul 2015 "..., 44}, {"rn", 2}, {"X-UA-Compatible: IE=Edge,chrome="..., 33}, {"rn", 2}, {"X-Refresh: h=23|g=103975.000|c=G"..., 98}, {"rn", 2}, {"Server: Apache", 14}, {"rn", 2}, {"Cache-Control: public, max- age=3"..., 33}, {"rn", 2}, {"Expires: Wed, 08 Jul 2015 04:25:"..., 38}, {"rn", 2}, {"Date: Wed, 08 Jul 2015 04:24:38 "..., 35}, {"rn", 2}, {"Vary: Accept-Encoding", 21}, {"rn", 2}, {"X-Cache: MISS from non_disclosed", 32}, {"rn", 2}, {"X-Cache-Lookup: MISS from non_di"..., 44}, {"rn", 2}, {"Via: 1.0 non_disclosed:3128 (squ"..., 35}, {"rn", 2}, ...], 46) = 33415 1436329479.808307 writev(17, [{" href="/reise/startseite/reise/r"..., 32768}], 1) = 32768 1436329479.808803 writev(17, [{"n>Ruland 342200236Manta, Manta 2342200234?</"..., 32768}], 1) = 32768 1436329479.809250 writev(17, [{" "..., 32768}], 1) = 32768 1436329479.809745 writev(17, [{"ss="kicker">Land Rover</span>n "..., 32768}], 1) = 32768 1436329479.810218 writev(17, [{"d.gif" data-img-src="http://bild"..., 32768}], 1) = 32768 1436329479.810640 writev(17, [{"echenland-gabriel_46440882-14362"..., 32768}], 1) = 32768 1436329479.811115 writev(17, [{".de/fotos/ondemand-23661532/Bild"..., 32768}], 1) = 32768 1436329479.811643 writev(17, [{"e2=' + window.location.pathname"..., 32768}], 1) = 32768 1436329479.811873 writev(17, [{"/faq-artikel-web/bildplus/faq-we"..., 6500}], 1) = 6500 1436329479.811979 poll([{fd=17, events=POLLIN}], 1, 50) = 1 ([{fd=17, revents=POLLIN}]) 1436329479.839042 read(17, "", 32767) = 0 1436329479.839113 close(17) = 0 ● 21 syscalls, ~12 minimum, 32ms von Anfang bis Ende (cache hit)
  • 6. Laptop als bild.de Server ab -H 'Accept-Encoding: gzip' -H 'Host: www.bild.de' -c 1000 -n 100000 localhost:8080/ ... Requests per second: 18175.82 [#/sec] (mean) Time per request: 55.018 [ms] (mean) Time per request: 0.055 [ms] (mean, across all concurrent requests) Transfer rate: 969279.78 [Kbytes/sec] received
  • 7. Varnish Performance ● Massiv multihreaded ● Moderne kernel-Interfaces ● mmap() ● event-interfaces Linux kqueues / Solaris ports ● 64bit ● Lock-freie Datenstrukturen ● Gzip-offload (seit 3.x)
  • 9. Konfiguration ● Einige globale Parameter ● Kein klassisches Config-File ● Konfiguration ist CODE ● Kompilierung: VCL → C → Maschinencode ● Shared object („library“) → zur Laufzeit ersetzen ● VCL subs = Hooks
  • 10. VCL sub recv_pool_www_special { # redirect all .bto content to .bild if (req.url ~ "^(.*).bto.([^.][^.]*)$") { # redirect to .bild.suffix set req.http.Location = "http://" req.http.Host pmatch.1 ".bild." pmatch.2; error 701 "Moved Permanently (bto)"; } call recv_pool_www_xmli; if (req.url ~ "^/no_url|^/gadget/version.xml") { error 410; } }
  • 11. In-line C C{ time_t tnow, tcheck; const time_t max_diff = 30; time(&tnow); tcheck = atol(VRT_r_pmatch(sp, &local_pmatch, 3)); if (labs(tnow - tcheck) > max_diff) { VRT_error(sp, 400, "G2O time delta exceeded"); VRT_done(sp, VCL_RET_ERROR); } }C
  • 12. VMODs ● VCL: set req.http.X-DeviceClass = dcs.type_class(dcs.classify()); ● VMOD: VCL_STRING vmod_type_class(dcs_ctx *ctx, VCL_INT e) { const VCL_INT t = dcs_match_type_id(e); (void) ctx; return dcs_type_class(t > 0 ? t : 0); } // ~ VCL_INT vmod_classify(dcs_ctx *ctx) { return dcs_varnish_classify(ctx); }
  • 13. Custom-Entwicklung für bild.de ● BildPlus: Paywall / Vauth ● Video Token Service ● Addefend Integration ● Bilder URL-Normalisierung / Parameter Check ● Akamai G2O ● URL-Kanonifizierung
  • 14. Custom-Entwicklung für bild.de ● VSLP Director ● Clustering ● Backend Conditional Requests
  • 15. VSLP Director ● Varnish Stateless Persistence Director ● Sharding über Backends ● Bei 404 Objekt vom „frischen“ Backend holen cobildweb212 cobildweb218 cobildweb221 cobildcache101 Server-ids beispielhaft
  • 17. Backend Conditional Requests ● If­Modified­Since und/oder If­None­Match → Backend ● Abgelaufene Objekte für eine begrenzte Zeit im Cache behalten (keep). ● Konditionaler Request ans Backend, falls Last­Modified und/oder Etag vorhanden ● UPLEX Entwicklung war experimenteller Branch ● Default seit 4.0
  • 18. Varnish bei bild.de ● Konfig-Generierung
  • 19. Konfig-Generierung ● Warum? ● 11 verschiedene Server – 8 prod – X dev (war: 7, jetzt 1, In Zukunft ?) – 1 stage ● Features an- und abschalten ● Varnish4 Migration aus einer Basis
  • 20. Konfig-Generierung ● VCL if (($ROLE == ROLE_STAGE) && $FEATURE[FEAT_STAGE_PROTECT]) { VCL <<EOF; sub vcl_recv { unset req.http.X-Auth-Saved; if (! (client.ip ~ relaunch_ip )) { ...
  • 21. Konfig-Generierung ● Perl generiert VCL ● my %vservernames_bypool = ( "sport" => [ "www.ein-herz-fuer-kinder.de", "sportbild.bild.de", "tv.device.bild.de", ], "ssl_channelizer" => [ "ssl-channelizer.bild.de", ]); ● VCL gen_pseudonyms_code(%vservernames_bypool, sub { ("set req.http.X-V-Pool = "". $_[0]."";n". "tcall vcl_recv_pool_". $_[0].";"); });
  • 22. Konfig-Generierung ... else if ((req.http.host == "www.ein-herz-fuer-kinder.de") || (req.http.host == "sportbild.bild.de") || (req.http.host == "tv.device.bild.de")) { set req.http.X-V-Pool = "sport"; call vcl_recv_pool_sport; } else if ((req.http.host == "ssl-channelizer.bild.de")) { set req.http.X-V-Pool = "ssl_channelizer"; call vcl_recv_pool_ssl_channelizer; } ...
  • 23. ABER Bild.de läuft noch auf unserem Varnish 2.0.3 fork :-( ● Migration auf 4.1 / master RSN