This document summarizes Mike Willbanks' presentation on Varnish to the MinneBar group on April 7, 2012. The presentation covered an introduction to Varnish including what it is and how it works, the benefits of using Varnish to accelerate websites ("The Good"), more advanced Varnish features like VCL and directors ("The Awesome"), and complex Varnish configurations involving ESI, purging, and embedding C code in VCL ("The Crazy"). It also provided overviews of various Varnish command line applications for monitoring and debugging Varnish like varnishtop, varnishstat, and varnishlog.
Looking at historic, current and evolving approaches, I will take you through from how we used to 'live' edit on one server with HTML in the code; to implementing Template Toolkit and 'front end / back end' servers; to the addition of version control; all the way through to distributed caching, file systems and processing (aka Six Apart worship) with 15+ servers.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Looking at historic, current and evolving approaches, I will take you through from how we used to 'live' edit on one server with HTML in the code; to implementing Template Toolkit and 'front end / back end' servers; to the addition of version control; all the way through to distributed caching, file systems and processing (aka Six Apart worship) with 15+ servers.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Want to get to production quickly? RAD tools like Spring Roo, with its support for beautiful, quick UI generation through addons like the Vaadin Roo addon, and Cloud Foundry, which take care of everything under the code, are an ideal combination. In this talk Josh Long, Spring Developer Advocate for SpringSource, introduces the 1-2-3 punch of Cloud Foundry, Roo and Vaadin.
Apache Camel: The Swiss Army Knife of Open Source Integrationprajods
The Camel project from Apache(camel.apache.org), is a very popular, light weight, open source integration framework.
This presentation shows some interesting features of Camel and the unique advantages that Camel brings to your integration projects. Some business
use cases are shown to explain how Camel makes open source integration a cakewalk.
Table of contents:
1. An overview of Apache Camel
2. Integration architecture explained
3. Using Camel in different integration architectures
3.a. In the Securities domain
3.b. In the Travel domain
4. High Availability and Load Balancing with Camel
Ror Seminar With agilebd.org on 23 Jan09Shaer Hassan
This presentation is done by Code71 Team to the IT community in Bangladesh. The presentation covers the basics of Ruby on Rails and the advantage of it over many other contemporary languages to build web applications. It also mentions the strength of RoR by siting great quotes and examples of great sites.
This presentation explains how to deploy and use the Integrated Caching feature on Netscaler. I gave this presentation to Citrix staff, customers and partners in worldwide in 2011. The presentation covers best practices and gotchas :) Integrated Caching is an excellent feature that can greatly improve the performance of your website.
Want to get to production quickly? RAD tools like Spring Roo, with its support for beautiful, quick UI generation through addons like the Vaadin Roo addon, and Cloud Foundry, which take care of everything under the code, are an ideal combination. In this talk Josh Long, Spring Developer Advocate for SpringSource, introduces the 1-2-3 punch of Cloud Foundry, Roo and Vaadin.
Apache Camel: The Swiss Army Knife of Open Source Integrationprajods
The Camel project from Apache(camel.apache.org), is a very popular, light weight, open source integration framework.
This presentation shows some interesting features of Camel and the unique advantages that Camel brings to your integration projects. Some business
use cases are shown to explain how Camel makes open source integration a cakewalk.
Table of contents:
1. An overview of Apache Camel
2. Integration architecture explained
3. Using Camel in different integration architectures
3.a. In the Securities domain
3.b. In the Travel domain
4. High Availability and Load Balancing with Camel
Ror Seminar With agilebd.org on 23 Jan09Shaer Hassan
This presentation is done by Code71 Team to the IT community in Bangladesh. The presentation covers the basics of Ruby on Rails and the advantage of it over many other contemporary languages to build web applications. It also mentions the strength of RoR by siting great quotes and examples of great sites.
This presentation explains how to deploy and use the Integrated Caching feature on Netscaler. I gave this presentation to Citrix staff, customers and partners in worldwide in 2011. The presentation covers best practices and gotchas :) Integrated Caching is an excellent feature that can greatly improve the performance of your website.
IRL: How Geeks Undermine Their Presentations & Conversations With Body Languagesarahnovotny
Watching my peers and staff speak and work with clients for many years, I have realized that many geeks are uncomfortable interacting IRL. But, you don’t have to be. There are some simple physical tricks to keeping an audience (of 1 or 1k) engaged and not undermining your skills and yourself. I’m going to talk about positioning of the body, head and hands in relation to the people you’re interacting with, in relation to the room, and in relation to yourself. I’ll outline some best practices, some behaviors to watch for in you, and even some things that you can watch for in other people.
Talk given at ClueCon 2016 that discusses FreeSWITCH and its place in a microservices architecture. Covers a specific deployment case using Docker and Adhearsion, along with certain features that make FreeSWITCH a model use-case for such a technology stack.
www.caucho.com
Using industry standard tool and methodology, Resin Pro web server was put to the test versus Nginx, a popular web server with a reputation for efficiency and performance. Nginx is known to be faster and more reliable under load than the popular Apache HTTPD. Benchmark tests between Resin and Nginx yielded competitive figures, with Resin leading with fewer errors and faster response times. In numerous and varying tests, Resin handled 20% to 25% more load while still outperforming Nginx. In particular, Resin was able to sustain fast response times under extremely heavy load while Nginx performance degraded.
My talk at ScaleConf 2017 in Cape Town on some tips and tactics for scaling WordPress, with reference to WordPress.com and the container-based VIP Go platform.
Video of my talk is here: https://www.youtube.com/watch?v=cs0DcY80spw
Rooting your internals - Exploiting Internal Network Vulns via the Browser Us...Michele Orru
Browser exploits are a primary attack vector to compromise a victims internal network, but they have major restrictions including; limited current browser exploits; the huge price for 0-day browser exploits; and exploit complexity due to sandboxing. So, instead of exploiting the victims browser, what if the victims browser exploited internal systems for you?
The new "BeEF Bind Exploit Proxy" module does this! This BeEF (Browser Exploitation Framework) module will allow penetration testers to proxy exploits through a victims web browser to compromise internal services. Not only this, but the new "BeEF Bind" shellcode also enables the communication channel to the attacker to pass back through the existing browser session.
This attack technique (Inter-protocol Exploitation) removes browser-based attacks from being dependent upon browser vulnerabilities. It increases the number of potential exploits to include many service vulnerabilities throughout the internal corporate network. This includes whatever service can be contacted via a browser request. This increases the success rate of client-side exploitation attempts by dramatically increasing the number of vulnerabilities accessible to the attacker.
So how does the new BeEF Bind Exploit Proxy work? BeEF is configured to use the BeEF Bind Exploit Proxy, and is set as the payload for XSS exploits or Phishing attacks. Once the victim visits the malicious site, their web browser becomes hooked and performs JavaScript port scanning across the internal corporate network looking for chosen open ports. Once a server has been identified, the BeEF server is notified and begins to send exploits through the hooked web browser to the service on the internal server. Each of these exploits are configured to use the new BeEF Bind shellcode.
Once an exploit has successfully triggered a vulnerability within the internal service, the BeEF Bind shellcode is executed. This shellcode is designed to setup a web-listener that proxies commands through to a shell on the compromised server. This allows the attacker to send commands through the hooked web browser to the BeEF Bind payload. The command is executed on the compromised server and returned to the web browser in HTTP responses. The hooked web browser is then able to receive the command output and proxy it back to the attacker at the BeEF server.
Penetration testers can now inject steroids into their XSS exploits by replacing simple alert boxes with demonstrations of actual compromised internal machines. They can also now increase the scope and success rate of their Phishing attacks to compromise internal servers. This new approach also minimizes the likelihood of IDS/IPS detection, and does not require an additional socket open back to the attacker via the firewall.
You Call that Micro, Mr. Docker? How OSv and Unikernels Help Micro-services S...rhatr
OSv is the new open source unikernel technology that combines the power of virtualization and micro-services architecture. This combination allows unmodified applications to be packaged just like Docker containers while at the same time outperform bare-metal deployments. Yes. You've heard it right: for the first time ever we can stop asking the question of how much performance would I lose if I virtualize. OSv lets you ask a different question: how much would my application gain in performance if I virtualize it. This talk will start by looking into the architecture of OSv and the kind of optimizations it makes possible for native, unmodified applications. We will then focus on JVM-specific optimizations and specifically on speedups available to micro-service oriented applications when they are being deployed on OSv.
Slides of my talk at QCon Hangzhou 2011, on the things that my team has been doing on the JVM, esp. on customization. Video available at http://www.infoq.com/cn/presentations/ms-jvm-taobao
- Emilien Macchi & Chmouel Boudjnah (eNovance)
Emilien et Chmouel sont tous deux développeurs OpenStack et sont de forts contributeurs aux projets Swift et Quantum. Ils introduiront le meetup en donnant une update sur les projets, les fonctionnalités de la release Grizzly et ce que nous pouvons attendre de la release Havana qui se dessinera à l'OpenStack Summit de Portland. Ils présenteront également un rapide bilan sur: "OpenStack: 3 ans après: qui sont les plus gros contributeurs du projet?"
ZF2 takes a different approach to services; there are several services out there and you should be providing the ability for ZF2 to integrate with this. ZF2 marries services with composer and a different packaging mechanism to ensure that services can be released without a specific framework version. This not only helps the framework but helps you prevent an API changing in between framework releases without having an issue of awaiting a framework release.
Presentation on sending push notifications with Zend Framework based on work to create Zend_Mobile_Push. Currently handles APNs (iPhone/iPad/iTouch), C2DM (Android) and MPNS (Windows Mobile). Additionally mentions of sending push notifications with WAP PAP (BlackBerry).
Mobile: for push and sync. Goes through implementing push notifications with PHP on the various 4 major players: Android (C2DM), Apple (APNS), Windows Phone (MPNS), BlackBerry (maybe works?).
So, you know how to deploy your code, what about your database? This talk will go through deploying your database with LiquiBase and DBDeploy a non-framework based approach to handling migrations of DDL and DML.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
Essentials of Automations: The Art of Triggers and Actions in FME
Varnish, The Good, The Awesome, and the Downright Crazy.
1. MinneBar April 7, 2012
Varnish, The Good, The
Awesome, and the
Downright Crazy
By Mike Willbanks
Software Engineering Manager
CaringBridge
2. Housekeeping…
• Talk
Slides will be online later!
• Me
Software Engineering Manager at CaringBridge
MNPHP Organizer
Open Source Contributor (Zend Framework and various others)
Where you can find me:
• Twitter: mwillbanks G+: Mike Willbanks
• IRC (freenode): mwillbanks Blog: http://blog.digitalstruct.com
• GitHub: https://github.com/mwillbanks
2
3. Agenda
• What the … is Varnish?
• The Good
“I don’t want to listen to you but only want to play on my laptop.”
The quick, easy and uninformed way... You listening over there?
• The Awesome
VCL’s, Directors and more… now we’re going places.
• The Crazy
ESI, Purging, VCL C, and VMOD… my head hurts, stop!
• Varnish Command Line Apps
varnishtop, varnishstat, etc.
• Questions
Although you can bring them up at anytime!
3
4. What the… is Varnish?
Official Statement
What the hell it means
Graphs, oh my!
5. Official Statement
“Varnish is a web application accelerator. You install it in
front of your web application and it will speed it up
significantly.”
5
6. What The Hell? Tell me!
• Varnish allow you to accelerate your website
By using memory and keeping in mind cookies, request headers
and more…
• It caches pages so that your web server can RELAX!
What about my apache, tomcat, uhhh… (mongrel|thin|goliath….)
Generally caching by TTL + HTTP Headers (cookies too!)
• A load banancer, proxy and more…
What? …. Yes, it can do that! #winning
6
7. A General Use Case
• CaringBridge Status Server
We need to get a message to our mobile users!
The system is down, or we want to be able to communicate a
message to them about some subject… maybe a campaign.
The apps and mobile site rely on an API
• Trouble in paradise? Few and far in between.
• We want to talk to our users? Of course!
Let an API talk to a server…
A story on crashing and burning
A story on a bad thing making a huge success!
7
8. The Graph - AWS
Req/s Peak Load
700 14
600 12
500 10
400 8
300 Req/s Peak Load
6
200 4
100
2
0
0
Small X-Large Small Varnish
Small X-Large Small Varnish
Time Requests
500 80000
450 70000
400
60000
350
300 50000
250 40000
Time Requests
200 30000
150
20000
100
50 10000
0 0
8 Small X-Large Small Varnish Small X-Large Small Varnish
9. The Raw Data
Small X-Large Small Varnish
Concurrency 10 150 150
Requests 5000 55558 75000
Time 438 347 36
Req/s 11.42 58 585
Peak Load 11.91 8.44 0.35
19,442
Comments failed reqs
9
10. The Good – Listen Up!
Installment
Documentation
Finding Existing VCL’s
11. Installment
• RTM : http://goo.gl/hl4Tt
Debian: sudo apt-get install varnish
EPEL: yum install varnish
• only 6.x otherwise you’ll be out of date!
WOOT Compiling #winning #git
• git clone git://git.varnish-cache.org/varnish-cache
• cd varnish-cache
• sh autogen.sh
• ./configure
• Make && make install
11
12. Documentation
• Reference Manual
https://www.varnish-cache.org/docs/3.0/reference/index.html
• Tutorial – more like a book version of the reference manual
https://www.varnish-cache.org/docs/3.0/tutorial/index.html
• Knock yourselves out! There is a ton of documentation
• Yes, this makes happy developers.
Documentation is very accurate, read carefully.
Focus heavily on VCL’s, that is generally what you need.
I’m attempting to show you some of how this works but you will
require the documentation to assist you.
12
13. Existing VCL’s – The truly lazy…
• VCL’s are available for common open source projects
Hi wordpress and drupal!
• https://www.varnish-cache.org/trac/wiki/VarnishAndWordpress
• https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal
Examples of all sorts of crazy
• https://www.varnish-cache.org/trac/wiki/VCLExamples
13
14. Wordpress = Bad Slashdot Bad!!!
backend default {
.host = "127.0.0.1“;
.port = "8080";
}
sub vcl_recv {
if (!(req.url ~ "wp-(login|admin)")) {
unset req.http.cookie;
}
}
sub vcl_fetch {
if (!(req.url ~ "wp-(login|admin)")) {
unset beresp.http.set-cookie;
}
}
14
15. The Awesome – Going Places
VCL
Directors
A Few Examples
17. VCL – Varnish Configuration Language
• VCL State Engine
Each Request is Processed Separately & Independently
States are Isolated but are Related
Return statements exit one state and start another
VCL defaults are ALWAYS appended below your own VCL
• VCL can be complex, but…
Two main subroutines; vcl_recv and vcl_fetch
Common actions: pass, hit_for_pass, lookup, pipe, deliver
Common variables: req, beresp and obj
More subroutines, functions and complexity can arise dependent
on condition.
17
18. VCL - Subroutines
• vcl_init – VCL is loaded, no request yet; VMOD initialization
• vcl_recv – Beginning of request, req is in scope
• vcl_pipe – Client & backend data passed unaltered
• vcl_pass – Request goes to backend and not cached
• vcl_hash – call hash_data to add to the hash
• vcl_hit – called on request found in the cache
• vcl_miss – called on request not found in the cache
• vcl_fetch – called on document retrieved from backend
• vcl_deliver – called prior to delivery of cached object
• vcl_error – called on errors
18
• vcl_fini – all requests have exited VCL, cleanup of VMOD’s
19. VCL - Variables
• Always Available • Backend Req Prepartion
now – epoch time bereq – backend request
• Backend Declarations • Retrieved Backend Request
.host – hostname / IP beresp – backend response
.port – port number • Cached Object
• Request Processing obj – Cached object, can only
client – ip & identity change .ttl
server – ip & port • Response Preparation
req – request information resp – http stuff
19
20. VCL - Functions
• hash_data(string) – adds a string to the hash input.
Request host and URL is default from the default vcl.
• regsub(string, regex, sub) – substitution on first occurance
sub can contain numbers 0-n to inject matches from the regex.
• regsuball(string, regex, sub) – substitution on all occurances
• ban(expression) – Ban all objects in cache that match
• ban(regex) – Ban all objects in cache that have a URL match
20
21. Directors
• Directors allow you to talk to the backend servers
• Directors are a glorified reverse proxy
Allows for certain types of load balancing
Allows for talking to a cluster
“A director is a logical group of backend servers
clustered together for redundancy. The basic role of
the director is to let Varnish choose a backend server
amongst several so if one is down another can be
used.”
21
22. Directors – The Types
• Random Director – picks a backend by random number
• Client Director – picks a backend by client identity
• Hash Director – picks a backend by URL hash value
• Round-Robin Director – picks a backend in order
• DNS Director – picks a backend by means of DNS
Random OR Round-Robin
• Fallback – picks the first “healthy” backend
22
23. Director - Probing
• To ensure healthy backends, you need to use probing.
It really sounds like a colonoscopy for servers.
• Variables
.url
.request
.window
.threshold
.intial
.expected_response
.interval
.timeout
23
26. ESI – Edge Side Includes
• ESI is a small markup language much like SSI (server side
includes) to include fragments (or dynamic content for that
matter).
• If don’t think you can use varnish because you have say,
user information displayed on every page; think again!
• Think of it as replacing regions inside of a page as if you
were using XHR (AJAX).
• Three Statements are Implemented
esi:include – Include a page
esi:remove – Remove content
<!-- esi --> - ESI disabled, execute normally
26
28. Using ESI
• In vcl_fetch, you must set ESI to be on
set beresp.do_esi = true;
By default, ESI will still cache, so add an exclusion if you need it
• if (req.url == “/show_username.php”) {
return (pass);
}
Varnish refuses to parse content for ESI if it does not look like XML
• This is by default; so check varnishstat and varnishlog
28
29. ESI – By Example
<html>
<head><title>Rock it with ESI</title></head>
<body>
<header>
<esi:include src="/user_header.php" />
<!-- Don't do this as you'd lose the advantage of varnish -->
<!--esi
<?php include 'user_header.php'; ?>
-->
</header>
<section id="main"></section
<footer></footer>
</body>
</html>
29
30. Purging – Beer anyone?
• Sometimes, you just need to purge.
Don’t drink too much tonight, ok?
• If you do… find a DD or a cab.
• The various ways of purging
varnishadm – command line utility
• It’s the ole finger in the back of the throat
Sockets (port 6082) – everyone likes a good socket wrench
• Sure, Ipecac is likely overkill.
HTTP – now that is the sexiness
• See, now we’re not forcing the point!
30
32. Sexy Purging
• Distributed Purging… now that’s a punch line!
Use a message queue (or gearman job server)
Have a worker that knows about the varnish servers
Submit the request to clear the cache in the asynchronously or
synchronously depending on your use case.
• Have enough workers to make this effective at purging the cache
quickly.
32
33. Embedding C in VCL – you must be crazy
• Before getting into VMOD; did you know you can embed C
into the VCL for varnish?
• Want to do something crazy fast or leverage a C library for
pre or post processing?
• I know… you’re thinking that’s useless..
On to the example; and a good one from the Varnish WIKI!
33
34. VCL - Embedded C for syslog – uber sexy
C{
#include <syslog.h>
}C
sub vcl_something {
C{
syslog(LOG_INFO, "Something happened at VCL line XX.");
}C
}
# Example with using varnish variables
C{
syslog(LOG_ERR, "Spurious response from backend: xid %s request %s %s
"%s" %d "%s" "%s"", VRT_r_req_xid(sp), VRT_r_req_request(sp),
VRT_GetHdr(sp, HDR_REQ, "005host:"), VRT_r_req_url(sp),
VRT_r_obj_status(sp), VRT_r_obj_response(sp), VRT_GetHdr(sp, HDR_OBJ,
"011Location:"));
}C
34
35. VMOD – Varnish Modules / Extensions
• Taking VCL embedded C to the next level
• Allows you to extend varnish and create new functions
• Now, if you are writing modules for varnish you have a
specialty use case!
Go read up on it!
https://www.varnish-cache.org/docs/trunk/reference/vmod.html
35
36. VMOD - std
• The VMOD std is shipped with varnish; it provides some
useful commands
toupper syslog
tolower fileread
set_up_tos duration
Random integer
Log collect
36
38. What is Varnish doing…
• What is varnish doing right now?
• How do I debug what is happening?
varnishtop
38
39. Logging
• Many times people want to log the requests to a file
By default Varnish only stores these in shared memory.
Apache Style Logs
• varnishncsa –D –a –w log.txt
39
40. Cache Warmup
• Need to warm up your cache before putting a sever in the
queue or load test an environment?
varnishreplay –r log.txt
• Replaying logs can allow you to do this. This is great for
when you are going to be deploying code to check for
performance issues.
40
41. Cache Hit Ratios? No Problem
• How to see your cache hit ratios…
varnishstat
• Want to parse them from XML?
varnishstat –x
41
42. Questions?
These slides will be posted to SlideShare & SpeakerDeck.
Slideshare: http://www.slideshare.net/mwillbanks
SpeakerDeck: http://speakerdeck.com/u/mwillbanks
Twitter: mwillbanks
G+: Mike Willbanks
IRC (freenode): mwillbanks
Blog: http://blog.digitalstruct.com
GitHub: https://github.com/mwillbanks