Pinterest - working with Varnish at scale
•
2 likes•161 views
This document discusses Pinterest's use of Varnish as a HTTP accelerator and cache in front of their backend servers. It covers upgrading from Varnish 3 to Varnish 4, scaling Varnish to handle high user loads, service discovery and configuration, monitoring and repairing Varnish, and TCP kernel tuning optimizations. The document also highlights some new features in Varnish 4 and changes to the VCL syntax.
Report
Share
Report
Share
Download to read offline
Recommended
RedisConf17 - Operationalizing Redis at Scale
This document discusses how Square operationalized Redis at scale. It describes their goals of standard setup and deployment, security through SSL/TLS, high availability, and monitoring. It then details how they used LXC containers, ghostunnel for SSL/TLS, disabled Redis persistence, relied on replication for durability, and implemented access controls. Looking forward, it discusses automating operations with SpinCycle and hopes for native Redis SSL/TLS and internal patches from Square.
RedisConf17 - Smartwaiver - Using Redis for Kiosk Registration Command and Co...
Smartwaiver uses Redis for kiosk registration and command/control features in their digital waiver application. Kiosks register with Redis to link to user accounts. Commands stored in Redis, like prepopulating waivers, allow remote control of kiosks. Over 2000 of Smartwaiver's 4000 kiosks have updated to use Redis for registration and responding to commands, improving the waiver signing process.
Better performances with HTTP/2
The HTTP/2 protocol is the latest evolution of the HTTP protocol addressing the issue of HTTP/TCP impedance mismatch. Web applications have been working around this problem for years employing techniques like concatenation or css spriting to reduce page load time and improve user experience. HTTP/2 is also a game changer on the server enabling increased concurrency. This talk will focus on the impact HTTP/2 will have on the server and examine how particularly well adapted the Vert.x concurrency model is to serve HTTP/2 applications.
A Kong retrospective: from 0.10 to 0.13
A short introductory talk given as part of the April 2018 Kong meetup "Introducing Kubernetes Ingress Controller for Kong".
This talk covers the new features and improvements made to Kong from 2017 to 2018, including the groundwork conducted by Kong Inc. and open source contributors that allowed for the development of the Kong Ingress Controller for Kubernetes.
The Kong Ingress Controller for Kubernetes was then announced during the meetup:
https://github.com/Kong/kubernetes-ingress-controller
RedisConf17 - Explosion of Data at the Edge in Equinix
This document discusses Equinix's plans to implement a microservices architecture and edge caching capabilities for their DCIM platform. The new architecture aims to address challenges from Equinix's large, global, and heterogeneous infrastructure by breaking the application into independent, containerized microservices. This will improve scalability, performance, fault tolerance, and deployments. The architecture will leverage edge processing by caching data and services at local data centers to reduce latency and enable offline functionality.
Portworx + Rancher Webinar
This document discusses Portworx, a container data services platform that provides data protection, access, performance and security for stateful containers. It can be deployed as a container and fingerprints servers and storage. It pools capacity and provides data services across environments. The document promotes using Portworx Developer for prototyping and evaluating Portworx Enterprise for production with its advanced features and support. It also discusses why Rancher is a good choice to launch containers and infrastructure services quickly on various clouds.
Citrix xen server
This document provides an overview of Citrix XenServer, a virtualization platform that allows multiple virtual machines to run on a single physical server. It discusses XenServer architecture, installation, configuration, networking, storage, virtual machine management, high availability, monitoring, troubleshooting and more. The document is a technical guide that details how to install, use and manage all aspects of the XenServer virtualization platform.
RedisConf18 - Redis Enterprise on Cloud Native Platforms
This document provides an introduction to cloud-native platforms and Kubernetes, and demonstrates how Redis Enterprise can run on these platforms. It discusses how Kubernetes provides orchestration of containers and manages the application lifecycle. It then demonstrates deploying Redis Enterprise on Kubernetes, showing how it uses a custom Kubernetes controller and operator to provide auto-bootstrapping of Redis clusters within Kubernetes pods. The demo shows creating a Redis database, service discovery, and benchmarking tool deployment on the Kubernetes-hosted Redis Enterprise clusters.
Recommended
RedisConf17 - Operationalizing Redis at Scale
This document discusses how Square operationalized Redis at scale. It describes their goals of standard setup and deployment, security through SSL/TLS, high availability, and monitoring. It then details how they used LXC containers, ghostunnel for SSL/TLS, disabled Redis persistence, relied on replication for durability, and implemented access controls. Looking forward, it discusses automating operations with SpinCycle and hopes for native Redis SSL/TLS and internal patches from Square.
RedisConf17 - Smartwaiver - Using Redis for Kiosk Registration Command and Co...
Smartwaiver uses Redis for kiosk registration and command/control features in their digital waiver application. Kiosks register with Redis to link to user accounts. Commands stored in Redis, like prepopulating waivers, allow remote control of kiosks. Over 2000 of Smartwaiver's 4000 kiosks have updated to use Redis for registration and responding to commands, improving the waiver signing process.
Better performances with HTTP/2
The HTTP/2 protocol is the latest evolution of the HTTP protocol addressing the issue of HTTP/TCP impedance mismatch. Web applications have been working around this problem for years employing techniques like concatenation or css spriting to reduce page load time and improve user experience. HTTP/2 is also a game changer on the server enabling increased concurrency. This talk will focus on the impact HTTP/2 will have on the server and examine how particularly well adapted the Vert.x concurrency model is to serve HTTP/2 applications.
A Kong retrospective: from 0.10 to 0.13
A short introductory talk given as part of the April 2018 Kong meetup "Introducing Kubernetes Ingress Controller for Kong".
This talk covers the new features and improvements made to Kong from 2017 to 2018, including the groundwork conducted by Kong Inc. and open source contributors that allowed for the development of the Kong Ingress Controller for Kubernetes.
The Kong Ingress Controller for Kubernetes was then announced during the meetup:
https://github.com/Kong/kubernetes-ingress-controller
RedisConf17 - Explosion of Data at the Edge in Equinix
This document discusses Equinix's plans to implement a microservices architecture and edge caching capabilities for their DCIM platform. The new architecture aims to address challenges from Equinix's large, global, and heterogeneous infrastructure by breaking the application into independent, containerized microservices. This will improve scalability, performance, fault tolerance, and deployments. The architecture will leverage edge processing by caching data and services at local data centers to reduce latency and enable offline functionality.
Portworx + Rancher Webinar
This document discusses Portworx, a container data services platform that provides data protection, access, performance and security for stateful containers. It can be deployed as a container and fingerprints servers and storage. It pools capacity and provides data services across environments. The document promotes using Portworx Developer for prototyping and evaluating Portworx Enterprise for production with its advanced features and support. It also discusses why Rancher is a good choice to launch containers and infrastructure services quickly on various clouds.
Citrix xen server
This document provides an overview of Citrix XenServer, a virtualization platform that allows multiple virtual machines to run on a single physical server. It discusses XenServer architecture, installation, configuration, networking, storage, virtual machine management, high availability, monitoring, troubleshooting and more. The document is a technical guide that details how to install, use and manage all aspects of the XenServer virtualization platform.
RedisConf18 - Redis Enterprise on Cloud Native Platforms
This document provides an introduction to cloud-native platforms and Kubernetes, and demonstrates how Redis Enterprise can run on these platforms. It discusses how Kubernetes provides orchestration of containers and manages the application lifecycle. It then demonstrates deploying Redis Enterprise on Kubernetes, showing how it uses a custom Kubernetes controller and operator to provide auto-bootstrapping of Redis clusters within Kubernetes pods. The demo shows creating a Redis database, service discovery, and benchmarking tool deployment on the Kubernetes-hosted Redis Enterprise clusters.
DalmatinerDB and cockroachDB monitoring plataform
Scalable monitoring solution based on dalmatinerDB and CockroachDB. Multi-protocols metrics are supported.
Group18_Awesome4some:Proxy server.ppt
This document discusses various types of proxy servers including anonymous proxy servers, HTTP proxy servers, FTP proxy servers, caching proxy servers, content filtering proxy servers, reverse proxy servers, and ICAP servers. It provides details on their functions and purposes. Team members on the Awesum4sum project are listed and an index outlines topics covered which include introduction to proxy servers, advantages and disadvantages, and a demonstration on CC Proxy.
Varnish extend
This document discusses Varnish Extend, which provides tools and services to flexibly deploy Varnish caching and other content delivery technologies. It can be used to build hybrid CDNs, leverage existing data centers for delivery, or create a fully custom DIY CDN. The key components are Varnish Plus for caching, Cedexis Radar for intelligent routing, and Cedexis Purge for unified purging across delivery architectures. An example implementation is shown with Varnish deployed at edge POPs and an origin server, using Varnish High Availability (VHA) for replication between nodes. Configuration details are provided for VCS statistics, Cedexis Radar and Openmix load balancing, and VHA. The document demonstrates how V
Service Discovery 101
This document summarizes a presentation on service discovery for microservice architectures. It outlines the need for service discovery due to services dynamically scaling and failing. It discusses common discovery strategies like hardcoded IPs, DNS, and DNS with load balancers that don't scale well. The recommended approach is to use a service registry like Consul, which allows for automated service discovery and failure detection in a distributed system. The presentation concludes with a demo of Consul's service discovery capabilities.
Content Delivery Network
A content delivery network (CDN) distributes content across multiple servers located around the world. This allows content to be served from servers closest to users, reducing latency and improving page load times. CDNs can increase website performance and reduce load on origin servers. They provide redundancy so if one server fails, users are redirected to another. Popular CDN providers include Akamai, Cloudflare, and Amazon Cloudfront.
Flight recordings and mission control through thick clouds
Java mission control and flight recorder provide monitoring and offline analysis capabilities for production Java systems. They can record system operations without requiring additional deployment. Jolokia provides a JSON over HTTP bridge for accessing JMX metrics from Kubernetes pods, bypassing firewall and connectivity limitations. It can be accessed via the Kubernetes API proxy or with tools like hawt.io.
Using Microservices Architecture and Patterns to Address Applications Require...
Using Microservices Architecture and Patterns to Address Applications Require...Prem Sankar Gopannan
Gives an overview about MEC application requirements and microservice patters. Provides a sample app and application of patterns to better design the MEC app. Provides an overview about state management of MEC appNetwork Service Mesh
This presentation provides an overview of Network Service Mesh, its relevance to EDGE/5G and usecases
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. It is used at Netflix to power various services by sitting on top of Redis and providing high availability, scalability, and tunable consistency. Conductor is a workflow orchestration engine used by Netflix that stores workflow definitions and state in Dynomite to allow reusable and controllable workflow processes.
Webinar Slides: Real time Recommendations with Redis, Java and Websockets
WebSockets connect the browser to your app server. But what if the processing happens on some other server? In that case you need to connect the worker process to the app process via a messaging system. After experimenting with RabbitMQ, we settled on Redis as a great pub sub and a caching system.
In this webinar you will learn:
* How to use Java, Spring, and Redis ( spring-websockets and spring-data-redis) to power a real time recommendations system
* How many users are currently using your system in real time
Orchestrating Shared Networks, Physical LB and DNS on Cloudstack
The adoption of Cloud Computing Solutions in datacenters worldwide has proven to be the right path, as it reduces costs, automates operation tasks and improve time-to-market for newly developed applications. However, cloud solutions are still in an early stage and do not meet all requirements for company environments, especially those with high capacity
With that in mind, Globo.com embraced an open cloud solutions, Cloudstack, and began developing together with the community to fulfill all its high availability, high capacity and high throughput requirements. Integration between Cloudstack and Globo.com’s infrastructure happens in all levels, with an special attention to network equipment management, DNS automation and load balancing.
HAProxyConf 2019: Building a Service Mesh at Criteo with Consul and HAProxy
At Criteo, we have been working on building a tight integration between HashiCorp Consul and HAProxy. In this talk, we will explain how we provision our HAProxy instances dynamically using Consul Connect, a new service mesh technology that allows HAProxy to talk to its peers from machine-to-machine without a traditional load balancer.
We explain the history of Discovery at Criteo and why we developed such service-mesh.
We also explain how the OpenSource HaProxy-Connect works internally and what features from HaProxy it uses (SPOE).
https://github.com/criteo/haproxy-consul-connect
https://www.haproxyconf.com/agenda/building-a-service-mesh-at-criteo-with-consul-and-haproxy/
MySQL on Docker - Containerizing the Dolphin
Docker is becoming more mainstream and adopted by users as a method to package and deploy self-sufficient applications in primarily stateless Linux containers. It's a great toolset on top of OS-level virtualization (LXC, a.k.a containers) and plays well in the world of micro services.
However, Docker containers are transient by default. If a container is destroyed, all data created is also lost. For a stateful service like a database, this is a major headache to say the least.
There are a number ways to provide persistent storage in Docker containers. In this presentation, we will talk about how to setup a persistence data service with Docker that can be torn down and brought up across hosts and containers.
We will touch upon orchestration tools, shared volumes, data-only-containers, security and configuration management, multi-host networking, service discovery and implications on monitoring when we move from host-centric to role-centric services with shorter life cycles.
HA System-First presentation
High-availability web server clusters use redundancy to ensure continuous service if any single component fails. They distribute workload across multiple servers for improved efficiency and scalability. The document proposes using Linux Virtual Server to implement such a cluster, with virtual servers distributing requests to real servers via network address translation, direct routing, or IP tunneling. Hardware requirements include servers, switches, and cabling to physically set up the redundant infrastructure.
Screen Shots of Building Linux IPv6 DNS Server
The document contains screenshots from building a Linux IPv6 DNS server. It shows the process of configuring the kernel for IPv6, adding IPv6 and IPv4 clients to the server, customizing the DNS configuration files, and activating the DNS server. The screenshots demonstrate registering domains, adding DNS records, and testing lookups from clients.
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControl
In any busy operations environment, there are countless tasks to perform - some monthly, or weekly, some daily or more frequently, and some on an ad-hoc basis. And automation is key to performing fast, efficient and consistently repeatable software deployments and recovery.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world. However, there are a small number of specialist domain-specific automation tools available also, and we are going to compare two of these products: MongoDB?s own Ops Manager, and ClusterControl from Severalnines.
We will cover:
* Installation and maintenance
* Complexity of architecture
* Options for redundancy
* Comparative functionality
* Monitoring, Dashboard, Alerting
* Backing up and restoring
* Automated deployment of advanced configurations
* Upgrading existing deployments
Participants should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
The document discusses various load balancing options for MySQL databases, including application connectors, reverse proxies, and SQL-aware proxies. It provides an overview of popular load balancers like HAProxy, MaxScale, ProxySQL, MySQL Router, and Nginx. These load balancers aim to improve high availability, performance, and scalability. They handle tasks like health checking, failover, read/write splitting, and query routing. The document analyzes the key features and limitations of each type of load balancer.
Another Day, Another Billion Packets
In this session, we walk through the Amazon VPC network presentation and describe the problems we were trying to solve when we created it. Next, we walk through how these problems are traditionally solved, and why those solutions are not scalable, inexpensive, or secure enough for AWS. Finally, we provide an overview of the solution that we've implemented and discuss some of the unique mechanisms that we use to ensure customer isolation, get packets into and out of the network, and support new features like VPC endpoints.
20190727 HashiCorp Consul Workshop: 管管你們家 config 啦
Hashicorp Consul 提供了相當豐富的 Service Mesh 功能，能夠替分散式服務快速地做服務發現、服務動態劃分和服務設定，Consul 也可以支援多種 runtime 平台，也和許多工具或雲服務商做很好的 Cloud-Native 整合。此 Workshop 主要分為幾個主軸:
- Why Configuration Management?
- Consul 基本使用操作: KV Store, Service Registering and Building a Consul Cluster
- 佈署 Consul 到 Production 環境前所需注意事項
2019 05-28 SRE Consul Criteo Meetup
This presentation explains the challenges we face at Criteo on discovering machines and services.
Criteo is using HashiCorp's Consul to discover services. We explain what is Consul, how it works, what are the challenges we faced and how we improved it.
We also explain how using it in combinaison with consul-templaterb can allow us performing Inversion Of Control for the whole infrastructure allowing us to use it as a database and iterating faster.
Araya
Este documento discute la morbilidad y mortalidad asociada con la diarrea aguda en niños en países en desarrollo. Reporta que en 1977-1978 hubo 5 millones de muertes anuales relacionadas con diarrea, disminuyendo a 3.5 millones en 1990 y 2.2 millones en 2000. Describe tasas altas de episodios de diarrea por niño por año en varios países y que la mayoría de las muertes ocurren en niños menores de 5 años. Finalmente, analiza estudios sobre tratamiento con hidratación oral y sol
Ingeniería Industrial: recursos electrónicos recomendados
Bibliografìa entregada a los estudiantes de Seminario de Investigación I de la Facultad de Ingeniería Industrial, en el marco de los talleres de capacitación impartidos en el 2011-2012.
More Related Content
What's hot
DalmatinerDB and cockroachDB monitoring plataform
Scalable monitoring solution based on dalmatinerDB and CockroachDB. Multi-protocols metrics are supported.
Group18_Awesome4some:Proxy server.ppt
This document discusses various types of proxy servers including anonymous proxy servers, HTTP proxy servers, FTP proxy servers, caching proxy servers, content filtering proxy servers, reverse proxy servers, and ICAP servers. It provides details on their functions and purposes. Team members on the Awesum4sum project are listed and an index outlines topics covered which include introduction to proxy servers, advantages and disadvantages, and a demonstration on CC Proxy.
Varnish extend
This document discusses Varnish Extend, which provides tools and services to flexibly deploy Varnish caching and other content delivery technologies. It can be used to build hybrid CDNs, leverage existing data centers for delivery, or create a fully custom DIY CDN. The key components are Varnish Plus for caching, Cedexis Radar for intelligent routing, and Cedexis Purge for unified purging across delivery architectures. An example implementation is shown with Varnish deployed at edge POPs and an origin server, using Varnish High Availability (VHA) for replication between nodes. Configuration details are provided for VCS statistics, Cedexis Radar and Openmix load balancing, and VHA. The document demonstrates how V
Service Discovery 101
This document summarizes a presentation on service discovery for microservice architectures. It outlines the need for service discovery due to services dynamically scaling and failing. It discusses common discovery strategies like hardcoded IPs, DNS, and DNS with load balancers that don't scale well. The recommended approach is to use a service registry like Consul, which allows for automated service discovery and failure detection in a distributed system. The presentation concludes with a demo of Consul's service discovery capabilities.
Content Delivery Network
A content delivery network (CDN) distributes content across multiple servers located around the world. This allows content to be served from servers closest to users, reducing latency and improving page load times. CDNs can increase website performance and reduce load on origin servers. They provide redundancy so if one server fails, users are redirected to another. Popular CDN providers include Akamai, Cloudflare, and Amazon Cloudfront.
Flight recordings and mission control through thick clouds
Java mission control and flight recorder provide monitoring and offline analysis capabilities for production Java systems. They can record system operations without requiring additional deployment. Jolokia provides a JSON over HTTP bridge for accessing JMX metrics from Kubernetes pods, bypassing firewall and connectivity limitations. It can be accessed via the Kubernetes API proxy or with tools like hawt.io.
Using Microservices Architecture and Patterns to Address Applications Require...
Using Microservices Architecture and Patterns to Address Applications Require...Prem Sankar Gopannan
Gives an overview about MEC application requirements and microservice patters. Provides a sample app and application of patterns to better design the MEC app. Provides an overview about state management of MEC appNetwork Service Mesh
This presentation provides an overview of Network Service Mesh, its relevance to EDGE/5G and usecases
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Dynomite is a framework that makes non-distributed databases distributed by adding a proxy layer, auto-sharding, replication across datacenters, and more. It is used at Netflix to power various services by sitting on top of Redis and providing high availability, scalability, and tunable consistency. Conductor is a workflow orchestration engine used by Netflix that stores workflow definitions and state in Dynomite to allow reusable and controllable workflow processes.
Webinar Slides: Real time Recommendations with Redis, Java and Websockets
WebSockets connect the browser to your app server. But what if the processing happens on some other server? In that case you need to connect the worker process to the app process via a messaging system. After experimenting with RabbitMQ, we settled on Redis as a great pub sub and a caching system.
In this webinar you will learn:
* How to use Java, Spring, and Redis ( spring-websockets and spring-data-redis) to power a real time recommendations system
* How many users are currently using your system in real time
Orchestrating Shared Networks, Physical LB and DNS on Cloudstack
The adoption of Cloud Computing Solutions in datacenters worldwide has proven to be the right path, as it reduces costs, automates operation tasks and improve time-to-market for newly developed applications. However, cloud solutions are still in an early stage and do not meet all requirements for company environments, especially those with high capacity
With that in mind, Globo.com embraced an open cloud solutions, Cloudstack, and began developing together with the community to fulfill all its high availability, high capacity and high throughput requirements. Integration between Cloudstack and Globo.com’s infrastructure happens in all levels, with an special attention to network equipment management, DNS automation and load balancing.
HAProxyConf 2019: Building a Service Mesh at Criteo with Consul and HAProxy
At Criteo, we have been working on building a tight integration between HashiCorp Consul and HAProxy. In this talk, we will explain how we provision our HAProxy instances dynamically using Consul Connect, a new service mesh technology that allows HAProxy to talk to its peers from machine-to-machine without a traditional load balancer.
We explain the history of Discovery at Criteo and why we developed such service-mesh.
We also explain how the OpenSource HaProxy-Connect works internally and what features from HaProxy it uses (SPOE).
https://github.com/criteo/haproxy-consul-connect
https://www.haproxyconf.com/agenda/building-a-service-mesh-at-criteo-with-consul-and-haproxy/
MySQL on Docker - Containerizing the Dolphin
Docker is becoming more mainstream and adopted by users as a method to package and deploy self-sufficient applications in primarily stateless Linux containers. It's a great toolset on top of OS-level virtualization (LXC, a.k.a containers) and plays well in the world of micro services.
However, Docker containers are transient by default. If a container is destroyed, all data created is also lost. For a stateful service like a database, this is a major headache to say the least.
There are a number ways to provide persistent storage in Docker containers. In this presentation, we will talk about how to setup a persistence data service with Docker that can be torn down and brought up across hosts and containers.
We will touch upon orchestration tools, shared volumes, data-only-containers, security and configuration management, multi-host networking, service discovery and implications on monitoring when we move from host-centric to role-centric services with shorter life cycles.
HA System-First presentation
High-availability web server clusters use redundancy to ensure continuous service if any single component fails. They distribute workload across multiple servers for improved efficiency and scalability. The document proposes using Linux Virtual Server to implement such a cluster, with virtual servers distributing requests to real servers via network address translation, direct routing, or IP tunneling. Hardware requirements include servers, switches, and cabling to physically set up the redundant infrastructure.
Screen Shots of Building Linux IPv6 DNS Server
The document contains screenshots from building a Linux IPv6 DNS server. It shows the process of configuring the kernel for IPv6, adding IPv6 and IPv4 clients to the server, customizing the DNS configuration files, and activating the DNS server. The screenshots demonstrate registering domains, adding DNS records, and testing lookups from clients.
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControl
In any busy operations environment, there are countless tasks to perform - some monthly, or weekly, some daily or more frequently, and some on an ad-hoc basis. And automation is key to performing fast, efficient and consistently repeatable software deployments and recovery.
There are many generic tools available, both commercial and open source, to aid with the automation of operational tasks. Some of these tools are even deployed in the database world. However, there are a small number of specialist domain-specific automation tools available also, and we are going to compare two of these products: MongoDB?s own Ops Manager, and ClusterControl from Severalnines.
We will cover:
* Installation and maintenance
* Complexity of architecture
* Options for redundancy
* Comparative functionality
* Monitoring, Dashboard, Alerting
* Backing up and restoring
* Automated deployment of advanced configurations
* Upgrading existing deployments
Participants should take away a clear understanding of the differences between these tools, and how they help automate and manage MongoDB operations.
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
The document discusses various load balancing options for MySQL databases, including application connectors, reverse proxies, and SQL-aware proxies. It provides an overview of popular load balancers like HAProxy, MaxScale, ProxySQL, MySQL Router, and Nginx. These load balancers aim to improve high availability, performance, and scalability. They handle tasks like health checking, failover, read/write splitting, and query routing. The document analyzes the key features and limitations of each type of load balancer.
Another Day, Another Billion Packets
In this session, we walk through the Amazon VPC network presentation and describe the problems we were trying to solve when we created it. Next, we walk through how these problems are traditionally solved, and why those solutions are not scalable, inexpensive, or secure enough for AWS. Finally, we provide an overview of the solution that we've implemented and discuss some of the unique mechanisms that we use to ensure customer isolation, get packets into and out of the network, and support new features like VPC endpoints.
20190727 HashiCorp Consul Workshop: 管管你們家 config 啦
Hashicorp Consul 提供了相當豐富的 Service Mesh 功能，能夠替分散式服務快速地做服務發現、服務動態劃分和服務設定，Consul 也可以支援多種 runtime 平台，也和許多工具或雲服務商做很好的 Cloud-Native 整合。此 Workshop 主要分為幾個主軸:
- Why Configuration Management?
- Consul 基本使用操作: KV Store, Service Registering and Building a Consul Cluster
- 佈署 Consul 到 Production 環境前所需注意事項
2019 05-28 SRE Consul Criteo Meetup
This presentation explains the challenges we face at Criteo on discovering machines and services.
Criteo is using HashiCorp's Consul to discover services. We explain what is Consul, how it works, what are the challenges we faced and how we improved it.
We also explain how using it in combinaison with consul-templaterb can allow us performing Inversion Of Control for the whole infrastructure allowing us to use it as a database and iterating faster.
What's hot (20)
Flight recordings and mission control through thick clouds
Flight recordings and mission control through thick clouds
Using Microservices Architecture and Patterns to Address Applications Require...
Using Microservices Architecture and Patterns to Address Applications Require...
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
RedisConf17 - Dynomite - Making Non-distributed Databases Distributed
Webinar Slides: Real time Recommendations with Redis, Java and Websockets
Webinar Slides: Real time Recommendations with Redis, Java and Websockets
Orchestrating Shared Networks, Physical LB and DNS on Cloudstack
Orchestrating Shared Networks, Physical LB and DNS on Cloudstack
HAProxyConf 2019: Building a Service Mesh at Criteo with Consul and HAProxy
HAProxyConf 2019: Building a Service Mesh at Criteo with Consul and HAProxy
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControl
Automating and Managing MongoDB: An Analysis of Ops Manager vs. ClusterControl
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
MySQL Load Balancers - MaxScale, ProxySQL, HAProxy, MySQL Router & nginx - A ...
20190727 HashiCorp Consul Workshop: 管管你們家 config 啦
20190727 HashiCorp Consul Workshop: 管管你們家 config 啦
Viewers also liked
Araya
Este documento discute la morbilidad y mortalidad asociada con la diarrea aguda en niños en países en desarrollo. Reporta que en 1977-1978 hubo 5 millones de muertes anuales relacionadas con diarrea, disminuyendo a 3.5 millones en 1990 y 2.2 millones en 2000. Describe tasas altas de episodios de diarrea por niño por año en varios países y que la mayoría de las muertes ocurren en niños menores de 5 años. Finalmente, analiza estudios sobre tratamiento con hidratación oral y sol
Ingeniería Industrial: recursos electrónicos recomendados
Bibliografìa entregada a los estudiantes de Seminario de Investigación I de la Facultad de Ingeniería Industrial, en el marco de los talleres de capacitación impartidos en el 2011-2012.
Carlos gamero esparza la paz es posible, pero - reportaje publicado en viva...
Carlos gamero esparza la paz es posible, pero - reportaje publicado en viva...Carlos Andrés Leondeurgel
Este documento resume un artículo crítico sobre Prem Rawat y su organización Elan Vital (antes conocida como Misión de la Luz Divina). El autor expresa sus dudas sobre un programa de televisión que promueve a Elan Vital, dado su propia y dolorosa experiencia como ex seguidor de 8 años. Describe cómo la organización se convirtió en un reflejo de las acciones cuestionables de su líder y cómo finalmente se alejó al enterarse de varios escándalos y actos de corrupción.Agrosierra
La empresa Agrosierra produce tres tipos de cacao y le envía una muestra y catálogo a Jennifer Díaz de Café y Chocolate S.A. para negociar una posible venta. Más tarde, Agrosierra adjunta el catálogo de productos y ofrece entregar un pedido en 3 días con 10% de descuento y un plazo de pago de 30-60 días. Finalmente, Agrosierra solicita dos referencias crediticias de Café y Chocolate S.A. para estudiar la aprobación de un crédito.
London Dine & Wine- A Bloomberg Brief Special Supplement
Discover the capital's secrets in Bloomberg Brief's special supplement London Dine & Wine. Inside you will find London's 10 most important restaurants for visitors, sommelier tips for picking a good wine, and much more.
To learn more about the Bloomberg Brief Newsletters and Supplements please visit:
http://www.bloombergbriefs.com/
Diari del 28 d'octubre de 2013
Aquí us podeu descarregar el diari d'avui. Que tingueu un Bondia!!! :)
Nervis villalobos
El documento es una queja extensa de una mujer sobre los derechos de las mujeres y la liberación femenina. Ella lamenta que las mujeres ya no puedan pasar los días tejiendo y cuidando el hogar, y en su lugar ahora deben trabajar, mantenerse delgadas y bellas, y competir con los hombres. Ella desea volver a los días en que las mujeres tenían el dominio en el hogar y dependían de los hombres.
Impactos de la minería javier lillo
El documento describe los principales impactos de la minería en el medio natural, incluyendo la contaminación atmosférica por emisiones de polvo y gases, la modificación y contaminación de suelos y el terreno, y las alteraciones a las aguas superficiales y subterráneas debido a cambios en la dinámica fluvial, pérdida de masas de agua, y contaminación por metales pesados y acidificación como resultado del drenaje ácido de mina.
Simone Bonavita - Il diritto all'oblio e gli strumenti di tutela
Slide relative al diritto all'oblio. Aspetti definitori e considerazioni anche alla luce nella nuova normativa europea. Interessante la parte iniziale sulle reti cognitive/non cognitive, molto spesso mai presa in considerazione nell'affrontare l'argomento.
Office Temping Guide
A full guide on everything you need to know when becoming an Office Angels temp.
All the basics from pay slips to employment benefits.
Curso Pnl
Este documento descreve um curso de 16 horas sobre Programação Neuro-Linguística (PNL). O curso ensina ferramentas da PNL para ajudar os alunos a alcançar objetivos de forma mais fácil, compreender e usar a comunicação de maneira eficaz, e transformar pensamentos para agir de maneira diferente e obter melhores resultados. O curso é voltado para gestores, terapeutas, professores e outros interessados em melhorar suas vidas de forma eficaz.
German vega rodgriguez
La adenomiosis es un engrosamiento del útero causado por el crecimiento anormal de tejido endometrial en las paredes musculares del útero. Puede causar sangrado menstrual abundante, dolor pélvico y dismenorrea. El diagnóstico definitivo requiere histerectomía, pero puede sospecharse mediante ultrasonido o resonancia magnética que muestren un engrosamiento del miometrio. No tiene cura y el tratamiento se enfoca en aliviar los síntomas.
The security monitoring and attack detection planning guide
This document provides guidance on implementing security monitoring and attack detection. It discusses approaches to security monitoring such as implementing security monitoring systems, correlating security audit events, and using independent software vendor solutions. It also covers issues and requirements for detecting policy violations, identifying external attacks, and implementing forensic analysis. Finally, it outlines the design of a security monitoring solution including elements, concepts, prerequisites, planning, architecture, and how the solution would work.
Apuntes De Matematicas Discretas
Este documento presenta un resumen de la introducción a un curso de Matemática Discreta. Explica brevemente que la Matemática Discreta estudia objetos discretos como los números naturales, que están separados entre sí a diferencia de los números reales que son continuos. También resume los temas que se cubrirán en el curso como lógica, algoritmos, combinatoria y grafos. Finalmente, indica que el curso está estructurado en 6 temas con introducciones, conceptos, ejemplos y ejercicios para cada uno.
Futuro Labs 2012: Reporte de competitividad de marcas en redes sociales- Febr...
Este reporte analiza la competitividad de marcas en las redes sociales Facebook, Twitter y Google+ en Perú durante febrero de 2012. En Facebook, los sectores con más "me gusta" fueron tiendas por departamentos, telecomunicaciones y cerveceras, mientras que aerolíneas tuvo la mayor interacción con usuarios. En Twitter, telecomunicaciones continuó como líder en seguidores e interacción, y bancos creció en seguidores. El crecimiento de seguidores en Google+ se desaceleró en comparación con meses anteriores.
Digital Research & Analysis
Este documento presenta información sobre diferentes temas relacionados con investigación de mercados, análisis de datos, estrategia de contenido digital y redes sociales. Incluye definiciones de términos como performance report, listening, benchmark, herramientas analíticas gratuitas y más. También describe arquetipos de personalidad que pueden usarse para definir la personalidad de una marca y ejemplos de cómo aplicar un benchmark para mejorar una estrategia.
Healthcare Career Outlooks - United States
This document provides an overview of various healthcare career outlooks and employment statistics in the United States according to sources from the Department of Labor. It discusses occupations such as nurses, nursing aides, physicians, dentists, pharmacists, emergency medical technicians, and other allied health roles. For each occupation, it provides data on employment projections, mean wages, and median compensation when available. It also briefly describes the peoplemenders.com website which provides healthcare career resources and networking opportunities.
Scarcity, trade off, opportunity cost
Scarcity means resources are limited so choices must be made about how to use natural resources, workers, and capital. A trade-off is giving up one thing to get another, like spending on education vs defense. Making choices always has an opportunity cost of forgoing other options.
Honra a tu padre y a tu madre
Este documento trata sobre el deber de honrar a los padres. Explora el significado del mandamiento bíblico de "honrar a tu padre y a tu madre", argumentando que implica más que simplemente proveer lo necesario, sino también mostrarles respeto, consideración, sumisión y cuidados con amor. Advierte que Dios castigará a aquellos que sean ingratos e irrespetuosos con sus padres.
Viewers also liked (20)
Ingeniería Industrial: recursos electrónicos recomendados
Ingeniería Industrial: recursos electrónicos recomendados
Carlos gamero esparza la paz es posible, pero - reportaje publicado en viva...
Carlos gamero esparza la paz es posible, pero - reportaje publicado en viva...
London Dine & Wine- A Bloomberg Brief Special Supplement
London Dine & Wine- A Bloomberg Brief Special Supplement
Simone Bonavita - Il diritto all'oblio e gli strumenti di tutela
Simone Bonavita - Il diritto all'oblio e gli strumenti di tutela
The security monitoring and attack detection planning guide
The security monitoring and attack detection planning guide
Futuro Labs 2012: Reporte de competitividad de marcas en redes sociales- Febr...
Futuro Labs 2012: Reporte de competitividad de marcas en redes sociales- Febr...
Similar to Pinterest - working with Varnish at scale
Pinterest - Working with Varnish at-scale
This document discusses Pinterest's use of Varnish cache to accelerate its website and APIs. It covers how Pinterest scaled Varnish to handle high user loads, implemented a resilient design using cloud components, and upgraded from Varnish 3 to Varnish 4. The document also provides information on monitoring Varnish, configuring HTTP accelerators, and upgrading related modules.
Murali updated Resume
E. Muralikumar provides his contact information and seeks a challenging career utilizing his 9+ years of experience in Linux system administration. He has expertise in Red Hat Enterprise Linux and basic knowledge of Solaris and AIX. His skills include firewalls, proxies, VPNs, load balancing, DNS, DHCP and various protocols. He is currently an associate consultant at HCL Technologies working on Sandisk projects involving Red Hat installation, configuration and migration.
Citrix Day 2014: NetScaler 10.5
Slides der Präsentation von Simeon Bosshard, Citrix, am Citrix Day 2014 von Digicomp.
Das HTML 5 GUI im Release 10.5 des NetScaler ist eine Neuerung, aber nicht die einzige. Erfahren Sie mehr über die wichtigsten Änderungen wie etwa MobileStream, Cisco RISE und ACI Integration sowie die Erweiterungen im Authentication-Bereich. Natürlich kommen die Neuerungen in den Core-Bereichen Loadbalancing, SSL Offloading etc. ebenfalls nicht zu kurz.
Xen Cloud Platform by Tim Mackey
Xen Cloud Platform (XCP) is an open-source virtualization platform based on Citrix's XenServer. It provides a complete virtualization stack including automation, resource pooling, and event management. XenServer has had significant adoption with over 1 million downloads and is used by over 50% of Fortune 500 companies. While many organizations use VMware for critical workloads, Citrix and Microsoft are often used alongside for non-critical systems and cost savings due to their lower prices.
RapidScale CloudServer
Your servers, our Cloud, the best service in the industry. RapidScale gives you on-demand capacity with the best infrastructure tools and utilities in the industry.
The goal of CloudServer is to provide a flexible, standardized and virtualized operating environment that allows applications to perform at their peak without downtime. Infrastructure as a Service (IaaS) abstracts hardware (server, storage and network infrastructure) into a pool of computing, storage and connectivity capabilities that are delivered as services for a usage-based (metered) cost. Leveraging our infrastructure allows you to operate like an enterprise-level organization, no matter what size you are.
With CloudServer, you will also receive the benefits of Disaster Recovery. There are a myriad of ways to lose equipment and data, whether it is fire, theft or malfunctioning sprinklers. RapidScale’s CloudServer houses your data in secure Tier 3, Class 1 data centers that are mirrored and securely backed up. This gives your services the ability to be quickly back on-line and running smoothly in the event of a disaster. Our infrastructure resides in multiple data centers so clients can be assured that their business critical applications and data will be fully redundant in a different data center, making it easier to resume business as usual.
CloudServer has a pay-as-you-go model which allows you to scale up or down depending on your needs. Quickly seize new opportunities and get projects up and running with the scalability of our solution. When it comes to pricey hardware, you can save a lot by using our infrastructure and buying only what you use. With IaaS, eliminate the many negative aspects of IT expansion, especially issues with in-house infrastructure, while still receiving the processing power and storage space that you need to run.
CloudServer is based off of enterprise-class infrastructure at a fraction of the costs with 24x7x365 friendly and reliable support. We care about your business’ uptime objectives and security, which is why RapidScale has worked hard to become a leader in CloudServer. With your applications built into a private cloud environment that includes encryption, firewalls and advanced around-the-clock monitoring, they will remain safe and secure. Operate worry-free, knowing that you always have RapidScale’s support.
With your infrastructure hosted off-site, you free up your IT staff to focus on value-added tasks, such as planning and development, instead of running around patching machines and fixing bugs. Reduce risk, eliminate headaches and save money with CloudServer.
Decisions behind hypervisor selection in CloudStack 4.3
As presented at the 2014 CloudStack Collaboration Conference in Denver (CCCNA14), this deck covers the matrix of functions and features within each supported hypervisor in CloudStack 4.3. This deck forms an excellent reference document for those seeking to provide multi-hypervisor support within their Apache CloudStack based cloud, and for those seeking to determine which feature elements are supported by a given hypervisor.
Resume (6)
Santhosh Kumar Routhu is a system engineer with over 4 years of experience in Linux, VMware, networking, VCS, Hadoop, SQL Server and WAS application server. He has an MCA with 70% and certifications in RHCE and MCP. His experience includes providing administration support, monitoring systems, performing maintenance, supporting applications, and issue resolution. He is proficient in technologies such as Linux, Windows, VMware, Java, networking and databases.
Iaas on xcp
The document discusses building an Infrastructure as a Service (IaaS) cloud using the Xen Cloud Platform (XCP). It describes the key components required, including hardware, networking, storage, hypervisor, service management, and user interface/API. It then outlines VMOps' cloud stack built on XCP, including its multi-site redundant deployment architecture and network and storage virtualization approaches. Finally, it notes some limitations of using XCP and opportunities for future enhancements.
Stream processing on mobile networks
In this presentation I describe the architecture of two of our Flink projects. Both developed for our customers from telco industry.
RUCUG: 6. Fabian Kienle - NetScaler and Branch Repeater for Hyper-V
Fabian Kienle из Citrix объяснит зачем нужен NetScaler и Branch Repeater для Hyper-V и как это работает
Netflix Massively Scalable, Highly Available, Immutable Infrastructure
Netflix uses a microservices architecture and immutable infrastructure approach. It loads content across multiple AWS regions for high availability and scales services dynamically. Netflix employs techniques like caching, adaptive streaming, and content delivery networks to optimize the user experience of streaming video globally to over 140 million subscribers.
What’s New in NGINX Ingress Controller for Kubernetes Release 1.5.0
On-Demand Recording:
https://www.nginx.com/resources/webinars/whats-new-nginx-ingress-controller-kubernetes-version-150/
Kubernetes is the leading orchestration platform for deploying, scaling, and managing containerized applications. Infrastructure operators constantly impose new application delivery requirements as they adopt Kubernetes for production workloads. The NGINX Ingress controller is the most popular ingress load balancer for Kubernetes, providing a complete and supported solution for delivering your containerized applications to clients.
Attend this webinar to learn about the latest developments in NGINX Ingress Controller for Kubernetes Release 1.5.0.
OSCON2014: Understanding Hypervisor Selection in Apache CloudStack
A presented at OSCON 2014, this deck covers the matrix of capabilities each supported hypervisor brings to the Apache CloudStack table when building a cloud.
Xenserver 5 Selling And Positioning
The document discusses the server virtualization market opportunity and Citrix's XenServer 5.0 product. It notes the three main players in the virtualization market are VMware, Microsoft, and Citrix. XenServer 5.0 offers virtualization for every server in the enterprise at low cost and with features including high availability, disaster recovery, and dynamic workload streaming.
Plandas-CacheCloud
This document discusses PlanDas Cache Cloud, a caching solution. It begins by covering concepts like availability, performance, reliability, and manageability as they relate to caching. It then discusses the differences between distributed and global caching approaches. The document outlines how caching can improve performance for web services and help address bottlenecks. It introduces the PlanDas Cache Cloud architecture, which uses consistent hashing for high availability. The document shows how the solution provides a global cache, multi-tenancy, and high performance. It also covers the web management interface and similarities to Redis APIs. Finally, it shares performance test results on AWS and physical machines that show throughput scaling as nodes are added.
Server Monitoring from the Cloud
Learn how to monitor your Windows/Linux server availability and performance from the cloud and gain insight into metrics that matter the most.
Backend Server Validation
This document describes an automated backend server validation solution. It aims to validate that servers provisioned in data centers match customer orders to save time and costs over manual validation. The solution structure includes a test computer communicating with the portal database/API and provisioned servers via SSH tunnels. It retrieves order info from the portal and server info from the data center to validate a match through JUnit test cases. Validation approaches differ for Linux and Windows, using Secure Shell commands or SikuliX/RDPClip respectively to retrieve server data.
Hypervisor Selection in CloudStack and OpenStack
Deploying a successful cloud is a function of the capabilities of both the virtualization layer and the cloud orchestration platform. In this deck, presented at the annual Deep Dive Day hosted by the Boston Virtualization User Group (virtg.com), I covered CloudStack 4.3 and OpenStack Havana. The deck doesn't seek to define a "best" option, but to provide the information data center architects and system administrators require regardless of preference for KVM, XenServer, vSphere or Hyper-V.
Hypervisor Selection in CloudStack and OpenStack
- The document discusses considerations for selecting a hypervisor for use in cloud platforms like CloudStack and OpenStack including defining service offerings, tenancy requirements, and virtualization infrastructure needs. It then provides information on specific hypervisor options like XenServer, vSphere, KVM, and Hyper-V comparing features relevant to CloudStack and OpenStack.
Hacking apache cloud stack
The document provides an agenda and overview of a session on hacking Apache CloudStack. The agenda includes introductions, a session on introducing CloudStack, and a hands-on session with DevCloud. The overview discusses what CloudStack is, how it works as an orchestration platform for IAAS clouds, its architecture and core components, and how users can consume and manage resources through it.
Similar to Pinterest - working with Varnish at scale (20)
Decisions behind hypervisor selection in CloudStack 4.3
Decisions behind hypervisor selection in CloudStack 4.3
RUCUG: 6. Fabian Kienle - NetScaler and Branch Repeater for Hyper-V
RUCUG: 6. Fabian Kienle - NetScaler and Branch Repeater for Hyper-V
Netflix Massively Scalable, Highly Available, Immutable Infrastructure
Netflix Massively Scalable, Highly Available, Immutable Infrastructure
What’s New in NGINX Ingress Controller for Kubernetes Release 1.5.0
What’s New in NGINX Ingress Controller for Kubernetes Release 1.5.0
OSCON2014: Understanding Hypervisor Selection in Apache CloudStack
OSCON2014: Understanding Hypervisor Selection in Apache CloudStack
More from Varnish Software
Ask Me Anything on authentication & authorisation in Varnish
This document summarizes an Ask Me Anything session on authentication and authorization with Varnish Cache. The presenter discusses how Varnish can be used as an authentication portal by validating signatures in VCL. JSON Web Tokens (JWT) can be parsed and validated in Varnish using regular expressions and digest functions. In the future, a Policy Engine module will simplify configuring JWT authentication. While Varnish only supports HTTP protocols natively, modules could enable support for LDAP or RADIUS. Rate limiting of APIs can also be configured through the upcoming Policy Engine.
Boozt.com Use Case
Watch the recorded talk here: https://www.youtube.com/watch?v=Xs4s0Vy6XbY&list=PLAygWJGZrhWBkIkl_Pu3_AZzWwtBt_uyd&index=4
Akamai connector for varnish
This document discusses using the Akamai Connector for Varnish to simplify operations between a Varnish cache and Akamai's edge caching network. It highlights common challenges with duplicating caching logic and managing lifecycle changes. The connector architecture allows caching policies to be defined in VCL and automatically propagated to Akamai. An example demo purges a file from both Varnish and Akamai caches. Attendees are encouraged to join the early access program to try the integration.
Varnish High Availability
Varnish is a proxy server that can be configured for high availability (HA) to increase its availability, cache capacity, and reduce backend load. Varnish Plus is a small external daemon that provides HA by replicating the cache across multiple Varnish servers. It uses an asynchronous replication method with a simple architecture and easy configuration. A demo showed that combining replication with sharding increased availability to 3x and cache capacity to 4x while keeping the backend load at 1x.
PostNord: Copy On Write
This document discusses using a copy-on-write cache accelerator pattern and Varnish cache to optimize a microservice that allows customers to specify a favorite delivery location. Implementing a persistent cache with Varnish Cache Plus and transaction logging would make the solution robust and scalable by handling cache updates asynchronously and repopulating the cache if needed. This caching approach is well-suited for services with an expected low cache hit rate and improves performance by consuming fewer resources than directly calling the microservice.
Streaming with Varnish
Streaming media over HTTP has become widespread due to its efficiency and ability to cache content globally using CDNs. This allows media to be streamed far and wide without buffering issues. While early solutions like Move Networks and Apple HLS paved the way starting in 2008, technologies like Microsoft Smooth Streaming, Netflix's own solution, Adobe HDS, and MPEG-DASH soon followed to make adaptive streaming over HTTP the dominant approach.
Edgestash
This document introduces Edgestash, a templating system that uses JSON data and EdgeSide Includes (ESI) to render templates on the edge cache. It discusses how Edgestash works using Varnish Cache to parse responses, index JSON, and execute templates. The document outlines the benefits of Edgestash like faster load times and less CPU usage. It also covers the syntax used in Edgestash templates and example Varnish Configuration Language code.
What is new in varnish plus
- The document appears to be a presentation about intelligent TCP acceleration and parallel ESI features in Varnish software.
- It discusses Varnish Plus for intelligent TCP acceleration across edge nodes to optimize content delivery.
- It also describes a new parallel ESI feature that allows Varnish to process multiple ESI includes at once for better performance compared to processing them one at a time.
- There is a Q&A section at the end to discuss these topics further.
Varnish Extend demo
This document summarizes a presentation about Varnish Extend and its components. Varnish Extend allows for multi-CDN, hybrid CDN, and DIY CDN configurations using Varnish Plus and Cedexis. Cedexis provides real-user monitoring, programmable load balancing, and purging capabilities. The demo architecture shows how Varnish Plus can be deployed on the edge and shield tiers for caching and load balancing traffic to origin servers.
Varnish extend introduction
Varnish Extend is a bundled software and services solution that allows users to build a content delivery network (CDN) in four hours. It uses Varnish Plus for intelligent caching, Cedexis for global load balancing and purging, and can direct global traffic to servers optimally. The solution aims to remove the need for an "all or nothing" CDN strategy by strengthening origin servers and adding secondary datacenters. It leverages CDNs only when primary servers cannot optimally serve traffic. The components used include Varnish Cache Plus, Varnish High Availability (VHA), and Cedexis Openmix for global load balancing.
Cedexis introduction
This document summarizes the services provided by Cedexis to deliver optimized user experiences on the web, mobile and video. Cedexis gathers real-time performance data from over 500 million user sessions across networks globally to monitor over 125 CDNs and cloud providers. It uses this community-based data to detect outages and issues in real-time. Cedexis then provides intelligent routing of users to the best performing platforms based on customized business criteria to ensure high availability, performance and cost efficiency across devices and locations. Major global brands trust Cedexis to optimize the delivery of their digital experiences.
Secure content caching
- Modern caching allows for security by using VCL (Varnish Configuration Language) for advanced caching logic and client identification.
- Client identification involves classifying clients based on information like cookies, local storage, or encryption/hashes of values. This allows origin servers to dynamically vary responses based on the identified client.
- Content access control lists (ACLs) can be applied to cached responses based on the identified client information, and client invalidation involves expiring client identification tokens or invalidating cached responses based on session identifiers.
Microservices
This document discusses using microservices with Varnish caching for improved performance and scalability. It describes moving from a monolithic Java application to many small, stateless services with Varnish caching all data. Key-value caching is implemented using X-Key to invalidate cache dynamically based on HTTP headers. This allows related cached objects to be automatically invalidated on a cache purge. The architecture has proven effective over 10 years in production by reducing complexity, improving debugging and scaling through statelessness and caching with Varnish and X-Key.
Varnishtest
This document summarizes David Porter's talk about leveraging the power of Varnish Configuration Language (VCL) and Varnishtest. It outlines how the company improved their practices around traffic routing, caching, and testing of VCL configurations by adopting automation tools like Puppet and Jenkins for configuration management and continuous integration, using Docker for testing environments, and implementing a software development life cycle with peer review and version control for VCL files. This allowed them to move from a problematic, manual process to one where configurations are more robust, standardized across environments, and collaboratively developed and tested.
Lightning fast with Varnish
The document provides an overview of using Varnish to accelerate web applications. Some key points:
- Varnish works by caching responses from backend servers in memory for future requests, reducing load on backends. It respects HTTP caching best practices like expiration, conditional requests, and cache variations.
- Configuration is done through Varnish Configuration Language (VCL) scripts which control caching behavior. Common tasks covered include setting cache rules, purging content, and normalizing requests.
- Techniques discussed include caching only safe requests, stripping cookies, blacklisting/whitelisting URLs, and using Edge Side Includes (ESI) to break pages into cacheable components. Guidelines are provided for optimizing caching of dynamic content.
E-commerce use case
Sheego originally built their e-commerce site on Intershop but moved to the OXID framework, which does not have built-in caching like Intershop. In 2014, Sheego started using open-source Varnish caching to improve site performance and in 2015 upgraded to Varnish Plus for professional support, monitoring tools, and high availability capabilities. This has led to advanced caching, less staff time needed for monitoring and support, and improved availability of content for their customers.
Access control
This document discusses different access control methods for purging content in Varnish, including IP-based access control, basic authentication, and cookie-based access control. It argues that cookie-based access control provides the optimal solution, describing how to generate and authenticate random cookies signed with a secret key to control access. The document then outlines how this approach is used to build a "Varnish auth tool kit" or "VARNISH PAYWALL" system for metered or subscription-based access control of content.
Varnish TLS
Hitch TLS is a small and fast TLS terminator that is bundled with Varnish Plus. It allows client-side TLS termination with Varnish Cache Plus handling encryption and decryption. TLS can also be used to encrypt connections to backends by adding ".ssl = 1" to backend definitions in Varnish. Hitch TLS supports features like OCSP stapling, PROXY protocol, and run-time reloads for updating certificates without interrupting service. Performance testing shows it can handle high throughput workloads with good scalability on commodity hardware.
MSE
This document summarizes the Massive Storage Engine 2.0, which was built to address scaling issues with file- and memory-based backends in handling workloads with gigabytes of content. It features allocation that is fragmentation-proof and can scale to over 100 terabytes, with an LFU eviction approach. The architecture uses threads for reliable allocation across multiple segments with reduced locking. It also supports an optional persistent datastore by mirroring metadata to disk in an asynchronous manner with minimal impact to performance. Evaluation showed it handles larger files well and recovers quickly from crashes by reading the stored book of metadata.
Debugging varnish
This document discusses using varnishlog to debug VCL code that is not behaving as expected. It provides examples of how to use varnishlog to investigate 500 responses, misses, timing details, and requests over 5 seconds. It also discusses timestamps in varnishlog output and how they can help debug timing issues. Examples are given for reproducing issues using varnishtest. The document encourages reaching out for support and exploring panic dumps and core files if Varnish crashes.
More from Varnish Software (20)
Ask Me Anything on authentication & authorisation in Varnish
Ask Me Anything on authentication & authorisation in Varnish
Recently uploaded
UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Presentation of the OECD Artificial Intelligence Review of Germany
Consult the full report at https://www.oecd.org/digital/oecd-artificial-intelligence-review-of-germany-609808d6-en.htm
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Infrastructure Challenges in Scaling RAG with Custom AI models
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
ここ3000字までしか入らないけどタイトルの方がたくさん文字入ると思います。
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Recently uploaded (20)
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Pinterest - working with Varnish at scale
- 1. Thriving in a cloud provider ecosystem with Varnish & upgrading from Varnish 3 to 4 Working with Varnish at-scale
- 2. Pinterest is a platform to Discover, Save, and Do the things you love 2:50 PM 100% 2
- 3. 75+ Billion Pins categorized by people into more than 1 Billion Boards 100 Million Active Users 3
- 4. 4 Varnish in Numbers 180k requests representing unique user actions
- 5. 5 Resilient Design Resilient systems, built on top of unreliable cloud components
- 6. 6 Scaling for Users Keeping backend servers up-to-date for varnish
- 7. 7 Service Discovery Serving configuration data at scale with high availability
- 8. 8 Varnish Service Unique service hooks, and inspecting the varnish daemon
- 9. 9 Reliability Monitoring, repairing and establishing a development environment for varnish
- 10. 10 Http Accelerators Working with http headers and understanding the varnish state diagram
- 11. Varnish state diagram ◊ vcl_recv vcl_hash ◊ vcl_hit / vcl_miss / vcl_pass ◊ vcl_fetch ◊ vcl_deliver
- 12. TCP kernel tuning ◊ net.core.somaxconn ◊ net.ipv4.tcp_max_syn_backlog ◊ net.core.netdev_max_backlog ◊ net.ipv4.ip_local_port_range ◊ net.ipv4.tcp_tw_reuse ◊ net.ipv4.tcp_tw_recycle net.ipv4.tcp_orphan_retries ◊ net.ipv4.tcp_slow_start_after_idle
- 13. 13 Varnish 4 Upgrades A few features to look forward to & working with varnish packages
- 14. 14 Upgrading VMODs VMODS must be compiled against the varnish version they will be run with.
- 15. 15 VCL Syntax Updates varnish3to4 direct syntax updates and the vmod_director addition
- 16. Pinterest believes having a team of dedicated, talented, authentic people working collectively to innovate and build enables us to move fast. Karan Gupta Jayme Cox Josh Enders Jeremy Carrol Doc (Rogier Mulhuijzen - Fastly)
- 17. Thanks to Varnish-Cache for providing a place for us to come together as a community at the Varnish Summit. With special thanks to Alexandra Lehner