Globo.com & Varnish

L
HTTP accelerator
          Cachos 2.0
    Luis Henrique Okama
         Tollef Fog Heen
          Mario Carvalho
What is the problem?
  GET / HTTP/1.1
    Hang on, I need to look up a few hundred things in my
    database and then do a lot of editing controlled by a
    scripting language
  HEAD / HTTP/1.1
    Hang on, I need to look up a few hundred things in my
    database and then do a lot of editing controlled by a
    scripting language, and then I will throw the result
    away.
Why are CMS-es slow?

   Complex content generation process
   Single database prevents clustering
   Expensive software ditto

This means: we need server side caching
Why are the existing solutions not good enough?

  Squid, ancient design, forward proxy
  Apache, not what it's built for, not what it's good at
  Akamai (and similar), expensive, vendor lock-in
What is Varnish?

 Dedicated HTTP accelerator
 Focus on server-side speedups
 Policy control
 High performance
 Varnish Configuration Language
 Shared memory log
High performance

 11 syscalls + 7 locks for a cache hit
 Work with the OS, not against it.
 Multi-CPU, multi-core
 64 bit
 Use advanced OS features:
    Accept filters
    madvise(MADV_RANDOM)
    kqueue, epoll
 Don't copy data if you don't have to
 Use workspaces, not malloc/free
 Compiled configuration
Policy control

  Override TTLs
  Add, remove or change headers
  Strip cookies
  Rewrite URLs
  Invalidate objects in the cache
Varnish Configuration Language

 Simple domain specific language
 Compiled via C language to binary
     Transparently!
 Dynamically loaded
 Multiple configs loaded concurrently
 Instant switch from one VCL to another.
     Can be done from VCL
VCL Example

  acl some_acl { "10.0.0.0/8" }
  if (client.ip ~ some_acl) {
      pass;
  }

  if (req.http.host ~ "foo.com$") {
      set req.http.host =
          regsub(req.http.host,
              "foo.com", "bar.org");
  }

  if (obj.valid) {
      set obj.ttl = 10m;
      deliver;
  }
Shared memory logfile

 Fast
 Custom log tailers
    varnishtop
    varnishlog
    varnishhist
    varnishncsa
    varnishstat
Globo.com cache infrastructure
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Comparative
  scalability
  setup time
  cost
  open source
  throughput
  physical space
Some data
Conclusion
  software customization
  work set
Real life performance
Questions
Contacts

  Luis H. Okama
  okama@corp.globo.com (www.globo.com)
  Mario Carvalho
  mariocar@corp.globo.com (www.globo.com)
  Tollef Fog Heen
  tfheen@redpill-linpro.com (www.varnish-cache.com)

  @ Stand Globo.com
1 of 23

Recommended

RGW Beyond Cloud: Live Video Storage with Ceph - Shengjing Zhu, Yiming Xie by
RGW Beyond Cloud: Live Video Storage with Ceph - Shengjing Zhu, Yiming XieRGW Beyond Cloud: Live Video Storage with Ceph - Shengjing Zhu, Yiming Xie
RGW Beyond Cloud: Live Video Storage with Ceph - Shengjing Zhu, Yiming XieCeph Community
118 views16 slides
Apache Traffic Server by
Apache Traffic ServerApache Traffic Server
Apache Traffic Serversupertom
20.8K views36 slides
Global deduplication for Ceph - Myoungwon Oh by
Global deduplication for Ceph - Myoungwon OhGlobal deduplication for Ceph - Myoungwon Oh
Global deduplication for Ceph - Myoungwon OhCeph Community
108 views12 slides
Velocity 2010 - ATS by
Velocity 2010 - ATSVelocity 2010 - ATS
Velocity 2010 - ATSLeif Hedstrom
2.3K views34 slides
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai by
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu ChaiRADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu ChaiCeph Community
93 views16 slides
Automatic Operation Bot for Ceph - You Ji by
Automatic Operation Bot for Ceph - You JiAutomatic Operation Bot for Ceph - You Ji
Automatic Operation Bot for Ceph - You JiCeph Community
98 views19 slides

More Related Content

What's hot

Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau by
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex LauDoing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex LauCeph Community
184 views15 slides
The Browser Does Not Protect You by
The Browser Does Not Protect YouThe Browser Does Not Protect You
The Browser Does Not Protect YouAli Huseyn Aliyev
75 views11 slides
Experiences building a distributed shared log on RADOS - Noah Watkins by
Experiences building a distributed shared log on RADOS - Noah WatkinsExperiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah WatkinsCeph Community
98 views80 slides
Troubleshooting redis by
Troubleshooting redisTroubleshooting redis
Troubleshooting redisDaeMyung Kang
2.6K views101 slides
3 Ways to Improve Performance from a Storage Perspective by
3 Ways to Improve Performance from a Storage Perspective3 Ways to Improve Performance from a Storage Perspective
3 Ways to Improve Performance from a Storage PerspectivePerforce
3.3K views22 slides
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan by
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John HaanBasic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John HaanCeph Community
155 views48 slides

What's hot(20)

Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau by Ceph Community
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex LauDoing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Doing QoS Before Ceph Cluster QoS is available - David Byte, Alex Lau
Ceph Community 184 views
Experiences building a distributed shared log on RADOS - Noah Watkins by Ceph Community
Experiences building a distributed shared log on RADOS - Noah WatkinsExperiences building a distributed shared log on RADOS - Noah Watkins
Experiences building a distributed shared log on RADOS - Noah Watkins
Ceph Community 98 views
Troubleshooting redis by DaeMyung Kang
Troubleshooting redisTroubleshooting redis
Troubleshooting redis
DaeMyung Kang2.6K views
3 Ways to Improve Performance from a Storage Perspective by Perforce
3 Ways to Improve Performance from a Storage Perspective3 Ways to Improve Performance from a Storage Perspective
3 Ways to Improve Performance from a Storage Perspective
Perforce3.3K views
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan by Ceph Community
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John HaanBasic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan
Basic and Advanced Analysis of Ceph Volume Backend Driver in Cinder - John Haan
Ceph Community 155 views
Day 2 General Session Presentations RedisConf by Redis Labs
Day 2 General Session Presentations RedisConfDay 2 General Session Presentations RedisConf
Day 2 General Session Presentations RedisConf
Redis Labs885 views
Redis Day Keynote Salvatore Sanfillipo Redis Labs by Redis Labs
Redis Day Keynote Salvatore Sanfillipo Redis LabsRedis Day Keynote Salvatore Sanfillipo Redis Labs
Redis Day Keynote Salvatore Sanfillipo Redis Labs
Redis Labs987 views
Accelerating HBase with NVMe and Bucket Cache by Nicolas Poggi
Accelerating HBase with NVMe and Bucket CacheAccelerating HBase with NVMe and Bucket Cache
Accelerating HBase with NVMe and Bucket Cache
Nicolas Poggi1.2K views
Performance & Scalability Improvements in Perforce by Perforce
Performance & Scalability Improvements in PerforcePerformance & Scalability Improvements in Perforce
Performance & Scalability Improvements in Perforce
Perforce3.2K views
RBD: What will the future bring? - Jason Dillaman by Ceph Community
RBD: What will the future bring? - Jason DillamanRBD: What will the future bring? - Jason Dillaman
RBD: What will the future bring? - Jason Dillaman
Ceph Community 290 views
Common Support Issues And How To Troubleshoot Them - Michael Hackett, Vikhyat... by Ceph Community
Common Support Issues And How To Troubleshoot Them - Michael Hackett, Vikhyat...Common Support Issues And How To Troubleshoot Them - Michael Hackett, Vikhyat...
Common Support Issues And How To Troubleshoot Them - Michael Hackett, Vikhyat...
Ceph Community 86 views
Caching with Memcached and APC by Ben Ramsey
Caching with Memcached and APCCaching with Memcached and APC
Caching with Memcached and APC
Ben Ramsey10.5K views
Tarantool: как сэкономить миллион долларов на базе данных на высоконагруженно... by Ontico
Tarantool: как сэкономить миллион долларов на базе данных на высоконагруженно...Tarantool: как сэкономить миллион долларов на базе данных на высоконагруженно...
Tarantool: как сэкономить миллион долларов на базе данных на высоконагруженно...
Ontico2.6K views
RedisConf17- Using Redis at scale @ Twitter by Redis Labs
RedisConf17- Using Redis at scale @ TwitterRedisConf17- Using Redis at scale @ Twitter
RedisConf17- Using Redis at scale @ Twitter
Redis Labs3.8K views
Oscon 2010 - ATS by Leif Hedstrom
Oscon 2010 - ATSOscon 2010 - ATS
Oscon 2010 - ATS
Leif Hedstrom131.9K views
Scaling Apache Pulsar to 10 Petabytes/Day by ScyllaDB
Scaling Apache Pulsar to 10 Petabytes/DayScaling Apache Pulsar to 10 Petabytes/Day
Scaling Apache Pulsar to 10 Petabytes/Day
ScyllaDB655 views
Perforce BTrees: The Arcane and the Profane by Perforce
Perforce BTrees: The Arcane and the ProfanePerforce BTrees: The Arcane and the Profane
Perforce BTrees: The Arcane and the Profane
Perforce795 views

Viewers also liked

Scel adc by
Scel adcScel adc
Scel adcRaul Pereira dos Santos
2K views1 slide
Empleo ENFINET 241 by
Empleo  ENFINET 241Empleo  ENFINET 241
Empleo ENFINET 241Alberto (Santander) Felpete
955 views15 slides
Abadia De Melk by
Abadia De MelkAbadia De Melk
Abadia De MelkJosé Roberto Cordeiro
535 views25 slides
Previsiones de la demanda energía by
Previsiones de la demanda energíaPrevisiones de la demanda energía
Previsiones de la demanda energíaAelasoft
354 views19 slides
Casos| 2da Sesión| Alejandra Jazmín Simental Franco by
Casos| 2da Sesión| Alejandra Jazmín Simental FrancoCasos| 2da Sesión| Alejandra Jazmín Simental Franco
Casos| 2da Sesión| Alejandra Jazmín Simental Francondimexico
1.4K views26 slides
Prezentacja korzysci Fm Mobile dla Biznesfm.pl by
Prezentacja korzysci Fm Mobile dla Biznesfm.plPrezentacja korzysci Fm Mobile dla Biznesfm.pl
Prezentacja korzysci Fm Mobile dla Biznesfm.plbiznesfm.pl
771 views60 slides

Viewers also liked(20)

Previsiones de la demanda energía by Aelasoft
Previsiones de la demanda energíaPrevisiones de la demanda energía
Previsiones de la demanda energía
Aelasoft 354 views
Casos| 2da Sesión| Alejandra Jazmín Simental Franco by ndimexico
Casos| 2da Sesión| Alejandra Jazmín Simental FrancoCasos| 2da Sesión| Alejandra Jazmín Simental Franco
Casos| 2da Sesión| Alejandra Jazmín Simental Franco
ndimexico1.4K views
Prezentacja korzysci Fm Mobile dla Biznesfm.pl by biznesfm.pl
Prezentacja korzysci Fm Mobile dla Biznesfm.plPrezentacja korzysci Fm Mobile dla Biznesfm.pl
Prezentacja korzysci Fm Mobile dla Biznesfm.pl
biznesfm.pl771 views
CTO-CybersecurityForum-2010-Will Gardner by segughana
CTO-CybersecurityForum-2010-Will GardnerCTO-CybersecurityForum-2010-Will Gardner
CTO-CybersecurityForum-2010-Will Gardner
segughana401 views
Presentación Institucional 2014 by Microsip
Presentación Institucional 2014Presentación Institucional 2014
Presentación Institucional 2014
Microsip833 views
Kemira IR Deck - May 2015 by SYGroup
Kemira IR Deck - May 2015Kemira IR Deck - May 2015
Kemira IR Deck - May 2015
SYGroup1.1K views
SUNOPTICS VERSUS BRISTOLITE - CUANDO LO BARATO SALE CARO, EL SALVADOR by luminariasboreal
SUNOPTICS VERSUS BRISTOLITE - CUANDO LO BARATO SALE CARO, EL SALVADORSUNOPTICS VERSUS BRISTOLITE - CUANDO LO BARATO SALE CARO, EL SALVADOR
SUNOPTICS VERSUS BRISTOLITE - CUANDO LO BARATO SALE CARO, EL SALVADOR
luminariasboreal2.4K views
Hiv aids sample questions based on neet pg , usmle, plab and fmge pattern (mc... by Medico Apps
Hiv aids sample questions based on neet pg , usmle, plab and fmge pattern (mc...Hiv aids sample questions based on neet pg , usmle, plab and fmge pattern (mc...
Hiv aids sample questions based on neet pg , usmle, plab and fmge pattern (mc...
Medico Apps653 views
Diapositivas de Web profunda by mcmazon
Diapositivas de Web profundaDiapositivas de Web profunda
Diapositivas de Web profunda
mcmazon629 views

Similar to Globo.com & Varnish

MNPHP Scalable Architecture 101 - Feb 3 2011 by
MNPHP Scalable Architecture 101 - Feb 3 2011MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011Mike Willbanks
1K views53 slides
DrupalCampLA 2011: Drupal backend-performance by
DrupalCampLA 2011: Drupal backend-performanceDrupalCampLA 2011: Drupal backend-performance
DrupalCampLA 2011: Drupal backend-performanceAshok Modi
3.8K views49 slides
UnConference for Georgia Southern Computer Science March 31, 2015 by
UnConference for Georgia Southern Computer Science March 31, 2015UnConference for Georgia Southern Computer Science March 31, 2015
UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin
1K views76 slides
Experience In Building Scalable Web Sites Through Infrastructure's View by
Experience In Building Scalable Web Sites Through Infrastructure's ViewExperience In Building Scalable Web Sites Through Infrastructure's View
Experience In Building Scalable Web Sites Through Infrastructure's ViewPhuwadon D
849 views32 slides
Clug 2011 March web server optimisation by
Clug 2011 March  web server optimisationClug 2011 March  web server optimisation
Clug 2011 March web server optimisationgrooverdan
265 views30 slides
WE18_Performance_Up.ppt by
WE18_Performance_Up.pptWE18_Performance_Up.ppt
WE18_Performance_Up.pptwebhostingguy
683 views43 slides

Similar to Globo.com & Varnish(20)

MNPHP Scalable Architecture 101 - Feb 3 2011 by Mike Willbanks
MNPHP Scalable Architecture 101 - Feb 3 2011MNPHP Scalable Architecture 101 - Feb 3 2011
MNPHP Scalable Architecture 101 - Feb 3 2011
Mike Willbanks1K views
DrupalCampLA 2011: Drupal backend-performance by Ashok Modi
DrupalCampLA 2011: Drupal backend-performanceDrupalCampLA 2011: Drupal backend-performance
DrupalCampLA 2011: Drupal backend-performance
Ashok Modi3.8K views
UnConference for Georgia Southern Computer Science March 31, 2015 by Christopher Curtin
UnConference for Georgia Southern Computer Science March 31, 2015UnConference for Georgia Southern Computer Science March 31, 2015
UnConference for Georgia Southern Computer Science March 31, 2015
Experience In Building Scalable Web Sites Through Infrastructure's View by Phuwadon D
Experience In Building Scalable Web Sites Through Infrastructure's ViewExperience In Building Scalable Web Sites Through Infrastructure's View
Experience In Building Scalable Web Sites Through Infrastructure's View
Phuwadon D849 views
Clug 2011 March web server optimisation by grooverdan
Clug 2011 March  web server optimisationClug 2011 March  web server optimisation
Clug 2011 March web server optimisation
grooverdan265 views
Planning for-high-performance-web-application by Nguyễn Duy Nhân
Planning for-high-performance-web-applicationPlanning for-high-performance-web-application
Planning for-high-performance-web-application
Nguyễn Duy Nhân1.8K views
Drupalcamp Estonia - High Performance Sites by Exove
Drupalcamp Estonia - High Performance SitesDrupalcamp Estonia - High Performance Sites
Drupalcamp Estonia - High Performance Sites
Exove1.1K views
Drupalcamp Estonia - High Performance Sites by drupalcampest
Drupalcamp Estonia - High Performance SitesDrupalcamp Estonia - High Performance Sites
Drupalcamp Estonia - High Performance Sites
drupalcampest865 views
Krug Fat Client by Paul Klipp
Krug Fat ClientKrug Fat Client
Krug Fat Client
Paul Klipp885 views
SharePoint 2010 Boost your farm performance! by Brian Culver
SharePoint 2010 Boost your farm performance!SharePoint 2010 Boost your farm performance!
SharePoint 2010 Boost your farm performance!
Brian Culver3.9K views
Making it fast: Zotonic & Performance by Arjan
Making it fast: Zotonic & PerformanceMaking it fast: Zotonic & Performance
Making it fast: Zotonic & Performance
Arjan8.6K views
Varnish http accelerator by no no
Varnish http acceleratorVarnish http accelerator
Varnish http accelerator
no no3K views
Data processing at the speed of 100 Gbps@Apache Crail (Incubating) by DataWorks Summit
Data processing at the speed of 100 Gbps@Apache Crail (Incubating)Data processing at the speed of 100 Gbps@Apache Crail (Incubating)
Data processing at the speed of 100 Gbps@Apache Crail (Incubating)
DataWorks Summit992 views
Lamp Stack Optimization by Dave Ross
Lamp Stack OptimizationLamp Stack Optimization
Lamp Stack Optimization
Dave Ross4.5K views
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010 by Bhupesh Bansal
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Bhupesh Bansal1.3K views

Recently uploaded

Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ... by
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...ShapeBlue
144 views12 slides
"Surviving highload with Node.js", Andrii Shumada by
"Surviving highload with Node.js", Andrii Shumada "Surviving highload with Node.js", Andrii Shumada
"Surviving highload with Node.js", Andrii Shumada Fwdays
53 views29 slides
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ... by
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...ShapeBlue
79 views17 slides
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T by
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TShapeBlue
112 views34 slides
Future of AR - Facebook Presentation by
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook PresentationRob McCarty
62 views27 slides
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue by
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlueMigrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlueShapeBlue
176 views20 slides

Recently uploaded(20)

Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ... by ShapeBlue
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
ShapeBlue144 views
"Surviving highload with Node.js", Andrii Shumada by Fwdays
"Surviving highload with Node.js", Andrii Shumada "Surviving highload with Node.js", Andrii Shumada
"Surviving highload with Node.js", Andrii Shumada
Fwdays53 views
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ... by ShapeBlue
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...
Import Export Virtual Machine for KVM Hypervisor - Ayush Pandey - University ...
ShapeBlue79 views
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T by ShapeBlue
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
ShapeBlue112 views
Future of AR - Facebook Presentation by Rob McCarty
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
Rob McCarty62 views
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue by ShapeBlue
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlueMigrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
Migrating VMware Infra to KVM Using CloudStack - Nicolas Vazquez - ShapeBlue
ShapeBlue176 views
Digital Personal Data Protection (DPDP) Practical Approach For CISOs by Priyanka Aash
Digital Personal Data Protection (DPDP) Practical Approach For CISOsDigital Personal Data Protection (DPDP) Practical Approach For CISOs
Digital Personal Data Protection (DPDP) Practical Approach For CISOs
Priyanka Aash153 views
NTGapps NTG LowCode Platform by Mustafa Kuğu
NTGapps NTG LowCode Platform NTGapps NTG LowCode Platform
NTGapps NTG LowCode Platform
Mustafa Kuğu365 views
Initiating and Advancing Your Strategic GIS Governance Strategy by Safe Software
Initiating and Advancing Your Strategic GIS Governance StrategyInitiating and Advancing Your Strategic GIS Governance Strategy
Initiating and Advancing Your Strategic GIS Governance Strategy
Safe Software140 views
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And... by ShapeBlue
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...
Enabling DPU Hardware Accelerators in XCP-ng Cloud Platform Environment - And...
ShapeBlue63 views
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... by ShapeBlue
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
ShapeBlue117 views
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue by ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlueVNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
VNF Integration and Support in CloudStack - Wei Zhou - ShapeBlue
ShapeBlue163 views
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue by ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlueWhat’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
What’s New in CloudStack 4.19 - Abhishek Kumar - ShapeBlue
ShapeBlue222 views
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue by ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlueCloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
ShapeBlue94 views
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading... by The Digital Insurer
Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...Webinar : Desperately Seeking Transformation - Part 2:  Insights from leading...
Webinar : Desperately Seeking Transformation - Part 2: Insights from leading...
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue by ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlueCloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
CloudStack Object Storage - An Introduction - Vladimir Petrov - ShapeBlue
ShapeBlue93 views
Keynote Talk: Open Source is Not Dead - Charles Schulz - Vates by ShapeBlue
Keynote Talk: Open Source is Not Dead - Charles Schulz - VatesKeynote Talk: Open Source is Not Dead - Charles Schulz - Vates
Keynote Talk: Open Source is Not Dead - Charles Schulz - Vates
ShapeBlue210 views
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ... by ShapeBlue
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
ShapeBlue85 views

Globo.com & Varnish

  • 1. HTTP accelerator Cachos 2.0 Luis Henrique Okama Tollef Fog Heen Mario Carvalho
  • 2. What is the problem? GET / HTTP/1.1 Hang on, I need to look up a few hundred things in my database and then do a lot of editing controlled by a scripting language HEAD / HTTP/1.1 Hang on, I need to look up a few hundred things in my database and then do a lot of editing controlled by a scripting language, and then I will throw the result away.
  • 3. Why are CMS-es slow? Complex content generation process Single database prevents clustering Expensive software ditto This means: we need server side caching
  • 4. Why are the existing solutions not good enough? Squid, ancient design, forward proxy Apache, not what it's built for, not what it's good at Akamai (and similar), expensive, vendor lock-in
  • 5. What is Varnish? Dedicated HTTP accelerator Focus on server-side speedups Policy control High performance Varnish Configuration Language Shared memory log
  • 6. High performance 11 syscalls + 7 locks for a cache hit Work with the OS, not against it. Multi-CPU, multi-core 64 bit Use advanced OS features: Accept filters madvise(MADV_RANDOM) kqueue, epoll Don't copy data if you don't have to Use workspaces, not malloc/free Compiled configuration
  • 7. Policy control Override TTLs Add, remove or change headers Strip cookies Rewrite URLs Invalidate objects in the cache
  • 8. Varnish Configuration Language Simple domain specific language Compiled via C language to binary Transparently! Dynamically loaded Multiple configs loaded concurrently Instant switch from one VCL to another. Can be done from VCL
  • 9. VCL Example acl some_acl { "10.0.0.0/8" } if (client.ip ~ some_acl) { pass; } if (req.http.host ~ "foo.com$") { set req.http.host = regsub(req.http.host, "foo.com", "bar.org"); } if (obj.valid) { set obj.ttl = 10m; deliver; }
  • 10. Shared memory logfile Fast Custom log tailers varnishtop varnishlog varnishhist varnishncsa varnishstat
  • 12. Comparative scalability setup time cost open source throughput physical space
  • 13. Comparative scalability setup time cost open source throughput physical space
  • 14. Comparative scalability setup time cost open source throughput physical space
  • 15. Comparative scalability setup time cost open source throughput physical space
  • 16. Comparative scalability setup time cost open source throughput physical space
  • 17. Comparative scalability setup time cost open source throughput physical space
  • 18. Comparative scalability setup time cost open source throughput physical space
  • 20. Conclusion software customization work set
  • 23. Contacts Luis H. Okama okama@corp.globo.com (www.globo.com) Mario Carvalho mariocar@corp.globo.com (www.globo.com) Tollef Fog Heen tfheen@redpill-linpro.com (www.varnish-cache.com) @ Stand Globo.com