Successfully reported this slideshow.
Your SlideShare is downloading. ×

Load balancing in the SRE way

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 11 Ad

Load balancing in the SRE way

Download to read offline

This is a lightning talk topic I presented at Triangle DevOps meetup to share ideas about operating load balancer by following SRE practices. It highlighted continuous delivery process and test driven development when we develop this feature. There're lots of online cloud-based services we used to achieve our goal including AWS, Google Cloud, Librato, Logentries, SignalSciences, BlueBoxCloud, Quay.io etc. And lots of open source software we love and use every day.

Disclaimer: It represents personal opinions only, not represent any points of view of my employer.

This is a lightning talk topic I presented at Triangle DevOps meetup to share ideas about operating load balancer by following SRE practices. It highlighted continuous delivery process and test driven development when we develop this feature. There're lots of online cloud-based services we used to achieve our goal including AWS, Google Cloud, Librato, Logentries, SignalSciences, BlueBoxCloud, Quay.io etc. And lots of open source software we love and use every day.

Disclaimer: It represents personal opinions only, not represent any points of view of my employer.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Viewers also liked (20)

Advertisement

Similar to Load balancing in the SRE way (20)

Recently uploaded (20)

Advertisement

Load balancing in the SRE way

  1. 1. Load Balancing in the SRE way Ke Zhu @shawnzhu Site (Un)Reliability Engineer at IBM
  2. 2. For What? • GitHub Enterprise Cluster • On Internet • Zero downtime • 100M+ HTTP requests per week • 30k+ attacks per week • 26k+ git clone per hour (https://help.github.com/enterprise/2.8/admin/guides/installation/maintenance-mode/)
  3. 3. Design Goals • Scripting Platform • traffic conducting via code • do social coding • Observable • Blue/Green deployment • High performance • Security from day one
  4. 4. Software Stack && 🎩 magic kernel parameters in /etc/sysctl.conf 🐰
  5. 5. Scripting Platform • OpenResty (Nginx + Lua) - https://openresty.org/en/ (Example: customized request rate limiting)
  6. 6. Blue/Green Deployment • Can not terminate any TCP connection • Two stacks: • load-balancer-green • load-balancer-blue (for experiment) • Cloud DNS • Switching A record + short TTL (~5m) • Simple/Weighted Routing policy • Run experiment by using docker image tags • Real time metrics collection by librato.com
  7. 7. • Test docker images • RSpec + Serverspec • Travis CI • Test docker host • RSpec + Serverspec • Test Kitchen Test Driven for Container
  8. 8. ❤vault • Secret mgmt via API - https://www.vaultproject.io/ • retrieve all secrets for provisioning load balancer via a single token with TTL 5min
  9. 9. Blocking mode in Production • Signal Sciences - https://signalsciences.net/
  10. 10. Summary • Conducting HTTPS traffic via Lua code • Blue-green deployment of Load balancer via DNS • Testing docker with RSpec + Serverspec • SignalSciences

×