Downloaded 19 times
More Related Content
Similar to Evolving Infrastructure
Evolving Infrastructure
- 1. Evolving Infrastructure Louis Dunne,Platform Engineering louis.dunne@workday.com
- 2. Workday Introduction • Background/ Our Own Cloud 2.0 Architecture • Config Management / Chef Build Pipeline / Chef Development Tools 3.0 Architecture • Release & Deployment Changes / Image Build Pipeline / Release Management / Planned vs. Unplanned Changes
- 3. Enterprise HCM & Financialssoftware in the cloud First release November 2006
- 6. Workday Introduction • Background/ Our Own Cloud 2.0 Architecture • Config Management / Chef Build Pipeline / Chef Development Tools 3.0 Architecture • Release & Deployment Changes / Image Build Pipeline / Release Management / Planned vs. Unplanned Changes
- 8. One of theDevOps mantras is Infrastructure is Code
- 9. If it’s code… →you need to test it → you need a build & test pipeline
- 10. Local Development Code Review C.I. Unit Tests All triggeredby developers pushing code Bronze Cookbook Artefacts Local Development Code Review C.I. Unit Tests Local Development Code Review C.I. Unit Tests
- 11. C.I. System Tests C.I. System Tests C.I. SystemTests Failures (back to dev) System Tests Triggered on the Hour Bronze Cookbook Artefacts Silver Cookbook Artefacts
- 12. C.I. Integration Tests Gold Cookbook Artefacts Silver Cookbook Artefacts Failures (back todev) Integration Tests Triggered Several Times a Day
- 13.
- 14.
- 15. Where Do WeRun Our ServerSpec Tests? • Lab Hardware? • Vagrant? • AWS?
- 16. The Lab? • HardwareBased • Can’t set machine state before the test run • Can’t reset machine state after the test run
- 17. Run In Vagrant? •Good for simple cases • Harder for integration testing a few dozens Chef roles • Prefer a hosted platform with longer running nodes for some services like artefact repos
- 19.
- 20.
- 21. Workday Introduction • Background/ Our Own Cloud 2.0 Architecture • Config Management / Chef Build Pipeline / Chef Development Tools 3.0 Architecture • Release & Deployment Changes / Image Build Pipeline / Release Management / Planned vs. Unplanned Changes
- 22. Platform Services Appliances Workday Linux Servers ImageBased Deployment Chef / Cobbler Based Deployment
- 23. ■ Cobbler forthe OS ■ Chef based deployments of system / infrastructure changes ■ Custom tooling for applications deployments ■ Cobbler / Chef for bare metal ■ Most services moving to image based deployments ■ Custom deployment tools to manage VM lifecycle 2.0 Deployments 3.0 Deployments
- 24. 1. Where inthe build & test pipeline do the Machine Images get created? 1. What technology & processes are used to create them?
- 25. • Early inthe pipeline • Application teams → image artefact • Image artefact → build & test pipeline
- 26. • Lots oftools to choose from: • Diskimage-builder • VMBuilder • Box Grinder • Packer • Imagefactory • We use Oz (https://github.com/clalancette/oz)
- 27.
- 28.
- 29. Gold Cookbook Artefacts Gold Application Artefacts Promote to Staging UNIT SYSTEMINTEGRATION UNIT INTEGRATION I N F R A S T R U C T U R E A P P L I C A T I O N S SYSTEM Promote to Production
- 30. Image Build Service SYSTEM TESTS UNITINTEGRATION I N F R A S T R U C T U R E SYSTEM Promote to Staging A P P L I C A T I O N S Base Image Base Image Application RPMs Application Image Application Image
- 31. The ability topush code to environments easily and quickly - push button deploys A stable framework for development, testing, deployment and auditing Deploy applications and operational environments in the same way each time Source code, tools and all components that make up releaseIdentifiability Reproducibility Consistency Agility
- 32. ■ Images ■ StartupProperties ■ Cookbooks ■ Roles ■ Data Bags ■ Encrypted Data Bags ■ RPMs ■ Other artefacts (e.g. ruby & python packages) Config Management Image Deployment
- 33. ■ Destroy andrecreate rather than change in place ■ All facets of the OS are captured by the image artefact ■ Server state mutated over time as updates are applied ■ Impractical to manage every last detail of the OS with config management Config Management Image Deployment
- 34. • The idealis… • Push all changes through the image pipeline for both planned and unplanned changes • If your pipeline is reliable with a quick turnaround you can use this for all changes • We also like to have a Break The Glass option Planned Vs Unplanned Changes
- 35. • For lowimpact changes only • Bash script → RPM • Agent on each VM can deploy RPMs • Trigger remotely via secure channel • Testable, easy to roll out, good for auditability Break The Glass
- 36.
- 37.
- 38. “If it hurts, domore of it”
- 39.