Your SlideShare is downloading. ×
0
The UnintendedBenefits of ChefStandardization FTW
whoamiClinton Wolfe, Web Architect @ OmniTImod perl developer since 1996cheffin since 2012@clintoncwolfegithub: clintoncwo...
$JOBomniti.comcustom web development - UX to hostingwe make OmniOS, an Illumos based OS distro
our dev clientsmostly brownfield worknew featuresoh no it wont scalemy nephew built it
client scalesome have hundreds of nodesmost clients have < 10 nodessome have many, want fewersome have few, need morewe sp...
process : anti-processeach customer has "unique" problemssome are, most arentwere flexible, urging gradual change
TMTOWTDI....dev environmentsasset statecronself-referential urlsCPAN/PECL/gem/egg/npm builds“You Only Deploy Once!"
clients: YODO!!!except for disaster recovery
clients: YODO!!!except for disaster recoveryexcept for onboarding devs
clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CI
clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling out
clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load t...
clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load t...
clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load t...
QA is hardcant make new QA environmentsQA env is preciousstate hard to manageIf testing is hard,no testing is donemijasper...
surely we can do betterWe want standardizationClient wants deployability (scale-out, resilience)Chef FTW
The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing ...
cheffing as discoveryExploring the existing app for on-boardingEven rsyncing a goldenimage is OKcapture it in chefGet depl...
The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing ...
app deployment in chef?most app deployment tools partially dictatelayout, push process, etcwho knows what we have been han...
cheffing an appidentify and decouple rolessetup user auth, systems stuffcode checkouttemplatize config filesservice defini...
The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing ...
finding variancesunderstand existing variancesstandardize the arbitraryjustify the ones that need to remainwriting a custo...
Our shared cookbooksOver 20 cookbooks used internally at thecompany, cross-clientAttribute-driven default recipesExposes p...
its still OK to puntWe will never be fully standardizedAs old projects mature, new projects arriveknown technical debtcan ...
The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing ...
cheffing: ci with jenkinsstatic checks on chef configscratchbuild Vagrant runsparametric builds and Vagrantfilesvagrant-te...
system testsHigh-level, near-english behavioral testsNot low-level unit tests against codebaseSupposed to be serving http?...
aside: our test harnessvagrant-rspec-ci gemprovision ok?services running?connect to port?Capybara for web UI testingJunit ...
Test spec exampledescribe "TrafficServer Service" dobefore(:all) do@vm = VagrantTestSubject::VM.attach()endit "should appe...
Shiny, Happy TeamsOps Team ImpactDev Team ImpactQA Team ImpactOrganizational ImpactLEGO Group
Ops Team ImpactChef everywhere! YAY!Chef for everything, especially wildly divergentapp deployments! BOO!Direct exposure t...
Dev Team ImpactCan get a new dev env anytime, have morethan one - YAYSame build everywhere – YAYI have to learn ruby? BOOS...
QA Team ImpactUse vagrant VMs as test subjectsQA VM provisioning can now be elastic,responsive to testing/release cyclesDe...
the VM as fixturedata loads - vary by test scenariodata loads - vary for scaleasset loadsrole permutationsos, vm, env perm...
aside: loading fixturesWe use Chef roles to define attributes that loadfixture stateWe use recipes to read the attributes ...
i accidentally the whole dbfixture loading must beused with careuse a safety flag toenable itNelson Yrizarry/brickshelf
Organizational ImpactStandardization eases:actual deploymenton-call ops can support the app moredevs responsible for deplo...
Thanks!Clinton Wolfeclinton@omniti.comfindmybrick.com
Upcoming SlideShare
Loading in...5
×

The unintended benefits of Chef

789

Published on

When most people talk about automating infrastructure, they focus on things like consistency, scalability, and flexibility. While fine goals, we recently converted several projects to Chef for both systems AND application deployment, and found that, with a little work, these tools could also help you enable better software quality assurance, load modeling, and even improve resource allocation.

By sharing cookbooks across projects, we were able to standardize practices and eliminate arbitrary differences, while using parameterization to perfectly isolate the special needs of each project. This allowed us to transfer knowledge among staff much more quickly. Pulling in and parameterizing application state – database contents, website assets, uploaded content – allowed us to spin up new environments with as much or as little state as needed. Integrating with Vagrant and Jenkins, we were then able to use chef to treat the entire image – system and application – as a test fixture. As each engineer (ops or dev) has visibility into the whole stack, we can more easily move people between dev and ops, or between projects.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
789
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "The unintended benefits of Chef"

  1. 1. The UnintendedBenefits of ChefStandardization FTW
  2. 2. whoamiClinton Wolfe, Web Architect @ OmniTImod perl developer since 1996cheffin since 2012@clintoncwolfegithub: clintoncwolfeLEGO group
  3. 3. $JOBomniti.comcustom web development - UX to hostingwe make OmniOS, an Illumos based OS distro
  4. 4. our dev clientsmostly brownfield worknew featuresoh no it wont scalemy nephew built it
  5. 5. client scalesome have hundreds of nodesmost clients have < 10 nodessome have many, want fewersome have few, need morewe specialize in doingmore with lessmijasper/brickshelf
  6. 6. process : anti-processeach customer has "unique" problemssome are, most arentwere flexible, urging gradual change
  7. 7. TMTOWTDI....dev environmentsasset statecronself-referential urlsCPAN/PECL/gem/egg/npm builds“You Only Deploy Once!"
  8. 8. clients: YODO!!!except for disaster recovery
  9. 9. clients: YODO!!!except for disaster recoveryexcept for onboarding devs
  10. 10. clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CI
  11. 11. clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling out
  12. 12. clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testing
  13. 13. clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testingexcept for upgrade testing
  14. 14. clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testingexcept for upgrade testingexcept for datacenter moves
  15. 15. QA is hardcant make new QA environmentsQA env is preciousstate hard to manageIf testing is hard,no testing is donemijasper/brickshelf
  16. 16. surely we can do betterWe want standardizationClient wants deployability (scale-out, resilience)Chef FTW
  17. 17. The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  18. 18. cheffing as discoveryExploring the existing app for on-boardingEven rsyncing a goldenimage is OKcapture it in chefGet deployable (in Vagrant ) ASAPmijasper/customminifigs.co.uk
  19. 19. The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  20. 20. app deployment in chef?most app deployment tools partially dictatelayout, push process, etcwho knows what we have been handedbut chef can do pretty much anything!
  21. 21. cheffing an appidentify and decouple rolessetup user auth, systems stuffcode checkouttemplatize config filesservice definitionsIterate, standardize, simplify!
  22. 22. The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  23. 23. finding variancesunderstand existing variancesstandardize the arbitraryjustify the ones that need to remainwriting a customcookbook is a red flagjokeith/brickshelf
  24. 24. Our shared cookbooksOver 20 cookbooks used internally at thecompany, cross-clientAttribute-driven default recipesExposes project variances in clear relief... yetwe can easily extend them if neededShared roles/handlers/databags, too
  25. 25. its still OK to puntWe will never be fully standardizedAs old projects mature, new projects arriveknown technical debtcan be healthymijasper/brickshelf
  26. 26. The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  27. 27. cheffing: ci with jenkinsstatic checks on chef configscratchbuild Vagrant runsparametric builds and Vagrantfilesvagrant-test-subjectmijasper/brickshelf
  28. 28. system testsHigh-level, near-english behavioral testsNot low-level unit tests against codebaseSupposed to be serving http? Check for it!Easy to add more later - start with minimum
  29. 29. aside: our test harnessvagrant-rspec-ci gemprovision ok?services running?connect to port?Capybara for web UI testingJunit output for pretty Jenkins graphs
  30. 30. Test spec exampledescribe "TrafficServer Service" dobefore(:all) do@vm = VagrantTestSubject::VM.attach()endit "should appear as a healthy service" do@vm.should have_running_service("trafficserver")endit "should be listening on external_ip:80" do@vm.should be_listening_on_external_ip(80)endit "should be the right process name on port 80" doprocess = @vm.process_name_listening(127.0.0.1, 80)process.should match(//opt/ts/bin/traffic_manager/)endit "should respond with HTTP 200 to / on port 80" do@vm.http_get(/).should be_http_not_found # from rspec-http gemendend
  31. 31. Shiny, Happy TeamsOps Team ImpactDev Team ImpactQA Team ImpactOrganizational ImpactLEGO Group
  32. 32. Ops Team ImpactChef everywhere! YAY!Chef for everything, especially wildly divergentapp deployments! BOO!Direct exposure to dev team deployment needs& tooling... a more devvy ops team
  33. 33. Dev Team ImpactCan get a new dev env anytime, have morethan one - YAYSame build everywhere – YAYI have to learn ruby? BOOSome awareness of practices on other projects,improved perspectiveMuch more visibility into how systems areprovisioned – a more opsy dev team
  34. 34. QA Team ImpactUse vagrant VMs as test subjectsQA VM provisioning can now be elastic,responsive to testing/release cyclesDev and QA exposure to systems BDTDev and ops teams more QA-y: devopsqa!
  35. 35. the VM as fixturedata loads - vary by test scenariodata loads - vary for scaleasset loadsrole permutationsos, vm, env permutationsclusters of related machines (db, web, CDN,client)
  36. 36. aside: loading fixturesWe use Chef roles to define attributes that loadfixture stateWe use recipes to read the attributes andconverge to the needed test state•special purge-all role clears all state•use env var to pass fixture list into vagrant forprovision
  37. 37. i accidentally the whole dbfixture loading must beused with careuse a safety flag toenable itNelson Yrizarry/brickshelf
  38. 38. Organizational ImpactStandardization eases:actual deploymenton-call ops can support the app moredevs responsible for deployabilityops can participate in app managementreduced ramp-up time onto existing projects
  39. 39. Thanks!Clinton Wolfeclinton@omniti.comfindmybrick.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×