The unintended benefits of Chef
Upcoming SlideShare
Loading in...5
×
 

The unintended benefits of Chef

on

  • 920 views

When most people talk about automating infrastructure, they focus on things like consistency, scalability, and flexibility. While fine goals, we recently converted several projects to Chef for both ...

When most people talk about automating infrastructure, they focus on things like consistency, scalability, and flexibility. While fine goals, we recently converted several projects to Chef for both systems AND application deployment, and found that, with a little work, these tools could also help you enable better software quality assurance, load modeling, and even improve resource allocation.

By sharing cookbooks across projects, we were able to standardize practices and eliminate arbitrary differences, while using parameterization to perfectly isolate the special needs of each project. This allowed us to transfer knowledge among staff much more quickly. Pulling in and parameterizing application state – database contents, website assets, uploaded content – allowed us to spin up new environments with as much or as little state as needed. Integrating with Vagrant and Jenkins, we were then able to use chef to treat the entire image – system and application – as a test fixture. As each engineer (ops or dev) has visibility into the whole stack, we can more easily move people between dev and ops, or between projects.

Statistics

Views

Total Views
920
Views on SlideShare
837
Embed Views
83

Actions

Likes
1
Downloads
3
Comments
0

3 Embeds 83

http://www.opscode.com 65
http://www.getchef.com 16
https://www.getchef.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The unintended benefits of Chef The unintended benefits of Chef Presentation Transcript

  • The UnintendedBenefits of ChefStandardization FTW
  • whoamiClinton Wolfe, Web Architect @ OmniTImod perl developer since 1996cheffin since 2012@clintoncwolfegithub: clintoncwolfeLEGO group
  • $JOBomniti.comcustom web development - UX to hostingwe make OmniOS, an Illumos based OS distro View slide
  • our dev clientsmostly brownfield worknew featuresoh no it wont scalemy nephew built it View slide
  • client scalesome have hundreds of nodesmost clients have < 10 nodessome have many, want fewersome have few, need morewe specialize in doingmore with lessmijasper/brickshelf
  • process : anti-processeach customer has "unique" problemssome are, most arentwere flexible, urging gradual change
  • TMTOWTDI....dev environmentsasset statecronself-referential urlsCPAN/PECL/gem/egg/npm builds“You Only Deploy Once!"
  • clients: YODO!!!except for disaster recovery
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devs
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CI
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling out
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testing
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testingexcept for upgrade testing
  • clients: YODO!!!except for disaster recoveryexcept for onboarding devsexcept for CIexcept for scaling outexcept for load testingexcept for upgrade testingexcept for datacenter moves
  • QA is hardcant make new QA environmentsQA env is preciousstate hard to manageIf testing is hard,no testing is donemijasper/brickshelf
  • surely we can do betterWe want standardizationClient wants deployability (scale-out, resilience)Chef FTW
  • The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  • cheffing as discoveryExploring the existing app for on-boardingEven rsyncing a goldenimage is OKcapture it in chefGet deployable (in Vagrant ) ASAPmijasper/customminifigs.co.uk
  • The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  • app deployment in chef?most app deployment tools partially dictatelayout, push process, etcwho knows what we have been handedbut chef can do pretty much anything!
  • cheffing an appidentify and decouple rolessetup user auth, systems stuffcode checkouttemplatize config filesservice definitionsIterate, standardize, simplify!
  • The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  • finding variancesunderstand existing variancesstandardize the arbitraryjustify the ones that need to remainwriting a customcookbook is a red flagjokeith/brickshelf
  • Our shared cookbooksOver 20 cookbooks used internally at thecompany, cross-clientAttribute-driven default recipesExposes project variances in clear relief... yetwe can easily extend them if neededShared roles/handlers/databags, too
  • its still OK to puntWe will never be fully standardizedAs old projects mature, new projects arriveknown technical debtcan be healthymijasper/brickshelf
  • The Plan!Focus on deployabilityIntegrate systems and app deploymentCustom Recipes – choosing your battlesAutomate testing early
  • cheffing: ci with jenkinsstatic checks on chef configscratchbuild Vagrant runsparametric builds and Vagrantfilesvagrant-test-subjectmijasper/brickshelf
  • system testsHigh-level, near-english behavioral testsNot low-level unit tests against codebaseSupposed to be serving http? Check for it!Easy to add more later - start with minimum
  • aside: our test harnessvagrant-rspec-ci gemprovision ok?services running?connect to port?Capybara for web UI testingJunit output for pretty Jenkins graphs
  • Test spec exampledescribe "TrafficServer Service" dobefore(:all) do@vm = VagrantTestSubject::VM.attach()endit "should appear as a healthy service" do@vm.should have_running_service("trafficserver")endit "should be listening on external_ip:80" do@vm.should be_listening_on_external_ip(80)endit "should be the right process name on port 80" doprocess = @vm.process_name_listening(127.0.0.1, 80)process.should match(//opt/ts/bin/traffic_manager/)endit "should respond with HTTP 200 to / on port 80" do@vm.http_get(/).should be_http_not_found # from rspec-http gemendend
  • Shiny, Happy TeamsOps Team ImpactDev Team ImpactQA Team ImpactOrganizational ImpactLEGO Group
  • Ops Team ImpactChef everywhere! YAY!Chef for everything, especially wildly divergentapp deployments! BOO!Direct exposure to dev team deployment needs& tooling... a more devvy ops team
  • Dev Team ImpactCan get a new dev env anytime, have morethan one - YAYSame build everywhere – YAYI have to learn ruby? BOOSome awareness of practices on other projects,improved perspectiveMuch more visibility into how systems areprovisioned – a more opsy dev team
  • QA Team ImpactUse vagrant VMs as test subjectsQA VM provisioning can now be elastic,responsive to testing/release cyclesDev and QA exposure to systems BDTDev and ops teams more QA-y: devopsqa!
  • the VM as fixturedata loads - vary by test scenariodata loads - vary for scaleasset loadsrole permutationsos, vm, env permutationsclusters of related machines (db, web, CDN,client)
  • aside: loading fixturesWe use Chef roles to define attributes that loadfixture stateWe use recipes to read the attributes andconverge to the needed test state•special purge-all role clears all state•use env var to pass fixture list into vagrant forprovision
  • i accidentally the whole dbfixture loading must beused with careuse a safety flag toenable itNelson Yrizarry/brickshelf
  • Organizational ImpactStandardization eases:actual deploymenton-call ops can support the app moredevs responsible for deployabilityops can participate in app managementreduced ramp-up time onto existing projects
  • Thanks!Clinton Wolfeclinton@omniti.comfindmybrick.com