DevOps for Drupal: Why We Cook With Chef


Published on

DevOps for Drupal presentation given at DrupalCon 2013 in Portland. Promet Source shares secrets for automation and how to make your infrastructure hum.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

DevOps for Drupal: Why We Cook With Chef

  1. 1. DevOps for Drupal: WhyWe Cook with ChefWednesday May 22ndDevOps TrackBeginner
  2. 2. Overview: PurposeWhy are we all here?•  Interested in DevOps practices•  Interested in automation tools like Chef•  Interested in using them with Drupal+
  3. 3. Overview: SharingHow are we going to do this?•  Team style presentation and discussion withviews from both sides - Development andOperations•  Share our experiences and examples... wewill be keeping it real•  We are NOT doing a tool comparison+
  4. 4. Overview: LearnWhat do we hope you will learn?•  Why DevOps matters•  Why automation is a must•  Why you can use Chef to help yourselves,make life easier, and save time+
  5. 5. For more info on Chef, check out fellowDevOps track session:The Joy of Cooking - Whip Up a DrupalEnvironment with Chef•  Thursday May 23rd at 2:15pm•  OR 203•  Presented by Opscodes Nathen HarveyOverview: More on Chef +
  6. 6. Who We AreA software development shop...•  specializing in custom Drupal development,systems integration, mobile, DevOps, and24x7 support•  based in Chicago with 30+ team membersworldwide...and yes we are hiring! Come talk to us!+
  7. 7. Who We ArePlaying the part of:Moderator andProduct Owner@ Promet Source:Director of Uhlinger
  8. 8. Who We ArePlaying the part of:Development@ Promet Source:Solutions wamiltontwitter/hub: @winmillwill+Will Milton
  9. 9. Who We ArePlaying the part of:Operations@ Promet Source:Systems Palmier
  10. 10. Audience Quiz TimeHow many are Developers? (a Will)+CC Image courtesy of Pedro Lozano on Flickr
  11. 11. Audience Quiz TimeHow many are Sysadmins? (a Greg)+CC Image courtesy of Sharyn Morrow on Flickr
  12. 12. Audience Quiz TimeHow many are both? (changing Prod servers)+CC Image courtesy of Arthur Caranta on Flickr
  13. 13. Audience Quiz TimeHow many are other? (the Product Manager)+
  14. 14. Audience Quiz TimeHow many are using automated configurationmanagement tools and processes now inyour everyday work with Drupal?How many are using Chef?+
  15. 15. DevOps: What It IsKey Culture•  Foster a collaborative working relationship betweenDevelopment and OperationsKey Concept•  Infrastructure as codeKey Results•  Higher deployment rates•  Better reliability, stability, and resilience of Production+
  16. 16. DevOps: 3 Key Patterns1. Make environments available early in theDevelopment process2. Shorten and amplify feedback loops3. Create reusable deployment procedures+
  17. 17. DevOps: What It IsntIt is not a collection of bash, Drush, insertscripts here, etc.+DevOps is not about the tools... they dont fix everythingmagicallyBut DevOps relies on some tools to implement itsprinciples
  18. 18. DevOps: Why Is It ImportantOne-off environments and complicated manualbuilds and deployments = FAILUREMust automate to meet business demands --support agile, lean, faster, better...Every environment is Production to someoneDrupal is not different -- all of these apply+
  19. 19. A Classic ScenarioJay: Will, you said this was working.Will: It works in Dev.Jay: Greg, can you look into why it isnt workingin Prod?Greg: Ok. (later on) @#$%! Dev and Prod arenot the same!+
  20. 20. Another Classic OneThe Blog Post: Installing Drupal on AWS1. Start wizard2. 50+ steps later3. A Drupal install+
  21. 21. Why We Chose ChefThe orange color matches oursof course!We already had experience (Marius)doing DevOps work with Chef inBay Area with leading tech companiesWe build Drupal applications -- we found Chefto be a good fit for application configurationmanagement+
  22. 22. The Operations part......out with the old and in with the new+
  23. 23. Our Operations StoryHow we started with Chef at PrometInconsistency was killing our timeHow realistic was automation and configurationmanagement of all of our stuff+
  24. 24. Wasting TimeYou are not in the sudoers file. This incident willbe reported.$ which git$Fatal error: Class PDO not found inblahblahblahAll of these messages diverted your work+
  25. 25. Manual Config Run Amok•  Dev environments != Prod environments•  Shared Dev/Prod/?? environments werecontinually bloated•  Over time modifications amounted to moreinconsistency between environments•  Documentation was lackingo  No records explaining changeso  Constant paper trail "paving"o  No great place for it anyways (what wiki?!?)•  More customizations -> High Fragility+
  26. 26. Typing Faster != Solution•  Runbooks, Bash and SSHo  Process to stand up new infrastructure was donemanually via a "runbook”•  Any configuration time was repeatedextensively•  Environment “discovery” bled timeo  Snooping around to analyze how things were initiallystood up and potentially modified(caching, etc.)•  Endless SSH-ing into servers+
  27. 27. Chef Migration•  In with the Newo  2 dedicated servers§  VM hostso  Similar model, consolidatedo  Entirely Chef managedo  Individual client Chef-spun Dev instances•  Stand Up, QA, and Cutovero  Virtualize legacyo  Considerations mainly PHP•  Monitoring customizations•  Special clientso  Non-Drupalo  Contractually unique+
  28. 28. Chef MigrationAnd migrating wasnt a complete nightmare....+
  29. 29. Righting the Ship•  Server configuration in Gito  Mitigated "paper trail"o  Team awareness of configs by notificationso  PHP, Apache, MySQL•  System config changes pushed to Chef forServer propagationo  No more endless SSH-ingo  No more runbook config for every servero  Undoes "helpful" customizations by others•  Direct between me and Config Mgmto  Configuration entirely “local” with Vagranto  Git driven accountability and awareness (--stat)o  `knife` is to Chef as drush is to Drupal+
  30. 30. Client Ownership•  Clients maintain ownership of their assetso  Any platform with a knife plugino  Legacy apps migratedo  Non-Drupal or Promet "old" clients•  Give us their Keyso  Let us be the Designated Drivero  Developers drop in, get pitted+
  31. 31. Client Ownership•  Excessive customization (not perfect)o  Non-Drupal clients or other worko  More so, use clients Hosted Chef•  Contract flexibilityo  No Dysfunctional Marriage•  Honor prior client / hosting relationshipso  Help clients where they are+
  32. 32. Things That Are Solved•  User Configso  .gitconfigo  .tmux.confo  .ssh/configo  shell customizationso  Not necessary but....doable•  Server Configso  logwatcho  mailo  mysqlo  apacheo  ssho  automysqlbackup+
  33. 33. Sysadmin TransformationHave time to explore/make new toolsDo support in a consistent fashionAutomate all the Things!"Grow" your team and infrastructure not withnumbers but with talentWe do more work with less resources!+
  34. 34. Sysadmin Team EvolutionWe even spun-off a Startup company forinfrastructure and applications automationand scaling using Chef in the Bay Area (non-Drupal)I dont care what you automate with... pick oneand do it!+
  35. 35. The Dev part......its not what you think+
  36. 36. What I Thought I Would Say•  What it was like...•  How it sucked for Devs•  How we changed it•  How it sucked in new ways•  How that lead to the new thing•  Repeat....•  Where we are now•  Whats really fun about this for Devs+
  37. 37. The Problem•  We went through too much shit•  Its not even fun to talk about•  Its probably boring•  Someone else already said what I was tryingto justify•  Youre probably the choir if youre here+
  38. 38. Drupal: OH @crell"If youre not using Features, you are not doingDrupal professionally"Also:•  If you dont have a one step build, you arentusing Features correctly=> Sucking at doing Drupal professionally•  Configuration module is also probably fine•  Its going into core+
  39. 39. So Now My Presentation Is•  What were doing now•  Why you shouldnt NOT do what were doing•  Actual examples of how it has helped•  Why Chef is really awesome•  A surprise+
  40. 40. What Were Doing Now•  Everything you need in the repoo  Any scripts or scripting exampleso  README fileso  (small(-ish)) csvs/xml/etco  ...pretty much any text file that is not too large and isuseful•  ...Still need a way to automate sourcingarchives of files and sql...•  Vagrantfile + Berksfile+
  41. 41. Berks What?+
  42. 42. +
  43. 43. Kirkberks? +
  44. 44. What a Dev Has to Do onOur Team•  Install git somehow•  Install the vagrant package•  Install the vagrant-berkshelf plugin•  git clone/pull && cd-to-the-root•  vagrant up (or reload)•  edit the hosts file or use dnsmasq or similar+
  45. 45. Some Concrete Examples ofThis Helping Us Do Better•  Auditso  Not perfect...o  But now no one has to repeat what is automated•  Every single deployment ever sinceo  Your Production push should not be the first timeyou attempt syncing to a Production environmento  ...unless you crave excitement•  Last minute accommodationso  Ever test RewriteBase?+
  46. 46. Add the .htaccessdirective... +...change the docroot in code!
  47. 47. Potential Examples: YouCant Prove the Negative•  PECL/Pear packageso  Shame on professionals who cant figure it out...o  ...but shame on me for betting the project•  Anything to do with email ever since ever•  Anything to do with external serviceso  ...memcachedo  ...SOLRo  ...even MySQL+
  48. 48. Why You Shouldnt NOT DoAs We DoChallenges you can deal with, excusesthat are bullshitOn stupid things people say re: vagrant andchef•  We make great docs!•  My native environment is fine•  Virtualization consumes resources•  I dont have enough time+
  49. 49. +
  50. 50. My Native Environment isFine1. No, it isnt•  Your job is to ship to your downstreamo  ...not manicure and fetishize your tools•  You are getting strictly less help than youcould geto  ...even if youre getting a lot of helpo  ...even if youre paying for help•  You are keeping yourself from helping+
  51. 51. My Native Environment isFine (contd)2.Your native environment is personal•  You may even obsess over it•  You may be completely wrong•  You will be happier without each other+
  52. 52. Virtualization ConsumesResources +
  53. 53. I Dont Have do everything perfectly•  We didnt either, and were doing far betterthan we were•  We do not have a massive continuousdelivery, cloud orchestration and scalinginfrastructure•  ... But now what we have is in code+
  54. 54. Chef is Really Awesome...Puppet is probably cool too•  Stewardship and leadership•  Constantly evolving ecosystem•  A community hell-bent on sharing•  Kicks ass even if you dont+
  55. 55. So Easy I Can Share It withYou!Dont try it on this connectionbash <(curl -L -s to work already!+
  56. 56. Why You Should Use thisProject (or something like it)We must advance the state of the art for Drupaldeployment and application managementThe CMS that is easiest to flexibly deploy willbe the CMS of the futureAll the features and D8 config exports in theworld cant help you if you cant automate it+
  57. 57. Other Projects/Strategies•  If you have a dope Jenkins workflow...o  help me automate it with Chef so we can all use it!•  Please use Puppet if it suits youo  and be vocal about hardcoded conventions thatmake your work hard•  Kraftwagen -•  Ariadne by patcon -
  58. 58. The Wrap-Up part......what should you remember+
  59. 59. Key Summary PointsProcess and team is the focus•  Get your Devs and Sysadmins in the sameroom and start communicatingTools are there to help•  Start automating nowChef is an awesome tool•  It works and it helps+
  60. 60. Q&AAsk us questions nowTalk to us afterwardStop by and see us at Booth 201Thanks!+
  61. 61. Feedback?Evaluate session