Puppet for SysAdmins


Published on

Stephen Wallace at Puppet Camp Sydney 2013

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Puppet for SysAdmins

  1. 1. Puppetfor SysadminsPuppetCamp 2013 - Sydney
  2. 2. Och Aye The Noo! (oh yes, right now!)Stephen Wallace20 years experiencePredom *nix / sysadmin backgroundSenior management - teams up to 20Large heavily integrated systemsHealth, Online Gaming, RecruitmentICE Systems 18 mths....one of the Puppet go to guysstephen.wallace@icesystems.com.auLinkedin: http://au.linkedin.com/in/stephenwallace
  3. 3. Who am I talking to?SysadminOperations management...and the Puppet newbs
  4. 4. Why should I listen?
  5. 5. Spot the challenge!An extract from my Linkedin profile(au.linkedin.com/in/stephenwallace) Harvey Nash, Database Administrator, London Hays Personnel Services, IT Manager ANZ, Sydney Saffron Consulting, Unix/Linux Consultant, Sydney Business Systems Manager, Corporate Express, Sydney Unix Services Manager, NHS, Glasgow Technical Operations Manager, Casino.com, Gibraltar IT Operations Manager, Healthecare Aust, Sydney Solution Architect, ICE Systems, SydneyNot a whisper of dev/proggie anywhere!
  6. 6. Puppet ...for Non-proggiesPuppetCamp 2013 - Sydney
  7. 7. Begin with the end in mind “Would you tell me, please, which way I ought to go from here?” “That depends a good deal on where you want to get to“, said the cat. “I don’t much care where”, said Alice. “Then it doesn’t matter which way you go”, said the cat. (Alices Adventures in Wonderland, chapter 6)...the importance of goals
  8. 8. Goals for everybody?
  9. 9. Goals for the talkers? (aka ops management...)AvailabilityScalabilitySupportabilityPredictabilityAgilityKeepTheCostsDownAbility...Smiling end users…and Cloudability
  10. 10. Goals for doers? (aka sysadmins...)Predictable / stable infrastructureReusable wins.....everybody listening?Fix stuff onceReduce support workload, so can finally get to that moreinteresting project stuff!Monitoring....that someone else maintains :)Documentation...dittoSmiling end users…resulting in...Weekends / sleepPub by 5 (past 4 preferably)Spot the cuttlefish!
  11. 11. Research-Centric Academic InstitutionSysadmin lead initiativeSmall teamNo opex budgetIncreasing number of users / power usersProvisioning demands increasingDiversity of platform support requirementsAlso, legacy *nix env, with people based SPOFUnique huh? Didnt think so...
  12. 12. Threat or an opp?Dev lead initiativesPublishing/media company – Window + Linux webstack. One of the biggest website in ANZ. Production.Insurance brokerage – current project.Ecommerce – Small team of developers. UsingPuppet/AWS to spin up and tear down on demand.Production.
  13. 13. What I did NOT need Puppeteer skills Neckbeard Apple Macbook
  14. 14. What I DID need• A paradigm shift, and a willingness to learn a few new tricks ALSO… Willingness to learn Willingness to collaborate
  15. 15. Times are a ChanginWe used to receive hardware, rack n stack,folder of CDs for each operating systemPass over to the apps teamPass over to the database chapsMake sure that the monitoring is configuredDraw some diagramsPass to testingProgress through change management
  16. 16. Nowadaze…Launch the hyperviser or AWS ManagementConsole, and click a button to spin up 1 or1001 new nodes via templates.Opportunity, or threat?People are people....carefully does it
  17. 17. Initial reaction to the word... “Devops”...The objection is rarely the objection!Long live the onion!
  18. 18. The Challenges & The Tools – aka “the useful stuff”“Were not proggies.....were sys admins!”Keep Calm, and Carry On
  19. 19. “Proggie syntax is WAY to hard...”Ops boys and girls love scripting...right?...especially when maintaining more than a handful of boxes...its just a different, and not hugely difficult scripting language...with Puppet Enterprise, it can be GUI driven
  20. 20. Puppet can write Puppet code! puppet resource user...handy! puppet resource - types (lots of fun for all the family) ...and theres lots of “inspirational” stuff from the Forge ( http://forge.puppetlabs.com )! puppet module search mysql (gem install puppet-module)
  21. 21. Native tools for syntax checkingpuppet parser validate blah.pp
  22. 22. Syntax Highlighting ToolsPut a bit of colour in your life...vim-puppetsyntastic - similar
  23. 23. There will still be a few sceptics
  24. 24. Were gonna need a bigger boat Geppetto IDE? Have a look! Self contained. Syntax examples with mouseover Autocorrects stuff! Integrates with source control Integrates into CI Integrates with the Forge!
  25. 25. A question of style? Weve all been there....your old sysadmin leaves...and the newb arrives to investigate the scripts...AAAARARRRRGGHPuppet Lint!
  26. 26. Finding this all a bit testing? Rspec-puppet - why test your modules, using yet another language? How do I test that my new apache module will...create the following file resource: /etc/apache2/sites-available/blah file ….when run on a Debian host?Check it out...http://rspec-puppet.com/
  27. 27. Maybe you didnt hear me“Were not proggies...were sys admins!”
  28. 28. “Whats wrong with flat files and dirs”Nothing...Puppet works like that out of the box!Hiera!Not actually a Scandinavian greeting
  29. 29. HieraHow can it benefit the ops crowd?A simple pluggable, hierarchicaldatabaseIncluded with Puppet Enterprise, geminstall for opensourceAllows data to be separated from codeCleaner, more supportable manifests
  30. 30. How does Hiera work?HTTP VS APACHE2?hiera.conf…you know conf files...right?!:hierarchy: - nodes/%{fqdn} - %{operatingsystem} - common:backends: - yaml:yaml: :datadir: /etc/puppetlabs/hieradataroot@hpmini:~# facter operatingsystemUbunturoot@hpmini:/etc/puppetlabs/hieradata/Ubuntu# cat common.yamlwebserver: apache2
  31. 31. Why Hiera?Good question!Either have lengthy case statements...
  32. 32. Complex thingocentos, redhat, oel, oraclelinux, linux: { $supported = true $webserver = [ "apache2" ] $svc_name = "apache2" $config = "/etc/apache2/httpd.conf" if $::operatingsystemrelease =~ /^5/ { $config_tpl = httpd.conf.el.erb } elsif $::operatingsystemrelease =~ /^6/ { $config_tpl = httpd.conf.el6.erb } else { fail("the webserver module doesnt know whattemplate to use for your ${::oper
  33. 33. ...or you can have thisroot@hpmini:/etc/puppetlabs/hieradata/Ubuntu# cat common.yamlwebserver: apache2$webserver = hiera(webserver,httpd) if $webserver != nil { package { $webserver: ensure => present }hiera.conf - %{CustomFact-Environment}/common
  34. 34. Augeus! Love those conf files... Configuration editing tool. It parses configuration files in their native formats and transforms them into a tree. Configuration changes are made by manipulating this tree and saving it back into native config files. $defaultrunlevel = hiera(defaultrunlevel,3) augeas { "runlevel": context => "/files/etc/inittab", changes => [ "set id/runlevels ${defaultrunlevel}", ], } root@hpmini:/etc/puppetlabs/hieradata/production# grep defaultrunlevel common.yaml defaultrunlevel: 42
  35. 35. Ruby? Sounds ScaryTheres no place like
  36. 36. Ruby – Not that scaryDeep breath...yum install blah.rpm (familiar?)apt-get install stuff...gem install puppet-lint (!)
  37. 37. But wait! Theres more...Cheatsheets! http://docs.puppetlabs.comProvisioning:Foreman (libvirt, EC2, RHEV)Vagrant (Oracle Virtualbox)...and now Razor (EMC). Early days, but growing popularityPuppet Cloud Provisioner (AWS & VMWare)
  38. 38. Version Control...not just for proggies! git very popular. github very useful. Stash growing in popularity...Aussie Aussie Aussie!...Handy for sysadmins shell scripts aswell!
  39. 39. But were an enterprise shop ...Puppets not quite there yet, Shirley? Client side Ruby brush n scrub up. Load reduced. All major Linux flavours Solaris Select legacy o/s, Windows included ;)
  40. 40. Puppet Server Side Grown Up PuppetDB – also with an API into Puppet Inventory Service. Ability to feed CMDB with most up to date facts Phusion Passenger Enterprise app server for Ruby ActiveMQ Enterprise message broker REST API
  41. 41. PE also includes MCollective How can I... See how many of my machines are running RHEL 6.1? See how many are running a particular release of a software package? Monitor all of my servers for a resource level? Mcollective can help manage, monitor, control Puppet, collect performance and inventory data...so...so...
  42. 42. But but butIve been using vi, shell scripts, and port 22 without mercy for years!....my father did the same....and his father before him...
  43. 43. Reinvent yourself every 2 years
  44. 44. The Holy Trinity IT Ops Management Challenges Responding toProductivity and Efficiency Business Needs Configuration Drift Lack of Visibility
  45. 45. What its like in the trenches Application Application Systems Application DatabaseService Desk Support Developer Administrator Developer AdministratorLog call. The Java Stop working Stop what Manual DBA analyzesconsole says monitoring on new code to they’re doing to investigation audit logs whicheverything is tools don’t troubleshoot. identify and establishes not points to bad green. show anything Need gather application query. either. Call the production production logs problem. developer. logs! for developer. Now Escalate. Escalate. Escalate. Respond. Escalate. what?
  46. 46. Monitoring detects the service outage - Nagios
  47. 47. Root Cause Analysis - Splunk • IT Operations review monitoring info, and feed relevant details into Splunk for event correlation over all enterprise devicesA config file hasbeen manuallyupdated….badly,causing the outage
  48. 48. Fix It Once!• Write a manifest to manage the files and fix the issue• Maybe use Puppet to write the Nagios cfg “Exported resources” / templates• Monitoring = green lights
  49. 49. “Were not proggies... were sys admins!!!”I know how you feel!Training did help,eventually ;)Productive withPuppet in a dayor two.
  50. 50. Keep Calm and Carry On ...againCan be completely shell driven – tickServer side install from CLI – tickClient side install from CLI, with answers file,and SSH bulk install possible - tickCan even be run in standalone via cron....canbe kept simple, and add value immediatelyEven reads like native language! Dare I say it,English
  51. 51. Building a module, the easy way A simple manifest `puppet resource service ssh` - outputs valid syntax A module `puppet module generate x-blah` - Rename module dir and make sure this is reflected in the class name in manifests/init.pp and tests/init.pp A class already in the `tests` directory. Puppet apply -v ./init.pp --noop
  52. 52. Any other useful stuff?What about DR? (Dont forget your certs!)Automated doco?/etc/puppet/modules/ssh/manifests# puppet doc ./init.ppUnit testing -> rspec-puppet - Mere mortal translation required!Dependency diagrams - dot -Tpng /var/opt/lib/pe- puppet/state/graphs/resources.dot -o /tmp/configuration.pngStill sounds risky? --noopEven the CAB will like you!
  53. 53. Puppet Enterprise, why do people choose it?Many of the tools are pre-compiledLow stress, upgradable solutionCeiling install...peace of mindDashboardMcollectiveCloud Provisioner - AWS & VMWareSupportable :)So what your next step?
  54. 54. Start small, and KISS!“A journey of a 1000 miles...” A.N. Other Smart PersonJust start!Download the training VMDo the tutorialsUse the Forge for inspiration!Use Google Groups – puppet-users@googlegroups.comto start...then puppet-dev@ when confidence starts to growUse the supporting toolsetsUse –noop! Derisk n smilePadding on the right shoulder might also be useful...
  55. 55. A Quick Recap Ops Manager Goals...revisitedAvailabilityScalabilitySupportability (style, newbies and secure commercial)PredictabilityAgility (all systems, now!)CloudabilityKeepTheCostsDownAbility...Smiling end uses (board happy too!)Legacy migration?
  56. 56. A Quick Recap Sysadm Goals...revistedPredictable / stable infrastructureReusable wins...version controlled, reusable modulesFix stuff once...what a concept :)Reduce support workload, so can finally get to that more interesting project stuff!Monitoring...that someone else maintains ….well, sortaDocumentation...dittoSmiling end users, resulting in...Weekends / sleep
  57. 57. “Were not proggies... were sys admins!”Hmmmm…The good news is that there are plenty options!Retooling, or....
  58. 58. ReferencesLearn Puppet in small chunks... http://info.puppetlabs.com/download-pdfs.html - leading to http://puppetlabs.com/misc/pdf-doc/http://forge.puppetlabs.com/http://theforeman.orghttp://docs.puppetlabs.com/guides/tools.htmlpuppet-rspec - easy intro http://puppetlabs.com/blog/the-next- generation-of-puppet-module-testing/ Puppet for sysadmins...
  59. 59. Get to the pub by 5 Who spotted the cuttlefish? ps. We are hiring!Contact: stephen.wallace @icesystems.com.au