Puppet getting started by Dirk Götz
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Puppet getting started by Dirk Götz

  • 1,223 views
Uploaded on

Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will......

Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will also cover an exemplary design for scaling the puppet master and for development livecycle of modules. Furthermore an example for design of modules will be given.

More in: Technology , Spiritual
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,223
On Slideshare
1,223
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
29
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. www.netways.de // blog.netways.de // @netways PUPPET GETTING STARTED 27 NOVEMBER 2013 | PUPPET CAMP DIRK GÖTZ | NETWAYS GMBH Make IT do more with less
  • 2. www.netways.de // blog.netways.de // @netways AGENDA ■ Brief introduction ■ Configuration management ■ Components ■ Design your environment ■ Design your workflow ■ Design your module Make IT do more with less
  • 3. www.netways.de // blog.netways.de // @netways BRIEF INTRODUCTION Make IT do more with less
  • 4. www.netways.de // blog.netways.de // @netways BRIEF INTRODUCTION TO NETWAYS • Founded in 1995 • Open source since 1997 • 40 employees • Specialised in open source systems management and open source data center infrastructure Make IT do more with less
  • 5. www.netways.de // blog.netways.de // @netways NETWAYS CONFERENCES Puppet Camp 2014 • 11 April 2014 • 110 attendees (April 2013) • 170 attendees (November 2014) Open Source Data Center Conference • 8 – 10 April 2014 • 120 attendees (2013) • 2 tracks of presentations & workshops Make IT do more with less
  • 6. www.netways.de // blog.netways.de // @netways CONFIGURATION MANAGEMENT Make IT do more with less
  • 7. www.netways.de // blog.netways.de // @netways CONFIGURATION MANAGEMENT Types of „Automation“ Manual configuration Golden images Self-made scripts Software solutions for i in $(cat host.cfg) do ssh user@$i uname -a done Make IT do more with less
  • 8. www.netways.de // blog.netways.de // @netways THE PUPPET WAY 1. Define 2. Simulate 4. Report 3. Enforce Current State Desired State Make IT do more with less
  • 9. www.netways.de // blog.netways.de // @netways THE PUPPET WAY - DEFINE ■ Using Puppet Domain Specific Language (DSL) ■ Describe relationship between resources ■ Create modular reuseable code Make IT do more with less
  • 10. www.netways.de // blog.netways.de // @netways THE PUPPET WAY - SIMULATE ■ Simulate deployment ■ Without changes to your system ■ Test and fix it before enforcing $ sudo puppet apply ntp.pp --verbose --noop Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb Info: Loading facts in /var/lib/puppet/lib/facter/vmware.rb Info: Loading facts in /var/lib/puppet/lib/facter/last_run.rb Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb Info: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rb Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb Error: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomain Error: Could not find template 'ntp/ntp.conf.erb' at /tmp/ntp.pp:9 on node puppet.localdomain Make IT do more with less
  • 11. www.netways.de // blog.netways.de // @netways THE PUPPET WAY - ENFORCE ■ Compares current state and desired state ■ Changes to desired state Current State Desired State ■ Idempotency $ sudo puppet agent -t Info: Retrieving plugin Info: Loading facts in /var/lib/puppet/lib/facter/root_home.rb Info: Loading facts in /var/lib/puppet/lib/facter/pe_version.rb Info: Loading facts in /var/lib/puppet/lib/facter/vmware.rb Info: Loading facts in /var/lib/puppet/lib/facter/last_run.rb Info: Loading facts in /var/lib/puppet/lib/facter/facter_dot_d.rb Info: Loading facts in /var/lib/puppet/lib/facter/puppet_vardir.rb Info: Loading facts in /var/lib/puppet/lib/facter/puppi_projects.rb Info: Loading facts in /var/lib/puppet/lib/facter/concat_basedir.rb Info: Caching catalog for puppet.localdomain Info: Applying configuration version '1384768191‚ Notice: Finished catalog run in 0.54 seconds Make IT do more with less
  • 12. www.netways.de // blog.netways.de // @netways THE PUPPET WAY - REPORT ■ Report changes and metrics ■ Many different presentation formats Make IT do more with less
  • 13. www.netways.de // blog.netways.de // @netways COMPONENTS – BASIC TOOLS Make IT do more with less
  • 14. www.netways.de // blog.netways.de // @netways PUPPET WORKFLOW SSL-Encryption Make IT do more with less
  • 15. www.netways.de // blog.netways.de // @netways FACTER ■ Open source Ruby library by Puppet Labs ■ Provides information on your host ■ CLI returns key-value pairs $ facter architecture => x86_64 augeasversion => 0.9.0 blockdevice_sr0_model => QEMU DVD-ROM blockdevice_sr0_size => 1073741312 blockdevice_sr0_vendor => QEMU blockdevices => sr0,vda domain => localdomain facterversion => 1.7.3 filesystems => ext4,iso9660 fqdn => puppet.localdomain $ facter osfamily RedHat Make IT do more with less
  • 16. www.netways.de // blog.netways.de // @netways PUPPET ■ Open source Ruby library by Puppet Labs ■ Client server based • REST-API • X509 certificates ■ Platform independent • Puppet Domain Specific Language • Providers for different platforms Make IT do more with less
  • 17. www.netways.de // blog.netways.de // @netways PUPPET DOMAIN SPECIFIC LANGUAGE ■ Abstracts resources ■ Resource types • • • • • user, group, file, package, service, exec, …and many more possible to create your own ■ Type and title/name pairs must be unique Make IT do more with less
  • 18. www.netways.de // blog.netways.de // @netways PUPPET PROVIDERS ■ Resource Abstraction Layer • Resource types • Providers Make IT do more with less
  • 19. www.netways.de // blog.netways.de // @netways PUPPET – MORE ABSTRACTION ■ Class groups resources in one manifest ■ Syntax constructs possible Make IT do more with less
  • 20. www.netways.de // blog.netways.de // @netways PUPPET – SYNTAX CONSTRUCTS ■ Variables (but are more like constants) ■ Logical structures • selectors • case statements • if/elseif/else statements ■ Dependencies • require/before • subscribe/notify ■ Inheritance Make IT do more with less
  • 21. www.netways.de // blog.netways.de // @netways PUPPET – MORE ABSTRACTION ■ Module groups classes and corresponding files modulename |-- files – static files |-- lib – custom facts, functions, resources |-- manifests – manifests containing classes |-- spec – tests for rspec-puppet |-- templates – dynamic files |-- tests – examples declaring classes Make IT do more with less
  • 22. www.netways.de // blog.netways.de // @netways PUPPET FILESERVER ■ File transfer from master • puppet:///modules/modulename/filename ■ “Magic“ mount point for modules ■ Additional mount points possible ■ Recursion for directories possible Make IT do more with less
  • 23. www.netways.de // blog.netways.de // @netways PUPPET PLUGIN SYNC ■ Option on the agent ■ Syncs from all modules • custom facts • custom resources (types and providers) ■ Before the actual Puppet run occurs Make IT do more with less
  • 24. www.netways.de // blog.netways.de // @netways PUPPET TEMPLATES ■ Function executed on server ■ ERB (Plain text with embedded Ruby) • • • • • variables (including facts) conditionals iterations access to tags and classes usage of Puppet functions ■ Combination of multiple templates possible Make IT do more with less
  • 25. www.netways.de // blog.netways.de // @netways PUPPET NODE DECLARATION ■ Connects system (node) with functionality (classes) • Single hosts • Mulitiple hosts (by regular expression) • Default • Inheritance possible Make IT do more with less
  • 26. www.netways.de // blog.netways.de // @netways COMPONENTS – BASIC GUI Make IT do more with less
  • 27. www.netways.de // blog.netways.de // @netways BASIC GUI – ADDED FUNCTIONALITY ■ Reporting target • Collects reports • Graphical presentation ■ External Node Classifier (ENC) • Graphical node declaration • Adds groups Make IT do more with less
  • 28. www.netways.de // blog.netways.de // @netways PUPPET - REPORTING ■ Enable on agent ■ Choose target on server • • • • • • http/https log tagmail store rrdgraph puppetdb ■ Multiple targets possible ■ Create your own Make IT do more with less
  • 29. www.netways.de // blog.netways.de // @netways PUPPET - ENC ■ External source for node declaration • Script returning yaml • LDAP ■ Merged with internal node declaration $/etc/puppet/node.rb puppet.localdomain --environment: production parameters: foreman_env: production owner_name: Admin User domainname: "" owner_email: root@localdomain root_pw: $1$default$hCkak1kaJPQILNmYbUXhD0 puppetmaster: "" classes: profiles::default: dn: cn=testserver,ou=Hosts,dc=madstop,dc=com objectClass: device objectClass: ipHost objectClass: puppetClient objectClass: top cn: testserver environment: testing ipHostNumber: 192.168.0.50 description: My test server l: dc1 puppetClass: testing puppetVar: owner_name=„Admin user“ Make IT do more with less
  • 30. www.netways.de // blog.netways.de // @netways PUPPET DASHBOARD ■ Open source ruby web interface • Version <= 1.2 by Puppet Labs • Version >= 2 by community http://puppetlabs.com/presentations/story-dashboard-20 Make IT do more with less
  • 31. www.netways.de // blog.netways.de // @netways COMPONENTS – ORCHESTRATION Make IT do more with less
  • 32. www.netways.de // blog.netways.de // @netways ORCHESTRATION ■ Centralized execution on multiple systems ■ Different solutions • • • • • • ssh loops func fabric capistrano mcollective … and many more Make IT do more with less
  • 33. www.netways.de // blog.netways.de // @netways MCOLLECTIVE ■ Open source Ruby framework by Puppet Labs ■ Facter for grouping ■ Middleware for communication $ mco package status httpd -F kernel=Linux * [ ============================================================> ] 3 / 3 centos63a ubuntu1204a master version = httpd-2.2.15-15.el6.centos.1 version = httpd-purged version = httpd-purged ---- package agent summary ---Nodes: 3 / 3 Versions: 1 * 2.2.15-15.el6.centos.1, 2 * purged Elapsed Time: 0.12 s Make IT do more with less
  • 34. www.netways.de // blog.netways.de // @netways PUPPET ENTERPRISE CONSOLE ■ Ruby web interface based on dashboard • Auditing • Live management Make IT do more with less
  • 35. www.netways.de // blog.netways.de // @netways COMPONENTS – DEPLOYMENT Make IT do more with less
  • 36. www.netways.de // blog.netways.de // @netways DEPLOYMENT ■ Config management needs running agent ■ (Agent needs running operating system) ■ Different solutions: • • • • • • Kickstart / Autoyast / Preseed / Jumpstart Cobbler / kickstand Razor Puppet Cloud Provisioner Foreman … and many more Make IT do more with less
  • 37. www.netways.de // blog.netways.de // @netways RAZOR ■ Open source Ruby library by Puppet Labs/EMC ■ Client • micro-kernel for PXE boot ■ Server • REST-API • CLI ■ Policy connects • host profiles • operating system • config management { "name": "centos-for-small", "repo": { "name": "centos-6.4" }, "installer": { "name": "centos" }, "broker": { "name": "noop" }, "enabled": true, "hostname": "host${id}.example.com", "root_password": "secret", "max_count": "20", "rule_number": "100", "tags": [{ "name": "small", "rule": ["<=", ["num", ["fact", "processorcount"]], 2]}] } Make IT do more with less
  • 38. www.netways.de // blog.netways.de // @netways CLOUD PROVISIONER ■ Open source Ruby library by Puppet Labs ■ Instantiates cloud instances • • • • EC2 VMware OpenStack Google Compute Engine ■ Installs Puppet using SSH Make IT do more with less
  • 39. www.netways.de // blog.netways.de // @netways FOREMAN ■ Open source Ruby web interface by Ohad Levy (Red Hat) • Reporting target • ENC • Provisioning Make IT do more with less
  • 40. www.netways.de // blog.netways.de // @netways FOREMAN – SMART PROXIES ■ Connects GUI and Backends • • • • • puppet puppetca tftp dhcp dns Make IT do more with less
  • 41. www.netways.de // blog.netways.de // @netways FOREMAN – COMPUTE RESOURCES ■ Integrates virtualization and cloud platforms • • • • • • • Libvirt oVirt / RHEV VMware EC2 Google Compute Engine OpenStack Rackspace Make IT do more with less
  • 42. www.netways.de // blog.netways.de // @netways COMPONENTS – MORE FEATURES Make IT do more with less
  • 43. www.netways.de // blog.netways.de // @netways EXPORTED RESOURCES ■ One node creates resource ■ Another node realizes that resource ■ Resource needs to be stored: • Stored configs (deprecated) • PuppetDB ■ Use cases: • • • • host entries sshkey management monitoring / backup other centralized services Make IT do more with less
  • 44. www.netways.de // blog.netways.de // @netways PUPPETDB ■ PostgreSQL and Java based data warehouse solution by Puppet Labs ■ Collects • Facts • Catalogs • Reports (optional) ■ Used for • Inventory service • Exported resources Make IT do more with less
  • 45. www.netways.de // blog.netways.de // @netways HIERA ■ Open source Ruby library by Puppet Labs ■ Hierarchical data look-up separates code and data ■ Integrated in Puppet >3 / addon for Puppet <3 $cat hiera.yaml --:backends: - yaml :yaml: :datadir: /etc/puppet/hieradata :hierarchy: - `hosts/${::fqdn}´ - `location/${::location}´ - common $cat hosts/specialhost.localdomain ntp::server = 192.168.23.23 $ cat location/rz2 ntp::server = 192.168.2.23 yum:mirror = 192.168.2.42 $ cat common ntp::server = 192.168.0.23 yum::mirror = 192.168.0.42 proxy::server = [`192.168.0.237´,`192.168.0.238´] Make IT do more with less
  • 46. www.netways.de // blog.netways.de // @netways COMPONENTS – SOFTWARE MANAGEMENT Make IT do more with less
  • 47. www.netways.de // blog.netways.de // @netways SOFTWARE MANAGEMENT ■ Provider needs defined sources for software ■ Local mirror reduces traffic ■ Many different tools • • • • • rsync / createrepo updian Spacewalk / Red Hat Satellite / Suse Manager pulp / katello … and many more Make IT do more with less
  • 48. www.netways.de // blog.netways.de // @netways PULP ■ Open source Python repository management by Red Hat ■ Server: • Import and upload of content • rpm and puppet modules • Publish web-based or ISO images ■ Client: • Server-side management and reporting Make IT do more with less
  • 49. www.netways.de // blog.netways.de // @netways KATELLO ■ Java web interface by Red Hat ■ Combines: • candlepin (subscription management) • pulp (software management) • Foreman + Puppet (configuration management) Make IT do more with less
  • 50. www.netways.de // blog.netways.de // @netways DESIGN YOUR ENVIRONMENT Make IT do more with less
  • 51. www.netways.de // blog.netways.de // @netways CHOICES ■ Support / packages needed? ■ Best method to report? ■ Resources to export? ■ Deployment needed? ■ Software management needed? ■ Node declaration or ENC? Make IT do more with less
  • 52. www.netways.de // blog.netways.de // @netways SCALE UP ■ Puppet easily scales up ■ Start simple, grow with your environment ■ Rule of thumb: number of nodes * catalog compile time (in seconds) number of masters = -----------------------------------------------------------------------------cores per master * run interval (in seconds) Make IT do more with less
  • 53. www.netways.de // blog.netways.de // @netways SCALE UP ■ Only one certficate authority • Only needed for registration ■ Load balance Puppet traffic ■ GUI only required for users • But ENC / Reporting always required ■ Orchestration Make IT do more with less
  • 54. www.netways.de // blog.netways.de // @netways SCALE UP – EXAMPLE: PUPPET ENTERPRISE Make IT do more with less
  • 55. www.netways.de // blog.netways.de // @netways DESIGN YOUR WORKFLOW Make IT do more with less
  • 56. www.netways.de // blog.netways.de // @netways EDITOR ■ vim – text editor • • • • vim-puppet – synthax highlight tabular – style guide conformity puppet-lint – style guide conformity syntastic – validation ■ Geppetto – eclipse based IDE synthax hightlight, style guide conformity, module creation Make IT do more with less
  • 57. www.netways.de // blog.netways.de // @netways STYLEGUIDE ■ Official guide http://docs.puppetlabs.com/guides/style_guide.html • • • • readability work with and without features simple and robust shareable / useable by others ■ Create your own • based on official guide • make your own rules Make IT do more with less
  • 58. www.netways.de // blog.netways.de // @netways TESTING ■ puppet parser validate – syntax ■ puppet-lint – style guide conformity ■ puppet apply --noop – simulate ■ vagrant – simulate ■ rspec-puppet – expected results Make IT do more with less
  • 59. www.netways.de // blog.netways.de // @netways VERSION CONTROL SYSTEM ■ Pre-Commit / Pre-Receive • use for validation and review ■ Post-Commit • checkout in environment ■ Versioning for change management ■ Solutions: • Version control: svn, git, bazaar • Validation & Review: gerrit, jenkins/hudson Make IT do more with less
  • 60. www.netways.de // blog.netways.de // @netways STAGING ■ Puppet environments • Different versions of modules ■ Use your own versioning • config_version: script returns version string Make IT do more with less
  • 61. www.netways.de // blog.netways.de // @netways DOCUMENTATION ■ Inline • RDoc markup • Console output • HTML generation ■ README • Markdown ■ Modulefile • used by Puppet module tool Make IT do more with less
  • 62. www.netways.de // blog.netways.de // @netways DESIGN YOUR MODULE Make IT do more with less
  • 63. www.netways.de // blog.netways.de // @netways CONSIDERATIONS ■ Where to start? ■ Use cases? • • • • different platforms different roles small adjustments one-time or multiple objects ■ ‚Part of it‘ or separate? • Dependencies ■ Templates or files? ■ Write your own? Make IT do more with less
  • 64. www.netways.de // blog.netways.de // @netways PUPPET FORGE ■ Community platform for modules • Web platform • Command line tool ■ Module information: • • • • • Author Project homepage and issue tracker Tags Releases and download count Test results Make IT do more with less
  • 65. www.netways.de // blog.netways.de // @netways PUPPET FORGE – EXAMPLE example42 ■ 96 modules on forge / more on github ■ always same layout ■ supports: Red Hat, CentOS, Fedora, Ubuntu, Debian, Mint, SLES, OpenSuSE, FreeBSD ■ adds monitoring, firewall and puppi (own script library) Make IT do more with less
  • 66. www.netways.de // blog.netways.de // @netways MODULE DESIGN – MORE ABSTRACTION ■ Classes abstract resources ■ Modules abstract classes ■ Nodes contain the logic Not maintainable! node 'basil.puppetlabs.vm' { class { 'apache’: version => 'latest’, } class { 'motd': } class { 'ssh’: } if $::operatingsystem == 'solaris' { class { 'users’: default_shell => '/bin/false’, } } else { class { 'users': } } Class['ssh’] -> Class['users’] } Make IT do more with less
  • 67. www.netways.de // blog.netways.de // @netways MODULE DESIGN – MORE ABSTRACTION ■ Classes abstract resources ■ Modules abstract classes ■ Profiles contain the logic • abstract the modules • separate implementation from technology class profiles::application { include tomcat include mysql include componenta } class profiles::application::x inherits profiles::application { include componentb componentb::resource { 'name': ensure => present, } } class profiles::application::y inherits profiles::application { include componentc include componentd } class profiles::application::z inherits profiles::application { include componentb include componentd include dependency Class['dependency'] -> Class['componentd'] } Make IT do more with less
  • 68. www.netways.de // blog.netways.de // @netways MODULE DESIGN – MORE ABSTRACTION ■ Classes abstract resources ■ Modules abstract classes ■ Profiles contain the logic ■ Roles contain business logic class role::webapp { include profiles::base include profiles::customapp include profiles::test_tools } • no logic, just profiles • separate business role from implementation ■ Node has exactly one role • No Puppet know-how needed for node declaration node ‘web1.example.com’ { include role:webapp } Make IT do more with less
  • 69. www.netways.de // blog.netways.de // @netways QUESTIONS & ANSWERS Make IT do more with less
  • 70. www.netways.de // blog.netways.de // @netways QUESTIONS & ANSWERS NETWAYS GmbH Deutschherrnstrasse 15-19 90429 Nürmberg Phone: +49 911 92885-0 Fax: +49 911 92885-77 Email: info@netways.de Website: www.netways.de Twitter: twitter.com/netways Facebook: facebook.com/netways Blog: blog.netways.de Make IT do more with less