A Presentation about Puppet that I've made at the OSSPAC conference
Upcoming SlideShare
Loading in...5
×
 

A Presentation about Puppet that I've made at the OSSPAC conference

on

  • 6,795 views

 

Statistics

Views

Total Views
6,795
Views on SlideShare
6,775
Embed Views
20

Actions

Likes
5
Downloads
221
Comments
0

3 Embeds 20

http://www.slideshare.net 15
http://www.techgig.com 3
http://basavrajb.blogspot.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • So, how do we do handle this process today At first, we need to input the server details in a few different locations (CMDB, DHCP, DNS etc)
  • So, how do we do handle this process today At first, we need to input the server details in a few different locations (CMDB, DHCP, DNS etc)
  • Some sites have a very optimized process, Effort involved Time, new hardware, downtimes, patching/updating, installation of new services (global rollouts….) TESTING Today, there is no easy way for a global team to push changes – e.g. clearcase team needs to ask each operation team to implement the changes “auditing” is a manual process – no transparency On a global scale, we don’t really know the configuration of each site SOX etc
  • Database is currently centralized. Performance issues? DNS – A, PTR, CNAME, TXT. DHCP – BOOT host, file, Vendor classes of sparc. Replaces RARP and bootparams LOGS – Updates the Gini DB with live data LSF – Opens and closes hosts during the build TFTP – manipulates the boot link based upon the host status CERTIFICATES – cleans old certificates and allows autosign CMDB – Updates the CMDB with live data

A Presentation about Puppet that I've made at the OSSPAC conference A Presentation about Puppet that I've made at the OSSPAC conference Presentation Transcript

  • Infrastructure Automation Ohad Levy – Infineon Technologies [email_address]
  • Automated
    • ”… a process which may once have been performed manually but has been altered in some way which allows a machine or computer to either wholly or partially manipulate the process to save time”
  • Infrastructure
    • “ Infrastructure is generally a set of interconnected structural elements that provide the framework supporting an entire structure”
    View slide
  • Automated Infrastructure
    • “ Having the basic services necessary for your infrastructure to operate largely without the aid of a keeper.”
    View slide
  • Grow – from This: http://i68.photobucket.com/albums/i2/RealConnections/googlecomputers.jpg
  • Large Scale Environment
  • Where does it fit in?
  • Typical System Lifecycle Pre/ Installation DNS/DHCP … Initial Configuration Kickstart Jumpstart Custom Scripts Fixes Updates Audits
  • System configuration possibilities
    • Manual System Configuration
      • „ log in and „just do it!““
        • + OK for small number of servers
        • + Quick and easy
        • - Very difficult to ensure similar configurations
        • - No auditing capability
        • - No change history
        • - No release documentation
        • - No way to deploy multiple servers quickly.
        • - No way to rebuild critical servers quickly.
  • System configuration possibilities
    • Install Time Auto-Configuration
      • „ Deploy using Kickstart, Jumpstart etc“
        • + Ensures that new systems are brought into proper state as part of installation process
        • + Possible to deploy multiple servers quickly
        • - No Validation that settings remain correct
        • - No easy way to deploy changes
        • - Auto-Configuration process is different between the sites  leads to different configurations
        • - No Auditing
        • - No change history
        • - No release documentation
  • Typical System Lifecycle Pre/ Installation DNS/DHCP … Initial Configuration Kickstart Jumpstart Custom Scripts Fixes Updates Audits Puppet
  • Puppet
    • What is Puppet ?
      • A product designed to deploy system configurations. It is –
        • Open source based on Ruby
        • Policy based
        • Runs every 30 minutes
        • An abstraction layer between the system administrator and the system
        • Capable to run on any UNIX operating system
  • Our Challenges
    • Keep our systems "harmonized“
    • Audit / Know what's going on each system
    • Replace a server if it dies or to be able to add another server that is exactly like it
    • Similar Applications, different OS's
    • Push out changes to all the servers that need a particular change
    • Stop duplicating effort
    • Go home early
  • How does Puppet works?
  • Puppet Types
    • A Type is a particular element that Puppet knows how to configure
    • Files (content, permissions, ownership)
    • Packages (installations, updates..)
    • Services (enabled/disabled, running/stopped)
    • Exec (run commands)
    • Full List: cron, exec, file, filebucket, group, host, interface, k5login, mailalias, maillist, mount, nagios*, package, service, sshkey, tidy, user, yumrepo, zone
  • Example: Managing sudoers file
    • file { /etc/sudoers:
    • owner => root,
    • group => root,
    • mode => 600,
    • source => puppet://server/files/sudoer
    • }
  • Dependencies
    • " require " and " before " / " after " settings ensures that types are
    • applied in the correct order
    • file { "/etc/sudoers":
    • ...
    • require => Package[sudo]
    • }
    • package { "sudo":
    • ensure => present,
    • before => File["/etc/sudoers"]
    • }
  • Dependencies - continued
    • " notify " and " subscribe " settings can trigger cascaded updates
    • Particularly useful in services, exec
    • file { "/etc/ssh/sshd_conf":
    • ...
    • notify => Service["sshd"]
    • }
    • service { "sshd":
    • subscribe => File["/etc/ssh/sshd_conf“]
    • }
  • What is Facter?
    • Facter gathers information about the client, which can be used as variables within puppet.
    • You can add custom facts as needed.
    • package {"sshd":
    • ensure => installed,
    • name => $operatingsystem ? {
    • solaris => "IFKLssh",
    • default => “openssh-server”
    • }
    • }
  • Example Facts
    • $ sudo facter
    • architecture => amd64
    • domain => sin.infineon.com
    • facterversion => 1.3.8
    • fqdn => sinn1636.sin.infineon.com
    • hardwaremodel => x86_64
    • hostname => sinn1636
    • ipaddress => 172.20.89.113
    • kernel => Linux
    • kernelrelease => 2.6.24-21-generic
    • lsbdistcodename => hardy
    • lsbdistdescription => Ubuntu 8.04.1
    • lsbdistid => Ubuntu
    • lsbdistrelease => 8.04
    • macaddress => 00:1c:25:14:26:ab
    • manufacturer => LENOVO
    • max speed => 2000 MHz
    • maximum memory module size => 4096 MB
    maximum total memory size => 8192 MB memoryfree => 988.30 MB memorysize => 1.94 GB operatingsystem => Debian operatingsystemrelease => 2.6.24-21-generic processor0 => Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz processor1 => Intel(R) Core(TM)2 Duo CPU T7300 @ 2.00GHz processorcount => 2 productname => 6458B43 ps => ps -ef puppetversion => 0.24.5 rubysitedir => /usr/local/lib/site_ruby/1.8 rubyversion => 1.8.6 serialnumber => L3A8632 swapfree => 912.23 MB swapsize => 956.99 MB type => Notebook uniqueid => 007f0101 vendor => LENOVO
  • What is a Class?
    • A named collection of type objects
    • Can include or inherit from other classes
    • class sudo_class {
    • include foo_class
    • file { "/etc/sudoers":
    • ...
    • }
    • package { "sudo":
    • ...
    • }
    • }
  • Class inheritance
    • class afile {
    • file { "/tmp/foo":
    • ensure => file
    • source => "/src/versionA"
    • }
    • }
    • class another_file inherits afile {
    • File ["/tmp/foo"] {
    • source => "/src/versionB"
    • }
    • }
  • What is a Node ?
    • A configuration block matching a client
    • Can contain types, classes
    • "default" node matches any client without a node block
    • node "ohad.myself" {
    • include sudo_class
    • include other_class
    • }
  • External Nodes
    • Node definitions can be defined outside of puppet – e.g. LDAP, external script…
    • Ideal when you have too many nodes..
  • Classes and definitions
    • Classes are groups of resources.
    • Definitions are similar to classes, but they can be instantiated multiple times with different arguments on the same node.
    • class apache2 {
    • define simple-vhost ( $admin = "webmaster", $aliases, $docroot) {
    • file { "/etc/apache2/sites-available/$name":
    • mode => "644",
    • require => [ Package["apache2"], Service["apache2"] ],
    • content => template ("apache/vhost.conf"),
    • }
    • }
    • }
  • Classes and definitions - Continued
    • node mywebserver {
    • include apache2
    • apache2::simple-vhost { "test.example.com":
    • docroot => "/var/www/test“
    • }
    • apache2::simple-vhost { "debian.example.com":
    • docroot =>"/var/www/debian“,
    • alias => [“debiantest.example.com”,”debian”],
    • admin => “system@example.com”
    • }
    • }
  • vhost.conf template
    • Puppet uses Ruby's ERB template system:
    • <VirtualHost *>
    • ServerAdmin <%= admin %>
    • DocumentRoot <%= docroot %>
    • ServerName <%= name %>
    • <% aliases.each do |al| -%>
    • ServerAlias <%= al %>
    • <% end -%>
    • ErrorLog &quot;|/usr/bin/cronolog /var/log/apache/<%=
    • name %>/%Y-%m/error-%d&quot;
    • CustomLog &quot;|/usr/bin/cronolog /var/log/apache/<%=
    • name %>/%Y-%m/access %d&quot; sane
    • </VirtualHost>
  • Template output
    • more /etc/apache2/sites-available/debian.example.com
    • <VirtualHost *>
    • ServerAdmin [email_address]
    • DocumentRoot /var/www/debian
    • ServerName debian.example.com
    • ServerAlias debiantest.example.com
    • ServerAlias debian
    • ErrorLog |/usr/bin/cronolog
    • /var/log/apache/debian.example.com/%Y-%m/error-%d“
    • CustomLog &quot;|/usr/bin/cronolog
    • /var/log/apache/debian.example.com/%Y-%m/access-%d“ sane
    • </VirtualHost>
  • OS API - It also works the other way around:
    • $ ralsh user levyo
    • user { 'levyo':
    • groups => ['tidc'],
    • ensure => 'present',
    • password => 'x',
    • shell => '/bin/bash',
    • uid => '49960',
    • comment => 'Ohad Levy, +65 xxxx xxxx',
    • gid => '11100',
    • home => '/home/levyo'
    • }
  • Puppet Modules
    • Modules allow you to group both the logic and the files for an application together.
    • Puppet automatically searches its module path to find modules.
    • Modules can contain four types of files, each of which must be stored in a separate subdirectory:
      • Manifests – must be stored in manifests/, and if you create manifests/init.pp then that file will be loaded if you import that moudle name directly (e.g. import “mymodule”)
      • Templates – must be stored under templates/, put in your manifest some where template(“mymodule/mytemplate”)
      • Files – stored in files/, can be accessed as: source => puppet://”mymodule”/myfile
      • Plugins – Additional types, provides or facts
  • File server and File Bucket
    • Puppet also includes a file server which you can use for transferring files from the server to the client.
    • If you configure it, puppet can also save a backup of each file that is changed on the client to the server. The backups go in a filebucket and can be retrieved later.
  • Getting Started
    • Install puppet (yum,pkg-get,…) or manually install ruby, facter and puppet
    • Setup a puppet server (puppetmaster / puppet-server).
    • Write a manifest for your node (classes etc)
    • Start puppet master on the server
    • Start puppetd on the client
  • Some more info
    • Puppet uses SSL for all communications, therefor it includes a CA, you can automatically sign CSR or use puppetca tool to mange them.
    • Storeconfig, option to save machine states(facts, configuration runs) and special facts (e.g. SSH keys) in a database.
    • Reporting, puppet has a few reporting options, most common are emails with changes, RRD files, yaml files and puppetshow web interface.
    • Puppet HA, load balancing etc
  • What are the benefits ?
    • Costs
      • Based on open source tools – No license cost
      • Reduced Labor  Reduced Cost
    • Productivity
      • Reproducibility, Accuracy and homogenous environment
      • (Reduce needed knowledge to operate our environment)
      • Allow easy ramp up of new sites and services
    • Change Management
      • Testing Processes, Auditing
      • Global control over servers
      • Harmonization  Consolidation
  • Conclusions
    • We're all stuck on the hamster wheel
    • Makes easy stuff easier, hard stuff possible
    • Harmonize! (not just today, also in the future)
    • Enable the business for consolidation
    • Similar projects
      • cfengine
      • bcfg2
    • Additional Resources
      • http://reductivelabs.com/trac/puppet
      • http://reductivelabs.com/trac/puppet/wiki/LanguageTutorial
      • http://reductivelabs.com/trac/puppet/wiki/CompleteConfiguration
    • #puppet on irc.freenode.org
  • Backup Slides Additional information
  • Configuration DB - Terminology
    • Module – A bundle of settings related to a single application
    • At Infineon we use 3 types of modules:
      • Host Type - e.g. Login Servers, Compute nodes…
      • Service - module which is needed by other modules, e.g. ssh, x11 etc.
      • Sites Modules - sites customizations
    • Versioning – The ability to refer (tag) to a certain state/PiT
      • Host Type and Service modules are associated with a version, e.g. Login Server v1.0
      • Site Modules are not tagged in order to reduce complexity
    • * Every change in a module, requires a check in action, which is identified by a user, and a comment (change log)
    • * A user access restriction should be applied to each module and to the tagging mechanism
  • Configuration DB - Terminology
    • Environments – The ability to select a group of modules in a certain version
    • Each site is associated with at least two environments – Production and Testing:
      • Testing – an environment which refer to versioned modules under a test tag.
      • Production – A selection of modules in a certain version that has already been approved for Production usage.
    • Development environment – The state where the modules are in the latest check-in version, but are not tagged (e.g. not a testing candidate yet).
    • Development  Testing  Production (Per Host Type)
  • Configuration DB - Terminology
    • Global modules / owners
      • Each owner is responsible for his module (or modules)
      • A special attentions is required for Service module as many other modules might depend on them
    • Site modules
      • As Site modules are not tagged (versioned) any change in the site specific settings, will be rolled out immediately.
  • Risks & Challenges
    • If the right processes are not in place, it would be possible to implement the wrong changes globally
    • Learning Curve
      • How our platform works
      • How to use version control
      • How to work in the new processes/workflows
    • Fear (Am I obsolete?, will I continue to learn?)
  • Why Did we choose Puppet
    • Open source
    • More flexible than other open source alternatives
    • Really works (Declarative language, Abstracts configuration as resources, Allows relationships, Transactional, Modular)
    • Provides an API to the OS, making it very simple to support other OS’s in the future
    • Puppet is now being adopted in many projects
    • For more info see http://reductivelabs.com/trac/puppet/wiki/CfengineVsPuppet
    • http://reductivelabs.com/trac/puppet/wiki/WhosUsingPuppet
  • Datacenters Challenges
    • Harmonization
    • Simplify Servers management
    • Implement and maintain Global standards
    • Accurate Configuration Database
    • Rapid recovery and deployment of services globally
    • Eliminates duplicate / unnecessary effort
    • Same look and feel everywhere (enables consolidation / global operation )
    • Audits / Compliance
  • Infineon Puppet Infrastructure
    • Puppeteer (Puppet Master of site PM’s)
    • Puppet Master per site (or based on size/load)
    • SQL Database for Web portal, Inventory
    • Configuration database (Version control & Ticketing system)
  • Gini’s Features
    • AD/LDAP Authentication
    • DHCP/DNS Connectors (API, + MS(yacks) DHCP, MS(yacks) DNS and infoblox)
    • Puppet Host type management
    • Disk Layout Management
    • Host Hardware management (needed for Solaris DHCP jumpstart, and general host info)
    • Multi OS management (e.g. RH, CentOs, Solaris.... in their various versions)
    • User role management (who is allowed to install, in which site, which host type etc)
    • Subnet and IP Management (auto assignment of IP address, DHCP reservation DNS records etc)
    • Multi Domain / site management
    • Hands-free auto OS installation (currently Kickstart and Jumpstart (sparc) using DHCP)
    • Management of Puppet Environments, choose an env per host (e.g. we have different production states for different sites, or for different host types).
    • Management of puppet certificates (also support chained certificates).
    • Import of Machine Facts (we cant use store config due to the large setup of our env)
    • Nice Graphs (with Gruff)
    • Puppet modules selection per host(for those who needs more than the default host type)
    • SP (Service Processor, iLOM etc) management
    • TFTP management
    • Status indications for machine state (pre installation, during puppet first run, etc)
    • ... and probably a lot of other small stuff that I cant think of now
  • Our Solution Web Portal Puppet Configuration DB Inventory DHCP/DNS .. Harmonized
  • Where does GINI live? GINI DB DNS TFTP CERTIFICATES CMDB DHCP LOGS LSF Future work