Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learning Puppet Chapter 1


Published on

A book for learning puppet by real example and by building code. Chapter 1 gives you basic introduction and sets you up with a server-agent using Vagrant so that you can do hands-on.

Published in: Software
  • Login to see the comments

Learning Puppet Chapter 1

  1. 1. LEARNING PUPPET - 01 SlideBook (Book on slides) Inspired by Slidedoc - -Vishal Biyani
  2. 2. PuppetPuppet – spreading wings
  3. 3. www.vishalbiyani.comLearning Puppet Why I wrote Puppet SlideBook? Going through tons of documentation and then doing some hands on just seemed counter intuitive Puppet documentation is very good – just that I wanted to learn it gradually and relate to it while building stuff 3|•You should definitely give a shot to Puppet learning VM: •Some basic awareness of “what is Puppet used for” is assumed in tutorial. Even if you don’t have hang on! Puppet learningVM* sounded great for this, but I wanted to build it from scratch and then learn components one by one. I wanted it to be fun & concise – maximum returns on easy to digest format with minimum text! Lastly since I thought of sharing my Puppet learning experience from which others might benefit.
  4. 4. www.vishalbiyani.comLearning Puppet Get Set Get source code at clone on your machine in a convenient directory. Configure the number of agents you want to spin up and RAM you want to allocate to master & agents inVagrantfile with parameters MASTER_MEMORY & AGENT_MEMORY. Ideally keep at least 1GB RAM for server, although in first few chapters 512M is fine too. Now all you need to bring up the whole setup is fire a simple command (Provided you have done the installation suggested in box on left side) InstallVirtualBox andVagrant 4| vagrant up The setup will take some time and will do following: Download a lightweight Linux machine image and create required number of master & agent instances. (~200MB download) Master instance will be installed and configured with Puppet Master and agent instances with Puppet Agent – they will also be connected to each other. (~ 100 MB download) InstallVirtualBox andVagrant on your machine before you start. You will need to know very basics of Git – and I will introduceVagrant, but otherwise much of tutorial is self contained and Puppet oriented. A very basic and sufficient introduction of Vagrant can be found at
  5. 5. www.vishalbiyani.comLearning Puppet MASTER_MEMORY=2048 AGENT_MEMORY=256 We start by setting some parameters in beginning of script Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| config.vm.define "puppet_master" do |pmaster| Then we start configuration for the master box. We name it pmaster and rest of configs will be pmaster.something TheVagrant (Black) magic! What’s going on in Vagrant? 5| Do not worry about learning Vagrant much – our aim is to focus on Puppet. This is only for information = "centos_6_3_x86_64" "private_network", ip: "#{PUPPET_MASTER_ADDRESS}" pmaster.vm.hostname = "“ Then we define a CentOS box and we provide an IP within a private network along with a domain name pmaster.vm.provider :virtualbox do |vb| vb.customize ["modifyvm", :id, "--memory", MASTER_MEMORY] end pmaster.vm.provision "shell", path: "scripts/" end We modify the RAM as per our need and finally we call an script on newly created box.We will look at this script shortly but it basically setups the whole box for us. A simple shell script –
  6. 6. Puppet setup + Basic puppet configurationconfiguration + Playing with Puppet
  7. 7. www.vishalbiyani.comLearning Puppet Puppet Master Installation in11 lines! 1 sudo rpm -ivh 2 sudo yum -y install puppetserver 3 # We are adding Puppet labs repo to RPM and then installing it. 4 5 sudo cp /vagrant/conf/puppet.conf /etc/puppet/puppet.conf 6 # Copying a config file, which we will look in details later 7 8 sudo echo " puppet puppetmaster" >> /etc/hosts 9 # Add IP of server with domain name across all machines 10 7| 10 11 sudo iptables -A INPUT -p tcp --dport 8140 -m state --state NEW -j ACCEPT 12 sudo service iptables save 13 sudo iptables -F 14 sudo service iptables save 15 # We are opening server's port 8140 to world & flushing iptables so that they behave well! 16 17 sudo puppet master start 18 # Started Puppet master 19 20 sudo cp /vagrant/puppet_data/site.pp /etc/puppet/manifests 21 sudo echo "*" > /etc/puppet/autosign.conf 22 # Copying some more conf file - more on it later
  8. 8. www.vishalbiyani.comLearning Puppet Installing & Connecting Puppet Agent 1 sudo rpm -ivh 2 sudo yum -y install puppet 3 # Add puppet repo to list & install Puppet (Client) 4 5 sudo cp /vagrant/conf/puppet.conf /etc/puppet/puppet.conf 6 sudo echo " puppet puppetmaster" >> /etc/hosts 7 # Copying some configuration files which we will see shortly. 8 9 sudo iptables -F 8| -: A NOTE OF CAUTION :- If you are going to use sudo before every puppet command – then use it uniformly for all commands. If you are not going to – then don’t do it for any command. Due to access permissions if you don’t use sudo then all directories will be created under $HOME/.puppet. So use one and leave other to avoid confusion! But decide right now. I am going to use sudo everywhere to avoid any issues at all 9 sudo iptables -F 10 sudo service iptables save 11 # Some iptables magic - nothing to worry here 12 13 sudo puppet agent -t 14 # Test run the puppet agent
  9. 9. www.vishalbiyani.comLearning Puppet 1 Puppet.conf is a configuration file which exists on every node – be it master or agent.Typical location is /etc/puppet/puppet.conf (Or /etc/puppetlabs/puppet/puppet.conf) Puppet.conf 2 There are three sections – [main] settings applicable to all nodes, [master] has settings only for master nodes and [agent] has settings meant for agent nodes 3 The only setting we are adding to default puppet.conf right now is “server =” – so that all nodes point to the server. 9| * - the other configuration of puppet called serverless Puppet – is in which you run puppet stand alone without need for a master. We will get a basic introduction of serverless puppet towards end of this chapter 4 To ensure that “” is resolved to a valid IP – we made an entry in /etc/hosts (Recall from previous page?) 5 In a typical server-agent setup* of Puppet there will be one or more master and n number of nodes.The node has to connect to master – and authenticate itself. 6 The autosign.conf that we configured in master script has “*” which means all nodes will be auto approved as soon as they connect to master – removing need for a manual approval. Just a convenience for our test.
  10. 10. www.vishalbiyani.comLearning Puppet Let’s play with what we setup! 1 Assuming you are in same directory where Vagrantfile is present fire command: vagrant status In my case I have one master and two agents configured so I get result like following, you will get a similar result: “puppet” is the command you will use irrespective of you are running on “master” or “agent” with those names as argument for example.We used following command to start puppet server: sudo puppet master start And to test agent (In shell scripts after boxes were provisioned): sudo puppet agent –t For any help simply type command “puppet help”, for a specific command help type “puppet help command_name” 3 To know various configurations of puppet there is a handy command: sudo puppet config print But that is going to print a whole lot of configurations, so we can choose to see only specific configurations: 4 10| #: To get “vagrant ssh” working on windows seamlessly there are some hacks listed at To get into any of boxes fire a ssh command with box name. If you are on windows OS check the bottom note #. I fired following command to get into master for example: vagrant ssh puppet_master Now you are in the Linux box that we just created and can fire any command. 2 So what exactly happens when we run “puppet agent -t” Puppet is getting info from server and applying configurations to node. 5 Remember the “autosign.conf” in which we added “*” – that ensured that all agents are automatically authenticated? Want to see them?Then fire the command: sudo puppet cert list –all If we had not configured that file then you would have to manually approve the certificate request by firing command like “sudo puppet cert sign node_name”. Get more familiarity with command by firing “sudo puppet help cert” 6
  11. 11. www.vishalbiyani.comLearning Puppet Puppet Terminology in short Everything in puppet is a resource – a file, a service, a package to be installed etc. Each resource has a “type” and other attributes. For example file is a type of resource or exec is a type which can execute external commands. Puppet provides lot of types in built plus we can write our own.* Manifest is where we write our Puppet code, typically extension is “.pp”.There might be classes etc. to provide structure to our code within manifest files. Manifests are compiled to catalog and then sent to nodes for actual execution. ERB – stands for Embedded RuBy. Used in templates with embedded code. Templates can be for a configuration file and code is resolved at runtime to resource manifest ERB template 11|* - Check all types that Puppet has built in at: Templates can be for a configuration file and code is resolved at runtime to populate appropriate values. Every system has certain facts – like IP address,OS type etc. which are reported back to server and can be used in code to reduce hard coding.We can also build custom facts of our own. Module is a logical unit of puppet code & configuration – which is self contained. Typically contains classes/manifests, configuration files and templates, files needed & any other libraries/plugins etc. Think of module as a logical packaging in other languages like JAR in Java or gem in Ruby (That is oversimplified but to get the point). For example you might write a module which can install and configureTomcat – so the module will have configuration files & manifests for doing that.You can also find modules built by community on Puppet Forge. ERB template facts Much more to come!! module
  12. 12. www.vishalbiyani.comLearning Puppet Getting hands dirty with Puppet 1 We talked about resources very briefly in previous slides. Puppet has certain in built resource types which it can manage, for example a file, a service or a group and so on.To know which all types puppet has in built fire following command: sudo puppet describe –list The manifest config gives you location of site.pp – think of this as “the king manifest” - a manifest which eventually encompasses all other manifests.We will play with site.pp in coming chapters. 2 Puppet module is another useful command and for now we will look at four usages of it which are helpful to us. sudo puppet module list Will list modules already installed on your server. Fire this command to see which modules are present on your setup sudo puppet module search puppetlabs Will search for modules on puppet Forge whose name contains 3 12| sudo puppet describe –list You will get a big list of things which puppet supports natively. If you want to know more about specific type for example to know more about “host”: sudo puppet describe host Each type has three main sections – description, parameters that it can take and provider (We will see provider a bit later) At this point in time don’t worry much about knowing everything about types – but this is a good command to recall when you want to inspect a type. Will search for modules on puppet Forge whose name contains “puppelabs” sudo puppet module generate Will generate a directory structure & bare bones files so you can wrote your own module.We will do this in coming chapters sudo puppet module install <ARGUMENTS> Install a module from Puppet forge or from a archive file. If you don’t want to execute/apply any code and just want to test your code you can pass the flag “--noop” which is dry run mode.The flag goes with almost all puppet commands and gives you a kind of simulation of what is going to happen without actually changing anything on system! 4
  13. 13. www.vishalbiyani.comLearning Puppet Curious case of Puppet Apply Most of what we have seen and will see through the book is puppet master-agent way of working but puppet can work on a standalone machine without needing a puppet master – called serverless puppet and this is achieved with puppet apply command. So how does it differ? The manifests/code is typically downloaded directly from a source code repository based on role etc. of node The catalog can be applied periodically, often through a cron job. You can pass a single manifest, include modules or pass a JSON catalog generated by compiling catalogs. (Catalog can be generated on puppet master by firing command “puppet master –compile”Why would someone use serverless 13| master by firing command “puppet master –compile” To apply from a manifest and to apply from a module by including a class respectively, code would look like below: There are lot more options and I suggest you take a quick look at documentation of puppet apply. (Of course by firing command “sudo puppet help apply” ☺) 3 $ puppet apply -l /tmp/action_log.log tomcat_manifest.pp 4 $ puppet apply --modulepath=/home/dev/modules -e "include tomcat" There is a good case study on using Puppet apply or masterless puppet at Why would someone use serverless puppet instead of a master-agent puppet?The reasons can be many and some of them may not be relevant as puppet evolves more. Some of points mentioned in presentation in the footnote for example are scalability to single point of failure if Master fails etc. As always there are multiple solutions to any problem and serverless puppet can be sometimes an easy and simple solution
  14. 14. www.vishalbiyani.comLearning Puppet But this is not only it.. Facter is a system profiling library which provides facts about the node. Imagine There is much more to Puppet than meets the eye Hiera is a key/value storage tool so you can store configurable data Mcollective is a orchestration framework which allows you to run PupetDB is the storage engine used by Puppet which also provides an What we have setup so far is bare minimum basic Puppet server and agent(s) – which is great for learning Puppet as beginner. But to harness the real power there are lot more things we will learn by end of this book. 14| about the node. Imagine having to hard code IP address of system?With facter you won’t ever need to do that store configurable data and retrieve when needed so that you can avoid hard coding and make code more configurable which allows you to run commands on set of servers in real time which also provides an API. Puppet – of course is the core declarative language framework which allows you to write code for controlling platform components Puppet Enterprise combines all previous components with a powerful UI – the Puppet Console, is free for upto 10 nodes Geppetto is a IDE for puppet so that you can write Puppet code with ease Puppet Forge is repository of modules (Reusable components) written by Puppetlabs team and community at large
  15. 15. www.vishalbiyani.comLearning Puppet What did we learn? Apart from core puppet there is an ecosystem of libraries and frameworks which allow you to do a vast number of things around infrastructure provisioning, handling and maintaining.We will learn most of these one at a time in coming chapters. Typically puppet runs on master- agent model.The agent connects to master using “server” setting in puppet.conf. Puppet can also be run in serverless mode – without a server.Which means puppet library for agent/server is same. Puppet.conf is the key 15| chapters.Puppet.conf is the key configuration file for controlling various parameters. -: SOMETIMES YOU WILL HALT THOSE VAGRANT BOXES:- When you halt the vagrant boxes – and then bring back up and it might seem like nothing is working. Don’t worry follow following steps: 1) Ensure puppet master is alive, else fire command “sudo puppet master start” on master box 2) For a given agent the certificates need to be generated fresh & needs cleaning up on master. So first on master machine “sudo puppet cert clean _AGENT_NAME_” 3) Then on agent “find /home/vagrant/.puppet/ssl -name _AGENT_NAME_.pem -delete” 4) And then fire on agent “sudo puppet agent -t” – and this should fix it. Option 2 1) If above steps don’t work for a given agent then destroy only that agent with “vagrant destroy _AGENT_NAME_” 2) And then bring up the agent with command “vagrant up _AGENT_NAME_”