Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Learning puppet chapter 2


Published on

A book for learning puppet by real example and by building code. Second chapters takes you through all basics of Puppet and enough ruby to work with Puppet.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Learning puppet chapter 2

  1. 1. LEARNING PUPPET - 02 SlideBook (Book on slides) Inspired by Slidedoc - -Vishal Biyani
  2. 2. Puppet languagelanguage for “No Ruby” person!
  3. 3. www.vishalbiyani.comLearning Puppet Puppet language Puppet is a DSL based on Ruby – an object oriented programming language. So do you need to know Ruby in and out to work with Puppet ? Not necessarily!The whole idea of this chapter is to introduce enough puppet & ruby to work with it. Puppet is declarative which has two implications • You define the final desired state not how to achieve it • The sequence in which tasks are carried out is not guaranteed unless you explicitly specify it. You don’t have to master all of the concepts right here – but do spend some time understanding syntax in this chapter.You can 3| spend some time understanding syntax in this chapter.You can always come back to understand any specific part which might not be clear. As with any language – there are sometimes multiple ways to do the same thing and this is true for Puppet/Ruby too. Lastly covering an entire language in such short span is a challenge. So do refer to guides & references: Some useful links: • provides ruby basics needed for Chef framework. A good quick introduction • Similar quick guide from Puppet documentation: /lang_visual_index.html • For the super curious once for quickly learning Ruby there is no better source than that I know of! Some language features are best explained within a context – with a use case or a problem. Hence those parts of puppet language have been intentionally skipped here and will be covered as we go.
  4. 4. www.vishalbiyani.comLearning Puppet Variables & Expressions Lot of times we also need to resolve variables within a string for ex: Variables can be declared with names starting with $ and value can be a literal value or a function/expression which resolves to a value. 1 $name = "vishal" 2 $number = 10+2 As long as the value that a variable resolves to is compatible with data type of intended target, it can be used as part of expression, function or attribute!You will also see variables in following 6 $f_name = "${conf_dir}/.conf" 3 $users = ['us1','us2'] 4 $users += ['us3'] Here we are resolving variable named conf_dir within a string – called interpolation. Similarly addition of variables of same type is easy: Now coming to one of most important features of variables – you can assign a variable only once in a given scope. In a new scope you can assign a new value to same variable, for example below the $name is reassigned but in a new scaope – scope of the class testClass: 1 $name = "Default Name" 2 class testClass { 3 $name = "Test Class Name" 4 } 4| attribute!You will also see variables in following form: 4 $tomcat::config::tc_port What does this mean? It means a variable called “tc_port” from class tomcat::config. We will see the classes & modules shortly.You can access variables from parent scopes but not from child scope (And we will scope in detail soon). For example you can access a top level variable from class level scope but not other way around. Also there is a special notation for variables at top scope – the scope is blank. In following example we are getting “osfamily” from top scope 4 $::osfamily 4 } Lastly – a variable can not be resolved unless it is defined.Which means unlike rest of Puppet – order matters while defining variables. Expressions Most expressions in puppet are usual suspects of any programming language like == (Equality), != (Non Equality), ,< , > , <= , >= , and, or, ! , +, -, /, *, % etc. Couple of them that need some attention are not found in some languages are: =~ Left hand is a string & right hand is a regex – if match is found then returns true !~ Left hand is a string and right is a regex – if match is found returns false in Left hand operand should be string and right hand could be string, array, hash. It checks if string on left side is “in” the right side variable. Returns true if found
  5. 5. www.vishalbiyani.comLearning Puppet Of Conditions We have quite a few friends in conditional statements in Puppet – if, unless, case & selectors.This part is quite easy so let’s run over it quick! Our if, elseif & else block is dead simple 1 if $osfamily == 'linux'{ 2 # Do something 3 } 4 elseif $osfamily == 'windows'{ 5 # Do something else 6 } 7 else 8 # Do something horrible here But if we use regular expression in if block – there Unless is exact opposite of if – but without additional else & regex magic: 1 unless $osfamily == 'linux'{ 2 # Only for Linux family things 3 } 1 case $osfamily { 2 'Linux' : {include role::linux} 3 'Windows': {include role::windows} 4 default : {include role::default} 5 } Case statement is also similar to other languages, almost like having multiple if conditions & a default condition which will be executed if none of previous conditions meet. And oh they also support regex capture variables if you are using a regular expression to evaluate the case expression! Finally the selectors!They are a special case of case conditional expression and differ in behavior. In case statement we execute a piece of code once the 5| But if we use regular expression in if block – there is some black magic happening. If you look at regular expression below – it is expecting string some and varying digits after it – like some01, some02 etc. Now these might be your hostnames and would be good to get them inside the execution block – as shown in comments within the block.They have a name: “Regex Capture variables” 10 if $hostname =~ /^some(d+)./ { 11 # Magic happens here with $0 and $1 12 # If some2 is passed then $1 has value of 2 13 # and $0 has value of some2 - entire match 14 } and differ in behavior. In case statement we execute a piece of code once the condition in a block is met, but in case of selector statement when the expression evaluates to true – a value is returned.This value can be assigned to a variable and then used later in flow. Imagine a situation where your windows user name and linux username for certain operation is different – and you want to use a selector to determine the user name dynamically based on OS as shown in example below: 1 $user_name = $osfamily ? { 2 'windows' => 'win_admin', 3 'linux' => 'root_user', 4 default => 'root', 5 } In above code – if $osfamily is windows then the username is different and if it is linux then it is a different user.
  6. 6. www.vishalbiyani.comLearning Puppet Functions & Classes Functions when it comes to puppet have a special meaning.They are already defined pieces of code which you can use to get things done. But more importantly – functions are evaluated at compilation time.Which means when your manifest convert to catalog – the functions either return their values or modify catalog based on function!We have already used a function on previous slide – which includes roles: A class is a logical grouping of resources which can be accessed by class name.There are two parts to it- first is you define the class and secondly you call (Called declare) it.The execution happens only when you make a call. Let’s first see how to define classes: 2 'Linux' : {include role::linux} Another function which returns a value of true or false based on input: 1 class test { 2 # Resource declarations here 3 } 4 class testParam ($name = 'default_value') { 5 # You can use the $name in any resource declaration 6 } In first example above we defined a class which does not take any parameters.Second one takes a parameter called “name” and if the caller does not provide any value then “default_value” is chosen.This is just to safeguard in case no value is provided & is a good practice to have a default. The resources inside class are defined as you would do normally and they 6| false based on input: 1 str2bool($is_virtual) Functions have following characteristics: • Rvalues type of function retuen some value (Like str2bool) •Statements kind of function will do some work but wont return anything (Like include) •Functions can take arguments – based on their definition of what kind of & how many arguments they take. •Functions are run at compilation time and hence on master – so any agent specific data/code should be done using functions. Instead use resource/custom facts (Which we will see soon)! The resources inside class are defined as you would do normally and they are belong/related to class. The classes should be in files with name of class & ideally should be in manifests directory within a module.You can of course place classes within main manifest or manifests which are imported into main manifest, but for now we won’t look at those cases. Puppet also has concept of class inheritance but it should be used sparingly. We will see the concept and when to use it in later chapters including how to override parent resource attributes
  7. 7. www.vishalbiyani.comLearning Puppet Declaring Classes Before we deep dive into declaring/calling classes let’s understand a fundamental difference due the way this evolved with Puppet: •Include-like way:This was introduced in 3.x & later versions.You can declare a class any number of times but it will be called only once. It uses a combination of external data & defaults to set values of parameters. (Good practice of externalizing configuration data from code). It is suggested to use this way. •Resource-like way:This is old way – in which Let’s say you want to include a class into another one, this can be done using “require” funcion which BTW is different than require used for ordering.The require’s behavior is include like and you can pass list separated by comma, array etc. There is third include like way of declaring classes which uses hiera but we will see that when we talk about hiera! Now let’s look at resource like way of declaring classes which is not suggested after version puppet 3. It is as good as declaring any other resource: 6 define nix::tomcat(){ 7 require java 8 } 10 class {‘nix::tomcat’} 7| •Resource-like way:This is old way – in which you declare a class the way you declare a resource.You MUST DECLARE ONLY ONCE!You can override parameters else they will be taken from external data or lastly form defaults! Now let’s see in how many different ways we can declare classes. Include: this is of course include like way to declare classes and can take many forms There is another way to assign classes to some nodes is via something called ENC - External Node Classifier!Why would you do that? External node classification is process of defining which node belong to which classes. Doing it separate from automation tool provides loose coupling & separation of concerns. ENC can be an external script or a system which when called gives data of which nodes need to be applied which classes and the automation tool accordingly applies the same. For example many organizations store the node data in LDAP and in such case you will fetch data from LDAP and then run puppet based on which nodes needs which class.This is a topic which needs more detailed treatment and we will not go into much details on this one at the moment. 1 include nix::tomcat 2 include nix::tomcat, nginx 3 clas_list = ['nix::tomcat','nginx'] 4 include clas_list 10 class {‘nix::tomcat’}
  8. 8. www.vishalbiyani.comLearning Puppet Resource & types Some more ground rules for resources: •You can not declare the same resource twice! Imagine if in one place you said file should be present and other place absent. Since puppet is declarative language – it has to have a clear state defined. Although you can add attributes to already defined resource for example: 1 file { "/home/vagrant/testFile" 2 ensure => present, 3 owner => 'root', 4 group => 'root', 5 mode => '0400', 6 } Lifecycle of a resource: When a resource declaration is executed, following things happen: • The resource’s current state is read. • If there is a different in current state vs. desired state then the change the resource to desired state. • The change in state is logged as event. This event will appear in log files & reports. The title of type in this case is Every resource has a type – in our example below it is file. It can be a service you want to manage or a package that needs installed etc. 8| attributes to already defined resource for example: 1 file { '/home/vagrant/testFile' 2 ensure => present, 3 } 4 File['/home/vagrant/testFile']{ 5 owner => 'root', 6 group => 'root', 7 mode => '0400', 8 } The title of type in this case is ‘/home/vagrant/testFile’ & rest of the details are ”attributes” with values (Equivalent to parameters in context of resources). Now the “ensure” attribute will ensure that the file is “present” – meaning it exists.The value of ensure might vary from type to type – for example for a service you might say “running” . So we are defining the desired state of the a resource. How to achieve it is what puppet does it for us on multiple platforms. So in short a resource has primarily three things – type, title & one or more attributes. Attributes also might vary form type to type. •The order in which you declare resources in file does not matter. Meaning during execution the order might change for execution optimization. So if you want to introduce sequence in resource execution – you will have to use relationships, which we will see shortly.
  9. 9. www.vishalbiyani.comLearning Puppet Objects & Scope 1/2 Now in real world scenarios when you configure nodes/machine – one node might be Database server while another might be a Application Server and so on. So you need to define what is a node’s identity! (Also called node classification). In Puppet we can define a node’s identity in a file /etc/puppet/manifests/site.pp – and this is our next level of structure. Let’s say you have a Database node – you might want to configure OS level resources first – which will be common across all type of nodes. After that you might apply database specific resource definitions.These two should be ideally in separate modules. So our node definition looks like: Remember that everything in Puppet is resource? But as we add more resources – there is a need of a structure to manage them. It’s almost like game of lego – you first get smaller blocks and combine them to make bigger structures. We can combine related resources together to create a class. And we can combine smaller classes to create bigger classes. For example we 50 #/etc/puppet/manifests/site.pp 51 node '' { 52 include os_common 53 include database 9| classes to create bigger classes. For example we might have a class for managing a file: 42 class base::home { 43 file { '/home/vagrant': 44 owner => 'root', 45 group => 'root', 46 mode => '0644', 47 } 48 } Next we combine classes & configuration data to create modules.These modules & classes can be called from outside. Again we won’t go to depths of module right away but consider that as next logical level of structure. 53 include database 54 } Here the modules os_common & and database is applied on a node named “” And now we have last structural element – “top”! It will be more clear when we talk about scope in next slide but for now it is a level which applies to all nodes in your infrastructure.
  10. 10. www.vishalbiyani.comLearning Puppet Objects & Scope 2/2 What is scope? A way to isolate & limit reach of things. In case of puppet specifically scope applies to variables & resource defaults. Let’s checkout various scope levels and their interplay: TopScope: is what you define at top most level and is accessible everywhere! Node Scope:Things that you define in your node definition exist at node scope. Code you define at node scope is available to classes included in that node but not at top scope. Local scope is something local to a class, resource declaration or a method etc. Overriding is the means to override a variable's value form parent scope. Let’s say we define “message” at node scope and need to override in a class: We saw four levels in last slide: Resource, Class, Node & top. Module is just a structural element which encompasses classes and is not a scope level really.Scopes form a hierarchy – for example you can access top scope code at node scope but not other way around. Basically a parent scope code/variable can be accessed at child level but child level can not access things at parent level. Below picture illustrates the scopes at various levels. topTop Scope 1 node { 2 $message = "This is node" 3 include 4 } 10| resource resource resource resource resource classclass module class filesconfig Node definition Local Scopes Node Scope 4 } 5 class test_class { 6 $message = "This is class overriding node" 7 notify{"Test message = $message":} 8 } What will get printed above? Message from class due to overriding. The scopes of top & class get names – i.e. they can be references using a notation like $::variable etc. But node & local scopes don’t get names hence they can not be referenced. Finally the variables which are not in your scope and have name (i.e. do not belong to node/local) can be accessed with their full names. For example let’s say the class tomcat has a parameter named port, we can call it with it’s fully qualified name: 1 include tomcat::config 2 $port = $tomcat::config::port_num
  11. 11. www.vishalbiyani.comLearning Puppet Relations & Ordering We saw that the sequence of execution of resources can be different than what we see in manifest. But there are times when you need to enforce order – and that’s where ordering attributes help us.There are four attributes – or metaparameters which enable ordering: Attribute Effect before Execute current resource before and Let’s say we want to ensure package “ssh” is installed first and then the file “/etc/ssh/ssh_config” is configured. So there are two ways to do it (With only relevant attributes: 10 file {'/etc/ssh/ssh_config' 11 ensure => present, 12 require => Package[ssh] 13 } 14 package {'ssh' 15 ensure => installed, 16 } 18 file {'/etc/ssh/ssh_config' 19 ensure => present, 20 } 21 package {'ssh' 22 ensure => installed, 23 before => File['/etc/ssh/ssh_config'] 24 } Similarly let’s say we want the service SSH to be running only after the file “/etc/ssh/ssh_config” is configured. In addition the service SSH should be 11| The notation -> (ordering) and ~> (Ordering + Refresh) are equivalent to the four attributes above based on direction you use them. You can use them or be more explicit and use the attribute names. These arrows are lovingly called “chaining arrows” before Execute current resource before and target resource later require Execute current resource later and target resource before notify Execute current resource before, target resource later & refresh target resource of current one changes subscribe Execute target resource before, then execute current resource, and refresh current if target has changed As you have noticed – before and require are same things but depends on where you declare it. Ditto with notify & subscribe but with a refresh included. Quite confusing? – let’s look at some example. “/etc/ssh/ssh_config” is configured. In addition the service SSH should be restarted every time the file changes. Again two ways we can configure it: 26 file {'/etc/ssh/ssh_config' 27 ensure => present, 28 notify => Service['ssh'], 29 } 30 service {'ssh' 31 ensure => running, 32 } 34 file {'/etc/ssh/ssh_config' 35 ensure => present, 36 } 37 service {'ssh' 38 ensure => running, 39 subscribe => File['/etc/ssh/ssh_config'] 40 } • You can refer an array of resources in value for attributes. For example you can restart SSH and some other service as file changes. • There are more complex scenarios where you might want to use “resource collectors” – but more on that later ☺
  12. 12. www.vishalbiyani.comLearning Puppet What did we learn? Although we covered quite a few things, we have omitted many details and certain concepts altogether in this chapter.This is intentional and we will pick up those concepts with examples so that they make lot more sense than bunch of theory concepts. Let’s start building stuff now! This chapter seems like lot of theory but as stated in introduction – do a sanity round and come back to this chapter every time you come across a certain concept and need help implementing it.We have covered quite a few things and we will start using them in 12| Let’s start building stuff now!we will start using them in upcoming chapters.