Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013

Loops and Unicorns
Future of the Puppet Language
Henrik Lindberg
Consulting Engineer | Puppet Labs
@hel

puppetconf.com #puppetconf
•  New parser
•  New Language Features
•  New approach
– Opt in to New/Experimental Features
– Maintaining Backwards Compatibility
Introduction

New Parser
The future is already here…

•  Rewritten from the ground up
•  Removes quirky grammar constraints
•  Improves error messages
•  Enabler for:
– opt-in to new and experimental features
– backwards compatibility
•  Use on command line, or in settings
New Parser
puppet apply –-parser future ...

De-Quirk
"This is a random number: ${fqdn_rand()}"
join([1,2,3])
$a = 0x0EH
$b = 0778
Interpolation and functions work:
Literal Array / Hash as argument in calls works:
Numbers must now be valid:

Array & Hash
$a = [1,2,3]
$b = [4,5,6]
$c = $a + $b
Concatenate Arrays:
Append to Array:
Merge Hash:
$d = $a << 4
$a = {name => 'mary'}
$b = {has => 'a little lamb'}
$c = $a + $b

Misc
unless $something { . . .}
else { . . . }
Unless Else:
Assignment is an expression:
Expression separator ';' :
$a = $b = 10
fqdn_rand($seed = 30)
$a = $x[1][1]
$a = $x[1];[1]

Error Reporting
Error: Could not parse for environment production:
Syntax error at 'node' at line 1 on node
kermit.example.com
puppet apply -e '$a = node "a+b" { }'
Then:
Now: Error: Invalid use of expression. A Node Definition
does not produce a value at line 1:6
Error: The hostname 'a+b' contains illegal characters
(only letters, digits, '_', '-', and '.' are allowed)
at line 1:11
Error: Classes, definitions, and nodes may only appear
at toplevel or inside other classes at line 1:6
Error: Could not parse for environment production:
Found 3 errors. Giving up on node kermit.example.com

•  Forgiving Grammar
•  Validation is separate
– Can validate a particular language version
– Language version != Puppet version
•  Evaluation is separate
– Can evaluate a particular language version way
– Language version != Puppet version
Separation of Language
Concerns

•  Iteration & Lambdas
•  Puppet Bindings
•  Heredoc support
•  Puppet Templates
New Language Features

Concepts
each($foo) |$x| { notice $x }
The lambda:
Inline call:
$foo.each |$x| { notice $x }
each($foo) |$x| { notice $x }

•  each
•  select – reject
•  collect
•  reduce
•  slice
Custom functions can take a lambda!
Iterating Functions

Each - Array
$a = ['a', 'b', 'c']
$a.each |$value| { notice $value }
Iterating over an array:
Produces:
Notice: Scope(Class[main]): a
Notice: Scope(Class[main]): b
Notice: Scope(Class[main]): c

Each – Array (with index)
$a = ['a', 'b', 'c']
$a.each |$index, $value| { notice "$index = $value" }
Iterating over an array – with index:
Produces:
Notice: Scope(Class[main]): 0 = a
Notice: Scope(Class[main]): 1 = b
Notice: Scope(Class[main]): 2 = c

Each - Hash
$a = {a => 10, b => 20, c => 30}
$a.each |$key, $value| { notice "$key = $value" }
Iterating over a hash – with key and value:
Produces:
Notice: Scope(Class[main]): a = 10
Notice: Scope(Class[main]): b = 20
Notice: Scope(Class[main]): c = 30

Each – Hash (elements)
$a = {a => 10, b => 20, c => 30}
$a.each |$elem| { notice "${elem[0]} = ${elem[1]}" }
Iterating over a hash elements:
Produces:
Notice: Scope(Class[main]): a = 10
Notice: Scope(Class[main]): b = 20
Notice: Scope(Class[main]): c = 30

Select / Reject
$a = [1, 2, 3]
notice $a.select |$value| { $v == 2 }
notice $a.reject |$value| { $v == 2 }
Select and Reject elements:
Produces:
Notice: Scope(Class[main]): 2
Notice: Scope(Class[main]): 1 3

Collect
$a = [1, 2, 3]
notice $a.collect |$value| { $v * 10 }
Transform each element with collect:
Produces:
Notice: Scope(Class[main]): 10 20 30

Reduce
$a = [1, 2, 3]
notice $a.reduce |$memo, $value| { $memo + $value }
Reduce all elements into one:
Produces:
Notice: Scope(Class[main]): 6

Examples
$usernames.each |$x| {
file { "/home/$x/.somerc":
owner => $x
}
}
Set ownership of some "rc-file" for each user:
Setting 'owner' and 'mode' from a Hash:
$users_with_mode = ['fred' => 0666, 'mary' => 0777 ]
$users_with_mode.each |$user, $mode| {
file {"/home/$user/.somerc":
owner => $user,
mode => $mode
}
}

Examples
$a.select |$x| { $x =~ /com$/ }.each |$x| {
file { "/somewhere/$x":
owner => $x
}
}
Filter and create resources:
Include classes based on array of roles:
$roles.each |$x| { include "role_$x" }

Custom Function (Ruby)
pp_block = args[-1]
Lambda always last argument:
Was it given?
Call it:
pp_block.is_a? Puppet::Parser::AST::Lambda
# in a custom function, self is scope)
pp_block.call(self, 'hello', 'world')

•  http://links.puppetlabs.com/arm2-iteration
•  http://links.puppetlabs.com/arm2-examples
ARM 2 - Iteration

Heredoc
Not yet on master branch
https://github.com/puppetlabs/puppet/pull/1659

Heredoc - Syntax
@( ["]<endtag>["] [:<syntax>] [/<escapes>] )
<text>
[|][-] <endtag>
ENDS-‐HERE

anything
not
in
<text>

"ENDS-‐HERE"

with
interpola2on

:json

syntax
check
result

/tsrn$L

turns
on
escape

/

turns
on
all

|

set
le7
margin

-‐

trim
trailing

t
tab

s
space

r
return

n
new-‐line

$
$

L
<end
of
line>

Heredoc – example
Example:
#.........1.........2.........3.........4.........5....
$a = @(END)
This is indented 2 spaces in the source, but produces
a result flush left with the initial 'T'
This line is thus indented 2 spaces.
| END

Heredoc – example
multiple on same line:
#.........1.........2.........3.........4.........5....
foo(@(FIRST), @(SECOND))
This is the text for the first heredoc
FIRST
This is the text for the second
SECOND

For more examples and details
•  http://links.puppetlabs.com/arm4-heredoc
ARM-4 Heredoc

Puppet Templates
Not yet on master branch
https://github.com/puppetlabs/puppet/pull/1660

•  Embedded Puppet (EPP) - like ERB
•  Same tags
§  <%, <%=, <%-, <%%, <%#
§  %>,-%>
•  Expressions are Puppet DSL
•  Parameterized
•  .epp ﬁle extension (by convention)
Puppet Templates

Use by calling
epptemplate(<name> [,<params_hash>])
inline_epptemplate(<text>[,<params_hash>])
Puppet Templates
$x = 'human'
inline_epptemplate('This is not the <%= $x %> you are
looking for.', { 'x' => 'droid'})
# => 'This is not the droid you are looking for.'

•  Parameterized
– declare parameters
– set default values
– parameter without value and no default = error
Puppet Templates
<%- ($x = 'human') -%>
This is not the <%= $x %> you are looking for.

•  http://links.puppetlabs.com/arm3-
puppet_templates
ARM-3 Puppet Templates

Puppet Bindings / Data in Modules

•  More powerful data bindings
•  For both Data, and Puppet Extensions
•  Composes Hiera2 data in modules and
environment + Hiera1
•  Bindings in Ruby
•  Opt in on command line or in settings
Puppet Binder
puppet apply --binder
puppet apply –-parser future ...

•  Default:
– All hiera-2 data (in default location) and all
(default) ruby bindings from all modules on
module path composed
– Site level hiera-2 data and ruby bindings
override contributions from modules.
•  Customize
– include alternatives, exclude bindings
– add / reorganize overriding "layers"
Configuration

•  Composable
•  Interpolation using Puppet DSL expressions
•  Changed hiera.yaml syntax (for version 2)
Hiera 2

Minimal opt-in (in a module)
---
version: 2
hiera.yaml:
data/common.yaml:
data/${osfamily}.yaml:
---
myclass::myparam: '1.2.3'
---
myclass::myparam: '2.4.6'

puppet_bindings
data_in_modules
ARM-8 & 9

Thank You
Henrik Lindberg
Consulting Engineer | Puppet Labs
@hel
Collaborate. Automate. Ship.

Follow us on Twitter @puppetlabs
youtube.com/puppetlabsinc
slideshare.net/puppetlabs
Collaborate. Automate. Ship.

Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013

More Related Content

What's hot

Similar to Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013

More from Puppet

Recently uploaded

Loops and Unicorns - The Future of the Puppet Language - PuppetConf 2013