Marc Cluet
https://www.flickr.com/photos/jsprig/5106529710/

Checklist
https://www.flickr.com/photos/timothytsuihin/2743062701/
https://www.flickr.com/photos/zachinglis/5507648594/
https://www.flickr.com/photos/epsos/8270256961/
https://www.flickr.com/photos/mini82/1361976303/

Who am I?
Marc Cluet (@lynxman)
Grumpy Engineer based in London
Co-­‐Founder of Ukon Cherry
17 years of experience as a SysAdmin
Founding member of Juju and MAAS
while at Canonical
Built a DevOps Engineering Team at
Rackspace
Been DevOps’in for the last 6 years

Who am I?
http://www.meetup.com/London-­‐DevOps/

What is Metadata?
https://www.flickr.com/photos/annarbor/4349876203/

What is Metadata?
Metadata is “data about data”
Structural Metadata
Descriptive Metadata

What is Metadata?
Structural Metadata
architecture => amd64
ipaddress => 10.0.115.197
Descriptive Metadata
$puppetversion = 3.7.5
$apacheversion = 2.4

Metadata purpose?
Abstract
Definitions
Unique
Data

Metadata purpose?
Abstract
Definitions
Unique
Data

Metadata in Puppet
https://www.flickr.com/photos/jimmcd/4859841581/

Metadata in Puppet
Variables
Facts
Hiera
PuppetDB

Variables
class apples {
$apples = 5
$string = “I have ${apples} apples”
}
$apples::apples
$apples::string
global

Scope
web1
mail
web2
apache
global

Exported resources
Class newclient {
@@sshkey { $::hostname:
type => dsa,
key => $mykey,
}
}

Exported resources
Server SSH bastion
class bastion {
Sshkey <<| |>>
}

Facts
Facter
Puppet

Facts
$ facter
interfaces => eth0,lo
ipaddress => 10.0.115.197
ipaddress_eth0 => 10.0.115.197
ipaddress_lo => 127.0.0.1
is_virtual => true
kernel => Linux
kernelmajversion => 3.13

Custom Facts
Where
/etc/facter/facts.d
What
Text File
Script
Output
role=webserver

Custom Facts
$ facter role
webserver

Scope
web1
facts
web2
apache
mail
global

Metadata Distribution
https://www.flickr.com/photos/wallyg/6271443142/

PuppetDB
Facter
Puppet
PuppetDB

PuppetDB Install
Debian/Ubuntu
$ apt-­‐get install puppetdb
RedHat
$ yum install puppetdb

PuppetDB Data
Keeps
Reports
Facts
Exported Resources (searchable)
Used by
Puppet Client (Exp Resources)
Reporting

Hiera
Facter
Puppet
Hiera PuppetDB

Hiera Install
$ gem install hiera

Hiera Config
/etc/hiera.yaml
-­‐-­‐-­‐
:backends:
-­‐ yaml
:hierarchy:
-­‐ "%{environment}/nodes/%{fqdn}”
-­‐ "%{environment}/roles/%{role}”
-­‐ "%{environment}/common”
-­‐ common
:yaml:
:datadir: /etc/puppet/hieradata

Hiera Data
/etc/puppet/hieradata/common.yaml
# Define our variable
example: foo

Hiera Data in Puppet
class test {
$hieravar = hiera(‘example’)
}

Hiera Data in Puppet
class test {
$hieravar = hiera(‘example’,’bar’)
}

Hiera Data Hierarchy
/etc/puppet/hieradata/common.yaml
# Apache variables
apache::rootdir: /www
apache::user: www-­‐data

Hiera Loops
/etc/puppet/hieradata/roles/apache.yaml
# Apache variables
vhost: site
rootdir: "%{hiera(’apache::rootdir')}/site"
user: "%{hiera(’apache::user')}"

Modules consume Hiera
https://forge.puppetlabs.com/mthibaut/users
/etc/puppet/hieradata/common.yaml
# Define our users
users_sysadmins:
marc:
ensure: present
uid: 10001
gid: staff
groups: sysadmin
comment: Marc Cluet
shell: /bin/bash

Hiera Security

Hiera Security
Hiera Security Projects
Hiera GPG
https://github.com/crayfishx/hiera-­‐gpg
Hiera eyaml
https://github.com/TomPoulton/hiera-­‐eyaml

Hiera eyaml Install
$ gem install hiera_eyaml
eyaml {createkeys
decrypt
edit
encrypt
recrypt}

Hiera eyaml
/etc/hiera.yaml
-­‐-­‐-­‐
:backends:
-­‐ eyaml
:hierarchy:
-­‐ "%{environment}/nodes/%{fqdn}”
-­‐ "%{environment}/roles/%{role}”
-­‐ "%{environment}/common”
-­‐ common
:eyaml:
:datadir: /etc/puppet/hieradata
:pkcs7_private_key: "/secure/private.pkcs7.pem”
:pkcs7_public_key: "/secure/public.pkcs7.pem"

Hiera eyaml
secretpassword: DEC::PCSK7[potato]!

Hiera eyaml
secretpassword: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoII
BejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNA
QEBBQAEggEAGPLCXn8goh27KO3msvCz6GJydTHTNrplQ23dR/d
fV0MuSmYwnASFa9RDpqe9K5fhs5XO5TfyQ5Uf4IAZxhnuDx+3Z
SiVY1lbXY6C3x6XeXUBN0jfB5FkdrR+mMYCzGRbVB3gPlM0I8g
2Wq2397h5zMHRkFizPr16vhKuQDxMeGlq8yfoZ4FqwhUteeYxP
MnCV7lx0I6Z/e8I4UZek4FQ7dMfuXNerdDAxx+UMZjRhK3trQl
R/x2TLS4vgh8m0Ml3F5q851W4s5O4ZE7/wvq//nvlr1RoH5EnP
pPQ6shv6R6I7lBfViqIRJalRdyTWVLpFgUknXKLRzoxH8S7IlD
5PzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBA66G5+tBaaF
9g8Lq4dTrwNgCCVP77OWTPoSWGyfxNkuTrmoOANAL98kVWXioO
re612Vw==]

Connecting Metadata
https://www.flickr.com/photos/lightspectrals/1447079898/

Automation metadata
Provisioner Puppet
Monitoring Services

Automation metadata
Provisioner Puppet
Monitoring Services
?

Metadata Partitioning
Service
Node A
Load
Balancer
Service
Node B
Service
Node C

Metadata Partitioning
Service
Node A
Load
Balancer
Service
Node C

Metadata partitioning
https://www.flickr.com/photos/genista/4346973713/

Detour!
https://www.flickr.com/photos/bionicteaching/14586204543/

Service Discovery

What is Service Discovery?
Service
Node A
Health
Check
Discovery
Agent
Discovery
Agent
Service
Node B
Health
Check
Service
Publication
Node A Node B

Metadata Unification
https://www.flickr.com/photos/bangorfuji9500/7152182001/

Metadata Unification
hiera_mysql
https://forge.puppetlabs.com/crayfishx/hiera_mysql
hiera_redis
https://forge.puppetlabs.com/rubyisbeautiful/hiera_redis
hiera_http
https://forge.puppetlabs.com/crayfishx/hiera_http
hiera_etcd
https://forge.puppetlabs.com/garethr/hiera_etcd
hiera_consul
https://forge.puppetlabs.com/lynxman/hiera_consul

Metadata Unification
Puppet
yaml
Hiera
consul
consul
k/v
1 2
3
2

Hiera Consul
/etc/hiera.yaml
-­‐-­‐-­‐
:backends:
-­‐ consul
:consul:
:host: 127.0.0.1
:port: 8500
:failure: graceful
:paths:
-­‐ /v1/catalog/service
-­‐ /v1/catalog/node

Hiera Consul
http://localhost:8500/v1/catalog/service/sensu
[
{
"ServicePort": 0,
"ServiceTags": [],
"ServiceName": "sensu",
"ServiceID": "sensu",
"Address": "10.0.115.197",
"Node": "ip-­‐10-­‐0-­‐115-­‐197“
}
]

Hiera Consul
http://localhost:8500/v1/catalog/service/sensu
[
{
"ServicePort": 0,
"ServiceTags": [],
"ServiceName": "sensu",
"ServiceID": "sensu",
"Address": "10.0.115.197",
"Node": "ip-­‐10-­‐0-­‐115-­‐197“
}
]

Hiera Consul
$sensu_service = hiera('sensu',[])
$sensu_service[ServicePort]
$sensu_service[ServiceTags]
$sensu_service[ServiceName]
$sensu_service[ServiceID]
$sensu_service[Address]
$sensu_service[Node]

Hiera Consul
$sensu_service = hiera('sensu',[])
$s_address = consul_info($sensu_service, 'Address')

Warnings & Advice
https://www.flickr.com/photos/catherinekolodziej/8866489274/

Facter
Facter needs to be FAST
Don’t put here long running scripts
Cron it up!
Text always works better

Hiera
Hiera is awesome!
Be careful of long query loops

Who’s the brain?
Provisioner Puppet Orchestration

Who’s the brain?
Provisioner Puppet Orchestration

Who’s the brain?
Provisioner Puppet Orchestration

Who’s the brain?
Provisioner Puppet Orchestration

Who’s the brain?
It’s your decision!
Keep coherency
Unified metadata is powerful

https://www.flickr.com/photos/mikko_luntiala/12691267935
https://www.flickr.com/photos/dullhunk/202872717
@lynxman
http://slideshare.net/lynxmanuk/
https://github.com/lynxman/
https://devroot.org/