Download as PDF, PPTX
![https://goo.gl/8LyZzN
can't connect to puppet
$ curl --cert certs/cottage.pem --key private_keys/cottage.pem.orig --cacert
certs/ca.pem https://puppet.example.com:8140/puppet/v3/environments | jq
…
"environments": {
"production": {
"settings": {
"modulepath": [
"/etc/puppetlabs/code/environments/production/modules",
"/etc/puppetlabs/code/modules",
"/opt/puppetlabs/puppet/modules"
],
"manifest": "/etc/puppetlabs/code/environments/production/manifests",
"environment_timeout": 0,
"config_version": ""
19](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-19-2048.jpg)
![https://goo.gl/8LyZzN
PSON
Puppet JSON → PSON
$ jq <pson>
$ curl … | jq
JQ? sed for JSON
$ jq .environment catalog.pson
$ jq .classes catalog.pson
$ jq '.resources | length' catalog.pson
$ jq '.resources[] | select(.type == "Stage")' catalog.pson
21](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-21-2048.jpg)
![https://goo.gl/8LyZzN
PSON
Puppet JSON → PSON
$ jq <pson>
$ curl … | jq
JQ? sed for JSON
$ jq .environment catalog.pson
$ jq .classes catalog.pson
$ jq '.resources | length' catalog.pson
$ jq '.resources[] | select(.type == "Stage")' catalog.pson
Sometimes it's good to
go...
22](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-22-2048.jpg)
![https://goo.gl/8LyZzN
Compiling
$ sudo puppet master
--compile problem.example.com
--debug --trace
--logdest /tmp/problem.puppet.log
--environment sandbox
{
"data": {
"resources": [
{
"title": "main",
"exported": false,
"tags": ["stage"],
"type": "Stage",
"parameters": {
"before": "Stage[post]",
JSON
Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding
file resource 'bucketdir':
'File[/var/lib/puppet/bucket]{:loglevel=>:debug, :group=>"puppet",
:ensure=>:directory, :links=>:follow, :owner=>"puppet", :backup=>false,
:mode=>"750", :path=>"/var/lib/puppet/bucket"}'
Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding
file resource 'publickeydir':
'File[/var/lib/puppet/ssl/public_keys]{:loglevel=>:debug,
:group=>"puppet", :ensure=>:directory, :links=>:follow,
:owner=>"puppet", :backup=>false, :mode=>"755",
:path=>"/var/lib/puppet/ssl/public_keys"}'
Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding
file resource 'plugindest':
'File[/var/lib/puppet/lib]{:loglevel=>:debug, :ensure=>:directory,
:links=>:follow, :backup=>false, :path=>"/var/lib/puppet/lib"}'
Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding
file resource 'fileserverconfig':
'File[/etc/puppet/fileserver.conf]{:loglevel=>:debug, :ensure=>:file,
:links=>:follow, :backup=>false, :path=>"/etc/puppet/fileserver.conf"}'
43](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-43-2048.jpg)
: Executing 'echo $((`date +%S` /
86400))'
Debug: Executing 'echo $((`date +%S` / 86400))'
Notice: /Stage[main]/Main/Exec[epoch]/returns: executed successfully
44](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-44-2048.jpg)
![https://goo.gl/8LyZzN
help yourself
function verb() {
erb -P -x -T '-' $1 | ruby -c
is_ok $?
}
function vyaml() {
ruby -ryaml -e "YAML.load_file '$1'"
is_ok $?
}
function ppv() {
puppet parser validate $*
is_ok $?
}
49
function vepp() {
puppet epp validate $*
is_ok $?
}
function is_ok() {
if [ $1 == 0 ]; then
if [[ "$TERM" == *"xterm"* ]] ||
[[ "$TERM" == *"vt100"* ]]; then
echo -e "033[1mSyntax 033[32;1mOk!033[0m"
else
echo "Syntax Ok!"
fi
else
return $1
fi
}](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-49-2048.jpg)
![https://goo.gl/8LyZzN
Bad/Missing Variable
$one = "1"
file {"pcone":
path => "/tmp/pc$one",
ensure => 'directory',
}
file {"pc1":
path => "/tmp/pc1",
ensure => 'file',
}
Info: Caching catalog for puppet.example.com
Error: Evaluation Error: Error while evaluating a Resource
Statement, Cannot alias File[pc1] to ["/tmp/pc1"] at
/root/pc.pp:6; resource ["File", "/tmp/pc1"] already
declared at /root/pc.pp:2 at /root/pc.pp:6:3 on node
puppet.example.com
55](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-55-2048.jpg)
![https://goo.gl/8LyZzN
Bad/Missing Variable
$PC = 'puppetconf'
pc {'one':
place => "/tmp/$PC",
type => "directory",
}
pc {'two':
place => "/tmp/$PC",
type => "file",
}
define pc (
String $place,
String $type,
) {
file {"$title":
path => $place,
ensure => $type,
}
}
Info: Caching catalog for puppet.example.com
Error: Evaluation Error: Error while evaluating a Resource
Statement, Evaluation Error: Error while evaluating a
Resource Statement, Cannot alias File[two] to
["/tmp/puppetconf"] at /root/define.pp:5; resource ["File",
"/tmp/puppetconf"] already declared at /root/define.pp:5 at
/root/define.pp:5:2 at /root/define.pp:15 on node
puppet.example.com
56](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-56-2048.jpg)
![https://goo.gl/8LyZzN
Checking
exec{'before resolv.conf':
command => '/usr/local/bin/puppet-debug before resolv.conf',
require => Class['debug']
} -> file { '/etc/resolv.conf':
source => template("dns/resolv.conf"),
noop => true,
}
class debug {
file {'puppet-debug':
path => '/usr/local/bin/puppet-debug',
source => 'puppet:///modules/debug/puppet-debug',
mode => 0755,
}
}
59](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-59-2048.jpg)
![https://goo.gl/8LyZzN
pry demo
#
69
#
…
From:
/etc/puppetlabs/code/environments/production/modules/pry
/lib/puppet/parser/functions/pry.rb @ line 4
#<Module:0xfb588d1>#real_function_pry:
2: newfunction(:pry) do |args|
3: require 'pry'
=> 4: binding.pry
5: end
[1] pry(#<Puppet::Parser::Scope>)>
puppet agent -tpuppetserver foreground
…
Puppet Server has successfully started and is now ready
to handle requests
exit
…
Puppet Compiled Catalog for xxx.example.com in y.z
seconds
Info: Caching catalog for xxx.example.com
Info: Applying configuration version 'XXX'](https://image.slidesharecdn.com/track2thursuphill-161102164906/75/PuppetConf-2016-Puppet-Troubleshooting-Thomas-Uphill-Wells-Fargo-69-2048.jpg)
Uploaded byPuppet
PuppetConf 2016: Puppet Troubleshooting – Thomas Uphill, Wells Fargo
This document provides a comprehensive guide on troubleshooting Puppet, covering issues related to connections, certificates, and catalog compilations. It includes step-by-step instructions for identifying problems, verifying configurations, and using various tools for diagnostics. Key recommendations emphasize understanding the current state, implementing changes incrementally, and utilizing low-level tools to address issues effectively.
More Related Content
Similar to PuppetConf 2016: Puppet Troubleshooting – Thomas Uphill, Wells Fargo
![[DevFest Strasbourg 2025] - NodeJs Can do that !!](https://cdn.slidesharecdn.com/ss_thumbnails/devfeststrasbourg2025-nodejscandothat-251127142731-da65b6fd-thumbnail.jpg?width=640&height=640&fit=bounds)
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
PuppetConf 2016: Puppet Troubleshooting – Thomas Uphill, Wells Fargo
- 1.
- 2. https://goo.gl/8LyZzN Mastering Puppet /Puppet Cookbook Packt Pub
- 3.
- 4. When you haveeliminated all which is impossible, then whatever remains, however improbable, must be the truth Sir Arthur Conan Doyle 4
- 5. https://goo.gl/8LyZzN Troubleshooting 101 ● documentcurrent state ● discover recent changes ○ audit everything, even things you don't touch ○ never assume it's magic ● change one thing at a time ○ if it doesn't fix the problem, revert it 5 1 une uno um unis odin jeden jedward yksi 一 एक אחד واﺣد
- 6. https://goo.gl/8LyZzN Troubleshooting 101 ● don'tbe afraid of diving deep ○ you can't fix what you don't understand ● use low level tools 6
- 7. https://goo.gl/8LyZzN TAKE HOME puppet isan HTTPS service 7
- 8.
- 9. https://goo.gl/8LyZzN can't find puppet $puppet config print server yourface $ puppet config print confdir /home/thomas/.puppetlabs/etc/puppet # puppet config print server puppet # puppet config print ca_server puppet # puppet config print confdir /etc/puppetlabs/puppet 9
- 10. puppet agent Runsas root… puppetserver Runs as puppet. 10 Basic UNIX permissions
- 11.
- 12. https://goo.gl/8LyZzN can't find puppet dns files /etc/hosts nsswitch.conf #host puppet Host puppet not found: 3(NXDOMAIN) # nslookup puppet Server: 10.0.2.3 Address: 10.0.2.3#53 ** server can't find puppet: NXDOMAIN # dig puppet # getent hosts puppet 127.0.0.1 puppet.example.com puppet localhost localhost.localdomain localhost4 localhost4.localdomain4 # ping puppet PING puppet.example.com (127.0.0.1) 56(84) bytes of data. 64 bytes from puppet.example.com (127.0.0.1): icmp_seq=1 ttl=64 time=0.035 ms 12
- 13. https://goo.gl/8LyZzN can't connect topuppet node puppet server network pixies 8140 server masterport 13
- 14. https://goo.gl/8LyZzN can't connect topuppet Verify you can connect: # nc -v puppet.example.com 8140 Ncat: Version 7.12 ( https://nmap.org/ncat ) Ncat: Connected to 192.168.1.1:8140 14
- 15. https://goo.gl/8LyZzN can't connect topuppet Q: How can you know you are talking to puppet? A: Use the REST API 15
- 16. https://goo.gl/8LyZzN can't connect topuppet $ curl -k https://puppet.example.com:8140/puppet-ca/v1/certificate/ca?environment=production -----BEGIN CERTIFICATE----- MIIFhDCCA2ygAwIBAgIBATANBgkqhkiG9w0BAQsFADAoMSYwJAYDVQQDDB1QdXBw ZXQgQ0E6IHB1cHBldC5leGFtcGxlLmNvbTAeFw0xNjEwMDgyMDEyNDRaFw0yMTEw … qsHOzPEQwJS5q1yPciBQ5jsKCB5vXJEdnPMhA5GBx9rp91hBsiWURlyWt0DMqMTq VqcUEGNc267LrCzKJsFBHCNO53e8z2I0 -----END CERTIFICATE----- masterportca_server puppet or puppet-ca api endpoint key environment https://docs.puppet.com/puppet/latest/reference/http_api/http_api_index.html 16
- 17. https://goo.gl/8LyZzN can't connect topuppet $ gnutls-cli --port 8140 puppet.example.com --x509cafile=ca.pem $ openssl s_client -connect puppet.example.com:8140 GET /path/to/thing/you/want Accept: PSON/YAML/Text/??? puppet master (puppetserver) nodeHTTPS/8140 17 lsof -i :8140 mtr traceroute ping
- 18. https://goo.gl/8LyZzN in order toauthenticate... you must first authenticate. 18
- 19. https://goo.gl/8LyZzN can't connect topuppet $ curl --cert certs/cottage.pem --key private_keys/cottage.pem.orig --cacert certs/ca.pem https://puppet.example.com:8140/puppet/v3/environments | jq … "environments": { "production": { "settings": { "modulepath": [ "/etc/puppetlabs/code/environments/production/modules", "/etc/puppetlabs/code/modules", "/opt/puppetlabs/puppet/modules" ], "manifest": "/etc/puppetlabs/code/environments/production/manifests", "environment_timeout": 0, "config_version": "" 19
- 20. https://goo.gl/8LyZzN API FTW $ curl--cacert certs/ca.pem --cert certs/cottage.pem --key private_keys/cottage.pem -H 'Accept: pson' https://puppet.example.com:8140/puppet/v3/catalog/cottage?environment=production > cottage.pson 20 catalog
- 21. https://goo.gl/8LyZzN PSON Puppet JSON →PSON $ jq <pson> $ curl … | jq JQ? sed for JSON $ jq .environment catalog.pson $ jq .classes catalog.pson $ jq '.resources | length' catalog.pson $ jq '.resources[] | select(.type == "Stage")' catalog.pson 21
- 22. https://goo.gl/8LyZzN PSON Puppet JSON →PSON $ jq <pson> $ curl … | jq JQ? sed for JSON $ jq .environment catalog.pson $ jq .classes catalog.pson $ jq '.resources | length' catalog.pson $ jq '.resources[] | select(.type == "Stage")' catalog.pson Sometimes it's good to go... 22
- 23.
- 24. https://goo.gl/8LyZzN can't get certificate alreadysigned (clean) dates off - expired CA, expired cert openssl x509 -in cert.pem -text puppet cert fingerprint host.example.com puppet cert print host.example.com puppet cert clean host.example.com ntpq -p chronyc sources rm /etc/puppetlabs/puppet/ssl/*/hostname* 24
- 25.
- 26.
- 27.
- 28. https://goo.gl/8LyZzN x.509 Public Key Infrastructure CertificateAuthority Trust https://www.flickr.com/photos/genvessel/431100596 28
- 29.
- 30.
- 31.
- 32. https://goo.gl/8LyZzN OpenSSL View Certificate: openssl x509-in ca.pem -text -noout |less Validity X509v3 Key Usage View Fingerprint: openssl x509 -in ca.pem -fingerprint -noout openssl x509 -in ca.pem -fingerprint -noout -sha256 SHA1 (default) Verify Certificate: openssl verify -CAfile ca.pem puppet.pem 32
- 33. https://goo.gl/8LyZzN OpenSSL (Puppet shortcuts) ViewCertificate: puppet cert print ca View Fingerprint: puppet cert fingerprint ca List Certificates: puppet cert list -a 33
- 34.
- 35. https://goo.gl/8LyZzN CRL # openssl crl-in crl.pem -text -noout |grep Serial Serial Number: 02 # openssl x509 -in cert.pem -text -noout |grep Serial Serial Number: 3 (0x3) 35
- 36. https://goo.gl/8LyZzN CRL # cat ca_crt.pemca_crl.pem >combined.pem # openssl verify -CAfile combined.pem -crl_check cottage.pem cottage.pem: OK # puppet cert clean cottage Notice: Revoked certificate with serial 3 Notice: Removing file Puppet::SSL::Certificate cottage at 'puppet/ssl/ca/signed/cottage.pem' Notice: Removing file Puppet::SSL::Certificate cottage at 'puppet/ssl/certs/cottage.pem' # cat ca_crl.pem ca_crt.pem >combined.pem again? # openssl verify -CAfile combined.pem -crl_check cottage.pem cottage.pem: CN = cottage error 23 at 0 depth lookup:certificate revoked 36
- 37. https://goo.gl/8LyZzN Modulus n = pq OpenSSL #openssl rsa -noout -modulus -in ca_key.pem |sha256sum 69578e29c08c130d37c7c0141134f1cc4778445c7b7d1d96d253b86d6bf4ca38 # openssl x509 -noout -modulus -in ca_crt.pem |sha256sum 69578e29c08c130d37c7c0141134f1cc4778445c7b7d1d96d253b86d6bf4ca38 BIG *RSE PRIME 37
- 38. https://goo.gl/8LyZzN Modulus n = pq OpenSSL #openssl rsa -noout -modulus -in ca_key.pem |sha256sum 69578e29c08c130d37c7c0141134f1cc4778445c7b7d1d96d253b86d6bf4ca38 # openssl x509 -noout -modulus -in ca_crt.pem |sha256sum 69578e29c08c130d37c7c0141134f1cc4778445c7b7d1d96d253b86d6bf4ca38 BIG *RSE PRIME $ puppet agent -t Error: Could not request certificate: The certificate retrieved from the master does not match the agent's private key. Certificate fingerprint: D4:D3:76:F1:6B:51:83:3C:4B:72:69:BF:BC:B0:80:94:79:75:1A:3B:D8:29:F5:EF:81:2C:44:3 5:21:93:CE:FD To fix this, remove the certificate from both the master and the agent and then start a puppet run, which will automatically regenerate a certificate. On the master: puppet cert clean cottage On the agent: 1a. On most platforms: find /home/thomas/.puppetlabs/etc/puppet/ssl -name cottage.pem -delete 1b. On Windows: del "homethomas.puppetlabsetcpuppetsslcertscottage.pem" /f 2. puppet agent -t 38 The certificate retrieved from the master does not match the agent's private key.
- 39. https://goo.gl/8LyZzN OpenSSL recap ● x509 viewcertificate check expiration check serial number ● crl revoked cert serial# ● verify verify cert with CA and/or CRL ● modulus how the cert was encrypted 39
- 40.
- 41.
- 42. https://goo.gl/8LyZzN problem/bugfixes environments configure r10kto make environments automatically environment.conf - modulepath (debug module) ← what is that? 42
- 43. https://goo.gl/8LyZzN Compiling $ sudo puppetmaster --compile problem.example.com --debug --trace --logdest /tmp/problem.puppet.log --environment sandbox { "data": { "resources": [ { "title": "main", "exported": false, "tags": ["stage"], "type": "Stage", "parameters": { "before": "Stage[post]", JSON Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding file resource 'bucketdir': 'File[/var/lib/puppet/bucket]{:loglevel=>:debug, :group=>"puppet", :ensure=>:directory, :links=>:follow, :owner=>"puppet", :backup=>false, :mode=>"750", :path=>"/var/lib/puppet/bucket"}' Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding file resource 'publickeydir': 'File[/var/lib/puppet/ssl/public_keys]{:loglevel=>:debug, :group=>"puppet", :ensure=>:directory, :links=>:follow, :owner=>"puppet", :backup=>false, :mode=>"755", :path=>"/var/lib/puppet/ssl/public_keys"}' Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding file resource 'plugindest': 'File[/var/lib/puppet/lib]{:loglevel=>:debug, :ensure=>:directory, :links=>:follow, :backup=>false, :path=>"/var/lib/puppet/lib"}' Thu Oct 23 14:34:24 -0700 2014 Puppet (debug): Using settings: adding file resource 'fileserverconfig': 'File[/etc/puppet/fileserver.conf]{:loglevel=>:debug, :ensure=>:file, :links=>:follow, :backup=>false, :path=>"/etc/puppet/fileserver.conf"}' 43
- 44. https://goo.gl/8LyZzN Apply $ sudo puppetapply --debug --trace --environment sandbox code.pp $date = "+%S" exec {'epoch': command => "echo $((`date $date` / 86400))", path => '/bin:/usr/bin:/sbin:/usr/sbin', } Info: Applying configuration version '1415729233' Debug: Exec[epoch](provider=posix): Executing 'echo $((`date +%S` / 86400))' Debug: Executing 'echo $((`date +%S` / 86400))' Notice: /Stage[main]/Main/Exec[epoch]/returns: executed successfully 44
- 45.
- 46.
- 47. catalog fails to compile duplicateresource modulepath/bad module name fails to apply unpredictable exec bad/broken service bad/missing variable 47
- 48.
- 49. https://goo.gl/8LyZzN help yourself function verb(){ erb -P -x -T '-' $1 | ruby -c is_ok $? } function vyaml() { ruby -ryaml -e "YAML.load_file '$1'" is_ok $? } function ppv() { puppet parser validate $* is_ok $? } 49 function vepp() { puppet epp validate $* is_ok $? } function is_ok() { if [ $1 == 0 ]; then if [[ "$TERM" == *"xterm"* ]] || [[ "$TERM" == *"vt100"* ]]; then echo -e "033[1mSyntax 033[32;1mOk!033[0m" else echo "Syntax Ok!" fi else return $1 fi }
- 50. https://goo.gl/8LyZzN duplicate resource separate intosubclass ( package {'httpd'} ) virtual resources ( @user, @package, @service) modulepath puppet config print modulepath root@puppet:~# puppet config print modulepath --environment production /etc/puppet/environments/production/public:/etc/puppet/environments/production /modules root@puppet:~# puppet config print modulepath --environment master /etc/puppet/modules:/usr/share/puppet/modules 50
- 51.
- 52.
- 53. https://goo.gl/8LyZzN Unpredictable exec #!/bin/bash echo $JAVA_HOME/home/javadev/.bashrc JAVA_HOME=/your/face It works for me! puppet runs as puppet 53
- 54. https://goo.gl/8LyZzN Broken Service service provider hasstatus=> true /sbin/service $service status /etc/init.d/$service status /usr/bin/systemctl is-active $service 54
- 55. https://goo.gl/8LyZzN Bad/Missing Variable $one ="1" file {"pcone": path => "/tmp/pc$one", ensure => 'directory', } file {"pc1": path => "/tmp/pc1", ensure => 'file', } Info: Caching catalog for puppet.example.com Error: Evaluation Error: Error while evaluating a Resource Statement, Cannot alias File[pc1] to ["/tmp/pc1"] at /root/pc.pp:6; resource ["File", "/tmp/pc1"] already declared at /root/pc.pp:2 at /root/pc.pp:6:3 on node puppet.example.com 55
- 56. https://goo.gl/8LyZzN Bad/Missing Variable $PC ='puppetconf' pc {'one': place => "/tmp/$PC", type => "directory", } pc {'two': place => "/tmp/$PC", type => "file", } define pc ( String $place, String $type, ) { file {"$title": path => $place, ensure => $type, } } Info: Caching catalog for puppet.example.com Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Cannot alias File[two] to ["/tmp/puppetconf"] at /root/define.pp:5; resource ["File", "/tmp/puppetconf"] already declared at /root/define.pp:5 at /root/define.pp:5:2 at /root/define.pp:15 on node puppet.example.com 56
- 57. https://goo.gl/8LyZzN Printing - Notify notify{"$variable": } 57
- 58.
- 59. https://goo.gl/8LyZzN Checking exec{'before resolv.conf': command =>'/usr/local/bin/puppet-debug before resolv.conf', require => Class['debug'] } -> file { '/etc/resolv.conf': source => template("dns/resolv.conf"), noop => true, } class debug { file {'puppet-debug': path => '/usr/local/bin/puppet-debug', source => 'puppet:///modules/debug/puppet-debug', mode => 0755, } } 59
- 60. https://goo.gl/8LyZzN Debug Script… justan example #!/bin/bash LOG=$(mktemp /tmp/puppet-debug.XXXXXX) echo Puppet Debug -- $@ -- $(date) | tee $LOG echo "-- Disk --" | tee -a $LOG df -h |tee -a $LOG df -i |tee -a $LOG echo "-- Mem --" | tee -a $LOG free | tee -a $LOG echo "-- Files --" | tee -a $LOG PUPPET=$(pgrep puppet) for proc in $PUPPET do lsof -p $proc |tee -a $LOG done Puppet Debug -- before resolv.conf -- Fri Oct 24 01:13:34 EDT 2014 -- Disk -- Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup-lv_root 6.7G 2.5G 3.9G 39% / tmpfs 246M 0 246M 0% /dev/shm /dev/vda1 485M 80M 380M 18% /boot Filesystem Inodes IUsed IFree IUse% Mounted on /dev/mapper/VolGroup-lv_root 440640 79253 361387 18% / tmpfs 62783 1 62782 1% /dev/shm /dev/vda1 128016 50 127966 1% /boot -- Mem -- total used free shared buffers cached Mem: 502268 415488 86780 0 22176 172036 -/+ buffers/cache: 221276 280992 Swap: 835580 0 835580 -- Files -- COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME puppet 2058 root cwd DIR 253,0 4096 14 /root puppet 2058 root rtd DIR 253,0 4096 2 / puppet 2058 root txt REG 253,0 10600 36617 /usr/bin/ruby puppet 2058 root mem REG 253,0 156928 4134 /lib64/ld-2.12.so puppet 2058 root mem REG 253,0 1926680 6282 /lib64/libc-2.12.so 60
- 61. https://goo.gl/8LyZzN Printing - Template -scope.to_hash - reject a few - sort - print, one per line file { "/tmp/puppet-debug.txt": content => inline_template("<% vars = scope.to_hash.reject { |k,v| !( k.is_a?(String) && v.is_a?(String) ) }; vars.sort.each do |k,v| %><%= k %>=<%= v %>n<% end %>"), } vars = scope.to_hash.reject { |k,v| !( k.is_a?(String) && v.is_a?(String) ) }; vars.sort.each do |k,v| k=vn end 61
- 62. https://goo.gl/8LyZzN Printing - Template _timestamp=2014-10-2322:29:52 -0700 architecture=x86_64 augeasversion=1.0.0 bios_release_date=01/01/2011 bios_vendor=Bochs bios_version=Bochs blockdevice_vda_size=8589934592 blockdevice_vda_vendor=6900 blockdevices=vda caller_module_name= clientcert=cookbook.example.com clientnoop=false clientversion=3.7.1 concat_basedir=/var/lib/puppet/concat domain=example.com environment=production facterversion=2.2.0 filesystems=ext4,iso9660 fqdn=cookbook.example.com gid=root hardwareisa=x86_64 hardwaremodel=x86_64 hostname=cookbook id=root interfaces=eth0,lo 62
- 63.
- 64. https://goo.gl/8LyZzN Scope The scene: roles andprofiles ntp server class role::ntp { include ntp } class ntp { include ntp::server } 64
- 65. https://goo.gl/8LyZzN Scope The solution: fully scopeeverything remember scope class role::ntp { include ::ntp } class ntp { include ntp::server } 65
- 66.
- 67. https://goo.gl/8LyZzN pry IRB replacement REPL available atruntime 67 pryrepl.org require 'pry' … # amazing code here # wow, much amaze … binding.pry …
- 68. https://goo.gl/8LyZzN pry demo 68 module Puppet::Parser::Functions newfunction(:pry)do |args| require 'pry' binding.pry end end modules/pry/lib/puppet/parser/functions/pry.rb node default { … pry() … } manifests/site.pp
- 69. https://goo.gl/8LyZzN pry demo # 69 # … From: /etc/puppetlabs/code/environments/production/modules/pry /lib/puppet/parser/functions/pry.rb @line 4 #<Module:0xfb588d1>#real_function_pry: 2: newfunction(:pry) do |args| 3: require 'pry' => 4: binding.pry 5: end [1] pry(#<Puppet::Parser::Scope>)> puppet agent -tpuppetserver foreground … Puppet Server has successfully started and is now ready to handle requests exit … Puppet Compiled Catalog for xxx.example.com in y.z seconds Info: Caching catalog for xxx.example.com Info: Applying configuration version 'XXX'
- 70. https://goo.gl/8LyZzN where to gofor help ● IRC #puppet / #puppet-dev ● slack puppetcommunity.slack.com #pug #puppet ● google group / mail list https://groups.google.com/forum/#!forum/puppet-users ● PUG https://www.meetup.com/Seattle-Puppet-Meetup/ 70
- 71. https://goo.gl/8LyZzN Summary Puppet is anHTTPS service End-to-end (gethostbyname, nc mtr) OpenSSL is your friend (x509,crl, verify, s_client make a debug class remember scope basic UNIX permissions 71
- 72.