Downloaded 26 times
![PuppetModule-ServiceChecks
consul::service { 'nginx':
port => '80',
checks => [
{
script => "/usr/lib/nagios/plugins/check_http -H localhost
-w 20 -c 60 -t 20",
interval => '30s',
},
],
}](https://image.slidesharecdn.com/puppetcamp2015-servicediscoveryandpuppet-151103182502-lva1-app6891/75/Puppet-Camp-London-Fall-2015-Service-Discovery-and-Puppet-50-2048.jpg)
![PuppetHiera-Arrays
notice('Generating rabbitmq cluster members
based on Consul information')
$consul_service_array = hiera('rabbitmq',[])
$mq_cluster_nodes =
consul_info($consul_service_array, 'Address')
notice("Result: ${mq_cluster_nodes}")](https://image.slidesharecdn.com/puppetcamp2015-servicediscoveryandpuppet-151103182502-lva1-app6891/75/Puppet-Camp-London-Fall-2015-Service-Discovery-and-Puppet-55-2048.jpg)
![PuppetHiera-Arrays
notice("Generating neo4j_ha cluster members based on
Consul information")
$consul_service_array = hiera('neo4j_ha',[])
$consul_fields = [ 'Address', 'ServicePort' ]
$consul_ha_initial_hosts =
consul_info($consul_service_array, $consul_fields, ':')
$ha_initial_hosts = join($consul_ha_initial_hosts, ',')
notice("Result: ${ha_initial_hosts}")](https://image.slidesharecdn.com/puppetcamp2015-servicediscoveryandpuppet-151103182502-lva1-app6891/75/Puppet-Camp-London-Fall-2015-Service-Discovery-and-Puppet-56-2048.jpg)
![Hieraeyaml
mysecret: DEC::PKCS7[mypassword]!](https://image.slidesharecdn.com/puppetcamp2015-servicediscoveryandpuppet-151103182502-lva1-app6891/75/Puppet-Camp-London-Fall-2015-Service-Discovery-and-Puppet-72-2048.jpg)
![Hieraeyaml
mysecret:
ENC[PKCS7,MIIBmQYJKoZIhvcNAQcDoIIBijCCAYYCAQAxggEhMII
BHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAAikKizydVh0w
XQrvtoMC7vM9NxfksqwOX2jtajDYMMJwXXP/5zKHjnnGmr
+LSXFVkL52FuGentCdityjF0zZEvbZ2D95TWnRTinO9htteC8Ziwq
pTeuNJkTJikOEEZvHbNlp6eX381ppKoatV1k0EmIHwsnqeRJN5T9T
VScoXOb/
1Fre4H7TxSvvaFqo02MWUBaKkWECoEu2PLiuXWEoiLrkDq8pxhjYA
DvGUJLWC8PUSWT/94075z5UKHYBQgLlFrzG+89Rhm5keTy/
cuHsOK9d0nUScjd4m6duCEsvRT5SG/n6GwTEk/
cDMqIuvAwNETv2fdepu4z5nR383zlngDBcBgkqhkiG9w0BBwEwHQY
JYIZIAWUDBAEqBBAJCDkds8PbXeBUMZhFPxWTgDDH1pvUCbCLtWDN
VFkW2yZ1NYF06RuqsSTxofHfMwajC+BSPcTu7heMKQnbKP/KE6o=]](https://image.slidesharecdn.com/puppetcamp2015-servicediscoveryandpuppet-151103182502-lva1-app6891/75/Puppet-Camp-London-Fall-2015-Service-Discovery-and-Puppet-73-2048.jpg)
Uploaded byMarc Cluet
Puppet Camp London Fall 2015 - Service Discovery and Puppet
This document discusses service discovery and how it can be implemented using Consul. It begins with an introduction to the presenter and overview of service discovery challenges. The main points are: - Consul is a service discovery tool that allows services to register themselves and discover other services via API or DNS queries. It supports health checking and secure key-value storage. - Consul uses agents running on each node that register services and perform health checks. Services can be discovered via the REST API or DNS queries. It provides a strongly consistent key-value store. - Puppet can integrate with Consul for service discovery via Puppet modules, Hiera backend, or direct API access. This allows dynamically generating configurations from service information in
More Related Content
![[1C1]Service Workers](https://cdn.slidesharecdn.com/ss_thumbnails/1c1serviceworkers-140927225946-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Puppet Camp London Fall 2015 - Service Discovery and Puppet
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Puppet Camp London Fall 2015 - Service Discovery and Puppet
- 1.
- 2.
- 5. Engineer based inLondon Co-Founder of Ukon Cherry Contractor at TrainLine 19 years of experience as a SysAdmin Founding member of Juju and MAAS while at Canonical Built a DevOps Engineering Team at Rackspace Been DevOps’in for the last 6 years WhoamI?
- 6.
- 7.
- 8.
- 9. Cloud is hard! •Old style DNS • TTL is a b*tch • Health Checks • Metadata Storage TheProblem
- 10.
- 11. Automatically define yourservices • Active Health Checking • Dynamically updated service lists • Can be DNS accessible (if needed) • API accessible (win!) ServiceDiscovery
- 12.
- 13. ServiceDiscovery Service Publication Service Node A Service Node B HealthCheck Health Check Discovery AgentDiscovery Agent
- 14. ServiceDiscovery Service Publication Service Node A Service Node B HealthCheck Health Check Discovery AgentDiscovery Agent
- 15. ServiceDiscovery Service Publication Service Node A Service Node B HealthCheck Health Check Discovery AgentDiscovery Agent
- 16. ServiceDiscovery Service Publication Service Node A Service Node B HealthCheck Health Check Discovery AgentDiscovery Agent
- 17.
- 18.
- 19. ServiceDiscovery Service Node A Service Node C Service NodeB Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 20. ServiceDiscovery Service Node A Service Node C Service NodeB Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 21. ServiceDiscovery Service Node A Service Node C Service NodeB Service: web 10.10.10.1 10.10.10.2 10.10.10.3
- 22.
- 23.
- 24.
- 25.
- 26. ServiceDiscoverySolutions API + DNSpublication k/v Strongly Consistent Host + Service checks API publication k/v Consistent Container checks API publication Container checks Auto-Heal Puppet module Hiera access Puppet module Hiera access Puppet module
- 27.
- 28. It's a ServiceDiscovery System (duh!) • Service Publications (DNS + API) • k/v Storage (strongly consistent) • Health Checks • With encryption! (whaaaaaat) WhatisConsul?
- 29.
- 30.
- 31.
- 32. ConsulArchitecture Consul Server A Consul Agent ConsulServer B Consul Server C Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent
- 33. ConsulArchitecture Consul Server A Consul Agent ConsulServer B Consul Server C Consul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul AgentConsul Agent Consul Servers Another DC
- 34.
- 35. service name: web zone:eu-west-1 <servicename>.service.<zone>.consul web.service.eu-west-1.consul ConsulDNSpublisher
- 36.
- 37. { "check": { "id": "mem-util", "name":"Memory utilization", "script": "/usr/local/bin/check_mem.py", "interval": "10s" } } ConsulHealthChecks
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45. Puppet+Consul You can integrateat several levels • Puppet Module • KyleAnderson-consul • Hiera • lynxman-hiera_consul
- 46. Puppet+Consul You can integrateat several levels • k/v access • venmo-consulr • Templates with Consul • ghdbaston-consul_template
- 47. PuppetModule-Install Puppet Forge makesit very easy to install $ puppet module install KyleAnderson-consul
- 48. PuppetModule-DNS $ puppet moduleinstall saz-dnsmasq include dnsmasq dnsmasq::conf { 'consul': ensure => present, content => 'server=/consul/127.0.0.1#8600', }
- 49. PuppetModule-NodeChecks consul::check { 'disk_space': script=> '/usr/lib/nagios/plugins/check_disk -w 5% -c 1% -x /run/lock -x /run/shm -x /run/user -x /dev -x /run -x /sys/fs/cgroup', interval => '30s', }
- 50. PuppetModule-ServiceChecks consul::service { 'nginx': port=> '80', checks => [ { script => "/usr/lib/nagios/plugins/check_http -H localhost -w 20 -c 60 -t 20", interval => '30s', }, ], }
- 51.
- 52. PuppetModule-Install Puppet Forge makesit very easy to install $ puppet module install lynxman-hiera_consul
- 53. PuppetHiera-Config :backends: - yaml - consul :yaml: :datadir:/etc/puppet/hieradata :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/catalog/service - /v1/catalog/node
- 54. PuppetHiera-Config :consul: :host: 127.0.0.1 :port: 8500 :failure:graceful :paths: - /v1/catalog/service - /v1/catalog/node
- 55. PuppetHiera-Arrays notice('Generating rabbitmq clustermembers based on Consul information') $consul_service_array = hiera('rabbitmq',[]) $mq_cluster_nodes = consul_info($consul_service_array, 'Address') notice("Result: ${mq_cluster_nodes}")
- 56. PuppetHiera-Arrays notice("Generating neo4j_ha clustermembers based on Consul information") $consul_service_array = hiera('neo4j_ha',[]) $consul_fields = [ 'Address', 'ServicePort' ] $consul_ha_initial_hosts = consul_info($consul_service_array, $consul_fields, ':') $ha_initial_hosts = join($consul_ha_initial_hosts, ',') notice("Result: ${ha_initial_hosts}")
- 57. PuppetHiera-k/vaccess Long live tothe API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/
- 58. PuppetHiera-k/vaccess Long live tothe API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - /v1/kv/web/
- 59. PuppetHiera-k/vaccess Long live tothe API! :consul: :host: 127.0.0.1 :port: 8500 :failure: graceful :paths: - "/v1/kv/%{env}/"
- 60.
- 61.
- 62.
- 63. PuppetSecurity git commit -m"All my passwords" git push
- 64.
- 65.
- 66.
- 67.
- 68.
- 69.
- 70.
- 71. Hieraeyaml Usage: eyaml <subcommand> Pleaseuse one of the following subcommands or help for more help: createkeys, decrypt, edit, encrypt, recrypt, version
- 72.
- 73.
- 74.
- 75.
- 76.
- 77.
- 79. Vault$ vault writesecret/foo value=bar Success! Data written to: secret/foo $ vault read secret/foo Key Value lease_id secret/foo/9c5f3cf1-1239-0160-4311-d6544fd1018c lease_duration 2592000 value bar $ vault delete secret/foo Success! Deleted 'secret/foo'
- 80. Vault $ vault readpostgresql/creds/production Key Value lease_id postgresql/creds/production/8ade2cde-5081- e3b7-af1a-3b9fb070df66 lease_duration 3600 password 56b43bc3-b285-4803-abdf-662d6a105bd0 username vault-root-1430141210-1847
- 81.
- 82.