1) Consul-templaterb is a Ruby-based templating engine for Consul that provides a more powerful and optimized templating experience than the default Golang one. 2) It handles complex templates, outputs to multiple formats like JSON and YAML, supports hot reloading, and can even create full-featured GUIs. 3) It is highly optimized for performance on large Consul clusters with thousands of services and nodes.

1. Pierre Souchay
Discovery Team @Criteo
Twitter: @vizionr
Github: pierresouchay
Consul
templates
on steroids
Leading Discovery Team @Criteo (SDKs + Consul)
Dealing with 240k+ services, 38k+ Consul nodes in 9 DCs
1st external contributor to Consul
Author of consul-templaterb

2. Why consul-templaterb?
Easier for complex templates
• ERB is nicer, more powerful than Golang templating
• Structured output formats (JSON, XML, YAML…)
• Hot-Reload, includes, lazy evaluation
• You can create a full featured GUI!
• Handle Consul compatibility for features (ex: meta)
• OSS with examples: https://github.com/criteo/consul-templaterb/

3. Why consul-templaterb?
Very High performance on large clusters
• More optimized for performance on large clusters: 800M/s -> 12k/s
• Bandwidth usage / endpoint (debug)
• Optimizations built-in per-endpoints

4. Why consul-templaterb?
Good compatibility with Consul-Template
• Most of features are present (process baby-sitting), reload commands…
• Many of the flags are compatible (using the same syntax, eg: --template)

5. How it works internally…
while (true) {
data = compute_template()
ready = download_update_data_async()
if (ready) {
write_output_to_file(data)
}
}

6. Well documented API
• API is fully documented
• https://github.com/criteo/consul-templaterb/blob/master/TemplateAPI.md
• Includes links to real world examples
• API includes regression tests
• All samples are always tested at build time and all working!
• Output of APIs is unit-tested
• Many samples (HTML, YAML, JSON, text, full HTML5 GUI)
• https://github.com/criteo/consul-templaterb/tree/master/samples

7. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
service(srv).each do |snode|
result[srv][snode.status] += 1
end
end
%><%= JSON.pretty_generate(result) %>

8. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
service(srv).each do |snode|
result[srv][snode.status] += 1
end
end
%><%= JSON.pretty_generate(result) %>
{
"chef-server": {
"passing": 11,
"warning": 0,
"critical": 0
},
"chef-webscale": {
"passing": 3,
"warning": 0,
"critical": 0
},
"chef-webscale-backend": {
"passing": 1,
"warning": 0,
"critical": 0
},
"host-arbitrage": {
"passing": 388,
"warning": 0,
"critical": 2
[…]

9. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
service(srv).each do |snode|
result[srv][snode.status] += 1
end
end
%><%= YAML.dump(result) %>

10. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
service(srv).each do |snode|
result[srv][snode.status] += 1
end
end
%><%= YAML.dump(result) %>
---
chef-server:
passing: 11
warning: 0
critical: 0
chef-webscale:
passing: 3
warning: 0
critical: 0
chef-webscale-backend:
passing: 1
warning: 0
critical: 0
host-arbitrage:
passing: 388
warning: 0
critical: 2
host-arbitrage-admin:
passing: 388
warning: 0
[…]

11. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
datacenters.each do |dc|
service(srv, dc:dc).each do |snode|
result[srv][snode.status] += 1
end
end
end
%><%= YAML.dump(result) %>

12. A nice API with sensitive defaults
<%
result = {}
services(tag:"webscale").each do |srv, tags|
result[srv] = {'passing' => 0, 'warning' => 0, 'critical' => 0}
datacenters.each do |dc|
service(srv, dc:dc).each do |snode|
result[srv][snode.status] += 1
end
end
end
%><%= YAML.dump(result) %>
---
chef-server:
passing: 25
warning: 0
critical: 0
chef-webscale:
passing: 3
warning: 0
critical: 0
chef-webscale-backend:
passing: 1
warning: 0
critical: 0
host-arbitrage:
passing: 4493
warning: 0
critical: 23
host-arbitrage-admin:
passing: 4493
warning: 0
[…]

13. Demo Time!
• Using ruby gems
gem install consul-templaterb
consul-templaterb --help
• Using docker for the GUI:
https://hub.docker.com/r/discoverycriteo/consul-templaterb

14. History of Services in a large Consul Cluster

15. Highly scalable UI (1000s of users)

16. See how we are using it to build new Stuff
Tomorrow at Sep 11 | 3:20 PM - 3:55 PM at Elwha A | Level 5
Inversion Of Control with Consul
https://hashiconf.hashicorp.com/schedule/inversion-of-control-with-consul
Slides: https://www.slideshare.net/PierreSouchay/2019-hashiconf-seattleconsulioc
Other resources:
https://www.hashicorp.com/resources/operating-consul-at-scale
https://www.hashicorp.com/resources/criteo-containers-consul-connect