The document discusses service-oriented architecture (SOA) and the use of Consul for service discovery and configuration management in microservices environments. It outlines configuration examples for server and client nodes, service registration, and health checks via HTTP and DNS interfaces. Additionally, it covers topics like distributed locking, leader election, and accessing key-value data in Consul.

1. Service Oriented Architecture
with Consul
Rajesh Sharma
@broncha
VantageBit

3. Problems when you scale a
SOA/microservices
Host resolution (service discovery)
Con guration updates

4. Solutions
Apache zookeeper
etcd
doozerd
consul

5. Consul
It is a tool for discovering and con guring services in
your infrastructure, with features like
Service Discovery
Health Checking
Key/Value Store

6. Installation
Just download a binary le and you are good to go.

7. Con guration

8. Server nodes
con g.json
{
"server": true,
"data_dir": "/var/consul",
"encrypt": "2346rxJm2aFuMNxWCKWnvPBA==",
"log_level": "INFO",
"enable_syslog": true,
"bind_addr": "172.31.0.195",
"start_join": ["172.31.12.223", "172.31.28.140"],
"ui": true
}

9. Client nodes

10. con g.json
{
"server": false,
"data_dir": "/var/consul",
"encrypt": "2346rxJm2aFuMNxWCKWnvPBA==",
"log_level": "INFO",
"enable_syslog": true,
"bind_addr": "172.31.21.186",
"start_join": ["172.31.28.140","172.31.12.223","172.31.0.195
}

11. services.json
{
"services": [
{
"id": "application@ip-172-31-21-186",
"name": "application",
"address": "172.31.21.186",
"port": 8080,
"checks": [
{
"name": "Application/Apache on port 8080",
"tcp": "localhost:8080",
"interval": "10s",
"timeout": "1s"
}
]
}
]
}

12. The HTTP Interface

13. Catalog API

14. /v1/catalog/nodes
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "baz",
"Address": "10.1.10.11",
"TaggedAddresses": {
"lan": "10.1.10.11",
"wan": "10.1.10.11"
},
"Meta": {
"instance_type": "t2.medium"
}
} ...
]

15. /v1/catalog/services
{
"consul": [],
"redis": [],
"postgresql": [
"primary",
"secondary"
]
}

16. /v1/catalog/service/<service>
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "192.168.10.10",
"Meta": {
"instance_type": "t2.medium"
},
"CreateIndex": 51,
"ModifyIndex": 51,
"ServiceAddress": "172.17.0.3",
"ServiceEnableTagOverride": false,
"ServiceID": "32a2a47f7992:nodea:5000",
"ServiceName": "foobar",
"ServicePort": 5000,
"ServiceTags": [
"tacos"
]
}

17. /v1/catalog/node/<node>
{
"Node": {
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "10.1.10.12"
},
"Services": {
"consul": {
"ID": "consul",
"Service": "consul",
"Port": 8300
},
"redis": {
"ID": "redis",
"Service": "redis",
"Tags": [
"v1"
],
"Port": 6379

18. Health Check API

19. /v1/health/node/<node>
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"CheckID": "serfHealth",
"Name": "Serf Health Status",
"Status": "passing",
"Notes": "",
"Output": "",
"ServiceID": "",
"ServiceName": ""
},
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",
"Status": "passing",
"Notes": "",

20. /v1/health/checks/<service>
[
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",
"Status": "passing",
"Notes": "",
"Output": "",
"ServiceID": "redis",
"ServiceName": "redis"
}
]

21. /v1/health/service/<service>
[
{
"Node": {
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "10.1.10.12"
},
"Service": {
"ID": "redis",
"Service": "redis",
"Tags": null,
"Address": "10.1.10.12",
"Port": 6379
},
"Checks": [
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",

22. /v1/health/service/<service>?passing
[
{
"Node": {
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "10.1.10.12"
},
"Service": {
"ID": "redis",
"Service": "redis",
"Tags": null,
"Address": "10.1.10.12",
"Port": 6379
},
"Checks": [
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",

23. The Key-Value Store API

24. GET /v1/kv/<key>
[
{
"CreateIndex": 100,
"ModifyIndex": 200,
"LockIndex": 200,
"Key": "zip",
"Flags": 0,
"Value": "dGVzdA==",
"Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e"
}
]
PUT /v1/kv/<key>
DELETE /v1/kv/<key>

25. The DNS Interface
Instead of making HTTP API requests to Consul, a
host can use the DNS server directly via name
lookups
redis.service.us-east-1.consul
master.rabbitmq.service.us-east-1.consul
slave.redis.service.consul
Consul will listen on 127.0.0.1:8600 for DNS
queries in the consul. domain
We can use dnsmasq to route DNS queries for
consul. domains to 127.0.0.1:8600
Unhealthy nodes are automatically ltered

26. Node Lookups
<node>.node[.datacenter].<domain>
$ dig @127.0.0.1 -p 8600 foo.node.consul ANY
; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY
; (1 server found)
;; truncating comment sections for brevity
;; QUESTION SECTION:
;foo.node.consul. IN ANY
;; ANSWER SECTION:
foo.node.consul. 0 IN A 10.1.10.12
;; AUTHORITY SECTION:
consul. 0 IN SOA ns.consul. postmaster.consul. 139283

27. Service Lookups
[tag.]<service>.service[.datacenter].<domain>
$ dig master.redis.service.consul
; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul
;; truncating comment sections for brevity
;; QUESTION SECTION:
;master.redis.service.consul. IN A
;; ANSWER SECTION:
master.redis.service.consul. 0 IN A 172.31.28.140
;; Query time: 2 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Feb 01 10:36:28 UTC 2017
;; MSG SIZE rcvd: 88

28. Distributed lock and Leader
election

29. Pick a key
service/<servicename>/leader`
Create a session
$ curl -X PUT -d '{"Name": "rabbitmq-service"}'
http://localhost:8500/v1/session/create
# response
{
"ID": "4ca8e74b-6350-7587-addf-a18084928f3c"
}

30. Acquire a session for given key
$ curl -X PUT -d <body>
http://localhost:8500/v1/kv/<key>?acquire=<session>
If return value is true key was acquired and this
node is the leader
Else some other node already acquired that key
and is the leader

31. Other nodes can discover the leader node using
the session info API
$ curl http://localhost:8500/v1/kv/<key>
[
{
"Session": "4ca8e74b-6350-7587-addf-a18084928f3c",
"Value": "Ym9keQ==",
"Flags": 0,
"Key": "<key>",
"LockIndex": 1,
"ModifyIndex": 29,
"CreateIndex": 29
}
]
If the key has no associated Session, then there is
no leader

32. The nodes would be watching the key to check if it
was released and try to become the leader
$ curl
http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a1
[
{
"LockDelay": 1.5e+10,
"Checks": [
"serfHealth"
],
"Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y"
"Name": "dbservice",
"ID": "4ca8e74b-6350-7587-addf-a18084928f3c",
"CreateIndex": 28
}
]

33. Questions?