R.I. Pienaar, creator of MCollective discusses his new project (Choria.io), and why NATS (www.nats.io) was the ideal choice for a simple, scalable middleware for Choria.

1. R.I.Pienaar
26 April 2017
How Choria Improves the
Operability of MCollective using
NATS

2. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Consultant for 20+ years
• Government, Finance, Health, Social Media,
Fortune 50, Startups
• DevOps, Automation, Architect,
Development
• Open Source @ github.com/ripienaar
• Architect and Author of many Puppet
related tools

3. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Overview
• Large scale Operations orientated RPC framework
• Provides ad-hoc orchestration within the Puppet eco
system
• Flexible discovery that integrates into many data sources
• Strong Authentication, Authorisation, Auditing
• Extendible using Ruby
• Communications over Middleware

4. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
$ mco rpc service restart service=httpd -W country=uk -W customer=acme
* [ ============================================================> ] 15 / 15
windev1.example.net Unknown Request Status
Cannot stop https, error was: Execution of 'C:/WINDOWS/system32/net.exe stop httpd'
returned 2: The service name is invalid.
Summary of Service Status:
running = 14
unknown = 1
Finished processing 15 / 15 hosts in 8352.48 ms
MCollective - CLI

5. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
$ mco rpc service restart service=httpd -W country=uk -W customer=acme
* [ ============================================================> ] 15 / 15
windev1.example.net Unknown Request Status
Cannot stop https, error was: Execution of 'C:/WINDOWS/system32/net.exe stop httpd'
returned 2: The service name is invalid.
Summary of Service Status:
running = 14
unknown = 1
Finished processing 15 / 15 hosts in 8352.48 ms
MCollective - CLI
Context aware summaries
“service” api
“restart” action discovery

6. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
$ irb -r mcollective
irb(main):001:0> include MCollective::RPC
=> Object
irb(main):002:0> client = rpcclient("service")
=> …
irb(main):003:0> client.fact_filter("cluster", "a")
=> nil
irb(main):004:0> client.discover
=> ["dev1-1.choria"]
irb(main):005:0> client.progress = false
=> false
irb(main):006:0> puts client.status(:service => "sshd").first.to_json
{"agent":"service","action":"status","sender":"dev1-1.choria","statuscode":
0,"statusmsg":"OK","data":{"status":"running"}}
MCollective - API
“service” api
discovery
“status” action
result from 1 node

7. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Web

8. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

9. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

10. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

11. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

12. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

13. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

14. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

15. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
MCollective - Installation

16. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar

17. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Modernising MCollective

18. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Playbooks
Ease of Installation
Modern PKISRV Records
AuditingPuppetDB Integration
Improved
Connectivity
Batteries Included
JSON

19. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Choria - Middleware Needs
• Easy to deploy
• Easy to secure using standard TLS
• Easy to operate and monitor
• Must be scalable and highly available
• Good documentation
• Responsive community
• Good Ruby support - but others needed too

20. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar

21. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
HTTP Monitoring
Single binary install
TLS
Good docs
and comm
Good trace logging
Full Mesh
Clustering
Many languages
supported
Scalable to
thousands

22. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
NATS - Configuration
$ gnatsd
--tls
--tlscert ssl/nats1.example.net.pem
--tlskey ssl/nats1.example.net.key
--tlscacert ssl/ca.pem
--tlsverify
-l logs/nats-0.log
-p 4222
-m 8222
--cluster nats://nats2.example.net:4223
--routes nats://nats2.example.net:4223
-DV

23. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
NATS - Configuration
port: 4222
monitor_port: 8222
tls {
cert_file: “ssl/nats1.example.net.pem"
key_file: “ssl/nats1.example.net.key"
ca_file: "ssl/ca.pem"
verify: true
}
cluster {
port: 4223
tls {
cert_file: “ssl/nats1.example.net.pem”
key_file: “ssl/nats1.example.net.key”
ca_file: "ssl/ca.pem"
verify: true
timeout: 2
}
routes = [
nats-route://nats2.example.net:4223
]
}

24. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
NATS - Monitoring
% curl -s localhost:8222/varz|
./jq '"port: (.port) uptime: (.uptime) connections: (.connections)”’
"port: 4222 uptime: 33d12h9m33s connections: 16"

25. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Federating NATS Clusters

26. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Choria - Federation

27. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Links
https://choria.io/
https://nats.io/
https://docs.puppet.com/mcollective
https://www.devco.net/

28. R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Questions?
twitter: @ripienaar
email: rip@devco.net
blog: www.devco.net
github: ripienaar
freenode: Volcane
slack.puppet.com: ripienaar
https://www.devco.net/