Introduction to MCollective - SF PUG

R.I.Pienaar
San Francisco - 29 May 2013
Introduction
to MCollective

R.I.Pienaar | rip@devco.net | http://devco.net | @ripienaar
Who am I?
• Puppet user since 0.22.x
• Architect of MCollective
• Author of Extlookup and Hiera
• Developer at Puppet Labs London
• Blog at http://devco.net
• Tweets at @ripienaar
• Volcane on IRC

What is MCollective?
Framework for building server
orchestration and parallel job execution
systems

Framework provides

• Server Addressing
Framework provides

• Logging
Framework provides

• Logging
• Connectivity
Framework provides

• Logging
• Connectivity
• Authentication,Authorization and Auditing
Framework provides

• Logging
• Connectivity
• Data Structures andValidation
Framework provides

• Logging
• Connectivity
• Generated or Custom User Interface
Framework provides

• Logging
• Connectivity
• Shell Completion
Framework provides

• Logging
• Connectivity
• Ruby based with debuggers and unit/integration testing
Framework provides

• Logging
• Connectivity
• Synchronous, Asynchronous and Hybrid comms modes
Framework provides

• Logging
• Connectivity
• Synchronous, Asynchronous and Hybrid comms modes
• ...everything is pluggable
Framework provides

CLI Overview

Default CLI
$ mco rpc puppet status
Discovering hosts using the mongo method .... 28
*
[
==========================================================
==> ] 28 / 28
devco.net
Applying: false
Daemon Running: false
Enabled: true
Idling: false
Status: stopped
.
.
Summary of Daemon Running:
stopped = 28
Summary of Enabled:
enabled = 28
Summary of Idling:
false = 28
Summary of Status:
stopped = 28
Finished processing 28 / 28 hosts in 270.99 ms
Pluggable source of
truth

Default CLI
*
[
==========================================================
==> ] 28 / 28
devco.net
Applying: false
Enabled: true
Idling: false
Status: stopped
.
.
stopped = 28
Summary of Enabled:
enabled = 28
Summary of Idling:
false = 28
Summary of Status:
stopped = 28
Completion Progress

Default CLI
*
[
==========================================================
==> ] 28 / 28
devco.net
Applying: false
Enabled: true
Idling: false
Status: stopped
.
.
stopped = 28
Summary of Enabled:
enabled = 28
Summary of Idling:
false = 28
Summary of Status:
stopped = 28
Results in a Human
friendly format

Default CLI
*
[
==========================================================
==> ] 28 / 28
devco.net
Applying: false
Enabled: true
Idling: false
Status: stopped
.
.
stopped = 28
Summary of Enabled:
enabled = 28
Summary of Idling:
false = 28
Summary of Status:
stopped = 28
Summary for the
entire result set

Default CLI
*
[
==========================================================
==> ] 28 / 28
devco.net
Applying: false
Enabled: true
Idling: false
Status: stopped
.
.
stopped = 28
Summary of Enabled:
enabled = 28
Summary of Idling:
false = 28
Summary of Status:
stopped = 28
28 RPC results
across 3 countries

Default CLI
$ mco rpc libvirt domaininfo domain=devco_net
*
[
==========================================================
==> ] 4 / 4
kvm1.devco.net
Autostart: true
CPU Time: 1652031000000000
Current Snapshot: false
Managed Save: false
Max Memory: 2097152
Memory: 2097152
Number of Snapshots: 0
Persistent: true
Snapshots: []
State Code: 1
State: Running
UUID: a6e10d5b-fa5a-8d0c-cbc2-d63b1043970a
VCPUs: 2
4 results, only
shows most
appropriate 1

Default CLI
$ mco rpc libvirt domaininfo domain=devco_net -j -I kvm1.devco.net
[
{
"statusmsg": "OK",
"data": {
"max_memory": 2097152,
"num_of_snapshots": 0,
"state": 1,
"memory": 2097152,
"cputime": 1652082980000000,
"has_current_snapshot": false,
"state_description": "Running",
"has_managed_save": false,
"autostart": true,
"persistent": true,
"vcpus": 2,
"uuid": "a6e10d5b-fa5a-8d0c-cbc2-d63b1043970a",
"snapshots": [
]
},
"sender": "kvm1.devco.net",
"agent": "libvirt",
"action": "domaininfo",
"statuscode": 0
}
]
Everything is DATA

Server Addressing

Class Filters
$ mco package status httpd -C /apache/
Server Addressing
Nodes with Puppet Classes /apache/
applied

Fact Filters
$ mco package status httpd -F country=uk
Server Addressing
Nodes with the custom fact “country”
set to “uk”

Identity Filters
$ mco package status httpd -I devco.net
Server Addressing
One speciﬁc node called “devco.net”

Simple Combined Fact and Class Filters
$ mco package .... -W “country=uk /apache/”
Server Addressing
Nodes in the UK with Puppet Classes
matching /apache/

Compound Statements for facts, classes and data
$ mco .. -S “((country=uk and /apache/)
or customer=acme)
and puppet().conﬁg_retrieval_time > 30"
Server Addressing
UK nodes with Apache in addition to all
nodes for “customer=acme” where
Puppet compiles are slow

Request Security

SSL Security setup for user “rip”
securityprovider = ssl
plugin.ssl_client_private = /home/rip/.mcollective.d/rip-private.pem
plugin.ssl_client_public = /home/rip/.mcollective.d/rip.pem
Authentication
Pluggable - PSK, SSL,AES, SSH, etc

Authorization rules for “puppet” agent
policy default deny
allow cert=rip runonce environment=production customer=acme
Authorization
Set by security system

policy default deny
allow cert=rip runonce environment=production customer=acme
Authorization
List of Actions

policy default deny
allow cert=rip * environment=production apache
allow cert=dev runonce maintenance().scheduled=true
Authorization
Facts, Classes and Data Plugins

Pluggable audit output - this is for Logstash
{'@source_host' => "devco.net",
'@tags' => [],
'@type' => 'mcollective-audit',
'@source' => 'mcollective-audit',
'@timestamp' => "2013-05-27T18:18:22.000000+0100",
'@ﬁelds' => {'uniqid' => "cebb1eed9d505ce1a699dce9c5e1fb39",
               'request_time' => "2013-05-27T18:18:22.000000+0100",
               'callerhost' => "devco.net",
               'caller' => "cert=rip",
               'agent' => "puppet",
               'action' => "runonce",
               'data' => {}},
               '@message' => "devco.net: cert=rip@devco.net invoked agent
puppet#runonce"}
Auditing

Quick Agent Guide

MCollective Agents
• Units of addressable logic
• An agent has actions - package agent has
update, status, install, uninstall etc
• Written in Ruby, actions can be Python,
PHP, Perl and others
• Deployable for interactive commands,
background daemons, SOA style services,
etc
• Optional custom user interfaces

Generate a skeleton agent
$ mco plugin generate agent nrpe actions=runcommand
Created plugin directory : nrpe
Created DDL ﬁle : nrpe/agent/nrpe.ddl
Created Agent ﬁle : nrpe/agent/nrpe.rb
Writing an Agent

DDL File describes the agent for UI
generation and input validation
metadata :name => "nrpe",
         :description => "NRPE Agent",
         :author => "R.I.Pienaar <rip@devco.net>",
         :license => "ASL2.0",
         :version => "0.1",
         :url => "http://devco.net",
         :timeout => 10
action "runcommand", :description => "Run a preconﬁgured NRPE command" do
    input :command,
          :prompt      => "Command",
          :description => "NRPE command to run",
          :type        => :string,
          :validation  => 'A[a-zA-Z0-9_-]+z',
          :optional    => false,
          :maxlength   => 50
    output :exitcode,
           :description  => "Exit Code from the Nagios plugin",
           :display_as   => "Exit Code",
           :default      => 3
    .
    .
    .
end
Writing an Agent

Agent logic
module MCollective
  module Agent
    class Nrpe<RPC::Agent
      action "runcommand" do
        reply[:exitcode] = run(get_nrpe_command(request[:command]),
                               :stdout => :output,
                               :stderr => :output,
                               :chomp  => true)
      end
      def get_nrpe_command(command)
        # not shown
      end
    end
  end
end
Writing an Agent

Ready to deploy...
Writing an Agent
$ mco plugin package
Created RPM and SRPM packages for mcollective-nrpe-agent
Created RPM and SRPM packages for mcollective-nrpe-common
$ ls -l *rpm
mcollective-nrpe-agent-0.1-1.noarch.rpm
mcollective-nrpe-agent-0.1-1.src.rpm
mcollective-nrpe-common-0.1-1.noarch.rpm
mcollective-nrpe-common-0.1-1.src.rpm

...but test your code ﬁrst using
mcollective-test gem
Writing an Agent
describe "nrpe agent" do
  describe "#runcommand" do
    before do
      @agent = MCollective::Test::LocalAgentTest.new("nrpe",
:agent_ﬁle => “agent/nrpe.rb”).plugin
    end
    it "should return correct status" do
      @agent.expects(:get_nrpe_command).returns("/bin/true")
      result = @agent.call(:runcommand, :command => "rspec")
      result.should be_successful
      result.should have_data_items(:exitcode=>0)
    end
  end
end
Custom matchers
and helpers

Post deployment, interact using
standard RPC user interface
Writing an Agent
$ mco rpc nrpe runcommand command=check_load
* [ ============================================================> ]
28 / 28
devco.net Request Aborted
UNKNOWN
Summary of Exit Code:
OK : 27
WARNING : 1
UNKNOWN : 0
CRITICAL : 0
Agents can provide custom
aggregation plugins

...or web interfaces with auto-generated
user interfaces based on the DDL
Writing an Agent

New in MCollective 2
• Entirely rewritten messaging layer
• Asynchronous mode
• Additional non-broadcast based comms
• Reliable messaging with TTLs and Queues
• Batched mode to affect nodes in groups
• Improved RabbitMQ and ActiveMQ support
• Pluggable discovery against your own source of truth
• Data plugins for discovery, data query and ACLs
• Plugin Generators and Packaging
• Improved Security
• MS Windows Support
• DDL based pluggableValidation on clients and servers

Future

Better Docs
• New Deployment Guide - http://docs.puppetlabs.com/
mcollective/deploy/
• Improved ActiveMQ docs for MCollective - http://
docs.puppetlabs.com/mcollective/deploy/middleware/
• ...agent development reference docs up next

More turnkey plugins
with dedicated
developer. Improved
plugin ecosystem.

Complex workﬂow
orchestration scripts

Improved support for
background and long
running commands

SOA like framework

Questions?
twitter: @ripienaar
email: rip@puppetlabs.com
blog: www.devco.net
github: ripienaar
freenode: Volcane
vagrant demo: http://srt.ly/mcvagrant

Introduction to MCollective - SF PUG

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Introduction to MCollective - SF PUG

Similar to Introduction to MCollective - SF PUG (20)

More from Puppet

More from Puppet (20)

Recently uploaded

Recently uploaded (20)

Introduction to MCollective - SF PUG