Managing large systems with
Marionette Collective

andreas.schmidt@cassini.de
twitter @aschmidt75




© 2012 Cassini Consulting GmbH
About myself
       Software Development
               C++, Java (EE)

       Web Operations
              Solaris, Linux, Shell

       Infrastructure Development
                 Puppet, Ruby



    About Cassini
        Systems Management
        IT Security
        Configuration Mgt & Deployment Automation

2
What is MCollective?
    What is needed to set it up?
    How does it work?
    Small demo



3
1 year later
                  Test / Production    200 hosts
                        stage
                      20 hosts
    Development
    Stage
    5 hosts


4
5
„                                     “
        ssh in a for loop is not a solution
                           - Luke Kanies

6
„  If you ever have to log on to a
    single box, you‘re doing it wrong.         “
                     Kristian Köhntopp, OSDC2012


7
„   mcollective is ssh on steroids  “
                               @kbuytaert




8
SSH …                      mcollective …

    is a protocol connecting   is a broadcast-oriented
    two endpoints              framework

    is used primarily to       transports structured
    transport text             data within an API

9   19.07.2012
Marionette Collective Overview




10
Architecture | What does it do?   Me


                  Messaging
                  Middleware




         My
       Hosts



11
Ruby 1.8.7+
     Architecture | Components           mcollective client
                                                    stomp
                   ActiveMQ | RabbitMQ




     Ruby 1.8.7+
     mcollective
     stomp
     facter
     agent code

12
Architecture | Collectives


          Collective




13
Architecture | Collectives


            Collective




     Subcollective



14
facter: Adressing by Facts
            operatingsystem=centos


            ipaddress=~^10.1.30




15
Walk-Through                         client sends request
                                          to middleware topic

                                      2
              All hosts watch                                             1
              topic and, reading
              request as it becomes              5
              available
                        3                                Client reads             6
                                                         all results from topic
                                                Results are delivered


                                                                 4   Host w/ matching
                                                                     facts execute
                                                                     agent call




16
DEMO

17
Backup // DEMO




18
Backup // DEMO




19
Backup // DEMO




20
Backup // DEMO




21
Backup // DEMO




22
Backup // DEMO




23
Backup // DEMO




24
Backup // DEMO




25
Backup // DEMO




26
Backup // DEMO




27
Backup // DEMO




28
Backup // DEMO




29
30
What about security?
     AES Security Plugin


              Client          Messaging           Node
                              Middleware
              Stomp                                  Stomp
                                                  Agent
           mcollective        Message       mcollectived server
                              Payload
          User SK              AES         User PK

          Server PK                        Server SK


     + Authorization Plugin
     + Auditing Plugin

31
MCollective 2.0
 • Direct Addressing Mode

 • Message TTLs

 • Complex Discovery Statements

 • Pluggable discovery




32
True Parallelism
     API w/ data validation
      Deterministic timing
         Added Velocity
       Scales really well
33
Q& A
Cassini Consulting

andreas.schmidt@cassini.de
twitter @aschmidt75




34

Berlin Devops: Managing systems with Marionette Collective