The document presents an introduction to orchestration using MCollective, focusing on its functionality as a framework for server orchestration and job execution. It explains the concept of finite state machines, agents, and the use of discovery plugins for filtering and determining system states. The document also includes an example deployment scenario highlighting the orchestration process during application updates on web servers.

1. Introduction to
orchestration using
MCollective
Pieter Loubser!
Puppet Camp London!
18 November 2013

2. Who am I?
Developer at Puppet Labs in London!
On the MCollective team for 2 years!
@pieterloubser on the twitter!
ploubser on IRC

3. Finite State Machines
Abstract machine that can only be in one of a finite
number of states!
It can move from one state to another when an
event or condition is triggered (transitions)

5. Old app
Magic?
Winning

6. Orchestration
It can be a Finite State Machine
Determine your states!
Determine your transitions

7. What is MCollective?
!
A framework for building server orchestration tools
and a parallel job execution system

8. The moving parts
Message Oriented Middleware!
MCollective Server (mcollectived)!
MCollective Client

9. Server
RPC Request
Server
RPC Request
Server
RPC Request
Middleware
RPC Request
Client

10. Request : Determine the status of the puppet service
Server
Agents
rpcutil
puppet
Actions
package
status
service
start
...
stop
...
Response

11. Server
stopped
Server
stopped
Middleware
Client
Process results
Server
running

13. Agents - State Transitions
Agents can change the state the world is in!
Agents can determine what state the world is in !
An agent consists of two parts!
The DDL file which describes the Agent

15. Agents - State Transitions
Agents can change the state the world is in!
Agents can determine what state the world is in !
An agent consists of two parts!
The DDL file which describes the Agent!
An implementation file that defines the actions

17. State
We’re not always looking at the system as
a whole
MCollective uses discovery plugins as the source of
truth!
But touching all the things can sometimes be a bad
idea!
There is a desire for fine grained filtering

18. Filters
Filter on fact sources

20. Filters
Filter on fact sources!
Filter on Puppet classes

22. Filters
Filter on fact sources!
Filter on Puppet classes!
Filter on identity

24. Filters
Filter on fact sources!
Filter on Puppet classes!
Filter on identity!
Complex filtering based on logical language

26. Data Plugins
Enables you to determine system state by
executing code in a similar manner to agents!
Consist of the same two parts as Agents!
DDL!
Implementation

30. Orchestration
A simple deployment scenario
5 web servers sitting behind a load balancer!
Mark a web server as down!
Update the application!
Mark the web server as up!
Profit

31. Group 1
web1
web2
web3
lb1
web4
web5

32. Ready for
upgrade
Removed from
load balancer
Pre deployment
Failed upgrade
Successful
upgrade
Ready for
load balancer
Failed to return
to load balancer
Failed to remove
from load balancer
Failed
deploy
Returned to
load balancer
Successful
deploy

33. The code
https://github.com/ploubser/puppetcamp-london

34. Here goes
everything…

35. Questions?