1
Mistral 

Workflow Service for OpenStack
Renat Akhmerov

Senior Software Engineer @ Mirantis
Dmitri Zimine
Chief Stormer @ StackStorm
2
Agenda
• Overview
• Workflow Service: what and why
• Architecture
• Roadmap
• Discussion
3
What is Mistral?
Mistral = Workflow Service for OpenStack
4
Workflow Service
!
• Service to define, execute, and monitor workflows
• Keeps state
• Carries data
• Provides goodies: HA, tracking, history, etc.
• Best for cross-service integration “orchestration”
5
Mistral Workflow
• Workflow: a graph of tasks
• Task Actions: call REST, run
command, create VM, send
email…
• Control flow:
• transitions
• conditions
• Data Flow: passing data
between tasks
• Triggers: cron, events, API
T1
T5
T3
T4
T2
Start
T6
T7
T8
T9
6
Workflow Definition Snippet
Workflow:!
tasks:!
...!
runJob:!
action: Demo.runJob!
on-error: deleteVM!
on-success: sendJobError!
publish:!
job_results: job_results!
!
deleteVM:!
action: Nova:deleteVM!
on-finish: end!
!
sendJobError:!
action: std.email!
parameters:!
to: admin@mycompany.com!
subject: Workflow completed!
body: | !
! ! Workflow {$.execution.id} completed !
with results {$.job_results}!
on-finish: deleteVM!
...
7
Data Flow Concept
RunVM
input:
image_id
SendMsg
input:
person.email
IsApprover
input:
person.name
GoOn
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com"
}
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com”
},
“vm_ip”: “10.0.0.3”
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com”
},
“vm_ip”: “10.0.0.3”
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com”
},
“vm_ip”: “10.0.0.3”,
“approver”: “true”
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com”
},
“vm_ip”: “10.0.0.3”,
“sent_msg”: “true”
“image_id”: “123”,
“person”: {
“name”: “John”,
“email”: “john@me.com”
},
“vm_ip”: “10.0.0.3”,
“approver”: “true”,
“sent_msg”: “true”
- task input - task result (added into context)
8
Architecture
Scheduler
Executor
API ServerEngine
Executor
!
...
workflows
executions
tasks
events
Workflow Queue
Task Queue
9
Dashboard
© MIRANTIS 2014
10
ROADMAP
NOW -> NEXT-> FUTURE
11
What Works Now (ver 0.0.2)
• API, Engine, Executor, Scheduler, scale up/down
• Workflow DSL *
• Workflow engine:
• Control flow - transitions, conditions *
• Data Flow
• Basic tasks: ssh, HTTP/REST, email, echo
• ‘Ad-hoc’ actions
• Keystone integration
• CLI and Python client
• Horizon dashboard *
12
Next
• Apply POC learnings
• Finalize workflow capabilities
• Simplify definition syntax
• Create content
• openstack action pack: nova, heat
• more standard actions
• Provide extensibility for writing custom actions
• Work on maturity, stability, performance
• Horizon Dashboard improvements
13
Future
• Ceilometer Triggers
• Event Listeners
• Rollback
• Horizon Dashboard - Graphical Workflow
Presentation
• “Dry-run” mode
• TaskFlow integration, when ready
14
More Info
Discussion etherpad: !
https://etherpad.openstack.org/p/juno-summit-mistral
!
More info:
• Mistral POC Demo: Expo E-26 booth
• Launchpad: https://launchpad.net/mistral
• Wiki: https://wiki.openstack.org/wiki/Mistral
• Screencast: https://www.youtube.com/watch?v=x-
zqz1CRVkI (search for “OpenStack Mistral POC Demo”)

Mistral Atlanta design session

  • 1.
    1 Mistral 
 Workflow Servicefor OpenStack Renat Akhmerov
 Senior Software Engineer @ Mirantis Dmitri Zimine Chief Stormer @ StackStorm
  • 2.
    2 Agenda • Overview • WorkflowService: what and why • Architecture • Roadmap • Discussion
  • 3.
    3 What is Mistral? Mistral= Workflow Service for OpenStack
  • 4.
    4 Workflow Service ! • Serviceto define, execute, and monitor workflows • Keeps state • Carries data • Provides goodies: HA, tracking, history, etc. • Best for cross-service integration “orchestration”
  • 5.
    5 Mistral Workflow • Workflow:a graph of tasks • Task Actions: call REST, run command, create VM, send email… • Control flow: • transitions • conditions • Data Flow: passing data between tasks • Triggers: cron, events, API T1 T5 T3 T4 T2 Start T6 T7 T8 T9
  • 6.
    6 Workflow Definition Snippet Workflow:! tasks:! ...! runJob:! action:Demo.runJob! on-error: deleteVM! on-success: sendJobError! publish:! job_results: job_results! ! deleteVM:! action: Nova:deleteVM! on-finish: end! ! sendJobError:! action: std.email! parameters:! to: admin@mycompany.com! subject: Workflow completed! body: | ! ! ! Workflow {$.execution.id} completed ! with results {$.job_results}! on-finish: deleteVM! ...
  • 7.
    7 Data Flow Concept RunVM input: image_id SendMsg input: person.email IsApprover input: person.name GoOn “image_id”:“123”, “person”: { “name”: “John”, “email”: “john@me.com" } “image_id”: “123”, “person”: { “name”: “John”, “email”: “john@me.com” }, “vm_ip”: “10.0.0.3” “image_id”: “123”, “person”: { “name”: “John”, “email”: “john@me.com” }, “vm_ip”: “10.0.0.3” “image_id”: “123”, “person”: { “name”: “John”, “email”: “john@me.com” }, “vm_ip”: “10.0.0.3”, “approver”: “true” “image_id”: “123”, “person”: { “name”: “John”, “email”: “john@me.com” }, “vm_ip”: “10.0.0.3”, “sent_msg”: “true” “image_id”: “123”, “person”: { “name”: “John”, “email”: “john@me.com” }, “vm_ip”: “10.0.0.3”, “approver”: “true”, “sent_msg”: “true” - task input - task result (added into context)
  • 8.
  • 9.
  • 10.
  • 11.
    11 What Works Now(ver 0.0.2) • API, Engine, Executor, Scheduler, scale up/down • Workflow DSL * • Workflow engine: • Control flow - transitions, conditions * • Data Flow • Basic tasks: ssh, HTTP/REST, email, echo • ‘Ad-hoc’ actions • Keystone integration • CLI and Python client • Horizon dashboard *
  • 12.
    12 Next • Apply POClearnings • Finalize workflow capabilities • Simplify definition syntax • Create content • openstack action pack: nova, heat • more standard actions • Provide extensibility for writing custom actions • Work on maturity, stability, performance • Horizon Dashboard improvements
  • 13.
    13 Future • Ceilometer Triggers •Event Listeners • Rollback • Horizon Dashboard - Graphical Workflow Presentation • “Dry-run” mode • TaskFlow integration, when ready
  • 14.
    14 More Info Discussion etherpad:! https://etherpad.openstack.org/p/juno-summit-mistral ! More info: • Mistral POC Demo: Expo E-26 booth • Launchpad: https://launchpad.net/mistral • Wiki: https://wiki.openstack.org/wiki/Mistral • Screencast: https://www.youtube.com/watch?v=x- zqz1CRVkI (search for “OpenStack Mistral POC Demo”)