Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Mistral OpenStack Meetup Feb 5

5,121 views

Published on

Presentation about Mistral (Workflow Service) from SF Bay Area OpenStack meetup held at Yahoo! on Feb 6th.

Published in: Technology
  • Be the first to comment

Mistral OpenStack Meetup Feb 5

  1. 1. Mistral 
 Workflow as a Service Renat Akhmerov
 Senior Software Engineer © MIRANTIS 2014 !1
  2. 2. Agenda • • • • • • • • Project Mission Project users Use Cases Main Features DSL, Data Flow concept, Architecture Current status Live Demo Roadmap and Contribution © MIRANTIS 2014 !2
  3. 3. Workflows all Around Find a job Stanford, Harvard, etc. Make money © MIRANTIS 2014 !3
  4. 4. Project Mission Provide a mechanism to define and execute tasks and workflows without writing the code © MIRANTIS 2014 !4
  5. 5. Who are Mistral Users? • OpenStack developers: Nova, Heat, Murano, Solum etc. • Administrators: capabilities like Cloud Cron to schedule periodic helper cloud tasks (local VM processes, backups etc.) • Enterprise application developers: automate business processes consisting of multiple distributed processing steps © MIRANTIS 2014 !5
  6. 6. Use Cases • Cloud Cron: a system administrator can schedule cloud tasks for periodical execution • Cloud Environment Deployment: tools like Heat can represent deployment steps as a workflow and use Mistral to run it • Live Migration: on CPU close to 100% run specific migration workflow • • Long-Running Business Process Big Data Analysis & Reporting © MIRANTIS 2014 !6
  7. 7. Cloud Cron • • Start a workflow on event (timer) • Wide range of tasks: local Single point of configuration and control for helper periodic jobs processes, OpenStack services (Nova, Cinder, Swift etc.), email, SMS, custom actions • • Monitoring & Management HA & Scalability © MIRANTIS 2014 !7
  8. 8. Main Features • • • RESTful Web service • • • • • Tasks associated with pluggable actions Based on Domain Specific Language (DSL) Runs workflows consisting of synchronous and asynchronous tasks Built-in actions: HTTP request, AMQP message Runs workflows on event (timer, ceilometer alarm) Provides history and state of currently running workflows HA and Scalability for workflow executions © MIRANTIS 2014 !8
  9. 9. Mistral Workflow • • • • • • • Graph of tasks T7 Task Actions (HTTP, AMQP, SSH, email, etc.) T9 T6 T8 Transitions depends Dependencies T3 Data Flow T2 T1 T4 Conditions start Events (triggers) © MIRANTIS 2014 T5 !9
  10. 10. Mistral DSL Capabilities • • Task dependencies (task3 depends on task2 and task1) • Conditional expressions in dependencies and direct transitions (based on Data Flow) • • • Task action declarations (signals over HTTP, AMQP) Direct transitions (on task1 success execute task2, on error task3) Standard actions (std:repeat, std:email) Cron events (run workflow hourly) © MIRANTIS 2014 !10
  11. 11. DSL Snippet Workflow: tasks: runJob: requires: createVM: $.vm_ip != null action: DemoApp:runJob on-success: deleteVM on-error: sendJobError ! ! deleteVM: action: Nova:deleteVM on-finish: end sendJobError: action: std:email parameters: address: admin@mycompany.com subject: Workflow error body: Error occurred while running a VM job in execution {$.execution.id} on-finish: deleteVM © MIRANTIS 2014 !11
  12. 12. Data Flow (concept) “image_id”: “123”, “person”: { “name”: “John”, “email”: “me@me.com” }, “vm_ip”: “10.0.0.3” IsMale input: person.name RunVM “image_id”: “123”, “person”: { “name”: “John”, “email”: “me@me.com” }, “vm_ip”: “10.0.0.3”, “male”: “true” GoOn input: “image_id”: “123”, image_id “person”: { “name”: “John”, “email”: “me@me.com” } SendMsg “image_id”: “123”, “person”: { “name”: “John”, “email”: “me@me.com” }, “vm_ip”: “10.0.0.3” - task input input: person.email “image_id”: “123”, “person”: { “name”: “John”, “email”: “me@me.com” }, “vm_ip”: “10.0.0.3”, “sent_msg”: “true” “image_id”: “123”, “person”: { “name”: “John”, “email”: “me@me.com” }, “vm_ip”: “10.0.0.3”, “male”: “true”, “sent_msg”: “true” - task result (added into context) © MIRANTIS 2014 !12
  13. 13. Architecture Workflow Queue Engine Scheduler API Server workflows executions tasks events Task Queue Task Executor ! ... Task Executor © MIRANTIS 2014 !13
  14. 14. Current Status • • • • Basic task dependency based model works Pilot is currently being worked on Launchpad home page o Code in stackforge code repos o o • • https://launchpad.net/mistral o https://github.com/stackforge/mistral https://github.com/stackforge/mistral-extra https://github.com/stackforge/python-mistralclient Description on OpenStack Wiki o wiki.openstack.org/wiki/Mistral Contributors: Mirantis, StackStorm © MIRANTIS 2014 !14
  15. 15. Live Demo Demo App 1. Upload workbook 4) sk (ta 4 Put Service on hold w REST endpoint rk flo 3 Mistral n Ru 5 Backup User Data Backup Service Data 2. HTTP wo 4 Execute Backup © MIRANTIS 2014 !15
  16. 16. Mistral Icehouse Roadmap • • End of February, 2014. Pilot. o Basic DSL capabilities (dependencies, direct transitions, conditionals, cron events) o Keystone Integration o REST API o Python Client April, 2014. Release 0.1 o Overall stability o Refined DSL o Mistral CLI o Horizon Dashboard (Workbook CRUD operations, workflow execution state, history) © MIRANTIS 2014 !16
  17. 17. Contribution Opportunities • • • • • • • • • Blueprints for additional capabilities Seed initial dev team Web UI (history, workflow lifecycle management) Ceilometer integration Integration with deployment tools (Puppet, Chef) Advanced capabilities of Python Client (building workflow step-by-step locally, callbacks etc.) Workflow rollback CLI for Mistral API CLI for Mistral action handlers © MIRANTIS 2014 !17
  18. 18. How to Become a Contributor? Install Python, git etc. Learn Python Register at Launchpad Make changes clone mistral repos Sign OpenStack CLA tor life -cy cle Buy a computer Co ntr ibu no Send patches Have coffee Tired? yes © MIRANTIS 2014 !18
  19. 19. Thanks for Attending! Project: launchpad.net/mistral IRC: #openstack-mistral Me: Renat Akhmerov Email: rakhmerov@mirantis.com © MIRANTIS 2014 Q&A !19

×