In this talk, I will introducing Mass, a Python package, that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, (visualization), handling failures, command line integration, and much more.
The purpose of Mass is to address all the plumbing typically associated with long-running batch processes. You want to chain many tasks, automate them, and failures will happen. These tasks can be anything, but are typically long running things like video encoding, rendering for computer graphics, DB dump, or complex algorithm on big data.
The main features of Mass:
* No centralized server.
* Pythonic code snippets to represent a job
* On-premise, cloud-based or even mixed mass workers are allowed.
Let's start your mass job and enjoy it!