Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Introduction to
Aaron Davidson
July 19, 2018
Outline
ML development challenges
How MLflow tackles these
Demo
Roadmap
Machine Learning
Development is Complex
ML Lifecycle
4
Delta
Data Prep
Training
Deploy
Raw Data
μ
λ θ Tuning
Scale
μ
λ θ Tuning
Scale
Scale
Scale
Model
Exchange
G...
ML Development Challenges
100s of software tools to leverage
Hard to track & reproduceresults: code, data, params,etc
Hard...
Introducing
Open machine learningplatform
• Works with any ML library& language
• Runs the same wayanywhere (e.g. any clou...
MLflow Design Philosophy
1. “API-first”, open platform
• Allow submittingruns,models,etc from anylibrary & language
• Exam...
MLflow Design Philosophy
2. Modular design
• Let people use different components individually(e.g.,use
MLflow’s project fo...
Why Open Source?
Everyone is solvinga similarproblem
Lots of benefits in having a common API across orgs
• Can open source...
MLflow Components
10
Tracking
Record and query
experiments: code,
data, config, results
Projects
Packaging format
for repr...
Notebooks
LocalApps
CloudJobs
Tracking Server
UI
API
MLflow	Tracking
Python or
REST API
Key Concepts in Tracking
Parameters: key-value inputs to your code
Metrics: numeric values (can update over time)
Artifact...
Project Spec
Code DataConfig
LocalExecution
Remote Execution
MLflow	Projects
Example MLflow Project
my_project/
├── MLproject
│
│
│
│
│
├── conda.yaml
├── main.py
└── model.py
...
conda_env: conda.ya...
Model Format
Flavor 2Flavor 1
Run Sources
InferenceCode
Batch & Stream Scoring
Cloud ServingTools
MLflow	Models
Simple mod...
Example MLflow Model
my_model/
├── MLmodel
│
│
│
│
│
└── estimator/
├── saved_model.pb
└── variables/
...
Usable by tools ...
Demo
Roadmap
Current Status
MLflow is still alpha, so expect things to break
• But send input or patches on GitHub!
Just made0.3.0 rele...
Longer-Term Roadmap
1. Improvingcurrent components
• Pluggable execution backends for mlflow.run
• Database-backed trackin...
Longer-Term Roadmap
2. MLflow Data component
• Let MLflowprojects load data from diverse formats (e.g. CSV vs
Parquet) so ...
Longer-Term Roadmap
3. Hyperparametertuning
• Integrate with common hyperparameter tuning libraries
• Make it easier to la...
Longer-Term Roadmap
4. Language and libraryintegrations
• Java and R are high on our list for APIs
• Built-in Spark MLlib ...
Contributingto MLflow
Submit issues and patches on GitHub
• We’re using it for all our development & issue tracking
• See ...
Conclusion
Powerful workflow tools can simplifythe ML lifecycle
• Improve usability for both data scientists and engineers...
Upcoming SlideShare
Loading in …5
×

of

MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 1 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 2 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 3 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 4 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 5 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 6 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 7 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 8 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 9 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 10 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 11 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 12 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 13 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 14 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 15 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 16 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 17 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 18 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 19 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 20 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 21 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 22 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 23 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 24 MLflow: Infrastructure for a Complete Machine Learning Life Cycle Slide 25
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

12 Likes

Share

Download to read offline

MLflow: Infrastructure for a Complete Machine Learning Life Cycle

Download to read offline

ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure.

In this talk, we will present MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

MLflow: Infrastructure for a Complete Machine Learning Life Cycle

  1. 1. Introduction to Aaron Davidson July 19, 2018
  2. 2. Outline ML development challenges How MLflow tackles these Demo Roadmap
  3. 3. Machine Learning Development is Complex
  4. 4. ML Lifecycle 4 Delta Data Prep Training Deploy Raw Data μ λ θ Tuning Scale μ λ θ Tuning Scale Scale Scale Model Exchange Governance
  5. 5. ML Development Challenges 100s of software tools to leverage Hard to track & reproduceresults: code, data, params,etc Hard to productionizemodels Needs large scale for best results
  6. 6. Introducing Open machine learningplatform • Works with any ML library& language • Runs the same wayanywhere (e.g. any cloud) • Designed to be useful for 1 or 100,000 person orgs
  7. 7. MLflow Design Philosophy 1. “API-first”, open platform • Allow submittingruns,models,etc from anylibrary & language • Example: a “model” can justbe a lambdafunction thatMLflow can thendeploy in many places (Docker, AzureML, Spark UDF, …) Key enabler: built aroundREST APIs and CLI
  8. 8. MLflow Design Philosophy 2. Modular design • Let people use different components individually(e.g.,use MLflow’s project format but not its deployment tools) • Easy to integrateinto existing ML platforms & workflows Key enabler: distinct components (Tracking/Projects/Models)
  9. 9. Why Open Source? Everyone is solvinga similarproblem Lots of benefits in having a common API across orgs • Can open source & share individualworkflow steps • ML tool developers can easily reach lots of users – E.g. a new ML library canuse MLflow Models to reach many serving tools
  10. 10. MLflow Components 10 Tracking Record and query experiments: code, data, config, results Projects Packaging format for reproducible runs on any platform Models General model format that supports diverse deployment tools
  11. 11. Notebooks LocalApps CloudJobs Tracking Server UI API MLflow Tracking Python or REST API
  12. 12. Key Concepts in Tracking Parameters: key-value inputs to your code Metrics: numeric values (can update over time) Artifacts: arbitrary files, including models Source: what code ran?
  13. 13. Project Spec Code DataConfig LocalExecution Remote Execution MLflow Projects
  14. 14. Example MLflow Project my_project/ ├── MLproject │ │ │ │ │ ├── conda.yaml ├── main.py └── model.py ... conda_env: conda.yaml entry_points: main: parameters: training_data: path lambda: {type: float, default: 0.1} command: python main.py {training_data} {lambda} $ mlflow run git://<my_project> mlflow.run(“git://<my_project>”, ...)
  15. 15. Model Format Flavor 2Flavor 1 Run Sources InferenceCode Batch & Stream Scoring Cloud ServingTools MLflow Models Simple model flavors usableby many tools
  16. 16. Example MLflow Model my_model/ ├── MLmodel │ │ │ │ │ └── estimator/ ├── saved_model.pb └── variables/ ... Usable by tools that understand TensorFlowmodel format Usable by any tool that can run Python (Docker,Spark,etc!) run_id: 769915006efd4c4bbd662461 time_created: 2018-06-28T12:34 flavors: tensorflow: saved_model_dir: estimator signature_def_key: predict python_function: loader_module: mlflow.tensorflow
  17. 17. Demo
  18. 18. Roadmap
  19. 19. Current Status MLflow is still alpha, so expect things to break • But send input or patches on GitHub! Just made0.3.0 release • SparkML integration (model logging & serving) • GCSartifact support • Doc, example and API improvements
  20. 20. Longer-Term Roadmap 1. Improvingcurrent components • Pluggable execution backends for mlflow.run • Database-backed tracking store (already a pluggable API) • Model metadata (e.g. required input schema) • Easier support for multi-step workflows
  21. 21. Longer-Term Roadmap 2. MLflow Data component • Let MLflowprojects load data from diverse formats (e.g. CSV vs Parquet) so you don’t have to pick a format in advance • Will build on Spark’s Data SourceAPI
  22. 22. Longer-Term Roadmap 3. Hyperparametertuning • Integrate with common hyperparameter tuning libraries • Make it easier to launch & track many runs in parallel (already possible but kind of awkward)
  23. 23. Longer-Term Roadmap 4. Language and libraryintegrations • Java and R are high on our list for APIs • Built-in Spark MLlib and PyTorchintegrations • Demonstrate how to use MLflow with other libraries (it’s easy) Let us know if you have other roadmap ideas!
  24. 24. Contributingto MLflow Submit issues and patches on GitHub • We’re using it for all our development & issue tracking • See CONTRIBUTING.rstfor how to run dev builds Join our mailinglist: tinyurl.com/mlflow-users Join our Slack: tinyurl.com/mlflow-slack
  25. 25. Conclusion Powerful workflow tools can simplifythe ML lifecycle • Improve usability for both data scientists and engineers • Same way that software dev lifecycle tools simplify dev MLflow is a lightweight, open platform that integrates easily into existing workflows
  • FranoisPaupier

    Feb. 20, 2020
  • dijonkock

    Sep. 4, 2019
  • rcolomina77

    Jul. 16, 2019
  • DmitryPogodin

    Jul. 4, 2019
  • zahernourredine

    Dec. 7, 2018
  • SorchaHealy

    Nov. 13, 2018
  • drewlll2ll

    Sep. 6, 2018
  • DanSun2

    Sep. 2, 2018
  • nickchervov

    Aug. 31, 2018
  • ssuser16965f

    Aug. 20, 2018
  • ssuser16965f

    Aug. 20, 2018
  • DylanCunningham1

    Aug. 7, 2018

ML development brings many new complexities beyond the traditional software development lifecycle. Unlike in traditional software development, ML developers want to try multiple algorithms, tools and parameters to get the best results, and they need to track this information to reproduce work. In addition, developers need to use many distinct systems to productionize models. To address these problems, many companies are building custom “ML platforms” that automate this lifecycle, but even these platforms are limited to a few supported algorithms and to each company’s internal infrastructure. In this talk, we will present MLflow, a new open source project from Databricks that aims to design an open ML platform where organizations can use any ML library and development tool of their choice to reliably build and share ML applications. MLflow introduces simple abstractions to package reproducible projects, track results, and encapsulate models that can be used with many existing tools, accelerating the ML lifecycle for organizations of any size.

Views

Total views

4,856

On Slideshare

0

From embeds

0

Number of embeds

12

Actions

Downloads

152

Shares

0

Comments

0

Likes

12

×