Introduction to machine learning on Azure data bricks environment gives you fair idea to kick start ML with the best tools and libraries available. It Covers: Why Machine learning Life Cycle Why Azure Databricks Available ML options in Azure Databricks MLFlow H20.ai

2. INTRODUCTION
• SOFTWARE ENGINEER WORKING WITH MICROSOFT
• 12 YEARS OF EXPERIENCE WORKING ON SQL, DWH,BI ,AZURE AND BIG DATA
TECHNOLOGIES LIKE SPARK, DATABRICKS, HDINSIGHT, IOT ETC.
• CATCH ME ON LINKEDIN OR VIA MY BLOG
• HTTPS://WWW.LINKEDIN.COM/IN/PRAMODSINGLA
• HTTPS://PRAMODSINGLA.COM/

3. TOPICS
Why Machine
learning
Life cycle of
machine learning
Challenges
Why Azure
Databricks
ML Using Azure
Databricks
Set up Databricks
Runtime ML
MLFlow H2O Demo
Available Machine
learning options in
Azure Databricks

4. WHAT AND WHY
MACHINE
LEARNING

5. DATA
SCIENCE
LIFE CYCLE

6. CHALLENGES
Collect, Clean and Process Data 80% time spent
Problem Formulation
Choosing algorithms and parameters
Speed (GPUs)
myriad tools and Libraries
Hard to track experiments
Hard to reproduce results
Hard to deploy ML

7. WHY AZURE DATABRICKS

8. AVAILABLE MACHINE LEARNING OPTIONS IN
AZURE DATABRICKS
•DATABRICKS RUNTIME ML
•APACHE SPARK MLLIB
•THIRD PARTY ML LIBRARY SUPPORT E.G. H2O
•MLFLOW

9. DATABRICKS
RUNTIME ML
PRELOADE
LIBARIRES
(SAMPLE
CLUSTER
CONFIG ON
NEXT SLIDE)
Category Libraries
Distributed Deep
Learning
Distributed training with Horovod and Spark:
Distributed TensorFlow and Keras prediction:
Deep Learning
Keras:
TensorFlow:
GPU libraries:
XGBoost XGBoost4j
Other machine learning
libraries
numpy
scikit-learn
scipy

11. MLFLOW
• AN OPEN SOURCE PLATFORM FOR MANAGING THE END-TO-END MACHINE LEARNING
LIFECYCLE
• ORGANIZED INTO THREE COMPONENTS: TRACKING, PROJECTS, AND MODELS.
• TRACKING  API AND UI FOR LOGGING PARAMETERS, DATA, CODE AND RESULTS .
TRACKING SERVER
• PROJECTS 
• CAPTURE WHOLE ENVIRONMENT INCLUDING DEPENDENT LIBRARIES (
PLATFORM AND DATA SCIENTIST INDEPENDENT).
• SIMPLY A DIRECTORY WITH CODE OR A GIT REPOSITORY, AND USES A
DESCRIPTOR FILE OR SIMPLY CONVENTION TO SPECIFY ITS DEPENDENCIES
AND HOW TO RUN THE CODE.
• MODELS MANAGING AND DEPLOYING MODELS FROM A VARIETY OF ML LIBRARIES
TO A VARIETY OF MODEL SERVING AND INFERENCE PLATFORMS

13. H2O SPARKLING
WATER
• BEST ML LIBRARIES FOR
SPARK
• LEADERS IN MAGIC QUADRANT
• SUPPORT BOTH SUPERVISED
AND UNSUPERVISED MODELS
• E.G. RANDOM FOREST, GLM,
GBM, XGBOOST, GLRM,
WORD2VEC AND MANY MORE

14. MLFLOW AND
H2O