SlideShare a Scribd company logo
INTRODUCTION
• SOFTWARE ENGINEER WORKING WITH MICROSOFT
• 12 YEARS OF EXPERIENCE WORKING ON SQL, DWH,BI ,AZURE AND BIG DATA
TECHNOLOGIES LIKE SPARK, DATABRICKS, HDINSIGHT, IOT ETC.
• CATCH ME ON LINKEDIN OR VIA MY BLOG
• HTTPS://WWW.LINKEDIN.COM/IN/PRAMODSINGLA
• HTTPS://PRAMODSINGLA.COM/
TOPICS
Why Machine
learning
Life cycle of
machine learning
Challenges
Why Azure
Databricks
ML Using Azure
Databricks
Set up Databricks
Runtime ML
MLFlow H2O Demo
Available Machine
learning options in
Azure Databricks
WHAT AND WHY
MACHINE
LEARNING
DATA
SCIENCE
LIFE CYCLE
CHALLENGES
Collect, Clean and Process Data 80% time spent
Problem Formulation
Choosing algorithms and parameters
Speed (GPUs)
myriad tools and Libraries
Hard to track experiments
Hard to reproduce results
Hard to deploy ML
WHY AZURE DATABRICKS
AVAILABLE MACHINE LEARNING OPTIONS IN
AZURE DATABRICKS
•DATABRICKS RUNTIME ML
•APACHE SPARK MLLIB
•THIRD PARTY ML LIBRARY SUPPORT E.G. H2O
•MLFLOW
DATABRICKS
RUNTIME ML
PRELOADE
LIBARIRES
(SAMPLE
CLUSTER
CONFIG ON
NEXT SLIDE)
Category Libraries
Distributed Deep
Learning
Distributed training with Horovod and Spark:
Distributed TensorFlow and Keras prediction:
Deep Learning
Keras:
TensorFlow:
GPU libraries:
XGBoost XGBoost4j
Other machine learning
libraries
numpy
scikit-learn
scipy
MLFLOW
• AN OPEN SOURCE PLATFORM FOR MANAGING THE END-TO-END MACHINE LEARNING
LIFECYCLE
• ORGANIZED INTO THREE COMPONENTS: TRACKING, PROJECTS, AND MODELS.
• TRACKING  API AND UI FOR LOGGING PARAMETERS, DATA, CODE AND RESULTS .
TRACKING SERVER
• PROJECTS 
• CAPTURE WHOLE ENVIRONMENT INCLUDING DEPENDENT LIBRARIES (
PLATFORM AND DATA SCIENTIST INDEPENDENT).
• SIMPLY A DIRECTORY WITH CODE OR A GIT REPOSITORY, AND USES A
DESCRIPTOR FILE OR SIMPLY CONVENTION TO SPECIFY ITS DEPENDENCIES
AND HOW TO RUN THE CODE.
• MODELS MANAGING AND DEPLOYING MODELS FROM A VARIETY OF ML LIBRARIES
TO A VARIETY OF MODEL SERVING AND INFERENCE PLATFORMS
H2O SPARKLING
WATER
• BEST ML LIBRARIES FOR
SPARK
• LEADERS IN MAGIC QUADRANT
• SUPPORT BOTH SUPERVISED
AND UNSUPERVISED MODELS
• E.G. RANDOM FOREST, GLM,
GBM, XGBOOST, GLRM,
WORD2VEC AND MANY MORE
MLFLOW AND
H2O
DPGD Microsoft Hyderabad 22nd Sept 2018

More Related Content

What's hot

Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sri Ambati
 
Azure databricks ml
Azure databricks mlAzure databricks ml
Azure databricks ml
Vishwas N
 
Azure cognitive service
Azure cognitive serviceAzure cognitive service
Azure cognitive service
Vishwas N
 
Scalable Machine Learning using R and Azure HDInsight - Parashar
Scalable Machine Learning using R and Azure HDInsight - ParasharScalable Machine Learning using R and Azure HDInsight - Parashar
Scalable Machine Learning using R and Azure HDInsight - Parashar
Parashar Shah
 
Big Data Advanced Analytics on Microsoft Azure 201904
Big Data Advanced Analytics on Microsoft Azure 201904Big Data Advanced Analytics on Microsoft Azure 201904
Big Data Advanced Analytics on Microsoft Azure 201904
Mark Tabladillo
 
Skutil - H2O meets Sklearn - Taylor Smith
Skutil - H2O meets Sklearn - Taylor SmithSkutil - H2O meets Sklearn - Taylor Smith
Skutil - H2O meets Sklearn - Taylor Smith
Sri Ambati
 
Intro to H2O Machine Learning in R at Santa Clara University
Intro to H2O Machine Learning in R at Santa Clara UniversityIntro to H2O Machine Learning in R at Santa Clara University
Intro to H2O Machine Learning in R at Santa Clara University
Sri Ambati
 
Elasticsearch in production Boston Meetup October 2014
Elasticsearch in production Boston Meetup October 2014Elasticsearch in production Boston Meetup October 2014
Elasticsearch in production Boston Meetup October 2014
beiske
 
Azure Machine Learning
Azure Machine LearningAzure Machine Learning
Azure Machine Learning
Dmitry Petukhov
 
Microsoft Machine Learning Server. Architecture View
Microsoft Machine Learning Server. Architecture ViewMicrosoft Machine Learning Server. Architecture View
Microsoft Machine Learning Server. Architecture View
Dmitry Petukhov
 
Intro to H2O in Python - Data Science LA
Intro to H2O in Python - Data Science LAIntro to H2O in Python - Data Science LA
Intro to H2O in Python - Data Science LA
Sri Ambati
 
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
Rui Romano
 
2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review
Hang Li
 
"Something worth ten Microsofts" - Azure Machine Learning 101
"Something worth ten Microsofts" - Azure Machine Learning 101 "Something worth ten Microsofts" - Azure Machine Learning 101
"Something worth ten Microsofts" - Azure Machine Learning 101
Haritha Thilakarathne
 
Introduction to Data Science and Machine Learning - Circles Hub Meetup
Introduction to Data Science and Machine Learning - Circles Hub MeetupIntroduction to Data Science and Machine Learning - Circles Hub Meetup
Introduction to Data Science and Machine Learning - Circles Hub Meetup
Ahmed Moatasem
 
Introduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and PythonIntroduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and Python
Sri Ambati
 
AI on the edge
AI on the edgeAI on the edge
AI on the edge
Riccardo Cappello
 
Machine Learning | Microsoft Azure
Machine Learning | Microsoft AzureMachine Learning | Microsoft Azure
Machine Learning | Microsoft Azure
Ben Zahra Anouar
 
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
Mark Tabladillo
 
Stacked Ensembles in H2O
Stacked Ensembles in H2OStacked Ensembles in H2O
Stacked Ensembles in H2O
Sri Ambati
 

What's hot (20)

Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
Sundar Ranganathan, NetApp + Vinod Iyengar, H2O.ai - Driverless AI integratio...
 
Azure databricks ml
Azure databricks mlAzure databricks ml
Azure databricks ml
 
Azure cognitive service
Azure cognitive serviceAzure cognitive service
Azure cognitive service
 
Scalable Machine Learning using R and Azure HDInsight - Parashar
Scalable Machine Learning using R and Azure HDInsight - ParasharScalable Machine Learning using R and Azure HDInsight - Parashar
Scalable Machine Learning using R and Azure HDInsight - Parashar
 
Big Data Advanced Analytics on Microsoft Azure 201904
Big Data Advanced Analytics on Microsoft Azure 201904Big Data Advanced Analytics on Microsoft Azure 201904
Big Data Advanced Analytics on Microsoft Azure 201904
 
Skutil - H2O meets Sklearn - Taylor Smith
Skutil - H2O meets Sklearn - Taylor SmithSkutil - H2O meets Sklearn - Taylor Smith
Skutil - H2O meets Sklearn - Taylor Smith
 
Intro to H2O Machine Learning in R at Santa Clara University
Intro to H2O Machine Learning in R at Santa Clara UniversityIntro to H2O Machine Learning in R at Santa Clara University
Intro to H2O Machine Learning in R at Santa Clara University
 
Elasticsearch in production Boston Meetup October 2014
Elasticsearch in production Boston Meetup October 2014Elasticsearch in production Boston Meetup October 2014
Elasticsearch in production Boston Meetup October 2014
 
Azure Machine Learning
Azure Machine LearningAzure Machine Learning
Azure Machine Learning
 
Microsoft Machine Learning Server. Architecture View
Microsoft Machine Learning Server. Architecture ViewMicrosoft Machine Learning Server. Architecture View
Microsoft Machine Learning Server. Architecture View
 
Intro to H2O in Python - Data Science LA
Intro to H2O in Python - Data Science LAIntro to H2O in Python - Data Science LA
Intro to H2O in Python - Data Science LA
 
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
Top 10 of Data & BI Summit Series: Power BI Tips & Tricks from the Trenches
 
2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review
 
"Something worth ten Microsofts" - Azure Machine Learning 101
"Something worth ten Microsofts" - Azure Machine Learning 101 "Something worth ten Microsofts" - Azure Machine Learning 101
"Something worth ten Microsofts" - Azure Machine Learning 101
 
Introduction to Data Science and Machine Learning - Circles Hub Meetup
Introduction to Data Science and Machine Learning - Circles Hub MeetupIntroduction to Data Science and Machine Learning - Circles Hub Meetup
Introduction to Data Science and Machine Learning - Circles Hub Meetup
 
Introduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and PythonIntroduction to Machine Learning with H2O and Python
Introduction to Machine Learning with H2O and Python
 
AI on the edge
AI on the edgeAI on the edge
AI on the edge
 
Machine Learning | Microsoft Azure
Machine Learning | Microsoft AzureMachine Learning | Microsoft Azure
Machine Learning | Microsoft Azure
 
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
201905 Azure Certification DP-100: Designing and Implementing a Data Science ...
 
Stacked Ensembles in H2O
Stacked Ensembles in H2OStacked Ensembles in H2O
Stacked Ensembles in H2O
 

Similar to DPGD Microsoft Hyderabad 22nd Sept 2018

Delivering Insights from 20M+ Smart Homes with 500M+ Devices
Delivering Insights from 20M+ Smart Homes with 500M+ DevicesDelivering Insights from 20M+ Smart Homes with 500M+ Devices
Delivering Insights from 20M+ Smart Homes with 500M+ Devices
Databricks
 
Making Data Scientists Productive in Azure
Making Data Scientists Productive in AzureMaking Data Scientists Productive in Azure
Making Data Scientists Productive in Azure
Valdas Maksimavičius
 
Large-Scale Data Science on Hadoop (Intel Big Data Day)
Large-Scale Data Science on Hadoop (Intel Big Data Day)Large-Scale Data Science on Hadoop (Intel Big Data Day)
Large-Scale Data Science on Hadoop (Intel Big Data Day)
Uri Laserson
 
Part 2: A Visual Dive into Machine Learning and Deep Learning 

Part 2: A Visual Dive into Machine Learning and Deep Learning 
Part 2: A Visual Dive into Machine Learning and Deep Learning 

Part 2: A Visual Dive into Machine Learning and Deep Learning 

Cloudera, Inc.
 
With Automated ML, is Everyone an ML Engineer?
With Automated ML, is Everyone an ML Engineer?With Automated ML, is Everyone an ML Engineer?
With Automated ML, is Everyone an ML Engineer?
Dan Sullivan, Ph.D.
 
Part 3: Models in Production: A Look From Beginning to End
Part 3: Models in Production: A Look From Beginning to EndPart 3: Models in Production: A Look From Beginning to End
Part 3: Models in Production: A Look From Beginning to End
Cloudera, Inc.
 
Unconference Round Table Notes
Unconference Round Table NotesUnconference Round Table Notes
Unconference Round Table Notes
Timothy Spann
 
Big Data Adavnced Analytics on Microsoft Azure
Big Data Adavnced Analytics on Microsoft AzureBig Data Adavnced Analytics on Microsoft Azure
Big Data Adavnced Analytics on Microsoft Azure
Mark Tabladillo
 
Machine Learning and AI
Machine Learning and AIMachine Learning and AI
Machine Learning and AI
James Serra
 
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
Bruce Kozuma
 
201908 Overview of Automated ML
201908 Overview of Automated ML201908 Overview of Automated ML
201908 Overview of Automated ML
Mark Tabladillo
 
Introduction to H2O and Model Stacking Use Cases
Introduction to H2O and Model Stacking Use CasesIntroduction to H2O and Model Stacking Use Cases
Introduction to H2O and Model Stacking Use Cases
Jo-fai Chow
 
Ai & Data Analytics 2018 - Azure Databricks for data scientist
Ai & Data Analytics 2018 - Azure Databricks for data scientistAi & Data Analytics 2018 - Azure Databricks for data scientist
Ai & Data Analytics 2018 - Azure Databricks for data scientist
Alberto Diaz Martin
 
Data Science and CDSW
Data Science and CDSWData Science and CDSW
Data Science and CDSW
Jason Hubbard
 
Teaching Apache Spark: Demonstrations on the Databricks Cloud Platform
Teaching Apache Spark: Demonstrations on the Databricks Cloud PlatformTeaching Apache Spark: Demonstrations on the Databricks Cloud Platform
Teaching Apache Spark: Demonstrations on the Databricks Cloud Platform
Yao Yao
 
Managing Enterprise Data Science 201904
Managing Enterprise Data Science 201904Managing Enterprise Data Science 201904
Managing Enterprise Data Science 201904
Mark Tabladillo
 
10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide
Databricks
 
Cloud ops
Cloud opsCloud ops
Cloud ops
jayaradhaa
 
Productionizing Machine Learning Pipelines with Databricks and Azure ML
Productionizing Machine Learning Pipelines with Databricks and Azure MLProductionizing Machine Learning Pipelines with Databricks and Azure ML
Productionizing Machine Learning Pipelines with Databricks and Azure ML
Databricks
 
Azure ML: from basic to integration with custom applications
Azure ML: from basic to integration with custom applicationsAzure ML: from basic to integration with custom applications
Azure ML: from basic to integration with custom applications
Davide Mauri
 

Similar to DPGD Microsoft Hyderabad 22nd Sept 2018 (20)

Delivering Insights from 20M+ Smart Homes with 500M+ Devices
Delivering Insights from 20M+ Smart Homes with 500M+ DevicesDelivering Insights from 20M+ Smart Homes with 500M+ Devices
Delivering Insights from 20M+ Smart Homes with 500M+ Devices
 
Making Data Scientists Productive in Azure
Making Data Scientists Productive in AzureMaking Data Scientists Productive in Azure
Making Data Scientists Productive in Azure
 
Large-Scale Data Science on Hadoop (Intel Big Data Day)
Large-Scale Data Science on Hadoop (Intel Big Data Day)Large-Scale Data Science on Hadoop (Intel Big Data Day)
Large-Scale Data Science on Hadoop (Intel Big Data Day)
 
Part 2: A Visual Dive into Machine Learning and Deep Learning 

Part 2: A Visual Dive into Machine Learning and Deep Learning 
Part 2: A Visual Dive into Machine Learning and Deep Learning 

Part 2: A Visual Dive into Machine Learning and Deep Learning 

 
With Automated ML, is Everyone an ML Engineer?
With Automated ML, is Everyone an ML Engineer?With Automated ML, is Everyone an ML Engineer?
With Automated ML, is Everyone an ML Engineer?
 
Part 3: Models in Production: A Look From Beginning to End
Part 3: Models in Production: A Look From Beginning to EndPart 3: Models in Production: A Look From Beginning to End
Part 3: Models in Production: A Look From Beginning to End
 
Unconference Round Table Notes
Unconference Round Table NotesUnconference Round Table Notes
Unconference Round Table Notes
 
Big Data Adavnced Analytics on Microsoft Azure
Big Data Adavnced Analytics on Microsoft AzureBig Data Adavnced Analytics on Microsoft Azure
Big Data Adavnced Analytics on Microsoft Azure
 
Machine Learning and AI
Machine Learning and AIMachine Learning and AI
Machine Learning and AI
 
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
2019-04-17 Bio-IT World G Suite-Jira Cloud Sample Tracking
 
201908 Overview of Automated ML
201908 Overview of Automated ML201908 Overview of Automated ML
201908 Overview of Automated ML
 
Introduction to H2O and Model Stacking Use Cases
Introduction to H2O and Model Stacking Use CasesIntroduction to H2O and Model Stacking Use Cases
Introduction to H2O and Model Stacking Use Cases
 
Ai & Data Analytics 2018 - Azure Databricks for data scientist
Ai & Data Analytics 2018 - Azure Databricks for data scientistAi & Data Analytics 2018 - Azure Databricks for data scientist
Ai & Data Analytics 2018 - Azure Databricks for data scientist
 
Data Science and CDSW
Data Science and CDSWData Science and CDSW
Data Science and CDSW
 
Teaching Apache Spark: Demonstrations on the Databricks Cloud Platform
Teaching Apache Spark: Demonstrations on the Databricks Cloud PlatformTeaching Apache Spark: Demonstrations on the Databricks Cloud Platform
Teaching Apache Spark: Demonstrations on the Databricks Cloud Platform
 
Managing Enterprise Data Science 201904
Managing Enterprise Data Science 201904Managing Enterprise Data Science 201904
Managing Enterprise Data Science 201904
 
10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide
 
Cloud ops
Cloud opsCloud ops
Cloud ops
 
Productionizing Machine Learning Pipelines with Databricks and Azure ML
Productionizing Machine Learning Pipelines with Databricks and Azure MLProductionizing Machine Learning Pipelines with Databricks and Azure ML
Productionizing Machine Learning Pipelines with Databricks and Azure ML
 
Azure ML: from basic to integration with custom applications
Azure ML: from basic to integration with custom applicationsAzure ML: from basic to integration with custom applications
Azure ML: from basic to integration with custom applications
 

More from Pramod Singla

9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
9\9 SSIS 2008R2_Training - Package Reliability and Package Execution9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
Pramod Singla
 
8\9 SSIS 2008R2_Training - Debugging_Package
8\9 SSIS 2008R2_Training - Debugging_Package8\9 SSIS 2008R2_Training - Debugging_Package
8\9 SSIS 2008R2_Training - Debugging_Package
Pramod Singla
 
7\9 SSIS 2008R2_Training - Script Task
7\9 SSIS 2008R2_Training - Script Task7\9 SSIS 2008R2_Training - Script Task
7\9 SSIS 2008R2_Training - Script Task
Pramod Singla
 
6.2\9 SSIS 2008R2_Training - DataFlow Transformations
6.2\9 SSIS 2008R2_Training - DataFlow Transformations6.2\9 SSIS 2008R2_Training - DataFlow Transformations
6.2\9 SSIS 2008R2_Training - DataFlow Transformations
Pramod Singla
 
6.1\9 SSIS 2008R2_Training - DataFlow Transformations
6.1\9 SSIS 2008R2_Training - DataFlow Transformations6.1\9 SSIS 2008R2_Training - DataFlow Transformations
6.1\9 SSIS 2008R2_Training - DataFlow Transformations
Pramod Singla
 
5\9 SSIS 2008R2_Training - DataFlow Basics
5\9 SSIS 2008R2_Training - DataFlow Basics5\9 SSIS 2008R2_Training - DataFlow Basics
5\9 SSIS 2008R2_Training - DataFlow Basics
Pramod Singla
 
4\9 SSIS 2008R2_Training - Expression and Variables
4\9 SSIS 2008R2_Training - Expression and Variables4\9 SSIS 2008R2_Training - Expression and Variables
4\9 SSIS 2008R2_Training - Expression and Variables
Pramod Singla
 
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
Pramod Singla
 
3.1\9 SSIS 2008R2_Training - ControlFlow asks
3.1\9 SSIS 2008R2_Training - ControlFlow asks3.1\9 SSIS 2008R2_Training - ControlFlow asks
3.1\9 SSIS 2008R2_Training - ControlFlow asks
Pramod Singla
 
2\9.SSIS 2008R2 _Training - Control Flow
2\9.SSIS 2008R2 _Training - Control Flow2\9.SSIS 2008R2 _Training - Control Flow
2\9.SSIS 2008R2 _Training - Control Flow
Pramod Singla
 
1\9.SSIS 2008R2_Training - Introduction to SSIS
1\9.SSIS 2008R2_Training - Introduction to SSIS1\9.SSIS 2008R2_Training - Introduction to SSIS
1\9.SSIS 2008R2_Training - Introduction to SSIS
Pramod Singla
 

More from Pramod Singla (11)

9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
9\9 SSIS 2008R2_Training - Package Reliability and Package Execution9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
9\9 SSIS 2008R2_Training - Package Reliability and Package Execution
 
8\9 SSIS 2008R2_Training - Debugging_Package
8\9 SSIS 2008R2_Training - Debugging_Package8\9 SSIS 2008R2_Training - Debugging_Package
8\9 SSIS 2008R2_Training - Debugging_Package
 
7\9 SSIS 2008R2_Training - Script Task
7\9 SSIS 2008R2_Training - Script Task7\9 SSIS 2008R2_Training - Script Task
7\9 SSIS 2008R2_Training - Script Task
 
6.2\9 SSIS 2008R2_Training - DataFlow Transformations
6.2\9 SSIS 2008R2_Training - DataFlow Transformations6.2\9 SSIS 2008R2_Training - DataFlow Transformations
6.2\9 SSIS 2008R2_Training - DataFlow Transformations
 
6.1\9 SSIS 2008R2_Training - DataFlow Transformations
6.1\9 SSIS 2008R2_Training - DataFlow Transformations6.1\9 SSIS 2008R2_Training - DataFlow Transformations
6.1\9 SSIS 2008R2_Training - DataFlow Transformations
 
5\9 SSIS 2008R2_Training - DataFlow Basics
5\9 SSIS 2008R2_Training - DataFlow Basics5\9 SSIS 2008R2_Training - DataFlow Basics
5\9 SSIS 2008R2_Training - DataFlow Basics
 
4\9 SSIS 2008R2_Training - Expression and Variables
4\9 SSIS 2008R2_Training - Expression and Variables4\9 SSIS 2008R2_Training - Expression and Variables
4\9 SSIS 2008R2_Training - Expression and Variables
 
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
3.2\9 SSIS 2008R2_Training - ControlFlow Tasks
 
3.1\9 SSIS 2008R2_Training - ControlFlow asks
3.1\9 SSIS 2008R2_Training - ControlFlow asks3.1\9 SSIS 2008R2_Training - ControlFlow asks
3.1\9 SSIS 2008R2_Training - ControlFlow asks
 
2\9.SSIS 2008R2 _Training - Control Flow
2\9.SSIS 2008R2 _Training - Control Flow2\9.SSIS 2008R2 _Training - Control Flow
2\9.SSIS 2008R2 _Training - Control Flow
 
1\9.SSIS 2008R2_Training - Introduction to SSIS
1\9.SSIS 2008R2_Training - Introduction to SSIS1\9.SSIS 2008R2_Training - Introduction to SSIS
1\9.SSIS 2008R2_Training - Introduction to SSIS
 

Recently uploaded

State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
kuntobimo2016
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
g4dpvqap0
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
fkyes25
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
ahzuo
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
javier ramirez
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
74nqk8xf
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
Bill641377
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
nuttdpt
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
Walaa Eldin Moustafa
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
mzpolocfi
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
74nqk8xf
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
bopyb
 

Recently uploaded (20)

State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023State of Artificial intelligence Report 2023
State of Artificial intelligence Report 2023
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
 
Natural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptxNatural Language Processing (NLP), RAG and its applications .pptx
Natural Language Processing (NLP), RAG and its applications .pptx
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...Population Growth in Bataan: The effects of population growth around rural pl...
Population Growth in Bataan: The effects of population growth around rural pl...
 
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
一比一原版(UCSB文凭证书)圣芭芭拉分校毕业证如何办理
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
一比一原版(Dalhousie毕业证书)达尔豪斯大学毕业证如何办理
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
 
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
 

DPGD Microsoft Hyderabad 22nd Sept 2018

  • 1.
  • 2. INTRODUCTION • SOFTWARE ENGINEER WORKING WITH MICROSOFT • 12 YEARS OF EXPERIENCE WORKING ON SQL, DWH,BI ,AZURE AND BIG DATA TECHNOLOGIES LIKE SPARK, DATABRICKS, HDINSIGHT, IOT ETC. • CATCH ME ON LINKEDIN OR VIA MY BLOG • HTTPS://WWW.LINKEDIN.COM/IN/PRAMODSINGLA • HTTPS://PRAMODSINGLA.COM/
  • 3. TOPICS Why Machine learning Life cycle of machine learning Challenges Why Azure Databricks ML Using Azure Databricks Set up Databricks Runtime ML MLFlow H2O Demo Available Machine learning options in Azure Databricks
  • 6. CHALLENGES Collect, Clean and Process Data 80% time spent Problem Formulation Choosing algorithms and parameters Speed (GPUs) myriad tools and Libraries Hard to track experiments Hard to reproduce results Hard to deploy ML
  • 8. AVAILABLE MACHINE LEARNING OPTIONS IN AZURE DATABRICKS •DATABRICKS RUNTIME ML •APACHE SPARK MLLIB •THIRD PARTY ML LIBRARY SUPPORT E.G. H2O •MLFLOW
  • 9. DATABRICKS RUNTIME ML PRELOADE LIBARIRES (SAMPLE CLUSTER CONFIG ON NEXT SLIDE) Category Libraries Distributed Deep Learning Distributed training with Horovod and Spark: Distributed TensorFlow and Keras prediction: Deep Learning Keras: TensorFlow: GPU libraries: XGBoost XGBoost4j Other machine learning libraries numpy scikit-learn scipy
  • 10.
  • 11. MLFLOW • AN OPEN SOURCE PLATFORM FOR MANAGING THE END-TO-END MACHINE LEARNING LIFECYCLE • ORGANIZED INTO THREE COMPONENTS: TRACKING, PROJECTS, AND MODELS. • TRACKING  API AND UI FOR LOGGING PARAMETERS, DATA, CODE AND RESULTS . TRACKING SERVER • PROJECTS  • CAPTURE WHOLE ENVIRONMENT INCLUDING DEPENDENT LIBRARIES ( PLATFORM AND DATA SCIENTIST INDEPENDENT). • SIMPLY A DIRECTORY WITH CODE OR A GIT REPOSITORY, AND USES A DESCRIPTOR FILE OR SIMPLY CONVENTION TO SPECIFY ITS DEPENDENCIES AND HOW TO RUN THE CODE. • MODELS MANAGING AND DEPLOYING MODELS FROM A VARIETY OF ML LIBRARIES TO A VARIETY OF MODEL SERVING AND INFERENCE PLATFORMS
  • 12.
  • 13. H2O SPARKLING WATER • BEST ML LIBRARIES FOR SPARK • LEADERS IN MAGIC QUADRANT • SUPPORT BOTH SUPERVISED AND UNSUPERVISED MODELS • E.G. RANDOM FOREST, GLM, GBM, XGBOOST, GLRM, WORD2VEC AND MANY MORE