Summer in Greece Festival
Microsoft Learn Student Ambassadors | GreeceMicrosoft Learn Student Ambassadors | Greece
Machine Learning
Services
in SQL Server 2019
1988 2000 Founderv 6.0 60 +2010
Antonios Chatzipavlis
Data Solutions Consultant & Trainer
Connect / Explore / Learn
V.102010 - 2020
A community for Greek professionals who use the
Microsoft Data Platform
Connect / Explore / Learn
WebminarsArticles
Videos News
EventsResources
@sqlschool@antoniosch
Sqlschoolgr
Page
Sqlschool
User Group
Sqlschool.gr
Group
SQLschoolGR
channel
Machine Learning Services
in
SQL Server 2019
Microsoft Learn Student Ambassadors | Greece
Data Analysis Scientific Process
• Create, Read, Update, Delete
• Programming Control
• Reporting
• Business Intelligence
• Statistics and Data Mining
• Story Telling
• Define the Question
• Create Hypothesis
• Create a Repeatable Test
• Publish Results
Data Science
Microsoft Learn Student Ambassadors | Greece
Descriptive
- Reporting and Scorecards
- Grouping Similarity Scores
Prescriptive
- Optimization and Simulation
- What, Why, When
Predictive
- Predicting the future from the past
Types of Analysis
Machine
Learning
Microsoft Learn Student Ambassadors | Greece
Machine Learning Algorithms
Which category?
How much/many?
Which Group?
Is it odd?
Which action?
Classification
Regression
Clustering
Anomaly
Reinforcement Learning
Microsoft Learn Student Ambassadors | Greece
Team
Data
Science
Process
https://bit.ly/2DkeHvL
Microsoft Learn Student Ambassadors | Greece
Collaboration and end-to-end efficiency
• Deploy training/scoring scripts in DB
• Make models accessible to any
applications
• Manage model version in DB
• Consume ML models from any apps
• Make simple stored proc calls
• Train models remotely on SQL Server
• Use R/Python code to embedded in T-SQL
stored procedure
• Explore and experiment with data
• Stay in your favorite IDE
• Use R, Python, T-SQL
Explore Train
Deploy
Share &
Consume
SQL Server
ML Services
App Developer
Data Scientist
Data Scientist
SQL Server DBA
Microsoft Learn Student Ambassadors | Greece
Evolution of SQL Server ML Services
Microsoft Learn Student Ambassadors | Greece
Machine Learning outside of database
Data
transformations
Model
training Model
Analytics Server
Scoring
Separate server or embedded logic
Application
Transactions
Data movement
Model
operationalization
New data
Predictions
Microsoft Learn Student Ambassadors | Greece
In-Database Machine Learning
Application
Data
transformations
Model
training
Scoring
Transactions
Microsoft Learn Student Ambassadors | Greece
How it works
Application
Server Box
Satellite
T-SQL
SQL Server
Launchpad Service
Direct
Communication
Performance
Microsoft Learn Student Ambassadors | Greece
• A database engine instance is required.
• Always On Availability Groups are supported for Machine Learning
Services.
• Always On Failover Cluster Instance (FCI) is supported with SQL Server
2019 and later.
• Don't install Machine Learning Services on a domain controller
• Side-by-side installation with other versions of Python and R is supported
but isn't recommended.
Pre-install checklist
Microsoft Learn Student Ambassadors | Greece
What we need to install?
Microsoft Learn Student Ambassadors | Greece
Set environment variables
Enable script execution
Restart the service
Post Installation actions
Microsoft Learn Student Ambassadors | Greece
Verify installation
R
Python
Microsoft Learn Student Ambassadors | Greece
• Create a login for SQLRUserGroup.
• Make sure that SQLRUserGroup has "Allow Log on locally" permissions.
• Firewall configuration for SQL Server Machine Learning Services.
• Ensure that SQL Server TCP network protocol is enabled.
• Ensure that SQL Server Remote Access is enabled.
• Manage disk quotas to avoid external scripts running tasks that exhaust
disk space
Additional configuration
Get Python package information
Get R package information
Microsoft Learn Student Ambassadors | Greece
Packages Version Description
revoscalepy 9.4.7
Used for remote compute contexts, streaming, parallel execution of rx
functions for data import and transformation, modeling, visualization,
and analysis.
microsoftml 9.4.7 Adds machine learning algorithms in Python.
Microsoft Python packages
Microsoft Learn Student Ambassadors | Greece
Packages Version Description
RevoScaleR 9.4.7
Used for remote compute contexts, streaming, parallel execution of rx
functions for data import and transformation, modeling, visualization, and
analysis.
sqlrutils 1.0.0 Used for including R script in stored procedures.
MicrosoftML 9.4.7 Adds machine learning algorithms in R.
olapR 1.0.0 Used for writing MDX statements in R.
Microsoft R packages
Microsoft Learn Student Ambassadors | Greece
Don't try to install Python packages directly in the SQL package library
using pip or similar methods.
Install Python / R packages with sqlmlutils
Python : https://bit.ly/3fn668o
R : https://bit.ly/2DE9IWb
Install new Python / R packages
For SQL Server 2019
Microsoft Learn Student Ambassadors | Greece
Code Execution
R / Python / Java
sp_execute_external_script
@language
@script
Microsoft Learn Student Ambassadors | Greece
Code Execution
Execute with Result Sets
EXECUTE sp_execute_external_script @language = N'Python'
, @script = N'OutputDataSet = InputDataSet;'
, @input_data_1 = N'SELECT * FROM T;'
WITH RESULT SETS(([ID] INT,[YOD] INT,[Name] nvarchar(200)));
GO
Microsoft Learn Student Ambassadors | Greece
• Generates a predicted value or scores
based on a stored model.
• Uses the native C++ extension
capabilities in SQL machine learning.
• Offers the fastest possible processing
speed of forecasting and prediction
workloads
• Support models in Open Neural
Network Exchange (ONNX) format or
models trained using the RevoScaleR
and revoscalepy packages.
PREDICT
T-SQL function
PREDICT
Code Execution
ML Samples
Microsoft Learn Student Ambassadors | Greece
Download them from GitHub https://bit.ly/2DyHLzc
In these reports, you can view details such as:
• Active Python or R sessions
• Configuration settings for the instance
• Execution statistics for machine learning jobs
• Extended events for R Services
• Python or R packages installed on the current instance
Monitor using SSMS Reports
Microsoft Learn Student Ambassadors | Greece
sys.dm_external_script_requests
Returns a row for each active worker account that is running an external script.
sys.dm_external_script_execution_stats
Returns one row for each type of external script request.
Monitor using DMVs
Microsoft Learn Student Ambassadors | Greece
Monitor using XE
event
SQLSatellite
predict%
https://bit.ly/33zlCf1
Monitor ML
Any
questions
Thank you!
@antoniosch - @sqlschool
./sqlschoolgr - ./groups/sqlschool
./c/SqlschoolGr
SQLschool.gr Group
Antonios Chatzipavlis
Data Solutions Consultant & Trainer
Machine Learning in SQL Server 2019

Machine Learning in SQL Server 2019

  • 1.
    Summer in GreeceFestival Microsoft Learn Student Ambassadors | GreeceMicrosoft Learn Student Ambassadors | Greece Machine Learning Services in SQL Server 2019
  • 2.
    1988 2000 Founderv6.0 60 +2010 Antonios Chatzipavlis Data Solutions Consultant & Trainer Connect / Explore / Learn
  • 3.
    V.102010 - 2020 Acommunity for Greek professionals who use the Microsoft Data Platform Connect / Explore / Learn WebminarsArticles Videos News EventsResources @sqlschool@antoniosch Sqlschoolgr Page Sqlschool User Group Sqlschool.gr Group SQLschoolGR channel
  • 4.
  • 5.
    Microsoft Learn StudentAmbassadors | Greece Data Analysis Scientific Process • Create, Read, Update, Delete • Programming Control • Reporting • Business Intelligence • Statistics and Data Mining • Story Telling • Define the Question • Create Hypothesis • Create a Repeatable Test • Publish Results Data Science
  • 6.
    Microsoft Learn StudentAmbassadors | Greece Descriptive - Reporting and Scorecards - Grouping Similarity Scores Prescriptive - Optimization and Simulation - What, Why, When Predictive - Predicting the future from the past Types of Analysis Machine Learning
  • 7.
    Microsoft Learn StudentAmbassadors | Greece Machine Learning Algorithms Which category? How much/many? Which Group? Is it odd? Which action? Classification Regression Clustering Anomaly Reinforcement Learning
  • 8.
    Microsoft Learn StudentAmbassadors | Greece Team Data Science Process https://bit.ly/2DkeHvL
  • 9.
    Microsoft Learn StudentAmbassadors | Greece Collaboration and end-to-end efficiency • Deploy training/scoring scripts in DB • Make models accessible to any applications • Manage model version in DB • Consume ML models from any apps • Make simple stored proc calls • Train models remotely on SQL Server • Use R/Python code to embedded in T-SQL stored procedure • Explore and experiment with data • Stay in your favorite IDE • Use R, Python, T-SQL Explore Train Deploy Share & Consume SQL Server ML Services App Developer Data Scientist Data Scientist SQL Server DBA
  • 10.
    Microsoft Learn StudentAmbassadors | Greece Evolution of SQL Server ML Services
  • 11.
    Microsoft Learn StudentAmbassadors | Greece Machine Learning outside of database Data transformations Model training Model Analytics Server Scoring Separate server or embedded logic Application Transactions Data movement Model operationalization New data Predictions
  • 12.
    Microsoft Learn StudentAmbassadors | Greece In-Database Machine Learning Application Data transformations Model training Scoring Transactions
  • 13.
    Microsoft Learn StudentAmbassadors | Greece How it works Application Server Box Satellite T-SQL SQL Server Launchpad Service Direct Communication Performance
  • 14.
    Microsoft Learn StudentAmbassadors | Greece • A database engine instance is required. • Always On Availability Groups are supported for Machine Learning Services. • Always On Failover Cluster Instance (FCI) is supported with SQL Server 2019 and later. • Don't install Machine Learning Services on a domain controller • Side-by-side installation with other versions of Python and R is supported but isn't recommended. Pre-install checklist
  • 15.
    Microsoft Learn StudentAmbassadors | Greece What we need to install?
  • 16.
    Microsoft Learn StudentAmbassadors | Greece Set environment variables Enable script execution Restart the service Post Installation actions
  • 17.
    Microsoft Learn StudentAmbassadors | Greece Verify installation R Python
  • 18.
    Microsoft Learn StudentAmbassadors | Greece • Create a login for SQLRUserGroup. • Make sure that SQLRUserGroup has "Allow Log on locally" permissions. • Firewall configuration for SQL Server Machine Learning Services. • Ensure that SQL Server TCP network protocol is enabled. • Ensure that SQL Server Remote Access is enabled. • Manage disk quotas to avoid external scripts running tasks that exhaust disk space Additional configuration
  • 19.
    Get Python packageinformation
  • 20.
    Get R packageinformation
  • 21.
    Microsoft Learn StudentAmbassadors | Greece Packages Version Description revoscalepy 9.4.7 Used for remote compute contexts, streaming, parallel execution of rx functions for data import and transformation, modeling, visualization, and analysis. microsoftml 9.4.7 Adds machine learning algorithms in Python. Microsoft Python packages
  • 22.
    Microsoft Learn StudentAmbassadors | Greece Packages Version Description RevoScaleR 9.4.7 Used for remote compute contexts, streaming, parallel execution of rx functions for data import and transformation, modeling, visualization, and analysis. sqlrutils 1.0.0 Used for including R script in stored procedures. MicrosoftML 9.4.7 Adds machine learning algorithms in R. olapR 1.0.0 Used for writing MDX statements in R. Microsoft R packages
  • 23.
    Microsoft Learn StudentAmbassadors | Greece Don't try to install Python packages directly in the SQL package library using pip or similar methods. Install Python / R packages with sqlmlutils Python : https://bit.ly/3fn668o R : https://bit.ly/2DE9IWb Install new Python / R packages For SQL Server 2019
  • 24.
    Microsoft Learn StudentAmbassadors | Greece Code Execution R / Python / Java sp_execute_external_script @language @script
  • 25.
    Microsoft Learn StudentAmbassadors | Greece Code Execution Execute with Result Sets EXECUTE sp_execute_external_script @language = N'Python' , @script = N'OutputDataSet = InputDataSet;' , @input_data_1 = N'SELECT * FROM T;' WITH RESULT SETS(([ID] INT,[YOD] INT,[Name] nvarchar(200))); GO
  • 26.
    Microsoft Learn StudentAmbassadors | Greece • Generates a predicted value or scores based on a stored model. • Uses the native C++ extension capabilities in SQL machine learning. • Offers the fastest possible processing speed of forecasting and prediction workloads • Support models in Open Neural Network Exchange (ONNX) format or models trained using the RevoScaleR and revoscalepy packages. PREDICT T-SQL function PREDICT
  • 27.
  • 28.
  • 29.
    Microsoft Learn StudentAmbassadors | Greece Download them from GitHub https://bit.ly/2DyHLzc In these reports, you can view details such as: • Active Python or R sessions • Configuration settings for the instance • Execution statistics for machine learning jobs • Extended events for R Services • Python or R packages installed on the current instance Monitor using SSMS Reports
  • 30.
    Microsoft Learn StudentAmbassadors | Greece sys.dm_external_script_requests Returns a row for each active worker account that is running an external script. sys.dm_external_script_execution_stats Returns one row for each type of external script request. Monitor using DMVs
  • 31.
    Microsoft Learn StudentAmbassadors | Greece Monitor using XE event SQLSatellite predict% https://bit.ly/33zlCf1
  • 32.
  • 33.
  • 34.
    Thank you! @antoniosch -@sqlschool ./sqlschoolgr - ./groups/sqlschool ./c/SqlschoolGr SQLschool.gr Group Antonios Chatzipavlis Data Solutions Consultant & Trainer