Diagrams.pptx

1. Flight Delay
Data
(CSV Files)
Historical
Airport
Weather Data
(CSV Files)
Azure Data
Factory
Azure
Databricks
(Spark SQL)
Visualize Delay
Predictions on
a Map
Flight Delays
Web Portal
3rd
Party API
Forecasted Airport
Weather
Create, train, & export ML
model
Azure Machine
Learning service
Store and operationalize
trained ML model
Operationalize ML
model
Copy data to blob storage
Execute Databricks Notebook for
batch scoring
Integration Runtime
Move on-premises data to
Azure Storage
Azure SQL
Database
Azure Monitor Azure Key Vault
Blob Storage
Containerized
AI Services
Flight Delay
Predictions

2. Flight Delay
Data
(CSV Files)
Historical
Airport
Weather Data
(CSV Files)
Blob Storage
Azure Data
Factory
Install Integration Runtime
to move on-premises data
to Azure Storage
Monthly copy
activity
Data Loading

3. Flight Delay
Data
(CSV Files)
Historical
Airport
Weather Data
(CSV Files)
Blob Storage
Historical Data Preparation
Azure
Databricks
(Spark SQL)
1) Explore & prepare data
2) Save prepared data
back to storage

4. Model Creation
Blob Storage Azure
Databricks
(Spark SQL)
1) Create and train machine learning
model from historical data
2) Export trained model and store
within Azure Machine Learning
service workspace model registry
0) Storage contains prepared
historical flight delay data
Azure Machine
Learning service
Store and operationalize
trained ML model
Containerized
AI Services
Flight Delay
Predictions
3) Deploy model using Azure ML
service to a container hosted
in Azure Kubernetes Service

5. On-Demand Delay Predictions
1) Query for weather forecast
2) Query for delay prediction
providing weather
forecast & flight data
Containerized
AI Services
Flight Delay
Predictions
Flight Delays
Web Portal
3rd
Party API
Forecasted Airport
Weather

6. Visualizing Bulk Delay Predictions
1) Load Power BI embedded
report
2) Report uses Power BI
Direct Query against
source and caches it
Flight Delays
Web Portal
Visualize Delay
Predictions on
a Map
Azure SQL
Database
Storage contains historical
flight delay data and bulk
delay predictions

7. Visualizing Bulk Delay Predictions
Notebook uses a JDBC
connection to SQL Database
to write bulk delay
predictions
Visualize Delay
Predictions on
a Map
Azure SQL
Database
Storage contains historical
flight delay data and bulk
delay predictions
Azure
Databricks
Azure Key Vault
Key Vault-backed Databricks
secret scope securely stores
JDBC connection string and
other secrets

8. Operationalizing machine learning
Azure Data
Factory
Azure
Databricks
(Spark SQL)
Blob Storage
Containerized
AI Services
Flight Delay
Predictions
Azure Machine
Learning service
Store and operationalize
trained ML model
Flight Delays
Web Portal
3rd
Party API
Forecasted Airport
Weather

9. Operationalizing machine learning
Azure Databricks
Workspace
Spark SQL
Explore & Prepare Data
Databricks notebook used
By AWT analysts

10. Machine learning model
Azure Databricks
Workspace
Spark SQL
Prepare Data
Spark MLlib two-class logistic
regression model created and
trained within Databricks notebook