Azure Data Factory can be used to build modern data warehouse patterns with Azure SQL Data Warehouse. It allows extracting and transforming relational data from databases and loading it into Azure SQL Data Warehouse tables optimized for analytics. Data flows in Azure Data Factory can also clean and join disparate data from Azure Storage, Data Lake Store, and other data sources for loading into the data warehouse. This provides simple and productive ETL capabilities in the cloud at any scale.

1. Data Quality Patterns in the Cloud
with Azure Data Factory
Mark Kromer
Sr. ProgramManager
Azure Data Factory

2. ADF: Simple and Productive ETL in the Cloud at any Scale

3. Modern Data Warehouse Pattern with ADF Mapping Data Flows
Applications
Dashboards
Business/custom apps
(structured)
Logs, files, and media
(unstructured)
r Azure Storage/
Data Lake Store
Azure Data
Factory
Load files into data
lake on a schedule
Azure Data
Factory
Extract and
transform
relational data
Azure SQL DW
Load processed
data into tables
optimized for
analytics
Clean and
join disparate
data
Databases
Azure Databricks

9. list of connectors
full list of compute environments data

12. Data Flow

13. Data Quality Patterns

17. • iif (length(title) == 0,toString(null()),title)

23. http://aka.ms/dataflowvideos
http://aka.ms/dataflowpatterns
https://docs.microsoft.com/en-us/azure/data-
factory/concepts-data-flow-overview
http://aka.ms/dataflowexpressions
https://aka.ms/dfperf
https://aka.ms/dflinks
https://github.com/kromerm/adfdataflowdocs/tree/master/sampledata
http://slideshare.net/kromerm