The document outlines components and processes involved in data extraction, transformation, and loading (ETL) using tools like SSIS, SQL Server, and Azure Data Factory. It discusses the creation of data pipelines, initial data sources, and processing services, as well as monitoring capabilities for managing and deploying these pipelines. Additionally, it highlights use cases such as customer profiling and analytics related to user activity and privacy considerations.

1. Platinum Sponsors
Titanium Sponsors

4. ETL Tool
(SSIS, etc)
EDW
(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed
Data
Transform
BI Tools
Data Marts
Data Lake(s)
Dashboards
Apps

5. ETL Tool
(SSIS, etc)
EDW
(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed
Data
Transform
BI Tools
Ingest (EL)
Original Data
Data Marts
Data Lake(s)
Dashboards
Apps

6. ETL Tool
(SSIS, etc)
EDW
(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed
Data
Transform
BI Tools
Ingest (EL)
Original Data
Scale-out
Storage &
Compute
(HDFS, Blob Storage,
etc)
Transform & Load
Data Marts
Data Lake(s)
Dashboards
Apps
Streaming data

7. ETL Tool
(SSIS, etc)
EDW
(SQL Svr, Teradata, etc)
Extract
Original Data
Load
Transformed
Data
Transform
BI Tools
Ingest (EL)
Original Data
Scale-out
Storage &
Compute
(HDFS, Blob Storage,
etc)
Transform & Load
Data Marts
Data Lake(s)
Dashboards
Apps
Streaming data

8. Azure Data Factory

10. • Compose storage, movement, and processing services into data pipelines
• Initial data sources
• SQL Server, SQL Server in IaaS Virtual Machines, Azure SQL Database,
Azure Blobs, and Azure Tables
• Initial processing services
• Hive, Pig, C# code running on HDInsight
• hybrid data movement
• PowerShell developer experience for pipeline composition and deployment
• Cluster management for on-demand or bring your own
• Rich visual monitoring experience for a single view of all pipelines and
datasets that provides lineage dependencies, health monitoring, and error
identification
• Consume datasets by BI tools and applications
Capabilities at Public Preview

12. Example: customer profiling, game analytics

13. Example: customer profiling, game analytics
Copy new users
to blob storage
Join and aggregate
activity per week
and user table
Weekly
Privacy:
Refresh:
Xbox New
Users
Game New
Users
Game Activity
Per Week
Daily
New User Activity Per
Week
Contains PII
Weekly, Mon
by 8AM

14. All data, and all systems