Azure Data Factory is a cloud-based data integration service that automates data pipelines for data storage, movement, and processing. It supports ETL and ELT processes, allowing integration from multiple sources, and features over 80 connectors, orchestration, and automation capabilities. The document discusses the advantages of Azure Data Factory over traditional SSIS, including scalability and cloud integration, as well as its key components like pipelines, activities, and datasets.

1. Azure Data Factory
Transform your Data

2. Speaker Introduction
C# Corner MVP(3), MCP, MCTS,MCSA, OCA
http://manojmittalblogs.blogspot.in
https://www.youtube.com/user/manojmcans/videos
/Manojmcans
In/manoj-mittal-mcp-mcts-oca-mcsa/
https://www.c-sharpcorner.com/members/manoj-mittal
Manoj Mittal
Sr. Technical Architect
@ Mindtree Ltd

3. Data Factory is a Cloud based data
integration service used to compose
data storage, data movement and
processing services into automated data
pipelines
Azure Data Factory

4. What is Data Factory
Cloud-based PaaS
Solution
(Pay what u used and scalable, Server
Less, No Patching)
Data Integration
Services
(Allow to integrate from multiple
sources, Generate and Provide
Common Platform)
Orchestration and
Automation
(80+ connector, Pipeline can be
automate whenever you need)
Data
Transformation
(HD Insights, Data Bricks, Azure
Function are data transform tool, Now
Data Flow help to aggregate the data
and transform )

5. ETL Vs ELT
ETL
(Transform i.e. filters, lookup, before
loading and designed for reliability i.e.
data well structured compliance )
ELT
(Transform after loading (on DW),
designed for agility i.e. Change freedom
to change query on fly as business need
frequently)
Azure Data Factory

6. SSIS Vs ADF
SSIS
Mode Code-free transformations
On Premises Connector( e.g. Excel not
available with data factory)
ADF
Much Higher Scalability
Cloud and SaaS Connectors
Event-based trigger
Can run SSIS Package

7. Data Factory Consideration
Two Versions
ADF v2 is the current and
improved version
Build Options
PowerShell, .NET, Python,
REST, ARM
Highly Integrated
DevOps, Key Vault,
Monitor, Automation..
No Data Storage
Need to persist data by
the end
Security Standards
HTTP/ TLS whenever
possible

8. Main Data Factory Elements
Activity
Trigger
Integration Runtime
Pipeline
Activity
Linked Services
Dataset

9. Data Factory Elements Relationship
PipelineActivityDataset
Linked
Service
Logical Group
Consumes
Produces
Runs On
Represent
Data On

10. Data Factory Elements Relationship
Azure Data Factory
Pipeline
Integration Runtime
Linked Service Data Set Copy Activity Linked ServiceData Set
Blob Storage Azure SQL
Database