This document provides an overview of Microsoft Azure Data Factory, a cloud-based integration service designed to automate data movement and transformation. It covers key concepts such as pipelines, datasets, and activities, as well as the differences between data lakes and data warehouses. The document highlights the importance of Azure Data Factory for modern data handling and offers a demo on moving data from SQL databases to blob storage.

1. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
f

2. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
Microsoft Azure
Data Factory

3. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
Microsoft Azure
Data Factory

4. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
03 Azure Data Factory Concepts
Microsoft Azure
Data Factory

5. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
03 Azure Data Factory Concepts
04 What is Data Lake? Microsoft Azure
Data Factory

6. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
03 Azure Data Factory Concepts
04 What is Data Lake?
05 Data Lake Concepts
Microsoft Azure
Data Factory

7. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
03 Azure Data Factory Concepts
04 What is Data Lake?
05 Data Lake Concepts
06 Data Lake Vs Data Warehouse
Microsoft Azure
Data Factory

8. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Offerings Of This Session
01 Why Azure Data Factory?
02 What Is Azure Data Factory?
03 Azure Data Factory Concepts
04 What is Data Lake?
05 Data Lake Concepts
06 Data Lake Vs Data Warehouse
07 Demo
Microsoft Azure
Data Factory

9. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
Why Azure Data Factory

10. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Why Data Factory?
Modern Data
handling requires you
to move from on
premise DB to Cloud
DW

11. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Why Data Factory?
Modern Data
handling requires you
to move from on
premise DB to Cloud
DW
This data needs
processing and goes
through a series of
steps, making the
process tedious

12. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Why Data Factory?
Modern Data
handling requires you
to move from on
premise DB to Cloud
DW
This data needs
processing and goes
through a series of
steps, making the
process tedious
Data Factory helps
you automate this
process and thus
serve the cause

13. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
What Is Data Factory?

14. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
It is a cloud based integration service that allows to create data driven workflows in the cloud for
orchestrating and automating data movement and data transformation.
Data Factory

15. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
It is a cloud based integration service that allows to create data driven workflows in the cloud for
orchestrating and automating data movement and data transformation.
Data Factory
Using Azure Data Factory, you can create and schedule
data-driven workflows (called pipelines) that can ingest
data from disparate data stores.

16. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
It is a cloud based integration service that allows to create data driven workflows in the cloud for
orchestrating and automating data movement and data transformation.
Data Factory
It can process and transform the data by using compute
services such as Azure HDInsight Hadoop, Spark, Azure
Data Lake Analytics, and Azure Machine Learning.
Using Azure Data Factory, you can create and schedule
data-driven workflows (called pipelines) that can ingest
data from disparate data stores.

17. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
How does it work?
The pipelines (data-driven workflows) in Azure Data Factory typically perform the following four steps:
Connect & Collect

18. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
How does it work?
The pipelines (data-driven workflows) in Azure Data Factory typically perform the following four steps:
Connect & Collect Transform & Enrich

19. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
How does it work?
The pipelines (data-driven workflows) in Azure Data Factory typically perform the following four steps:
Connect & Collect Transform & Enrich Publish

20. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Factory?
How does it work?
The pipelines (data-driven workflows) in Azure Data Factory typically perform the following four steps:
Connect & Collect Transform & Enrich Publish Monitor

21. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
Data Factory Concepts

22. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Factory Concepts
Pipeline
A pipeline is a
logical grouping of
activities that
performs a unit of
work

23. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Factory Concepts
Pipeline
A pipeline is a
logical grouping of
activities that
performs a unit of
work
Datasets
Datasets
represent data
structures within
the data stores

24. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Factory Concepts
Pipeline
A pipeline is a
logical grouping of
activities that
performs a unit of
work
Activity
Activities
represent
processing step in
a pipeline
Datasets
Datasets
represent data
structures within
the data stores

25. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Factory Concepts
Pipeline
A pipeline is a
logical grouping of
activities that
performs a unit of
work
Activity
Activities
represent
processing step in
a pipeline
Datasets
Datasets
represent data
structures within
the data stores
Linked Services
Information
needed to
connect to
external sources

26. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
What Is Data Lake?

27. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
What Is Data Lake?
It is an enterprise wide hyperscale repository for Big Data Analytics workloads. Azure Data Lake holds
data of any size, type and allows you to do operational and exploratory analytics

28. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
Data Lake Concepts

29. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Lake Concepts
Analytics Storage
HDInsight
Azure Data StoreAzure Data
Lake
Azure Data
Lake
Components

30. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Analytics on data of any size
All users productive on day one
Ready for your enterprise
Data Lake: Key
Things To Remember
Data Lake Concepts

31. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Structured Data
Semi Structured Data
Unstructured Data
Data Lake: Types Of
Data Stored
Data Lake Concepts

32. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Extents & Vertices
Supports parallel Reads and Writes
Supports Replication of data
Data Lake: How Is
Data Stored
Data Lake Concepts

33. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
Data Lake Vs Data Warehouse

34. Microsoft Azure Certification Training www.edureka.co/microsoft-azure-training
Data Lake Vs Data Warehouse
Complimentary to DW
Detailed Data
Schema on Read
One language to process Data of any
format
May be sourced to Data Lake
Filtered Summarized, Refined Data
Schema on Write
Processes SQL Complaint Data

35. Copyright © 2018, edureka and/or its affiliates. All rights reserved.
Demo- Move Data From SQL DB To Blog
Storage

36. Popularity