Azure Data Factory (ADF) is a powerful, cloud-based, fully managed serverless ETL data integration solution for ingesting, preparing, and transforming all the data at a scale. It provides a complete data integration and data transformation experience that can be used to move data between many business data sources, transform them at scale, and write the process data store of choice.

1. Agenda as part of Azure Data Engineering Course:
Topics Covered:
 Azure Data Lake Storage
 Azure Data Factory
 Azure Functions
 Azure Logic Apps
 Azure Data Bricks
 Delta Tables
 Azure SQL Database/Warehouse
1. Azure Data Lake
Architecture
Resource Group
Azure Data lake Account
Exploring Data Lake
Diff between the versions over Gen2
Benefits of Azure Data Lake
Practice Session
2. Azure Data Factory
Architecture
Pipelines
Linked Service
Input Dataset
Output Dataset
Activities in the Pipelines
Control Flow Activities
Integrating Logic Apps with ADF Pipelines

2. Web API’S
Data Flows
Practice Session
3. Azure Data bricks
Architecture
Integration with Spark
How to create Cluster in Cloud
Create & Execute Jobs, Commands
How to create Cluster, workspace & Notebooks
Integrate Data bricks with ADF Pipelines.
Delta Tables
Practice Session
4. Scala
OOPS
Object & Class
Case Class
Singleton & Companion Objects
Collections – List , Tuples , Set , Map
Loops
Functions
Exception Handling
String Functions
Arrays
Practice Session

3. 5. Apache Spark
Architecture
Features
Spark Context & Spark Sessions
Spark Core
Spark SQL
Spark SQL Joins
RDD
Data Frames & Datasets
Transformations & Actions
Case When with Dataframes
Storage Layers
Accumulators
Pair Functions
Repartition & Coalesce
Shuffle Partitions
Pivots & Flatten
String Functions
Drop Rows with Null Values
Single to Multiple Columns
Array, Nested Array to Single Array.
Map & Sort Functions
Aggregate Functions
Window Functions
Practice Session- Hands On
6. Azure SQL Server
Architecture
Basics of SQL

4. SQL Data Warehouse
Types of Indexes
Window Functions
Practice Session – Hands On