Data Management Activities, Extraction, Transformation and Loading (ETL)

1. Data Science
Data Management
• Data Management Activities
• Data Pipelines

2. Data Management
 Data management refers to the process of collecting, storing, organizing,
and maintaining data in a structured and efficient manner.
 It involves various activities such as data entry, data manipulation, data
analysis, data storage, data retrieval, data security, and data governance.
 The primary goal of data management is to ensure that data is accurate,
reliable, accessible, and secure, and that it meets the needs of the
organization or individuals using it.
 Effective data management practices are essential for businesses, research
institutions, governments, and other organizations to make informed
decisions, improve efficiency, and gain competitive advantages.

3. Data Management Activities
Data
Collection
Data Storage
Data Quality
Assurance
Data Security
Data
Organization
and
Cataloging
Data
Integration

4. Data Management Activities
Data
Governance
Data Privacy
and
Compliance
Data Retrieval
and Analysis
Data
Archiving and
Purging
Data
Documentation
Data Auditing
and
Monitoring

5. Data Pipelines
 Data pipelines are a series of processes and tools used to ingest, process,
transform, and move data from one or more sources to a destination,
typically a data storage or analytics system.
 These pipelines automate the flow of data, enabling organizations to
efficiently handle large volumes of data and ensure its quality and
accessibility for various purposes such as analysis, reporting, and machine
learning.
 Data pipelines often include steps such as data extraction, data cleansing,
data transformation, and data loading.

6. Data Pipelines
Operational
System
External
Data
Data
Warehouse
Logistics
Marketing
Performance
Evaluation
Data
analysis,
Time series
Analysis,
Data Mining
Optimization
.
Staging
Area
ETL

7. ETL Process
Extraction
Extract data from various sources.(spread sheets,
flat files, operational data, external data)
Reading data from source system and storing it in
staging area.
Transformation
The extracted data is transformed into a format that
is suitable for loading into the data warehouse.
Loading
After being extracted and transformed data are
loaded into the tables of data warehouse to make
them available to analyst and decision support
applications.

8. Thanks for Watching!
Please check the description box for the link to Machine Learning videos.