This presentation explains what data engineering is and describes the data lifecycles phases briefly. I used this presentation during my work as an on-demand instructor at Nooreed.com
3. Plan
• What is Data Engineering?
• Data Engineer vs. Data Scientist vs. Data Analyst
• Understanding Data Management (Data Layers, DQS, MDS,
Provenance)
• Distributed Computing
• Designing Data Pipelines (Choosing Paradigm / Technologies)
• Data Engineer Jobs / Required Skills
• Helpful Tips
• Online Courses
4/6/2022 3
5. What is Data Engineering?
4/6/2022 5
• Data Engineering is the act of:
• Collecting data
• Transforming (…) data
• Validating data
“Making data consumable”
6. What is a Data Engineer?
AI/ML Engineer BI Developer Data Analyst
Database
Administrator
Report Developer Data Developer
Data Architect Data Integration
Specialist
ETL Developer
Data Scientist
4/6/2022 6
7. Data Engineer vs. Data Scientist
4/6/2022 7
Source: https://elu.nl/careers-in-data-science-data-analyst-vs-data-engineer-vs-data-scientist/
13. Data Wrangling vs. Data Pre-processing
4/6/2022 13
Source: https://medium.com/swlh/data-pre-processing-data-wrangling-4a6a8624e747
Data Pre-processing Data Wrangling
39. 4/6/2022 39
• Coursera:
• Google Cloud - Data Engineering, Big Data, and Machine Learning on GCP
Specialization
• San Diego - Big Data Specialization
• Udacity:
• Data Engineering nanodegree
• DataCamp:
• Data Engineer with Python Track
• IBM – CognitiveClass.ai
• Free data science and data engineering courses
• Udemy:
• Data Science A-Z™: Real-Life Data Science Exercises Included