An introduction to data engineering. Summarize from a short course that I currently took.

1. An introduction to the role of Data Engineer,
the workflow, tools, and an introduction to
Python programming language
Basic Data Engineering

2. A data engineer main tasks
are developing and designing
data management
architecture and monitoring
the infrastructure within a
company.

3. Core Knowledge of
Data Engineering
Programming
Distributed
System
Analysis

4. Data Engineer Methodology:
AUTOMATION
Since data engineer deals with
many jobs at hand, this makes
automation become a popular
methodology to reduce people
intervention during the flow and
increase productivity.

5. Data Engineering Workflow
Data Source
ETL
(Extract,
Transform,
Load)
Data
Warehouse

6. A data engineer's must have skill.
Python
Programming Language

7. Python is open source and easy to
build packages, hence it is very
versatile. Python can help you
making fancy visualization,
connecting databases. We can use
python to practically build any
piece of software.
What Is
Python?

8. Python Lesson #1:
VARIABLE
A Python variable is a reserved memory
location to store values. In other words, a
variable in a python program gives data to the
computer for processing.
list_of_fruits = ['banana', 'apple', 'orange']
Variable Value
A professional programmer use an easy to understand name for the
variable to avoid confusion in the future. Avoid using unclear name
such as single alphabet, abbreviation, etc.

9. Text: str
Numeric: int, float, complex
Sequence: list, tuple, range
Mapping: dict
Set: set, frozenset
Boolean: bool
Binary: bytes, bytearray, memoryview
Python Lesson #2:
DATA TYPE
Unlike other programming language, in python we do ot need to
declare data type as python can define for itself

10. Python Lesson #3:
COMMENT
Comments in Python begin with a hash mark ( # ) and whitespace
character and continue to the end of the line. Because comments
do not execute, when you run a program you will not see any
indication of the comment there. Comments are in the source
code for humans to read, not for computers to execute.

11. PIPELINE TEAM
Ines Desdemarsa
Juliana Kristanti
Novita Sari