1. The data science process involves 6 steps: setting goals, retrieving data from internal and external sources, data preparation including cleansing, combining data through joins and appends, exploratory data analysis, building models through an iterative process of selecting techniques and variables, and presenting results for automation. 2. Data is stored internally in databases, data warehouses, data marts and data lakes, with databases for storage, warehouses for analysis, and marts serving specific business units. Data lakes contain raw data. 3. Building models is iterative, using either statistics or machine learning, to select variables and techniques and diagnose models for comparison.

1. Data Science Process

2. 1. Setting the research goal
1. Spend time understanding the goals and context of your research
2. Create a project charter

3. 2. Retrieving Data
1. Data within company
a. This data can be stored in official data repositories such as databases, data marts, data
warehouses, and data lakes maintained by a team of IT professionals.
b. The primary goal of a database is data storage, while a data warehouse is designed for
reading and analyzing that data.
c. A data mart is a subset of the data warehouse and geared toward serving a specific business
unit. While data warehouses and data marts are home to preprocessed data, data lakes
contains data in its natural or raw format.
2. Open source data

4. 3. Data Preparation
1. Cleansing

6. Why the errors should be corrected asap?
● Not everyone spots the data anomalies. Decision-makers may make costly mistakes
on information based on incorrect data from applications that fail to correct for the
faulty data.
● If errors are not corrected early on in the process, the cleansing will have to be done
for every project that uses that data.
● Data errors may point to defective equipment, such as broken transmission lines
and defective sensors.
● Data errors can point to bugs in software or in the integration of software that may
be critical to the company. While doing a small project at a bank we discovered that
two software applications used different local settings. This caused problems with
numbers greater than 1,000. For one app the number 1.000 meant one, and for the
other it meant one thousand.

7. Combining Data
1. Joining Tables
2. Appending Tables
3. Creating views

8. Data Transformation

10. 4. EDA

11. 5. Build the Model
Building a model is an iterative process. The way you build your model depends on
whether you go with classic statistics or the somewhat more recent machine
learning school, and the type of technique you want to use. Either way, most
models consist of the following main steps:
1 Selection of a modeling technique and variables to enter in the model
2 Execution of the model
3 Diagnosis and model comparison

14. 6 . Presentation and automation—
Presenting your results to the stakeholders and industrializing your analysis
process for repetitive reuse and integration with other tools.