The document outlines the typical data science project lifecycle and necessary data scientist skill set. It describes the main stages as business requirements, data acquisition, data preparation, hypothesis/modeling, evaluation/interpretation, deployment, and optimization. For each stage, it provides brief explanations of common tasks. It also maps out key skill areas for data scientists including programming, domain knowledge, data collection/wrangling, statistics, machine learning, visualization, and communication. Finally, it compares course offerings at major universities based on these skill areas.

1. Data Science Project Lifecycle
and Data Scientist Skill Set
Jason Geng @Data Application Lab
Miya Du @Data Science Association

2. Business
Requirement
Data
Acquisition
Data
Preparation
Hypothesis &
Modeling
Evaluation &
Interpretation
Deployment
Operations
Optimization

3. Business Requirements
 Data scientists need to work with business people and
those with expertise in understanding the data,
understanding the business
 Specify the business requirements
 For instance, the healthcare data

4. e.g. ‘DISCWT’:
‘This the discharge-level weight
on the HCUP nationwide data to
produce national estimates’
Understand the data:
Understand the Business:
Goal:
Predict Readmission Rate
Database:
Healthcare:
Readmissions Database
Modeling

5. Data Collection
 Data from product line
 Purchase third party data
 Social media (Facebook, LinkedIn)
 Web crawling
 Open source data (Opendata, U.S. Census Data)
Challenge
Data Storage
Data Management

6. Legacy data
OLTP Web Log
Web Crawler
Open Source
Third Party
Data
Social Media
Data
XML
CSV
LOG
SQL
…
Product Line
Business
Intelligence
Data Science
App

7. Data preparation (data wrangling)
 Cleaning data (semantic errors, missing entries, or inconsistent
formatting)
 Challenge: data integration
 80% time in project workflow
Data
Source A
Data
Source B
Data
Source B
ETL
Data
Warehouse

8. Feature engineering
Select or
creating
features
Research
feature
relevance
Experiment
and
validation
Change the
feature set
Go back to
feature
selection
step

9. Modeling
Reference Source: http://scikit-learn.org/stable/tutorial/machine_learning_map/

10. Deploy to product line

12. Machine Learning
Data Collection
Communication
& Storytelling
Data Wrangling
Product Development &
Feedback Analysis
Data Visualization
Statistics
Domain
Knowledge &
Business Mindset
Data Science
Skill Tree

13. Required Knowledge
Skillsets Knowledge
Domain Knowledge and
Business Mindset
Programming R, Python, NLP, Java, Distributed System
Industry
Various Concentrations(Finance, E-
Commerce, Geo, Biology, Medicine)
Data Collection &
Wrangling
Database Database Systems and Management
Big Data Big Data Processing and Analytics
Statistics Modeling, Inference and Optimization
Machine learning Data Mining and Machine Learning
Data Visualization Data Visualization and Exploratory Analytics
Communication and Storytelling Professional Speaking and Writing

14. Program Comparison
University Name Northwestern CMU
Johns
Hopkins
Columbia
University
Stanford Berkeley UW USC
Domain
Knowledge
& Business
Mindset
Programming ✓ ✓ ✓ ✓ ✓ ✓ ✓
Industry ✓ ✓ ✓ ✓ ✓
Data
Collection
&
Wrangling
Database ✓ ✓ ✓ ✓
Big Data ✓ ✓ ✓ ✓ ✓ ✓
Statistics ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
Machine learning ✓ ✓ ✓ ✓ ✓ ✓ ✓
Data Visualization ✓ ✓ ✓ ✓
Communication and
Storytelling
✓ ✓ ✓

15. Thank you!
https://www.DataAppLab.com
Feb 2017
PPT: Xiaolu Zhao @ Feb 16, 2017