Data Science not just for
Big Data
Gregory Piatetsky, @kdnuggets
Analytics, Big Data,
Data Mining, and Data Science Resour...
What do we call it?
Same Core Idea:
• Statistics, 1830• Data mining, 1980Finding Useful
• Knowledge Discovery in
Patterns ...
Big Data > Data Mining >
> Predictive Analytics , Data Science
Data mining
Data Mining

Big Data

Big Data

Google Trends ...
Data Science before “Big Data”
• Ancient astronomers
• Kepler laws of planetary motion
(1609), derived from observations b...
Ignaz Semmelweis – early data
scientist (1818-1865)
Graph
from
Wikipedia

Semmelweis found that the main difference betwee...
Data Science Application:
Process, not one step
CRISP-DM
process

© KDnuggets 2013

6
Data Science Application:
Process, not one step
CRISP-DM
process

Building
Predictive
Models

© KDnuggets 2013

Most fun f...
Data Science Basic Principles & Ideas
• Focus on actionable patterns
• Build predictive models - supervised learning
(trai...
What Changes in Data Science
with Big Data?
• Data munging becomes much more complex
• New algorithms, technology needed t...
Upcoming SlideShare
Loading in …5
×

data-science-not-just-for-big-data

2,274 views

Published on

I review what remains constant in Data Science regardless of data size and describe data science basic principles and ideas.

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total views
2,274
On SlideShare
0
From Embeds
0
Number of Embeds
819
Actions
Shares
0
Downloads
61
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

data-science-not-just-for-big-data

  1. 1. Data Science not just for Big Data Gregory Piatetsky, @kdnuggets Analytics, Big Data, Data Mining, and Data Science Resources © KDnuggets 2013 1
  2. 2. What do we call it? Same Core Idea: • Statistics, 1830• Data mining, 1980Finding Useful • Knowledge Discovery in Patterns in Data Data (KDD), 1989• • • • • Business Analytics, 1997Predictive Analytics, 2002Data Analytics,2011Data Science, 2011Big Data, 2012 © KDnuggets 2013 Different Emphasis 2
  3. 3. Big Data > Data Mining > > Predictive Analytics , Data Science Data mining Data Mining Big Data Big Data Google Trends search, Jan 2008- Sep 2013, Worldwide © KDnuggets 2013 3
  4. 4. Data Science before “Big Data” • Ancient astronomers • Kepler laws of planetary motion (1609), derived from observations by Tycho Brahe • Genetics – Gregor Mendel found patterns in inheritance of pea plants • Western Medicine • … © KDnuggets 2013 4
  5. 5. Ignaz Semmelweis – early data scientist (1818-1865) Graph from Wikipedia Semmelweis found that the main difference between clinics was that 1st had medical students who also examined cadavers, and inferred that students carried something on their hands from the autopsy. He proposed washing © KDnuggets 2013 hands after autopsy but was rejected and died in insane asylum 5
  6. 6. Data Science Application: Process, not one step CRISP-DM process © KDnuggets 2013 6
  7. 7. Data Science Application: Process, not one step CRISP-DM process Building Predictive Models © KDnuggets 2013 Most fun for data scientists, But only a small part of the process 7
  8. 8. Data Science Basic Principles & Ideas • Focus on actionable patterns • Build predictive models - supervised learning (train, test, x-validate) • Avoid overfitting • Calculating similarity of objects - unsupervised learning • Avoid information leakers • Select important variables/features • Model accuracy vs lift: how much more prevalent a pattern is than would be expected by chance • Estimate probability and cost/gain of actions • Help optimize decisions © KDnuggets 2013 8
  9. 9. What Changes in Data Science with Big Data? • Data munging becomes much more complex • New algorithms, technology needed to deal with Big Data Volume, Velocity, & Variety • New, effective algorithms that require Big Data: e.g.: deep belief networks, recommendations • Predictions become (somewhat ) more accurate • New things become visible: social networks, recommendations, mobility, knowledg e? • However, basic principles remain © KDnuggets 2013 9

×