data science history / data science @ NYT

data science @ The New York Times
and how a 164-year old content company became data-driven
chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins
references: bit.ly/icerm

“data science”
jobs, jobs, jobs

data science: mindset & toolset
drew conway, 2010

modern history:
2009

“data science”
blogs, blogs, blogs

“data science”
blogs, blogs, blogs
The first time I heard "data science" was in 2007 while reading a proposal that my adviser had passed along, outlining an academic program similar to what we think of as data
science.
The first time I heard "data science" was in 2007 while
reading a proposal that my adviser had passed along,
outlining an academic program similar to what we think of
as data science.

“data science”
ancient history: 2001

data science
context

home schooled

PhD in topology

“By the end of late 1945, I was a
statistician rather than a topologist”

invented: “bit”

invented: “software”

invented: “FFT”

“the progenitor of data science.” - @mshron

“The Future of Data Analysis,” 1962
John W. Tukey

introduces:
“Exploratory data anlaysis”

Tukey 1965, via John Chambers

TUKEY BEGAT S WHICH BEGAT R

Tukey 1972

? 1972

Jerome H. Friedman

Tukey 1975
In 1975, while at Princeton, Tufte was asked to teach a
statistics course to a group of journalists who were visiting
the school to study economics. He developed a set of
readings and lectures on statistical graphics, which he
further developed in joint seminars he subsequently taught
with renowned statistician John Tukey (a pioneer in the ﬁeld
of information design). These course materials became the
foundation for his ﬁrst book on information design, The
Visual Display of Quantitative Information

TUKEY BEGAT VDQI

Tukey 1977

TUKEY BEGAT EDA

fast forward -> 2001

“The primary agents for change should be
university departments themselves.”

histories
1. in academia -> Bell: as heretical
statistics (see also Breiman)
2. in industry: as job description
historical rant: bit.ly/data-rant

biology: 1892 vs. 1995
biology changed for good.

genetics: 1837 vs. 2012
ML toolset; data science mindset

ML toolset; data science mindset
arxiv.org/abs/1105.5821 ; github.com/rajanil/mkboost

data science: mindset & toolset

news: 20th century
church state

church

news: 21st century
church state
engineering

1851 1996
newspapering: 1851 vs. 1996

example:
millions of views per hour2015

data science: the web

is your “online presence”

is a microscope

is an experimental tool

is an optimization tool

1851 1996
newspapering: 1851 vs. 1996 vs. 2008
2008

“a startup is a temporary organization in search of a
repeatable and scalable business model” —Steve Blank

every publisher is now a startup

learnings

learnings
- supervised learning
- unsupervised learning
- reinforcement learning

learnings
- supervised learning
- unsupervised learning
- reinforcement learning
cf. modelingsocialdata.org

stats.stackexchange.com

from “are you a bayesian or a frequentist”
—michael jordan
L =
NX
i=1
' (yif(xi; )) + || ||

supervised learning, e.g.,

supervised learning, e.g.,
“the funnel”

interpretable supervised learning
supercoolstuff

interpretable supervised learning
supercoolstuff
arxiv.org/abs/q-bio/0701021

optimization & learning, e.g.,
“How The New York Times Works “popular mechanics, 2015

recommendation as supervised learning

unsupervised learning, e.g,
cf. daeilkim.com ; import bnpy

modeling your audience
bit.ly/Hughes-Kim-Sudderth-AISTATS15

(optimization, ultimately)

also allows recommendation as inference

Reporting
Learning
Test
aka “A/B
testing”;
business as
usual
(esp.
supervised)
Some of the most recognizable personalization in our service is the
collection of “genre” rows. …Members connect with these rows so
well that we measure an increase in member retention by placing
the most tailored rows higher on the page instead of lower.
reinforcement learning: from A/B to….

real-time A/B -> “bandits”
GOOG blog:

Reporting
Learning
Test
Optimizing
Exploreunsupervised:
supervised:
reinforcement:

common requirements in
data science:

common requirements in
data science:
1. people
2. ideas
3. things
cf. USAF

things:
what does DS team deliver?

things:
what does DS team deliver?
- build data prototypes
- build APIs
- impact roadmaps

- build data prototypes
cf. daeilkim.com

- in puppet, w/python2.7
- collaboration w/pers. team
- build APIs

- impact roadmaps
flickr/McJex

data skills
- data engineering
- data science
- data visualization
- data product
- data multiliteracies
- data embeds
cf. “data scientists at work”, ch 1

data science: people
- new mindset > new toolset

summary:
pay attention to:
1. people
2. ideas
3. things
cf. USAF

thanks to the data science team!

chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
@chrishwiggins

data science history / data science @ NYT

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Viewers also liked

Viewers also liked (20)

Similar to data science history / data science @ NYT

Similar to data science history / data science @ NYT (20)

More from chris wiggins

More from chris wiggins (14)

Recently uploaded

Recently uploaded (20)

data science history / data science @ NYT