Data Alchemy

Data Alchemy
(And A Boat Filled With Sauerkraut)
@lukasvermeer

Lukas Vermeer
Experiments at Booking.com
@lukasvermeer

SAUERKRAUT
Introduction
@lukasvermeer

HMS Endeavour
Our voyage starts when a ship sails from Plymouth on 26 August 1768.
@lukasvermeer

@lukasvermeer
- Wikipedia
“Provisions loaded at the outset of
the voyage included 6,000 pieces of
pork and 4,000 of beef, nine tons of
bread, five tons of flour, three tons
of sauerkraut, one ton of raisins
and sundry quantities of cheese,
salt, peas, oil, sugar and oatmeal.”

REVOLUTIONS
Chapter 1
@lukasvermeer

@lukasvermeer
- Thomas Kuhn
“Progress in science is not a simple
line leading to the truth.”

Aristarchus of Samos
c. 310 BC – c. 230 BC
@lukasvermeer

@lukasvermeer
- Archimedes
“[Aristarchus’] hypotheses are that
the fixed stars and the Sun remain
unmoved, that the Earth revolves
about the Sun on the circumference
of a circle, […]”

@lukasvermeer
- George Pólya
“His fame rests on his heliocentric
theory. […] Perhaps ‘theory’ is too
strong a word, for his proofs were
weak; yet it was a great idea.”

Claudius Ptolemy
c. 100 AD – c. 170 AD
@lukasvermeer

@lukasvermeer
- Claudius Ptolemy
“But it has escaped [heliocentric
proponents’] notice in the light of
what happens around us in the air
that such a notion would seem
altogether absurd.”

@lukasvermeer
- Claudius Ptolemy
“For the earth would always
outstrip them in its eastward
motion, so that all other bodies
would seem to be left behind and
to move towards the west.”

@lukasvermeer
- Claudius Ptolemy
No westward motion.
No stellar parallax.
Geocentric math works.
QED

@lukasvermeer
- Claudius Ptolemy
No observed westward motion.
No observed stellar parallax.
Geocentric math works to explain
what we have observed.
GED

Nicolaus Copernicus
1473 – 1543
@lukasvermeer

Dē Revolutionibus Orbium Coelestium
Copernicus's vision of the universe, published in 1543, the year of his
death, though he had formulated the theory several decades earlier.
@lukasvermeer

@lukasvermeer
- Andreas Osiander
“These hypotheses need not be
true nor even probable. On the
contrary, if they provide a calculus
consistent with the observations,
that alone is enough.”

Galileo Galilei
1564 – 1642
@lukasvermeer

@lukasvermeer
- Michael Fowler
“The real breakthrough that
ultimately led to the acceptance of
Copernicus’ theory was due to
Galileo, but was actually a
technological rather than a
conceptual breakthrough.”

Galileo did not invent the idea.
He built a better telescope.
@lukasvermeer

Galileo first observed the moons of Jupiter
This observation upset the notion that all celestial bodies revolve around
the Earth. Galileo published a full description in March 1610.
@lukasvermeer

Multiple models could probably
explain the data you already have.
Determining which one is closer to
the truth requires a directed effort
to collect new data (to the contrary).
@lukasvermeer

@lukasvermeer
Imagine we have two observations.

@lukasvermeer
This theory fits the data.

@lukasvermeer
But so does this theory.

@lukasvermeer
This observation doesn’t help.

@lukasvermeer
But this one does.

@lukasvermeer
Uh oh. Time for a paradigm shift!

@lukasvermeer
Data You
Have
Data You
Need

@lukasvermeer
Data You
Have
Data You
Need
Sauerkraut
Science

TRANSIT
Chapter 2
@lukasvermeer

“On the sizes and distances”
Aristarchus's 3rd-century BC calculations on the relative sizes of (from
left) the Sun, Earth and Moon, from a 10th-century AD Greek copy.
@lukasvermeer

@lukasvermeer
Aristarchus (3rd century BC)
Distance to the sun
380 - 1520
Earth Radii

Diagram from Edmund Halley's 1716 paper
Addressed to the Royal Society showing how the Venus transit could be
used to calculate the distance between the Earth and the Sun.
@lukasvermeer

Route of the first voyage of James Cook
An expedition to the south Pacific Ocean aboard HMS Endeavour, from
1768 to 1771. It was the first of three Pacific voyages of which Cook was
the commander.
@lukasvermeer

Three years of travel.
For two timestamps.
@lukasvermeer

@lukasvermeer
- James Cook
“Not a Clowd was to be seen the
Whole day and the Air was
perfectly clear, so that we had
every advantage we could desire in
Observing the whole of the
passage of the Planet Venus over
the Suns disk.”

The "black drop effect"
As recorded during the 1769 transit by James Cook.
@lukasvermeer

Right place, right time, right idea.
Insufficiently accurate telescope.
@lukasvermeer

@lukasvermeer
Jérôme Lalande (1771)
Distance to the sun
24 000
Earth Radii

Science is limited by data.
Data is limited by engineering.
@lukasvermeer

@lukasvermeer
- Elon Musk
“In the absence of the engineering,
you do not have the data. You just
hit a limit. You can be real smart
within the context of the limit of
the data you have, but unless you
have a way to get more data, you
can’t make progress.”

@lukasvermeer
Data You
Have
Data You
Need
Data You
COULD
CREATE

TRANSMUTATION
Chapter 3
@lukasvermeer

@lukasvermeer
- Michael Palmer
“Data is just like crude. It’s valuable,
but if unrefined it cannot really be
used. It has to be changed into gas,
plastic, chemicals, etc to create a
valuable entity that drives profitable
activity; so must data be broken
down, analyzed for it to have value.”

@lukasvermeer
- Wikipedia
“The philosopher's stone is a
legendary alchemical substance
capable of turning base metals
such as mercury into gold or silver.”

Data Alchemy
is not
Data Science
@lukasvermeer

Kaggle.com
“Your home for data science”.
@lukasvermeer

Something good, something bad
Hotel reviews on Booking.com.
@lukasvermeer

Sentiment Analysis
Excerpt from “Entity Based Sentiment Analysis on Twitter” by Siddharth
Batra and Deepak Rao (Stanford University).
@lukasvermeer

Kaggle is to real-life machine
learning as chess is to war
Intellectually challenging and great
mental exercise, but YOU DON'T
KNOW, MAN! YOU WEREN'T THERE!
@lukasvermeer

Sentiment Analysis
At Booking.com, we solve the sentiment analysis challenge at data
collection time.
@lukasvermeer

@lukasvermeer
Data You
Have
Data You
Need
Data You
COULD
CREATE
Delete or archive?

@lukasvermeer
Data You
Have
Data You
Need
Data You
COULD
CREATEKeep, or recreate?

@lukasvermeer
Data You
Have
Data You
Need
Data You
COULD
CREATE
Proxies?

Deciding which data to collect,
and how, is a fundamental step
in the scientific method.
Limited both by available
theories and engineering.
@lukasvermeer

Some of us are philosophers.
Some of us build telescopes.
@lukasvermeer

@lukasvermeer
- Voltaire
“Judge a man by his questions
rather than by his answers.”

COOK’S SECRET SECOND OBJECTIVE
Appendix A
@lukasvermeer

Terra Australis Nondum Cognita
1570 map by Abraham Ortelius depicting a large continent on the
bottom of the map and also an Arctic continent.
@lukasvermeer

A new map of the world
With Captain Cook's tracks, his discoveries and those of the other
circumnavigators. Published in 1800 by W. Palmer.
@lukasvermeer

@lukasvermeer
- Matthew Flinders (1814)
“There is no probability, that any
other detached body of land, of
nearly equal extent, will ever be
found in a more southern latitude”

EXPERIMENTING WITH SCURVY
Appendix B
@lukasvermeer

@lukasvermeer
- Johann Bachstrom (1734)
“This evil is solely owing to a total
abstinence from fresh vegetable
food, and greens; which is alone
the primary cause of the disease”

@lukasvermeer
- George F. M. Ball (2004)
“Twelve sailors with scurvy were divided
into six pairs and each pair was given a
different daily concoction in addition to a
common diet. Two fortunate patients
were each given two oranges and one
lemon every day; only these two
recovered, thus demonstrating the
efficacy of oranges and lemons.”

@lukasvermeer
- Wikipedia
“Malt and wort were top of the list
of the remedies Cook was ordered
to investigate. The others were
beer, Sauerkraut and Lind's rob.
The list did not include lemons.”

TLDR
Limes cure scurvy!
No they don’t.
Experiment shows they do!
Theory still says no.
We’re eating limes now!
What about these cheaper ones, though?
Experiment says equally effective!
So why did Scott’s expedition fail?
Breaking news: limes don’t cure scurvy!?
Told you so.
So maybe eat more fresh meat?
Works for Scott!
Hooray! Science saves the day!
Science also found this thing called “vitamin C”?
Yes! And guess what? Limes cure scurvy again!
…
@lukasvermeer
1734
1747
1795
1860
1903
1911
1932

Data Alchemy

More Related Content

More from Rising Media Ltd.

Recently uploaded

Data Alchemy