Ray poynter big data and advanced analytics

Ray Poynter, The Future Place – JMRX Lectures 2015
Big
Data
and
Advanced
Analy0cs

Ray Poynter
The Future Place
JMRX
–
Tokyo
–
May
12,
2015

Agenda

1.  What
is
big
data?

2.  The
strengths
of
big
data

3.  When
big
data
misses
the
mark

4.  The
challenges
of
big
data

5.  Linking
big
data,
analyEcs
and
market

research

6.  ImplicaEons
for
Japan

How
big
is
big
data?

BIG!

•  If
it
fits
in
Excel,
it
is
not
really
big
data

•  If
it
fits
in
SPSS,
it
is
not
really
big
data

•  If
you
have
a
profile
of
10
million
telco

customers,
it
is
not
really
big
data

•  If
you
have
second-‐by-‐second
locaEon
and

usage
data
for
10
million
telco
customers,
that

is
big
data

•  If
you
need
to
use
Hadoop,
it
is
big
data

What
is

?

InstrucEon

Results

IBM’s
four
Vs

What
does
Google
Know?

And
shares
with
you

Google
Ads

http://www.google.com/settings/ads/

Loca0on
History

https://maps.google.com/locationhistory
7
days
of
travel

Google
Search
History

https://www.google.com/history/

Google
Other

4.  Google
monthly
security
and
privacy
report

–
lisEng
all
the
services
you
use

hVps://www.google.com/seYngs/dashboard

5.  Apps
and
extensions
that
have
access
to

your
Google
data

hVps://security.google.com/seYngs/security/permissions

6.  Export
all
your
Google
data

hVps://www.google.com/takeout

7.  History
of
all
your
YouTube
searches

hVps://www.youtube.com/feed/history/search_history

The
Signal
and
the
Noise
–
Nate
Silver

Weather
forecas0ng

PMSL
–
Pressure
at
Mean
Sea
Level

Weather
forecas0ng
benchmarks

1. Same
as
today

2. Same
as
average
of
last
few
years
-‐
climate

What benchmark are you going to
use for your predictive analytics?

Google
Flu
Trends

http://www.google.org/flutrends/
December 2014

Google
Flu
Trends

http://www.google.org/flutrends/
May 2015

Google
Flu
Trends
-‐
Japan

Target

BPP
and
USA
Prices
Index

Tesco

•  Loyalty
card
data

•  Real-‐Eme
monitoring
of
refrigerators
across
120
stores
in

UK
and
Ireland
to
save
20million
Euros
a
year

–  70
million
data
points

•  Real-‐Eme
monitoring
of
lighEng
and
heaEng,
across
120

stores

–  Management
dashboard
shows
by
7am
which
stores

are
not
at
the
right
temperature

•  Buying
SocialmaEcs
–
programmaEc
and
re-‐targeEng
ads

INTERESTING
BIG
DATA
PROJECTS

Ebola
and
mobile
phones

Flowminder
using
data
from
telcos
to
map
populaEon
mobility
–
but
level
of
granularity

is
quite
large
–
journeys
over
20KM

The
smartphone

Accelerometer

Temperature

Gravity

Gyroscope

Light

Air
pressure

Proximity

Humidity

GPS

Call
acEvity

App
acEvity

Internet
usage

WiFi

Bluetooth

Cameras

Near
Field
Comms

GSM/CDMA

Aberdeen
tracks
‘hundreds
of
thousands’

Kingsgate
Shopping
Mall

Huddersﬁeld,
Yorkshire
(UK)

Mount
Sinai
Hospital

Mount
Sinai
are
using

Big
Data
approaches

to
uElise
data
to

make
personalised

health
and
treatment

predicEons.

Project
led
by
Jeﬀ

Hammerbacher,
a
30-‐
year-‐old
known
for

being
Facebook’s
ﬁrst

data
scienEst.

Big Data Failures

Earthquakes

Given the abruptly self-organizing nature of earthquakes, it is extremely
unlikely that precursors can attain such levels of accuracy. I therefore
conclude that prediction of major earthquakes is, in any practical sense,
impossible.
Russ Evans, 1998

Big
Data
thinks
I
am
a
pregnant
dude!

http://www.carolroth.com/blog/google-dude-target-pregnant/

Google
Flu
–
the
failures

Flu
Near
You
–
a
crowdsourced
approach

The
Samaritans
App

ScrapeGate

BIG
DATA
CHALLENGES

Correla0on
Annual
Chocolate
Consump0on
&
Nobel

Prizes
per
10
Million
of
Popula0on

New
England
Journal
of
Medicine.

Correla0on
and
Causa0on

1.  CorrelaEon
predicts
the
past

– Which
is
someEmes
enough

– Especially
when
the
past
repeats
itself

2.  CausaEon
is
needed
to
predict
new
futures

– But
causaEon
is
hard
to
establish
in
the
real
world

3.  Experiments
are
key

– Market
research
can
help

Perverse
incen0ves

•  Colonial
Hanoi
–
bounEes
for
rats
tails

•  Duplessis
Orphans
–
Canada
1945-‐60,
Orphans=70
cents,

mentally
ill
$2.25,
20K
children
conﬁned

•  Paying
for
acEon
(e.g.
doctors
and
ﬁremen)
decreases

prevenEon
work

•  Facebook
likes
correlate
with
success
–
unless
they
are
set
as

a
target,
creaEng
a
black
market
in
them

•  TwiVer
menEons
correlate
with
success
–
unless
they
are

benchmarked,
creaEng
value
in
them

•  Big
Data
benchmarks
change
behaviour
to
improve
scores,

rather
than
underlying
performance

Where
are
the
data
scien0sts
going
to

come
from?

Google Trends

MARKET
RESEARCH
AND

ANALYTICS

Aribu0on
Modelling

Knowing
where
to
dig
and
digging

Lucien
Bowater,
Director
Strategy
and

Insight
at
BSkyB
–
UK
media
company

MRS
Conference,
UK,

2013

Issue
MR
Digs

Big

Data

AcEon
CEO

PaVern

Big

Data

Digs
MR
AcEon
CEO

T
hVps://www.ted.com/talks/
ben_wellington_how_we_found_the_worst_place_to_park_in_new_york_city_using_big_data

Where
to
dig
and
digging

1.  When
is
rush
hour?
MR
asks
the
quesEon,

Big
Data
digs

2.  Why
is
this
ﬁre
hydrant
generaEng
so
much

money
in
parking
Eckets?
Big
Data
asks
the

quesEons,
but
it
was
Qual
that
answered
it.

Thank You!
Questions?

IMPLICATIONS
FOR
JAPAN?

1.  What
new
skills
do
we
need?

2.  What
new
people
do
we
need?

3.  What
new
tools
do
we
need?

4.  How
should
MR
work
with
Big
Data

and
Analy0cs?

Ray poynter big data and advanced analytics

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (7)

Similar to Ray poynter big data and advanced analytics

Similar to Ray poynter big data and advanced analytics (20)

More from MROC Japan

More from MROC Japan (20)

Recently uploaded

Recently uploaded (20)

Ray poynter big data and advanced analytics