SlideShare a Scribd company logo
1 of 25
Download to read offline
Big$Data$Then$And$Now$
Presenta(on*to*R:Evolu(on*Conference*!*5th*March*2014*!*Mar(n*Willcox,*Director*Big*Data*CoE*
(Interna(onal)*
The$technology$industry$thrives$on$hype:$“Big$Data”$is$
hot$right$now$
“Big$Data”$has$recently$overtaken$“cloud$compu>ng”$as$the$
most$hyped$expression$in$IT.$
We$are$currently$bombarded$with$“facts”$and$
opinions…$
“Unprecedented*data*growth…*that*con1nues,*
regardless*of*budget*constraints”*
Ten*Trends*&*Technologies*To*Impact*IT*Over*The*Next*5*Years*
David*Cappucio,*Research*VP*(Gartner),*January*9th*2013*
…on$both$sides$of$the$“Big$Data”$debate$
“Big*Data*is*bullshit…*it’s*really*just*
data.”*
Harper*Reed,*CTO*Obama*For*America*
Spoiler:$both$sides$are$half$right$
Yes*it’s*a*big*deal…* …no*it’s*not*unprecedented*
Big$Data,$circa$1986$
Deployment*of*EPoS*systems*in*the*late*80s*revolu(onises*Retail*
•  Enormous$(by$the$standards$of$the$day)$Teradata$system$enables$
WalMart$to$capture$store*/*SKU*/*day*level*aggregated$data$across$all$
its$stores$in$North$America;$
•  The*rest*is*Retail*history…*
Big DataBig Data“…really$we$got$big$by$replacing$inventory$with$
informa>on…”$–$Sam*Walton,*Founder,*WalMart*
From*transac1ons*I*to*interac1ons:*the*three*new*
waves*of*Big*Data*
Analysis$of$clickstream$data$enables$Amazon$
and$eBay$to$achieve$“mass$customisa>on”$of$
their$webSsites.$
Analysis$of$social$/$interac>on$data$enables$
Amazon,$Apple$and$LinkedIn$to$go$social$
(“people$who$like$what$you$like$also$like…”)$
Increasing$instrumenta>on$is$now$leading$to$
the$emergence$and$op>misa>on$of$“the$
Internet$of$Things”.$
People*
interac1ng*with*
things*
People*
interac1ng*with*
people*
Things*
interac1ng*with*
things*
*
(1)*
*
*
(2)*
*
*
(3)*
*
These*trends*are*real*and*accelera1ng*–*but*are*
they*about*“more”,*or*“different”?*
“Big$Data”$are$oVen$“unstructured”$and$difficult$to$store$and$
analyse$in$tradi>onal$database$technologies…$
I didn t say Bill was ugly.
I didn t say Bill was ugly.
I didn t say Bill was ugly.
I didn t say Bill was ugly.
I didn t say Bill was ugly.
I didn t say Bill was ugly.
…now$“new”$informa>on$management$strategies,$Analy>cs$and$
suppor>ng$technologies$are$enabling$us$to$extend$Enterprise$Analy>cs$
Structured(
data(
Mul,-structured(
data(
Non-tradi,onal((,me-series(/(path(/(graph)(analy,cs(
Coun,ng(things(and(sta,s,cal(analy,cs(
Business$
Intelligence$
&$Analy>cs$
Capture,*
Store,*
Refine*
Explora1on*&*Discovery*
Take$home$lesson$#1$
“Big*Data”*aren’t*just*“lots*more*data”;*“big”*
oZen*means*“different”.*
The$corollary$of$Moore’s$Law$
Simple*compu(ng*devices*are*now*incredibly*inexpensive*
An*iPad2*would*have*stayed*on*the*list*of*the*world’s*most*
powerful*supercomputers*through*1994.*
13 06/03/2014 Teradata Confidential
I$CAN$SENSE$YOUR$MOVEMENT$&$UNDERSTAND$YOUR$BEHAVIOR$
Shopping cart will track the
consumer’s every move
14 06/03/2014 Teradata Confidential
I$KNOW$WHO$YOU$ARE$FACIAL$RECOGNITION$
Retailers are testing
new facial recognition
technology
15 06/03/2014 Teradata Confidential
I$KNOW$YOU$AND$CAN$ENGAGE$VIA$MOBILE$
Mobile enabling you to
engage in the Store to
find the items you like
16 06/03/2014 Teradata Confidential
I$KNOW$WHAT$INTERESTS$YOU$
Improving on-shelf
availability with
cameras
17 06/03/2014 Teradata Confidential
Facebook-
enabled coat
hanger tracks
the number of
”likes”
I$CAN$UNDERSTAND$IF$YOU$ARE$SOCIALLY$INFLUENCED$
Photo © C&A Brazil/DDB Brazil
Take$home$lesson$#2$
The*(smart)*machines*are*coming,*bearing*data.**We*will*soon*
be*able*to*measure*anything*–*and*everything.*
New$sources$of$data$follow$the$same$trajectory$
From*byZproduct*to*raw*material;*from*BI*to*CI*
“We*are*used*to*the*idea*
of*deploying*new*
technology*to*improve*
produc(vity*and*
efficiency...*But*data*are*
no*longer*merely*the**
byZproduct*of*process*
improvement,*they*are*
becoming*the*raw*
material*of*business.”$
“Hot$right$now”$in$Retail$Big$Data$Discovery$Analy>cs$
Mass$personalisa>on$/$collabora>ve$
filtering$ Social$/$sen>ment$analysis$
Marke>ng$ahribu>on$/$PPC$analy>cs$ Golden$path$/$pathStoSchurn$analy>cs$
21 06/03/2014 Teradata Confidential
Tradi>onal$BI:$what$is$the$answer$to$the$ques>on?$
Discovery$&$Explora>on:$what$are$the$interes>ng$ques>ons?$
“Capture only what’s
needed”
IT delivers a platform for
storing, refining, and
analyzing all data sources
Business explores data for
questions worth answering
Big Data Analytics
Multi-structured & Iterative Analysis
IT structures the data to
answer those questions
Business determines what
questions to ask
Classic BI
Structured & Repeatable Analysis
“Capture in case it’s
needed”
22 06/03/2014 Teradata Confidential
EDW Model V Big Data Discovery
EDW Model
Highly Planned & Controlled
Slow Release Schedule
•  3x releases 2 years
High Central funding cost
Low Risk / High Success
Discovery Model
Small Iterative Projects
•  40+ Discoveries / 2 years
Low cost per project
•  $20k-$50k per project
BAU funded initiatives
•  $Project funded
•  $Central discovery team /
BIU for free thinking
High Risk / High Fail
•  Iterates to a new project
$5m
$5m
$5m
3 Releases Over 2 years
Release 1
Release 2
Release 3
40+ Projects Over 2 years
23 06/03/2014 Teradata Confidential
EDW Model V Big Data Discovery
EDW Model
•  EDW projects must succeed
•  Successful Discoveries
productionised as part of
release schedule
Discovery Model
•  Many projects “fail”
•  Failure is accepted as part
of the process and leads to
new innovations and
iterative projects
•  Successful projects are
often productionised on the
EDW for execution
$5m
$5m
$5m
3 Releases Over 2 years
Release 1
Release 2
Release 3
40+ Projects Over 2 years
Successful Project
Failed Project
Take$home$lesson$#3$
Enabling*innova1on*means*embracing*risk,*“failing*fast”*–*and*
moving*on.*
And$finally…$
Good*technology*is*necessary,*but*not*sufficient;*
organisa1on*and*culture*maber*more.*
@Willcoxmnk*

More Related Content

Similar to R:evolution 2014 - Martin Willcox

Big data - Aditya Yadav
Big data - Aditya YadavBig data - Aditya Yadav
Big data - Aditya YadavAditya Yadav
 
I started a VC fund - here is what happened
I started a VC fund - here is what happenedI started a VC fund - here is what happened
I started a VC fund - here is what happenedAndreas Klinger
 
Make sense of your big data - Pilato
Make sense of your big data - PilatoMake sense of your big data - Pilato
Make sense of your big data - PilatoCodemotion
 
Developer Report 2021 (Published: January 2022)
Developer Report 2021 (Published: January 2022)Developer Report 2021 (Published: January 2022)
Developer Report 2021 (Published: January 2022)Maria Xinhe Shen
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Maria Xinhe Shen
 
Elasticsearch : petit déjeuner du 13 mars 2014
Elasticsearch : petit déjeuner du 13 mars 2014Elasticsearch : petit déjeuner du 13 mars 2014
Elasticsearch : petit déjeuner du 13 mars 2014ALTER WAY
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Maria Xinhe Shen
 
Continuum Analytics and Python
Continuum Analytics and PythonContinuum Analytics and Python
Continuum Analytics and PythonTravis Oliphant
 
Towards a sustainable solution to open source sustainability
Towards a sustainable solution to open source sustainabilityTowards a sustainable solution to open source sustainability
Towards a sustainable solution to open source sustainabilityTobie Langel
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Maria Xinhe Shen
 
DataEd Slides: Leveraging Data Management Technologies
DataEd Slides: Leveraging Data Management TechnologiesDataEd Slides: Leveraging Data Management Technologies
DataEd Slides: Leveraging Data Management TechnologiesDATAVERSITY
 
What to expect when you are visualizing
What to expect when you are visualizingWhat to expect when you are visualizing
What to expect when you are visualizingKrist Wongsuphasawat
 
EarthCube DDMA AGU
EarthCube DDMA AGUEarthCube DDMA AGU
EarthCube DDMA AGUTanu Malik
 
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdf
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdfSR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdf
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdfHabibAbda
 
Industrial Tech Startups In The SF Bay Area
Industrial Tech Startups In The SF Bay AreaIndustrial Tech Startups In The SF Bay Area
Industrial Tech Startups In The SF Bay AreaChristian Dahlen
 
How to Use Data for Good
How to Use Data for Good How to Use Data for Good
How to Use Data for Good Experian_US
 
How to manage Crowdfunding for artists and musicians
How to manage Crowdfunding for artists and musiciansHow to manage Crowdfunding for artists and musicians
How to manage Crowdfunding for artists and musiciansVirginie BERGER
 

Similar to R:evolution 2014 - Martin Willcox (20)

Big data - Aditya Yadav
Big data - Aditya YadavBig data - Aditya Yadav
Big data - Aditya Yadav
 
I started a VC fund - here is what happened
I started a VC fund - here is what happenedI started a VC fund - here is what happened
I started a VC fund - here is what happened
 
Make sense of your big data - Pilato
Make sense of your big data - PilatoMake sense of your big data - Pilato
Make sense of your big data - Pilato
 
Developer Report 2021 (Published: January 2022)
Developer Report 2021 (Published: January 2022)Developer Report 2021 (Published: January 2022)
Developer Report 2021 (Published: January 2022)
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)
 
Elasticsearch : petit déjeuner du 13 mars 2014
Elasticsearch : petit déjeuner du 13 mars 2014Elasticsearch : petit déjeuner du 13 mars 2014
Elasticsearch : petit déjeuner du 13 mars 2014
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)
 
Continuum Analytics and Python
Continuum Analytics and PythonContinuum Analytics and Python
Continuum Analytics and Python
 
GITA April 2015 Newsletter
GITA April 2015 NewsletterGITA April 2015 Newsletter
GITA April 2015 Newsletter
 
Towards a sustainable solution to open source sustainability
Towards a sustainable solution to open source sustainabilityTowards a sustainable solution to open source sustainability
Towards a sustainable solution to open source sustainability
 
David pilato codemotion-telaviv
David pilato codemotion-telavivDavid pilato codemotion-telaviv
David pilato codemotion-telaviv
 
Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)Developer Report 2021 (Published: January 2021)
Developer Report 2021 (Published: January 2021)
 
DataEd Slides: Leveraging Data Management Technologies
DataEd Slides: Leveraging Data Management TechnologiesDataEd Slides: Leveraging Data Management Technologies
DataEd Slides: Leveraging Data Management Technologies
 
What to expect when you are visualizing
What to expect when you are visualizingWhat to expect when you are visualizing
What to expect when you are visualizing
 
EarthCube DDMA AGU
EarthCube DDMA AGUEarthCube DDMA AGU
EarthCube DDMA AGU
 
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdf
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdfSR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdf
SR-R-nKAnwar_PPM_Penulisan_ProposalLPDP.pdf
 
Industrial Tech Startups In The SF Bay Area
Industrial Tech Startups In The SF Bay AreaIndustrial Tech Startups In The SF Bay Area
Industrial Tech Startups In The SF Bay Area
 
Evolution of end-to-end: why the Internet is not like any other network
Evolution of end-to-end: why the Internet is not like any other networkEvolution of end-to-end: why the Internet is not like any other network
Evolution of end-to-end: why the Internet is not like any other network
 
How to Use Data for Good
How to Use Data for Good How to Use Data for Good
How to Use Data for Good
 
How to manage Crowdfunding for artists and musicians
How to manage Crowdfunding for artists and musiciansHow to manage Crowdfunding for artists and musicians
How to manage Crowdfunding for artists and musicians
 

Recently uploaded

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 

Recently uploaded (20)

What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 

R:evolution 2014 - Martin Willcox