SlideShare a Scribd company logo
1 of 51
DATA AND
DISILLUSIONMENT


SOLVEfor
INTERESTING
OTHERWISE LIFE IS DULL.
Volume
              (the “big”
                 part)


              Pick
              any
 Velocity
              two              Variety
(the “fast”                      (the
   part)                   “anything” part)
Big Data is the Third Age of computing


  Computing      Networking                         Big Data

    Automate      Interconnect                  Predict & change
     things          things                          things




                                 (Jim Stodgill of O’Reilly Radar said this.)
Enterprises expect Big Data to deliver better
decisions and improved customer experiences
      What tangible benefits do you hope to achieve
            through your big data initiatives?




                                   NewVantage Partners LLC www.newvantage.com
(And apparently Hadoop is winning)

         What data management approaches
                are you considering?




                              NewVantage Partners LLC www.newvantage.com
The
relational
database
is a general-
purpose
tool.
A library is
                                                                                a database
                                                                                optimized
                                                                                for retrieval




Photo by cybrgrrl (http://www.flickr.com/photos/cybrgrl/1295482521/) on Flickr
A change
counter is a
database
optimized for
insertion
An example:
eventual
consistency
“End of Day Balance will only appear for dates previous to
the last 2 business days.”
“Transactions from today are reflected in your balance, but
may not be displayed on this page if you recently updated
your bankbook, if a paper statement was recently issued, or
if a transaction is backdated. These transactions will appear
in your history the following business day.”
Relational




             BIG


                   Statistical
http://www.flickr.com/photos/jenny-pics/3239638494/sizes/l/




                            Breadcrumb trail
The average enterprise has 178 social
media accounts




            (According to @setlinger and the Altimeter group.)
Ward off disease.
            Pinpoint disasters.
A force     Reveal corruption.
for good.
            Make cities smarter.
            Improve how we teach.
Big healthcare
Big philanthropy
Big commuting
Erode our privacy.
           Justify prejudices.
A force    Polarize groups.
for bad.
           Leak private truths.
Big prejudice
“…nobody notices offers they do not
get. And if these absent opportunities
start following certain social patterns
(for example not offering them to
certain races, genders or sexual
preferences) they can have a deep civil
rights effect.”
                 Anders Sandberg, Oxford University
Personalization looks a lot
     like prejudice.
Big radio
Times a song in “heavy rotation”
is played each day
30

                           Every 55m


15



        Every 4h
0
          2007               2012
Humans are bad at data.
We prefer false positives.
Wooly mammoth



http://www.flickr.com/photos/pong/172438102/sizes/o/
Sun temple



http://www.flickr.com/photos/30787002@N02/3298693694/sizes/l/
Some proof.
It’s really hard to find people who can think
about data well

      How challenging is it to source data scientists?




                                     NewVantage Partners LLC www.newvantage.com
Mistake correlation for causality
Seek truthiness rather than fact
Find patterns where they don’t exist
Easily swayed by tone
Side with our tribes
Dig in and ignore new evidence
Athenian swimming pools
Volume
           Big
Variety    Data   Good
                  data
Velocity
Veracity
525,000 state & local officers
Under 25 officers per precinct
130 million incident reports
200,000 uses of force
31% keep computer files
Evidence.com
Hard drive
Big Data is not about data.
Big Data is about truth,
auditability, and the ability to
  analyze data on a level
        playing field.

   It’s about analysis for
          everyone.
Alistair Croll
                          @acroll
                          www.solveforinteresting.com

THANKS!                   alistair@solveforinteresting.com




SOLVEfor
INTERESTING
OTHERWISE LIFE IS DULL.

More Related Content

What's hot

What's hot (20)

How AI Can Help You Make Your Audience Sit Up and Take Notice
How AI Can Help You Make Your Audience Sit Up and Take NoticeHow AI Can Help You Make Your Audience Sit Up and Take Notice
How AI Can Help You Make Your Audience Sit Up and Take Notice
 
Beyond Measure, Erika Hall
Beyond Measure, Erika HallBeyond Measure, Erika Hall
Beyond Measure, Erika Hall
 
The ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | MaterclassThe ultimate guide to data storytelling | Materclass
The ultimate guide to data storytelling | Materclass
 
Storyfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to StoriesStoryfying your Data: How to go from Data to Insights to Stories
Storyfying your Data: How to go from Data to Insights to Stories
 
Facts, Figures & Fictions
Facts, Figures & Fictions Facts, Figures & Fictions
Facts, Figures & Fictions
 
Ask Measure Learn
Ask Measure LearnAsk Measure Learn
Ask Measure Learn
 
The value of storytelling through data
The value of storytelling through dataThe value of storytelling through data
The value of storytelling through data
 
Be Data Informed Without Being a Data Scientist
Be Data Informed Without Being a Data ScientistBe Data Informed Without Being a Data Scientist
Be Data Informed Without Being a Data Scientist
 
From Information to Insight: Data Storytelling for Organizations
From Information to Insight: Data Storytelling for OrganizationsFrom Information to Insight: Data Storytelling for Organizations
From Information to Insight: Data Storytelling for Organizations
 
Humanizing Data Storytelling for Greater Business Impact
Humanizing Data Storytelling for Greater Business ImpactHumanizing Data Storytelling for Greater Business Impact
Humanizing Data Storytelling for Greater Business Impact
 
Workshop Data Manager
Workshop Data ManagerWorkshop Data Manager
Workshop Data Manager
 
Bigit Keynote - Big Data & Critical Thinking
Bigit Keynote - Big Data & Critical ThinkingBigit Keynote - Big Data & Critical Thinking
Bigit Keynote - Big Data & Critical Thinking
 
Cornell 140721-open
Cornell 140721-openCornell 140721-open
Cornell 140721-open
 
Improve Customer Experiences With Big Data #DataTalk
Improve Customer Experiences With Big Data #DataTalkImprove Customer Experiences With Big Data #DataTalk
Improve Customer Experiences With Big Data #DataTalk
 
How To Get Into Data Science & Analytics - feliperego.com.au
How To Get Into Data Science & Analytics - feliperego.com.auHow To Get Into Data Science & Analytics - feliperego.com.au
How To Get Into Data Science & Analytics - feliperego.com.au
 
Is Big Data for Real?
Is Big Data for Real?Is Big Data for Real?
Is Big Data for Real?
 
Acceptance, Accessible, Actionable and Auditable
Acceptance, Accessible, Actionable and AuditableAcceptance, Accessible, Actionable and Auditable
Acceptance, Accessible, Actionable and Auditable
 
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
 
The Art of Speaking Data.
The Art of Speaking Data.The Art of Speaking Data.
The Art of Speaking Data.
 
Penguin, SEO and the Apocalypse
Penguin, SEO and the ApocalypsePenguin, SEO and the Apocalypse
Penguin, SEO and the Apocalypse
 

Similar to Big data tokyo (extended version)

Big data and enterprise search trends 120827nn
Big data and enterprise search trends 120827nnBig data and enterprise search trends 120827nn
Big data and enterprise search trends 120827nn
Cathy McKnight
 
Big Data in Practice.pdf
Big Data in Practice.pdfBig Data in Practice.pdf
Big Data in Practice.pdf
Tom Tan
 

Similar to Big data tokyo (extended version) (20)

The Advantages and Disadvantages of Big Data
The Advantages and Disadvantages of Big DataThe Advantages and Disadvantages of Big Data
The Advantages and Disadvantages of Big Data
 
L21 Big Data and Analytics
L21 Big Data and AnalyticsL21 Big Data and Analytics
L21 Big Data and Analytics
 
Data mining with big data implementation
Data mining with big data implementationData mining with big data implementation
Data mining with big data implementation
 
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
NPTEL BIG DATA FULL PPT  BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...NPTEL BIG DATA FULL PPT  BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
 
Mind and the machine
Mind and the machineMind and the machine
Mind and the machine
 
Big Data By Vijay Bhaskar Semwal
Big Data By Vijay Bhaskar SemwalBig Data By Vijay Bhaskar Semwal
Big Data By Vijay Bhaskar Semwal
 
BIG DATA AND HADOOP.pdf
BIG DATA AND HADOOP.pdfBIG DATA AND HADOOP.pdf
BIG DATA AND HADOOP.pdf
 
Big Data v. Small data - Rules to thumb for 2015
Big Data v. Small data - Rules to thumb for 2015Big Data v. Small data - Rules to thumb for 2015
Big Data v. Small data - Rules to thumb for 2015
 
sybca-bigdata-ppt.pptx
sybca-bigdata-ppt.pptxsybca-bigdata-ppt.pptx
sybca-bigdata-ppt.pptx
 
Big Data for One Big Family
Big Data for One Big FamilyBig Data for One Big Family
Big Data for One Big Family
 
Keynote on 2015 Yale Day of Data
Keynote on 2015 Yale Day of Data Keynote on 2015 Yale Day of Data
Keynote on 2015 Yale Day of Data
 
Demystifying Big Data for Associations
Demystifying Big Data for AssociationsDemystifying Big Data for Associations
Demystifying Big Data for Associations
 
"Big Data Dreams"
"Big Data Dreams""Big Data Dreams"
"Big Data Dreams"
 
Data mining with big data
Data mining with big dataData mining with big data
Data mining with big data
 
Data mining with big data
Data mining with big dataData mining with big data
Data mining with big data
 
Introduction to big data
Introduction to big dataIntroduction to big data
Introduction to big data
 
Big data and enterprise search trends 120827nn
Big data and enterprise search trends 120827nnBig data and enterprise search trends 120827nn
Big data and enterprise search trends 120827nn
 
Top 5 Truths About Big Data Hype and Security Intelligence
Top 5 Truths About Big Data Hype and Security IntelligenceTop 5 Truths About Big Data Hype and Security Intelligence
Top 5 Truths About Big Data Hype and Security Intelligence
 
Big Data: Friend, Phantom or Foe?
Big Data: Friend, Phantom or Foe?Big Data: Friend, Phantom or Foe?
Big Data: Friend, Phantom or Foe?
 
Big Data in Practice.pdf
Big Data in Practice.pdfBig Data in Practice.pdf
Big Data in Practice.pdf
 

More from Lean Analytics

More from Lean Analytics (17)

Lean Analytics for Startups and Enterprises
Lean Analytics for Startups and EnterprisesLean Analytics for Startups and Enterprises
Lean Analytics for Startups and Enterprises
 
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
Introduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SF
 
Lean Analytics @ MicroConf
Lean Analytics @ MicroConfLean Analytics @ MicroConf
Lean Analytics @ MicroConf
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
 
OnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
 
7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help
 
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)
 
Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine Montreal
 

Recently uploaded

CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Recently uploaded (20)

How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 

Big data tokyo (extended version)