• Save
The Paradox of Big Data: Misery or Magic?
Upcoming SlideShare
Loading in...5
×
 

The Paradox of Big Data: Misery or Magic?

on

  • 572 views

We are all becoming adherents of the data-driven life. Big data, data-driven decisioning, the quantified self are the buzzwords of today. Arguments not backed by data just do not cut it, whether it is ...

We are all becoming adherents of the data-driven life. Big data, data-driven decisioning, the quantified self are the buzzwords of today. Arguments not backed by data just do not cut it, whether it is in decision making, a business meeting, or a casual social conversation.

The Powerpoint Presentation of Suresh Shankar's Speech at Big Data Asia World 2013, Singapore.

Statistics

Views

Total Views
572
Views on SlideShare
562
Embed Views
10

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 10

https://twitter.com 10

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Paradox of Big Data: Misery or Magic? The Paradox of Big Data: Misery or Magic? Presentation Transcript

  • Big Data World Asia 2013 Suresh Shankar, Crayon Data 13 September 2013 1 The Paradox of Big Data: Misery or Magic?
  • 13 September 2013 2 Before we start, a personal perspective Loyalty is dead Big Data will flourish
  • 3 We live in the Age of MORE MORE MORE MORE Connected channels to interact with consumers Transparency driven by ubiquity of information Sources of influence Real-time environment than ever Personalisation expected by consumers Choices available to consumers
  • 4 Big data is changing life for businesses… and consumers Volume Velocity Variety Data drives the age of more
  • 13 September 2013 5 Reality: Data on anything & everything at our fingertips… Food & Beverage TravelHealthcare Fitness
  • 13 September 2013 6 Reality: we are all adherents of the “data-driven life” o  MedHelp - One of the largest forums for health information o  More than 30,000 new personal tracking projects are started by users every month Source: The New York Times, “The Data-Driven Life, By Gary Wolf, April 28, 2010 o  Fitbit – a personal activity tracker that measures the steps you walk, quality of sleep, calories burned and other personal fitness metrics. o  Achieve and share targets / goals o  Airbnb – platform for short-term lodging for guests o  Collects data across 250k listings in 30k cities to personalize stays based on preferences, social connections, rental history, reviews etc.
  • 13 September 2013 7 We are now at an inflection point in the big data revolution Each of us now leaves a trail of digital exhaust, an infinite stream of phone records, texts, browser histories, GPS data and other information, that will live on forever Every animate & inanimate object on earth will soon be generating data, including our homes, our cars, and YES, even our bodies. An average person today processes more data in a single day than a person in the 1500s did in an entire lifetime Source: The Human Face of Big Data
  • 8 ...but data can easily become misery Decision Paralysis Post-purchase dissonance Default to the easiest option Source: The Paradox of Choice, Barry Schwartz
  • 13 September 2013 9 At work… The latest numbers show that… …let’s look into the numbers in greater detail, let’s resolve this difference, let’s get accurate & up-to-date numbers next time Result: No conclusions, postponed decisions
  • 13 September 2013 10 At home… 74 Formulas Liquids / Powders / Tablets Cotton / Polyester 20x stains Plant based / Chemical Premium / Value Front loading / Top Loading Hot vs. Cold Source: http://www.goodhousekeeping.com/product-reviews Over 1,000 possible outcomes… Result: Buying default product because there were too many to choose from
  • 13 September 2013 11 At play… Which team is performing better? Who will win the match / series? What do the statistics tell us? Result: Selective use of facts. Who is the best player?
  • 13 September 2013 12 We are inundated with more and more data and we have less and less time to navigate and make sense of this data The Paradox of More
  • 13 September 2013 13 How can we make data magic? We need a different mind-set ‘what is the meaning of this analysis?’ ‘can we see some more data?’ ‘what if we analyse it differently?’ ‘…can we run some more analysis?’ ‘…is the data right?’ ‘why did this happen?’ Traditional mind-set
  • 13 September 2013 14 Big Data can set your mind free Stop trying to understand why something happened, and let statistical algorithms find patterns we cannot detect Leverage intelligent algorithms to reduce the number of choices to a relevant few More data beats better algorithms Look for confidence level, not certainty in predicting outcomes
  • 13 September 2013 15 Correlation is enough. We can stop looking for models. We can analyze the data without hypotheses about what it might show…. The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Chris Anderson, Wired Magazine Correlation = Causation Source: Wired Magazine, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete, Chris Anderson, June 2008
  • 13 September 2013 16 Correlation is enough. We can stop looking for models. We can analyze the data without hypotheses about what it might show…. The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Chris Anderson, Wired Magazine Correlation Trumps Causation Source: Wired Magazine, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete, Chris Anderson, June 2008
  • 13 September 2013 17 More data beats better algorithms In a nutshell, having more data allows the “data to speak for itself,” instead of relying on unproven assumptions and weak correlations More data can reveal non-linear relationships within a dataset
  • 13 September 2013 18 When people are given a moderate number of options (4 to 6) rather than a large number (20 to 30), they are more likely to make a choice, are more confident in their decisions, and are happier with what they choose. Sheena Iyengar, The Art of Choosing Less = More
  • 13 September 2013 19 The era of probabilistic evidence-based answers is here §  Extracts facts and understands the relationships in vast quantities of data §  Judges each possible answer in real time and decides which one is most likely the correct answer §  Maps possible answers to a probabilistic estimate that the response is correct with a “level of confidence” confidence
  • 13 September 2013 20 To make big data magic, we need to resolve the “Paradox of More” Algorithms that help data find data Bigger data sets merging external & enterprise data Make data visual and easy to understand
  • 13 September 2013 21 What if consumer’s decisions were easier? Enable consumers to make choices based on discovery, interest & serendipity… Her tastemakers / Friends…? Who / What influences her? Her tastes? Her past behaviour? Her context - location, weather, time…?
  • 13 September 2013 22 What if business decisions were easier? Mix and utilize vast amounts of internal & external datasets to find relationships Derive range of outcomes / possibilities at various confidence levels Predict the outcomes of decisions
  • 13 September 2013 23 Example of making big data magic: A Choice Engine to map & predict the world’s choices
  • Suresh Shankar suresh@crayondata.com +65 96639144 24 Thank you