Your SlideShare is downloading. ×
0
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
The Paradox of Big Data: Misery or Magic?
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The Paradox of Big Data: Misery or Magic?

511

Published on

We are all becoming adherents of the data-driven life. Big data, data-driven decisioning, the quantified self are the buzzwords of today. Arguments not backed by data just do not cut it, whether it is …

We are all becoming adherents of the data-driven life. Big data, data-driven decisioning, the quantified self are the buzzwords of today. Arguments not backed by data just do not cut it, whether it is in decision making, a business meeting, or a casual social conversation.

The Powerpoint Presentation of Suresh Shankar's Speech at Big Data Asia World 2013, Singapore.

Published in: Technology, Spiritual
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
511
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Big Data World Asia 2013 Suresh Shankar, Crayon Data 13 September 2013 1 The Paradox of Big Data: Misery or Magic?
  • 2. 13 September 2013 2 Before we start, a personal perspective Loyalty is dead Big Data will flourish
  • 3. 3 We live in the Age of MORE MORE MORE MORE Connected channels to interact with consumers Transparency driven by ubiquity of information Sources of influence Real-time environment than ever Personalisation expected by consumers Choices available to consumers
  • 4. 4 Big data is changing life for businesses… and consumers Volume Velocity Variety Data drives the age of more
  • 5. 13 September 2013 5 Reality: Data on anything & everything at our fingertips… Food & Beverage TravelHealthcare Fitness
  • 6. 13 September 2013 6 Reality: we are all adherents of the “data-driven life” o  MedHelp - One of the largest forums for health information o  More than 30,000 new personal tracking projects are started by users every month Source: The New York Times, “The Data-Driven Life, By Gary Wolf, April 28, 2010 o  Fitbit – a personal activity tracker that measures the steps you walk, quality of sleep, calories burned and other personal fitness metrics. o  Achieve and share targets / goals o  Airbnb – platform for short-term lodging for guests o  Collects data across 250k listings in 30k cities to personalize stays based on preferences, social connections, rental history, reviews etc.
  • 7. 13 September 2013 7 We are now at an inflection point in the big data revolution Each of us now leaves a trail of digital exhaust, an infinite stream of phone records, texts, browser histories, GPS data and other information, that will live on forever Every animate & inanimate object on earth will soon be generating data, including our homes, our cars, and YES, even our bodies. An average person today processes more data in a single day than a person in the 1500s did in an entire lifetime Source: The Human Face of Big Data
  • 8. 8 ...but data can easily become misery Decision Paralysis Post-purchase dissonance Default to the easiest option Source: The Paradox of Choice, Barry Schwartz
  • 9. 13 September 2013 9 At work… The latest numbers show that… …let’s look into the numbers in greater detail, let’s resolve this difference, let’s get accurate & up-to-date numbers next time Result: No conclusions, postponed decisions
  • 10. 13 September 2013 10 At home… 74 Formulas Liquids / Powders / Tablets Cotton / Polyester 20x stains Plant based / Chemical Premium / Value Front loading / Top Loading Hot vs. Cold Source: http://www.goodhousekeeping.com/product-reviews Over 1,000 possible outcomes… Result: Buying default product because there were too many to choose from
  • 11. 13 September 2013 11 At play… Which team is performing better? Who will win the match / series? What do the statistics tell us? Result: Selective use of facts. Who is the best player?
  • 12. 13 September 2013 12 We are inundated with more and more data and we have less and less time to navigate and make sense of this data The Paradox of More
  • 13. 13 September 2013 13 How can we make data magic? We need a different mind-set ‘what is the meaning of this analysis?’ ‘can we see some more data?’ ‘what if we analyse it differently?’ ‘…can we run some more analysis?’ ‘…is the data right?’ ‘why did this happen?’ Traditional mind-set
  • 14. 13 September 2013 14 Big Data can set your mind free Stop trying to understand why something happened, and let statistical algorithms find patterns we cannot detect Leverage intelligent algorithms to reduce the number of choices to a relevant few More data beats better algorithms Look for confidence level, not certainty in predicting outcomes
  • 15. 13 September 2013 15 Correlation is enough. We can stop looking for models. We can analyze the data without hypotheses about what it might show…. The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Chris Anderson, Wired Magazine Correlation = Causation Source: Wired Magazine, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete, Chris Anderson, June 2008
  • 16. 13 September 2013 16 Correlation is enough. We can stop looking for models. We can analyze the data without hypotheses about what it might show…. The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Chris Anderson, Wired Magazine Correlation Trumps Causation Source: Wired Magazine, “The End of Theory: The Data Deluge Makes the Scientific Method Obsolete, Chris Anderson, June 2008
  • 17. 13 September 2013 17 More data beats better algorithms In a nutshell, having more data allows the “data to speak for itself,” instead of relying on unproven assumptions and weak correlations More data can reveal non-linear relationships within a dataset
  • 18. 13 September 2013 18 When people are given a moderate number of options (4 to 6) rather than a large number (20 to 30), they are more likely to make a choice, are more confident in their decisions, and are happier with what they choose. Sheena Iyengar, The Art of Choosing Less = More
  • 19. 13 September 2013 19 The era of probabilistic evidence-based answers is here §  Extracts facts and understands the relationships in vast quantities of data §  Judges each possible answer in real time and decides which one is most likely the correct answer §  Maps possible answers to a probabilistic estimate that the response is correct with a “level of confidence” confidence
  • 20. 13 September 2013 20 To make big data magic, we need to resolve the “Paradox of More” Algorithms that help data find data Bigger data sets merging external & enterprise data Make data visual and easy to understand
  • 21. 13 September 2013 21 What if consumer’s decisions were easier? Enable consumers to make choices based on discovery, interest & serendipity… Her tastemakers / Friends…? Who / What influences her? Her tastes? Her past behaviour? Her context - location, weather, time…?
  • 22. 13 September 2013 22 What if business decisions were easier? Mix and utilize vast amounts of internal & external datasets to find relationships Derive range of outcomes / possibilities at various confidence levels Predict the outcomes of decisions
  • 23. 13 September 2013 23 Example of making big data magic: A Choice Engine to map & predict the world’s choices
  • 24. Suresh Shankar suresh@crayondata.com +65 96639144 24 Thank you

×