Your SlideShare is downloading. ×
Big data wonderland
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Big data wonderland

154
views

Published on

Two Views on the Big Data Revolution

Two Views on the Big Data Revolution


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
154
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Big Data Wonderland:! Two Views on the Big Data Revolution Strata New York October 2012Mark Madsen Marc DemarestThird Nature, Inc. Noumenal, Inc.mark@thirdnature.net marc@noumenal.com@markmadsen
  • 2. Preamble Twenty Years On •  We came up together in this industry in the early 1990s, as pointy-headed advocates of data and star schema design, trained by the deity himself, Ralph Kimball Our Alma Mater •  Back then, it was a simpler world...big iron, big DBMS, hand-coded ETL, star schema, a thousand rinky-dink query tools •  Mostly, conversation was dominated by ETL and schema design •  “There will never be a decisional St. Ralph database larger than 10 GB...”2 Third Nature, Inc. || Noumenal, Inc.
  • 3. Preamble Twenty Years On •  Twenty years on, we find ourselves with opposing view on what is either the biggest con, or the biggest sea-change, in our data warehousing odyssey Madsen as Jack Kilpatrick? •  Question: Is the big data revolution big, or a revolution? •  Question: do we have to change? and if so, how? •  Not a round table. A slugfest.... Demarest as Shana Alexander?3 Third Nature, Inc. || Noumenal, Inc.
  • 4. Regular Programming Is Suspended Demarest Madsen4 Third Nature, Inc. || Noumenal, Inc.
  • 5. Compromise You take the blue pill. The story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes. Remember, all I am offering is the truth: Demarest Madsen nothing more.5 Third Nature, Inc. || Noumenal, Inc.
  • 6. The Issues 1. Data As A Factor of Production RED BLUE Hype. For most companies, data is an asset supporting process, not a factor in the production of its products or services. Execute vs manage the business.6 Third Nature, Inc. || Noumenal, Inc.
  • 7. The Issues 1. Data As A Factor of Production RED BLUE Amen. Hype. This change has For most companies, been in process for data is an asset more than a decade. supporting process, Social media leads not a factor in the the way, but we’re all production of its affected. products or services. Execute vs manage the business.7 Third Nature, Inc. || Noumenal, Inc.
  • 8. The Issues 2. The Reality of Big Data RED BLUE No company escapes. Text, social, sensors, streaming -- the instrumentation of the real world transforms company decision-making processes.8 Third Nature, Inc. || Noumenal, Inc.
  • 9. The Issues 2. The Reality of Big Data RED BLUE No company escapes. Few companies transformed. Text, social, sensors, streaming -- the Social media help up instrumentation of a lot with no the real world quantification of transforms company benefits. decision-making processes. Management consultants? Asleep at the switch.9 Third Nature, Inc. || Noumenal, Inc.
  • 10. The Issues 3. The Commodity Hardware Revolution & Radical Scale-Out RED BLUE The current topology is alive and well. These commodity building blocks are, after all, just SMP platforms. Real problems are under-investment, bad design.10 Third Nature, Inc. || Noumenal, Inc.
  • 11. The Issues 3. The Commodity Hardware Revolution & Radical Scale-Out RED BLUE The new topology. The current topology is alive and well. Cheap compute, unintelligent direct- These commodity attach storage and building blocks are, free comms make after all, just SMP large scale-out grids platforms. the future. Real problems are under-investment, bad design.11 Third Nature, Inc. || Noumenal, Inc.
  • 12. The Issues 4. Merchant DBMSs RED BLUE Increasingly irrelevant. We’ve been over- structured and under-resourced for 20 years. CSV is still the international standard.12 Third Nature, Inc. || Noumenal, Inc.
  • 13. The Issues 4. Merchant DBMSs RED BLUE Increasingly Will rise to the irrelevant. challenge. We’ve been over- Any worthwhile structured and innovation will be under-resourced for absorbed by the 20 years. merchant DBMS players. CSV is still the international Even the big players standard. use these things.13 Third Nature, Inc. || Noumenal, Inc.
  • 14. The Issues 5. Query, Reporting & Dashboarding Tools RED BLUE Will rise to the challenge. We have two generations of analysts trained to feed using these tools. Big data offers no last-mile answers.14 Third Nature, Inc. || Noumenal, Inc.
  • 15. The Issues 5. Query, Reporting & Dashboarding Tools RED BLUE Ineffective, now and Will rise to the in the future. challenge. Can’t do real-time, We have two can’t visualize large generations of data sets, can’t analysts trained to support discovery feed using these and exploration. tools. Big data offers no last-mile answers.15 Third Nature, Inc. || Noumenal, Inc.
  • 16. The Issues 6. Structured Query Language RED BLUE Toast. Too complex, too hard to code, too hard to debug. A way of ensuring dependency on merchant DBMSs.16 Third Nature, Inc. || Noumenal, Inc.
  • 17. The Issues 6. Structured Query Language RED BLUE Toast. Tasty. Too complex, too Powerful, expressive hard to code, too language for hard to debug. A way complex analytical of ensuring problems. dependency on merchant DBMSs. Why do noSQL vendors reinvent it all the time?17 Third Nature, Inc. || Noumenal, Inc.
  • 18. The Issues 7. New Programming Models RED BLUE The “new model” looks a lot like SAS, only with java and no support. Open source doesn’t mean free. Or easy. The skills gap here is huge, we can’t fill it.18 Third Nature, Inc. || Noumenal, Inc.
  • 19. The Issues 7. New Programming Models RED BLUE Say hello to Pig. The “new model” looks a lot like SAS, New analytical only with java and problems no support. (decisioning, discovery, Open source doesn’t exploration) require mean free. Or easy. new languages, new tools and new The skills gap here is programming huge, we can’t fill it. models.19 Third Nature, Inc. || Noumenal, Inc.
  • 20. The Issues 8. Conventional DW Architecture RED BLUE A relic. Overly complex. Difficult to implement. Controlled by the supply side of the market, anyway.20 Third Nature, Inc. || Noumenal, Inc.
  • 21. The Issues 8. Conventional DW Architecture RED BLUE A relic. Perfectly viable. No need to change. Overly complex. Difficult to Some new implement. technologies may Controlled by the play roles, but we’re supply side of the good to go, generally. market, anyway. Built by developers for users. The new is built by developers for developers.21 Third Nature, Inc. || Noumenal, Inc.
  • 22. The Issues 9. The Cloud RED BLUE Don’t go there. Your inside-the- firewall apps remain the core information asset. Where is “there” anyway?22 Third Nature, Inc. || Noumenal, Inc.
  • 23. The Issues 9. The Cloud RED BLUE We all go there. Don’t go there. Most of the Your inside-the- interesting data is firewall apps remain there; it’s more the core information effective to move our asset. data, and our analyses, to where Where is “there” the data is, already. anyway?23 Third Nature, Inc. || Noumenal, Inc.
  • 24. The Issues 10. New Technologies RED BLUE Save Us. Best of breed integration led by in- house designers ins back, with a vengeance.24 Third Nature, Inc. || Noumenal, Inc.
  • 25. The Issues 10. Emerging Technologies RED BLUE Save Us. Distract Us. Best of breed We’ve already seen integration led by in- what best-of-breed house designers ins gives us: a circus. back, with a vengeance.25 Third Nature, Inc. || Noumenal, Inc.
  • 26. What We Really Think 1. Data As A Factor of Production 2. The Reality of Big Data 3. The Commodity Hardware Revolution 4. Merchant DBMSs 5. Query, Reporting & Dashboarding Tools 6. Structured Query Language 7. New Programming Models 8. Conventional DW Architecture 9. The Cloud 10. New Technologies26 Third Nature, Inc. || Noumenal, Inc.