Big Data? No. Big Decisions
    are What You Want.

              Stuart Miniman
          Wikibon, Senior Analyst
                    stu@wikibon.org
                    @stu

This presentation and more at http://wikibon.org/BigData
Big Questions

What is Big Data?
Evolution or Revolution of Business
 Intelligence (BI)?
Who is Using Big Data?
How Should Practitioners Proceed?
Massive Data Growth




      Source: http://wikibon.org/blog/infographics/
Transforming Data       Knowledge




       BI as we know it has failed.
The Old Way
       CRM
                          Data          ETL                                      Traditional
       ERP                                           Normalized Data             Data
                          Data      Data Quality
                                                                                 Warehouse
       Finance




                                  Business Analyst
Data Warehouse Administrator


                                                                 Business User
VOLUME
   TYPE
  SPEED
BIG DATA
BIG DATA
 Process and Analyze ALL Your Data
        Ask NEW Questions
       Ask MORE Questions
       Get Answers FASTER
       Get CLEARER Insight

MAKE BETTER BUSINESS
      DECISIONS
BIT FLIP
Subsets                 All Data
Historical              Near Real-time
Structured (database)   Structured/Unstructured
Data growth as a        Data as a new source of
  burden & challenge      competitive opportunity
Two NEW APPROACHES to BIG DATA
Hadoop is is open source framework for
processing and analyzing massive
amounts of distributed data.




                                  Next Generation Data Warehouses use
                                  massively parallel processing, columnar
                                  architectures and data compression to analyze
                                  not-quite-so-massive data in close to real-time.




These two approaches overlap in some areas
and compliment one another in other areas.
Data Scientists
10/90 rule for magnificent data success
  Over-invest in people, because
  without that investment big data
  will absolutely, positively, be a big
  disappointment for your
  company. Computers and
  artificial intelligence are simply
  not there yet. Hence your BFF is
  natural intelligence.
                    -AvinashKaushik


      http://www.kaushik.net/avinash/big-data-imperative-driving-big-action/
BIG MONEY in BIG DATA




                     CAGR of 58%
Revenue mix today: 44% services, 31% hardware, 25% software
Recommendation Engine
Use Hadoop to match and recommend
users to one another or to products and
services based on analysis of user profile
and behavioral data
IT: BIaaS




Predictions of Future
Equipment Failures
Large Media Company:
                 BIG DATA + WAN
             Site 1: Advertising                       Site 2: Content
             Analysis                                  Customization
                                            10
                     Hadoop             GbpsHadoop             Hadoop
                     Clusters             Traffic              Clusters
                       DC1                                       DC2
                       7PB                                       7PB
1                                  2                           3
    10 TB’s/day of source              5 TB’s/day for inter-        1 TB/day of
    data: browsing pattern,            cluster sync                results sent for
    click throughs, server logs                                    integrated analysis with
                                                                   structured data
Customer Experience Analytics
Integrating data from previously siloed channels
such as call center, online chat, Twitter, etc.




                              Source: Clickfox
New Revenue from Data

                      The Associated Press is combining a mix
                      of decades of historical news releases with
                      real-time additions to create new
                      monetization opportunities for it’s data
                      using a document-oriented database
                      (rather than traditional relational
                      database).



NYSE is delivering analytics on data that
is seeing massive growth that adds up to
Petabytes of information that can be
offered as a cloud service to traders.
BIG DATA Infrastructure
Network optimization (low latency)
Share-nothing storage
  – Bring the computation to the data
Massive compute requirements
  – Emerging opportunity in the cloud
BIG DATA Organization
Broad cross-silo impact
  – Tight coordination needed between business
    decision makers and technology/analyst
Organize for selling/buying data within
 organization and IT
  – Next-generation “chargeback”
What’s Your BIG DATA Strategy?

Enterprises should …
              EVALUATE
               ENGAGE
                PLAN
              CULTIVATE
              EXECUTE
               REPEAT
Creating a BIG DATA IT Plan
• Understand IO-centric technologies that
  allow near real-time big data processing
• Select key vendor partnerships
• Start with small projects of integrated
  design
• Investigate opportunities to deliver big
  data services for your industry
What’s Your BIG DATA Strategy?

Vendors should …
               LISTEN
             EDUCATE
             INNOVATE
                SELL
             SUPPORT
              REPEAT
Now Is the Era of
               BIG DATA
        Big Data is the new definitive source of
      competitive advantage across all industries.


         Special Thanks to David Floyer and Jeff Kelly


This presentation and more at http://wikibon.org/BigData

Big data? No. Big Decisions are What You Want

  • 1.
    Big Data? No.Big Decisions are What You Want. Stuart Miniman Wikibon, Senior Analyst stu@wikibon.org @stu This presentation and more at http://wikibon.org/BigData
  • 2.
    Big Questions What isBig Data? Evolution or Revolution of Business Intelligence (BI)? Who is Using Big Data? How Should Practitioners Proceed?
  • 3.
    Massive Data Growth Source: http://wikibon.org/blog/infographics/
  • 4.
    Transforming Data Knowledge BI as we know it has failed.
  • 5.
    The Old Way CRM Data ETL Traditional ERP Normalized Data Data Data Data Quality Warehouse Finance Business Analyst Data Warehouse Administrator Business User
  • 6.
    VOLUME TYPE SPEED BIG DATA
  • 7.
    BIG DATA Processand Analyze ALL Your Data Ask NEW Questions Ask MORE Questions Get Answers FASTER Get CLEARER Insight MAKE BETTER BUSINESS DECISIONS
  • 8.
    BIT FLIP Subsets All Data Historical Near Real-time Structured (database) Structured/Unstructured Data growth as a Data as a new source of burden & challenge competitive opportunity
  • 9.
    Two NEW APPROACHESto BIG DATA Hadoop is is open source framework for processing and analyzing massive amounts of distributed data. Next Generation Data Warehouses use massively parallel processing, columnar architectures and data compression to analyze not-quite-so-massive data in close to real-time. These two approaches overlap in some areas and compliment one another in other areas.
  • 10.
    Data Scientists 10/90 rulefor magnificent data success Over-invest in people, because without that investment big data will absolutely, positively, be a big disappointment for your company. Computers and artificial intelligence are simply not there yet. Hence your BFF is natural intelligence. -AvinashKaushik http://www.kaushik.net/avinash/big-data-imperative-driving-big-action/
  • 11.
    BIG MONEY inBIG DATA CAGR of 58% Revenue mix today: 44% services, 31% hardware, 25% software
  • 12.
    Recommendation Engine Use Hadoopto match and recommend users to one another or to products and services based on analysis of user profile and behavioral data
  • 13.
    IT: BIaaS Predictions ofFuture Equipment Failures
  • 14.
    Large Media Company: BIG DATA + WAN Site 1: Advertising Site 2: Content Analysis Customization 10 Hadoop GbpsHadoop Hadoop Clusters Traffic Clusters DC1 DC2 7PB 7PB 1 2 3 10 TB’s/day of source 5 TB’s/day for inter- 1 TB/day of data: browsing pattern, cluster sync results sent for click throughs, server logs integrated analysis with structured data
  • 15.
    Customer Experience Analytics Integratingdata from previously siloed channels such as call center, online chat, Twitter, etc. Source: Clickfox
  • 16.
    New Revenue fromData The Associated Press is combining a mix of decades of historical news releases with real-time additions to create new monetization opportunities for it’s data using a document-oriented database (rather than traditional relational database). NYSE is delivering analytics on data that is seeing massive growth that adds up to Petabytes of information that can be offered as a cloud service to traders.
  • 17.
    BIG DATA Infrastructure Networkoptimization (low latency) Share-nothing storage – Bring the computation to the data Massive compute requirements – Emerging opportunity in the cloud
  • 18.
    BIG DATA Organization Broadcross-silo impact – Tight coordination needed between business decision makers and technology/analyst Organize for selling/buying data within organization and IT – Next-generation “chargeback”
  • 19.
    What’s Your BIGDATA Strategy? Enterprises should … EVALUATE ENGAGE PLAN CULTIVATE EXECUTE REPEAT
  • 20.
    Creating a BIGDATA IT Plan • Understand IO-centric technologies that allow near real-time big data processing • Select key vendor partnerships • Start with small projects of integrated design • Investigate opportunities to deliver big data services for your industry
  • 21.
    What’s Your BIGDATA Strategy? Vendors should … LISTEN EDUCATE INNOVATE SELL SUPPORT REPEAT
  • 22.
    Now Is theEra of BIG DATA Big Data is the new definitive source of competitive advantage across all industries. Special Thanks to David Floyer and Jeff Kelly This presentation and more at http://wikibon.org/BigData

Editor's Notes

  • #2 Resources:http://wikibon.org/BigData
  • #3 Abstract: Everyone talks about big data, but big data isn’t really useful unless you can use it. What you need are big decisions. In this session, you will learn what constitutes big data, best practices to store it for retrieval, and how to use it to make business decisions. We will include a few case studies illustrating key points and provide a starting point on how to use big data to make big decisions
  • #4 Gigabytes to Petabytes to Exabytes to ZettabytesToday’s “Big Storage” is tomorrow’s ”Little Storage”
  • #5 Tools to understand data have been around for a long time – even in the 90’s we were learning how to “read the matrix”TRYING to do this isn’t new
  • #6 Structured, well defined questions, typically not agile
  • #10 Hadoop distributions include Cloudera, Hortonworks, MapRNGDW = EMC Greenplum, HP Vertica, Teradata Aster, IBM Netezza
  • #11 High demand for new skills – gap in the workforce
  • #12 Big names include IBM, Intel, Oracle, HP and “pure plays” like the vendors discussed on NGDW + Hadoop distribution slide
  • #13 Everyone is familiar with websites that are crunching massive amounts of data to help provide connections/insight
  • #14 Here’s an example from a large IT player who is “dogfooding” Big Data.
  • #15 All your data – all sources – all locations
  • #16 Understand your customers (ad placement, customer retention and much more)
  • #17 Data = opportunity
  • #18 Scalability, flexibility/extensible, robust architecture