Operationalizing the Buzz:
Big Data 2013
A Review of EMA / 9sight Research
Shawn P. Rogers
VP of Research
EMA
John Myers
Senior Analyst
EMA
Dr. Barry Devlin
Founder & Principal
9sight Consulting
© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Today’s Presenters
2 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Shawn Rogers, VP of Research, EMA
Shawn has more than 19 years of hands-on IT experience, with a focus on Internet-
enabled technology. In 2004 he co-founded the BeyeNETWORK and held the position of
Executive VP and Editorial Director. Shawn guided the company's international growth
strategy and helped the BeyeNETWORK grow to 18 Web sites around the world, making
it the largest and most read community covering the business intelligence, data
warehousing, performance management and data integration space.
John Myers, Senior Analyst, EMA
John has over 10 years of experience working in areas related to business analytics in
professional services consulting and product development roles. Additionally, John helps
organizations solve their business analytics problems, whether they relate to operational
platforms – such as customer care or billing – or applied analytical applications – such as
revenue assurance or fraud management.
Dr. Barry Devlin, Founder & Principal, 9sight Consulting
Barry has 30 years of experience in the IT industry, previously with IBM, as an architect,
consultant, manager and software evangelist. He is a founder of the data warehousing
industry and among the foremost authorities worldwide on business intelligence. He is a
widely respected consultant, lecturer and author of “Data Warehouse--from Architecture
to Implementation.”
Logistics for Today’s Webinar
3
• An archived version of the event
recording will be available at
www.enterprisemanagement.com
• Log questions in the Q&A panel
located on the lower right corner of
your screen
• Questions will be addressed during the
Q&A session of the event
Questions
Event recording
© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Research Sponsors
4 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Agenda
• Research Highlights
• Respondents
• Projects
• Big Data Requirements
• Speed of Response
• Load
• Economics
• Complex Workload
• Structure
• Hybrid Data Ecosystem
• Question and Answer
5 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Research Methodology
• Highly Qualified
• Trends year over year
• Global in Coverage
• Balanced – Business and IT
• Real World Project Insights
• Almost 600 active Big Data projects
• Sponsors, workloads and more
6 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
2013 Research Highlights
7 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Key messages
• Clear move from market hype to real implementation
• Maturing of solutions
• More projects in production
• Machine-generated data shows significant growth
8 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Two thirds of respondents have more than one
Big Data project.
9 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
In operation grows from 36% to 53%
10 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
The REAL Big Data Logical Architecture
• REAL = Realistic, Extensible,
Actionable, Labile
• Three interconnected pillars
of information
• Messages, events, measures and
transactions from real world
• Metadata = context-setting information
• Adaptive information processing
• Instantiation, assimilation and
reification – ETL, ELT, Virtualization
• See “Business unIntelligence – Insight and
Innovation Beyond Analytics and Big Data”
http://bit.ly/BunI-Technics
EventsMeasures Messages
Transactions
Reification
Utilization
Instantiation
Human-
sourced
(information)
Machine-
generated
(data)
Process-
mediated
(data)
Context-setting (information)
Assimilation
Transactional
(data)
11 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Machine-generated data jumps 14%
12 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Survey Respondents
13 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Respondent Perception of Corporate Innovation
14 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Innovators around the World
Slide 15 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Big Data Projects
16 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Project Sponsors
17 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Business Users Take the Lead
18 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Drivers of Big Data
19 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
20 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Importance of Data in Business
21 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Technical Motivators
22 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
23 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Big Data Size for 2013
5.5-100TB
1.1TB-2PB
30%
indicated 11-20%
of overall data
management was
allotted to Big Data
initiatives
Most Common Response
Majority Response
24 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Big Data Management Size for 2014
6-120TB
1.2TB-2.8PB
Most Common Response
Majority Response
31%
of Respondents
indicated 11-20%
growth for 2014
25 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
26 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Big Data Budgets for 2013
$550k-$5m
$10k-$20m
Most Common Response
Majority Response
40%
of Respondents
indicated 11-20% of
overall IT Budgets
were assigned to
Big Data
27 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Who’s paying?
$88k-$1.25m
49%
of Respondents
indicated non IT
sources of Big Data
Budget funding
28 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
29 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Big Data Platforms have Multiple Uses
30 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Increasing Workloads
31 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
32 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Expanding Beyond the Wall:
Rome
33 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Expanding Beyond the Wall:
When and Where to Apply Schemas
34 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Hybrid Data Ecosystem
35 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Hybrid Data Ecosystem 2012:
Creating a New Nucleus
36 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Hybrid Data Ecosystem 2013:
Distribution of Platforms
One Platform,
28.2%
Two Platforms,
32.1%
Three Platforms,
27.8%
Four Platforms,
4.3%
Five Platforms,
3.5%
Six Platforms,
1.5%
Seven Platforms,
0.4%
Eight Platforms,
2.3%
Sample Size = 259
37 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Hybrid Data Ecosystem 2013:
From Requirements to Consumers
Question and Answer
39 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
Question and Answer:
Please Log Questions in the Q&A Panel
Get the Big Data Operationalizing the
Buzz Research Report at
https://www.enterprisemanagement.
com/research
Available October 21, 2013!
40 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting

EMA Analyst Slides: 2013 Big Data Research Results

  • 1.
    Operationalizing the Buzz: BigData 2013 A Review of EMA / 9sight Research Shawn P. Rogers VP of Research EMA John Myers Senior Analyst EMA Dr. Barry Devlin Founder & Principal 9sight Consulting © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 2.
    Today’s Presenters 2 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting Shawn Rogers, VP of Research, EMA Shawn has more than 19 years of hands-on IT experience, with a focus on Internet- enabled technology. In 2004 he co-founded the BeyeNETWORK and held the position of Executive VP and Editorial Director. Shawn guided the company's international growth strategy and helped the BeyeNETWORK grow to 18 Web sites around the world, making it the largest and most read community covering the business intelligence, data warehousing, performance management and data integration space. John Myers, Senior Analyst, EMA John has over 10 years of experience working in areas related to business analytics in professional services consulting and product development roles. Additionally, John helps organizations solve their business analytics problems, whether they relate to operational platforms – such as customer care or billing – or applied analytical applications – such as revenue assurance or fraud management. Dr. Barry Devlin, Founder & Principal, 9sight Consulting Barry has 30 years of experience in the IT industry, previously with IBM, as an architect, consultant, manager and software evangelist. He is a founder of the data warehousing industry and among the foremost authorities worldwide on business intelligence. He is a widely respected consultant, lecturer and author of “Data Warehouse--from Architecture to Implementation.”
  • 3.
    Logistics for Today’sWebinar 3 • An archived version of the event recording will be available at www.enterprisemanagement.com • Log questions in the Q&A panel located on the lower right corner of your screen • Questions will be addressed during the Q&A session of the event Questions Event recording © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 4.
    Research Sponsors 4 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 5.
    Agenda • Research Highlights •Respondents • Projects • Big Data Requirements • Speed of Response • Load • Economics • Complex Workload • Structure • Hybrid Data Ecosystem • Question and Answer 5 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 6.
    Research Methodology • HighlyQualified • Trends year over year • Global in Coverage • Balanced – Business and IT • Real World Project Insights • Almost 600 active Big Data projects • Sponsors, workloads and more 6 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 7.
    2013 Research Highlights 7© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 8.
    Key messages • Clearmove from market hype to real implementation • Maturing of solutions • More projects in production • Machine-generated data shows significant growth 8 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 9.
    Two thirds ofrespondents have more than one Big Data project. 9 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 10.
    In operation growsfrom 36% to 53% 10 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 11.
    The REAL BigData Logical Architecture • REAL = Realistic, Extensible, Actionable, Labile • Three interconnected pillars of information • Messages, events, measures and transactions from real world • Metadata = context-setting information • Adaptive information processing • Instantiation, assimilation and reification – ETL, ELT, Virtualization • See “Business unIntelligence – Insight and Innovation Beyond Analytics and Big Data” http://bit.ly/BunI-Technics EventsMeasures Messages Transactions Reification Utilization Instantiation Human- sourced (information) Machine- generated (data) Process- mediated (data) Context-setting (information) Assimilation Transactional (data) 11 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 12.
    Machine-generated data jumps14% 12 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 13.
    Survey Respondents 13 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 14.
    Respondent Perception ofCorporate Innovation 14 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 15.
    Innovators around theWorld Slide 15 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 16.
    Big Data Projects 16© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 17.
    Project Sponsors 17 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 18.
    Business Users Takethe Lead 18 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 19.
    Drivers of BigData 19 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 20.
    20 © 2013Enterprise Management Associates, Inc. & 9sight Consulting
  • 21.
    Importance of Datain Business 21 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 22.
    Technical Motivators 22 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 23.
    23 © 2013Enterprise Management Associates, Inc. & 9sight Consulting
  • 24.
    Big Data Sizefor 2013 5.5-100TB 1.1TB-2PB 30% indicated 11-20% of overall data management was allotted to Big Data initiatives Most Common Response Majority Response 24 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 25.
    Big Data ManagementSize for 2014 6-120TB 1.2TB-2.8PB Most Common Response Majority Response 31% of Respondents indicated 11-20% growth for 2014 25 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 26.
    26 © 2013Enterprise Management Associates, Inc. & 9sight Consulting
  • 27.
    Big Data Budgetsfor 2013 $550k-$5m $10k-$20m Most Common Response Majority Response 40% of Respondents indicated 11-20% of overall IT Budgets were assigned to Big Data 27 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 28.
    Who’s paying? $88k-$1.25m 49% of Respondents indicatednon IT sources of Big Data Budget funding 28 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 29.
    29 © 2013Enterprise Management Associates, Inc. & 9sight Consulting
  • 30.
    Big Data Platformshave Multiple Uses 30 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 31.
    Increasing Workloads 31 ©2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 32.
    32 © 2013Enterprise Management Associates, Inc. & 9sight Consulting
  • 33.
    Expanding Beyond theWall: Rome 33 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 34.
    Expanding Beyond theWall: When and Where to Apply Schemas 34 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 35.
    Hybrid Data Ecosystem 35© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 36.
    Hybrid Data Ecosystem2012: Creating a New Nucleus 36 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 37.
    Hybrid Data Ecosystem2013: Distribution of Platforms One Platform, 28.2% Two Platforms, 32.1% Three Platforms, 27.8% Four Platforms, 4.3% Five Platforms, 3.5% Six Platforms, 1.5% Seven Platforms, 0.4% Eight Platforms, 2.3% Sample Size = 259 37 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 38.
    Hybrid Data Ecosystem2013: From Requirements to Consumers
  • 39.
    Question and Answer 39© 2013 Enterprise Management Associates, Inc. & 9sight Consulting
  • 40.
    Question and Answer: PleaseLog Questions in the Q&A Panel Get the Big Data Operationalizing the Buzz Research Report at https://www.enterprisemanagement. com/research Available October 21, 2013! 40 © 2013 Enterprise Management Associates, Inc. & 9sight Consulting