This document summarizes the results of a survey of 298 data management professionals about big data challenges and opportunities. It finds that big data is now present in all organizations, with 11% managing over a petabyte and another 20% having hundreds of terabytes. While bigger companies are dealing with more data currently, most companies will soon have petabyte-sized stores. However, the business does not fully understand big data's potential value yet. Capitalizing on big data does not require replacing existing infrastructure but integrating it. The survey also finds that relational databases and Hadoop are both important technologies for managing big data now and in the future.

2. 2
Big Data,
Big Challenges,
Big Opportunities
Joe McKendrick
Lead Analyst

3. 3
Survey on Big Data
• 298 data management and IT
managers/professionals
• Members of Independent Oracle Users Group
(IOUG); 98% run Oracle Databases
• Large organizations (>10,000 employees) 22%;
small firms (1-500 employees) 16%
• Major industries represented: manufacturing;
government/education/non-profit;
utility/telecommunications/transportation;
retail/wholesale; high-tech

4. 4
Observations
• Big Data is here now, and is flowing through all
organizations.
• With all this Big Data now on the scene, more
needs to be done to educate the business about
the potential of Big Data.
• Capitalizing on Big Data doesn’t mean making
huge financial investments or tearing down your
current infrastructure; rather, it can be
integrated and incorporated into your existing
assets.

5. 5
1.
Big Data is here now,
and is flowing through
all organizations.

6. 6
Total Amount of Data Managed Today
11% of organizations now
manage more than a petabyte of
data ...
...another 20% have data in the
hundreds of terabytes.

7. 7
It’s Mainly Larger Organizations, But...
28% of the largest
organizations have >1PB
8% of medium-size businesses
have >1PB
... Soon, most businesses of
all sizes will have data stores in
the PBs.

8. 8
Many types of data: transactional,
user-generated, machine generated

9. 9
Where It’s Coming From, Right Now:

10. 10
The Problem...
SILOS,
SILOS,
SILOS

11. 11
Growing Amounts of Unstructured Data

12. 12
Industries With the Most Unstructured Data

13. 13
2.
With Big Data now
on the scene, more needs
to be done to educate the business
about its potential.

14. 14
Barriers: Business Doesn’t Understand the Value
Yet—Thus, Budgets are Falling Short ...

15. 15
Most Data Executives Do Not Feel Their
Data Infrastructure Is or Will Be Capable
72% of survey respondents are not completely
confident in their IT infrastructure and their
database systems for managing Big Data now ...
81% are not completely confident in their IT
infrastructure and their database systems for
managing Big Data in three years.

16. 16
Where Confidence in Data Infrastructure
is Lowest

17. 17
3.
Capitalizing on Big Data
doesn’t mean making huge
financial investments or
tearing down your current
infrastructure—it can be
integrated into your
existing assets.

18. 18
Big data is a
“natural resource”—
and it’s unlimited!

19. 19
There is Business Value in Big Data
55% of survey respondents
acknowledge that Big Data is
either “extremely” or “very”
important to their
business.

20. 20
Industries Where Big Data Really Matters
“Speed and accuracy are of the essence in
winning new business and maintaining current
customers.”

21. 21
Survey Respondents
Say Big Data Helps Them:
Just a few
other areas of
Big Data value:
customer
profitability,
text analytics,
e-commerce,
risk
management!

22. 22
Different Industries, Different Motivations

23. 23
“As the big data applications begin to
come on line, priorities within the
security/compliance group will become
more risk oriented. This focus will
allow the business to focus resources
toward those items that pose the
highest degree of risk to data.
Additionally, conditions that could be
seen as possible threat vectors or the
beginnings of events can be found
easier.”

24. 24
Technology That Will Get Us There

25. 25
Big Data Foundation Being Built on
Existing, Proven Environments—
Relational Databases

26. 26
How Data is Integrated With BI Applications
32% pre-process
Big Data then load
into data warehouse for
integrated analysis, but
...
... 46% are still unsure
how this
will play out.

27. 27
Are Data Warehouses a Big Company Thing?
36% of large organizations (>10,000
employees) pre-process Big Data
then load into data warehouse for
integrated analysis.
26% of small firms (<100 employees) use
data warehouses to manage
Big Data.

28. 28
Hadoop—
especially for ad-hoc queries
and data mining

29. 29
Who Uses Hadoop?
(now/planned for this year)

30. 30
How Hadoop Is and Will Be Used

31. 31
Strive for a
“co-existence” strategy
between data systems—
not either/or.

32. 32
Managing and Staffing
Big Data Environments

33. 33
“We already have as close a
relationship with management as is
possible. We intend to keep it that
way by doing a great job on Big
Data, but we have no idea what the
percentage of the data flying past
[is]us good enough to capture.
Understanding the potential benefits
and liabilities of capturing a wide
range of data beyond traditional
transactions is an open-ended
subject.”

34. 34
Where Do Big Data Projects Originate?

35. 35
Even when business
takes the lead,
IT responsible for
implementation
In larger organizations, others also help oversee
implementations:
54% of respondents in large organizations say
BI/analytics team oversees Big Data projects.
62% of large organizations also charge Big Data
implementations to DBAs.

36. 36
Who Makes It Happen?

37. 37
Business-Side Driver of Big Data Initiatives

38. 38
Financial Decisions

39. Recommendations
• Develop a business case.
• Get business buy-in and support.
• Develop an integration strategy between
unstructured and “traditional” enterprise
data.
• Strive for a “co-existence” strategy between
data systems—not either/or.
• Develop an integrated information
management lifecycle strategy.