1   © 2011 Forrester Research, Inc. Reproduction Prohibited
Forrester’s View On Big Data-Trends in the Enterprise    Big data will disrupt the data management and analytics landscape...
Forrester’s View of Big Data3   3   © 2010 Forrester Research, Inc. Reproduction Prohibited         © 2011 Forrester Resea...
Big Data makes extreme scale economicalOur View of Big Data:The techniques and technologies that make capturingvalue from ...
Dimensions of Extreme Scale5   © 2011 Forrester Research, Inc. Reproduction Prohibited
The Big Data has not cross the chasm                                        • Social Sensitives       • Big Buckers       ...
What Does the Customer Look Like?7   7   © 2010 Forrester Research, Inc. Reproduction Prohibited         © 2011 Forrester ...
The Big Data customer has multiple personalitydisorderFirms and excited, confused and skeptical all at the same time… Fir...
They want more data, but aren’t sure how tomanage what they have                          61% believe data will           ...
Buyers really want this future…                         Current                                              Future       ...
Forrester’s initial assessment verticals                                                                                  ...
Various types of data are addressed with marketing    and operations being top                                   “What ent...
A ¾ are actually doing something with big data or at       least evaluating it.                                         “W...
What Does the Data Look Like?14 14   © 2010 Forrester Research, Inc. Reproduction Prohibited         © 2011 Forrester Rese...
Firms Are Storing Many Types Of Data “Overall, including the data/information on networked and direct-attached disk storag...
Data Is Growing Faster Than Firms Can Deal With      “For each data/information type, what do you expect will be the growt...
Buyers are interested in big data for a number of reasons     “What are the main business requirements or inadequacies of ...
Most Current Production Implementations Use Existing      Transactional Data Sources For Big Data Analysis        “What ty...
BI, ERP and CRM applications are the biggest      recipients of big data insights                                     “Do ...
A significant number of Forrester clients are still using      commercial technology for Big Data                         ...
What are the Technology Concerns with                     Big Data?21 21   © 2010 Forrester Research, Inc. Reproduction Pr...
Customer challenges for securing Big Data                                                                 • Customers are ...
Storage Efficiency Challenges for Big Data                                                                • Challenge: In ...
A Conservative Perspective of Storage $ Increase                   How do you expect your firm’s spending on the following...
Commodity vs Specialized Hardware: Pros and Cons                                                                • Opportun...
Potential future state architecture: ANY DATA                                                    Analytics Tools          ...
Moving Ahead27 27   © 2010 Forrester Research, Inc. Reproduction Prohibited         © 2011 Forrester Research, Inc. Reprod...
On-Premise vs Cloud: Pros and Cons                                                                • Challenge: Capex, spac...
The market is rapidly evolving to fill gaps      Other file storage systems that capture some Hadoop       market share. ...
Gaps in the Big Data market mean opportunities      Intelligent Management         – Storage efficiency         – Securit...
How Does This Role Evolve?  Data ownership is everyone’s responsibility  Important to bring together a team within IT as...
Questions, comments, discussion? 32   © 2011 Forrester Research, Inc. Reproduction Prohibited
Thank youVanessa Alvarez+1 617.613.6259valvarez@forrester.comTwitter: @vanessaalvarez1Blog: blog.forrester.comwww.forreste...
Big Data Is Disruptive Not Incremental Big data is not incremental solutions to old problems where data has grown bigger ...
…but the “why” is about speed to value35   © 2011 Forrester Research, Inc. Reproduction Prohibited
Security considerations for Big Data are nascent Current customer focus:     – What is Big Data, what can we do with it? ...
Key Pieces to the Big Data Puzzle  The role of SSD       – Prices coming down       – Commoditization of hardware happens...
Upcoming SlideShare
Loading in …5
×

Hadoop World 2011: Hadoop Trends & Predictions - Vanessa Alverez, Forrester

3,727
-1

Published on

Hadoop is making its way into the enterprise, as organizations look to extract valuable information and intelligence from the mountains of data in their storage environments. The way in which this data is analyzed and stored is changing, and Hadoop has become a critical part of this transformation. In this session, Vanessa will cover the trends we are seeing in the enterprise in regards to Hadoop adoption and how it’s being used, as well as predictions on where we see Hadoop and Big Data in general, going as we enter 2012.

Published in: Technology, Business
2 Comments
24 Likes
Statistics
Notes
No Downloads
Views
Total Views
3,727
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
2
Likes
24
Embeds 0
No embeds

No notes for slide

Hadoop World 2011: Hadoop Trends & Predictions - Vanessa Alverez, Forrester

  1. 1. 1 © 2011 Forrester Research, Inc. Reproduction Prohibited
  2. 2. Forrester’s View On Big Data-Trends in the Enterprise Big data will disrupt the data management and analytics landscape Vanessa Alvarez, Analyst November 8, 20112 2 © 2010 Forrester Research, Inc. Reproduction Prohibited 2009 © 2011 Forrester Research, Inc. Reproduction Prohibited2
  3. 3. Forrester’s View of Big Data3 3 © 2010 Forrester Research, Inc. Reproduction Prohibited © 2011 Forrester Research, Inc. Reproduction Prohibited3
  4. 4. Big Data makes extreme scale economicalOur View of Big Data:The techniques and technologies that make capturingvalue from data at extreme scale economical Big Data technology: Handles data at extreme scale. Can be characterized by: • Massive parallel computing to divide and conquer huge workloads. • Extremely flexible to allow unlimited data manipulation and transformation. • Massively scalable in terms of both technology and cost.4 © 2011 Forrester Research, Inc. Reproduction Prohibited
  5. 5. Dimensions of Extreme Scale5 © 2011 Forrester Research, Inc. Reproduction Prohibited
  6. 6. The Big Data has not cross the chasm • Social Sensitives • Big Buckers • Web 2.0s • Survivors • One footers • Data crunchers • The Awakened • The Hadoop Crowd • Proprietary Plays • The “I do tooers” • The “I guess we betters” • The NoSQL Crowd • The biggest & fastest Source: Forrester Research and http://en.wikipedia.org/wiki/Crossing_the_Chasm_(book)6 © 2011 Forrester Research, Inc. Reproduction Prohibited
  7. 7. What Does the Customer Look Like?7 7 © 2010 Forrester Research, Inc. Reproduction Prohibited © 2011 Forrester Research, Inc. Reproduction Prohibited7
  8. 8. The Big Data customer has multiple personalitydisorderFirms and excited, confused and skeptical all at the same time… Firms want more but don’t now what to do with what they have We sent our survey to thousands of clients – only got 60 completes The high number of partial completes indicates confusion Firms are experimenting with data they already have …but planning for new types of data They plan on keeping their data a long time8 © 2011 Forrester Research, Inc. Reproduction Prohibited
  9. 9. They want more data, but aren’t sure how tomanage what they have 61% believe data will 56% are concerned with change how business the ability to manage operates. current data. 330 303 Source: “Global Survey: The Business Impact of Big Data,” Avanade, November 20109 © 2011 Forrester Research, Inc. Reproduction Prohibited
  10. 10. Buyers really want this future… Current Future “No data is discarded anymore! U.S. xPress leverages a large scale of transaction data and a diversity of interaction data, now extended to perform big data processing like “Our oil rigs generate about 25,000 Hadoop …We assess driver data points per second and we only performance with image files and pick use about 5% of that information.” up customer behaviors from texts by customer service reps. U.S. xPress — super major energy saved millions of dollars per year by company executive …augmenting our enterprise data with sensor, meter, RFID tags, and geospatial data.” — CTO, energy company10 © 2011 Forrester Research, Inc. Reproduction Prohibited
  11. 11. Forrester’s initial assessment verticals ent nce om le ure ta inm e sa ura c ele g Ins hol or urin r & L , Ent e &T ect &W e& act es S s eis anc litie v ic blic nuf dia tail Ser Fin Me Ma Ut i Data Pu Re Marketing Operations Sales Risk Management IT Analytics Finance Product Development Customer Service Logistics Innovators Early Adopters Early Majority Late Majority & LaggardsSource: Forrester research – this data represents our preliminary qualitative assessment based interviews and inquiries 11 © 2011 Forrester Research, Inc. Reproduction Prohibited
  12. 12. Various types of data are addressed with marketing and operations being top “What enterprise areas does your Big Data initiative address?” Marketing 45% Operations 43% Sales 38% Risk management 35% IT analytics 33% Product development 32% Finance 32% Customer service 30% Logistics 22% Other 12% HR 12% Brand management 8% Base: 60 IT Professionals who are Forrester clientsSource: How Forrester Clients Are Using Big Data, September 2011 (Upcoming) 12 © 2011 Forrester Research, Inc. Reproduction Prohibited
  13. 13. A ¾ are actually doing something with big data or at least evaluating it. “What is the status of your Big Data initiative?” Other Dont know Testing 2% 2% 2% Piloting 18% Evaluating In production 53% 23% Base: 60 IT Professionals who are Forrester clientsSource: How Forrester Clients Are Using Big Data, September 2011 (Upcoming) 13 © 2011 Forrester Research, Inc. Reproduction Prohibited
  14. 14. What Does the Data Look Like?14 14 © 2010 Forrester Research, Inc. Reproduction Prohibited © 2011 Forrester Research, Inc. Reproduction Prohibited 1
  15. 15. Firms Are Storing Many Types Of Data “Overall, including the data/information on networked and direct-attached disk storage that you know about, how much data/information does your firm currently maintain? “ 0 to 100GB 100GB to 999GB 1TB to 9 TB 10TB to 99TB 100TB to 9PB 10PB or more 1% Overall — company wide 5% 22% 33% 29% 6% Database systems 13% 23% 27% 18% 12% 2% General file storage, including file servers 5% 20% 34% 21% 13% 2% Server backups for disaster recovery and continuity 10% 19% 31% 21% 12% 2% Archiving, all forms including compliance, discovery, and 14% 19% 29% 18% 11% 2% infrequently accessed dataDigital and Web content repositories, such as video, audio, 29% 22% 25% 10% 6% 1% images, and Web pages PC backups for disaster recovery and continuity 41% 20% 17% 10%4% Enterprise content management 31% 20% 21% 11% 5% 1% Other data 35% 18% 17% 8%4% 1% Base: 1,252 North American and European IT executives and technology decision-makers Source: Forrester’s Forrsights Hardware Survey, Q3 2010 15 © 2011 Forrester Research, Inc. Reproduction Prohibited
  16. 16. Data Is Growing Faster Than Firms Can Deal With “For each data/information type, what do you expect will be the growth at your company over the next 12 months?” More than 100% 50% to 100% 26% to 50% 11% to 25% Grow 1% to 10% No growth 2% Overall — company wide 7% 16% 42% 31% 2% 1% Database systems 4% 11% 34% 44% 4% 1% General file storage, including file servers 3% 10% 33% 46% 4% 2% Archiving, all forms including compliance, discovery, and 4% 11% 28% 44% 9% infrequently accessed data 1% Server backups for disaster recovery and continuity 3% 11% 32% 40% 10% 1%Digital and Web content repositories, such as video, audio, 3% 10% 24% 44% 15% images, and Web pages 1% Enterprise content management 3% 9% 19% 50% 15% 1% Other data 4% 14% 51% 23% 1% PC backups for disaster recovery and continuity 5% 18% 44% 26% Base: 1,252 North American and European IT executives and technology decision-makers Source: Forrester’s Forrsights Hardware Survey, Q3 2010 16 © 2011 Forrester Research, Inc. Reproduction Prohibited
  17. 17. Buyers are interested in big data for a number of reasons “What are the main business requirements or inadequacies of earlier-generation BI/DW/ET technologies, applications, and architecture that are causing you to consider or implement Big Data?” In traditional BI and DW applications, requirements come first, applications come later. In other worlds requirements drive applications. Big Data turns this model upside down, where free form exploration using Big Data technology to prove a certain hypothesis or to find a pattern, often results in specifications for a more traditional BI/DW application Data volume 75% Analysis-driven requirements (Big Data) vs. 58% requirements driven analysis (traditional … Data diversity, variety 52% Velocity of change and scope/requirements 38% unpredictability Cost. Big Data solutions are less expensive 30% than traditional ETL/DW/BI solutions Other 10% Dont know 3% Cost is also a factor, in many cases, dealing with data using big data technologies is simpliy cheaper and faster than other methods. Base: 60 IT Professionals Source: How Forrester Clients Are Using Big Data, September 2011 (Upcoming)17 © 2011 Forrester Research, Inc. Reproduction Prohibited
  18. 18. Most Current Production Implementations Use Existing Transactional Data Sources For Big Data Analysis “What types of data/records are you planning to analyze using Big Data technologies?” Most big data use cases hype its application for analysis of new, raw data from social media, sensors and web traffic, but we found that firms are being very practical, with early adopters using it for operating on enterprise data they already have. Transactional data from enterprise applications 72% Sensor/machine/device data 42% Unstructured content from email, office… 35% Social media (Facebook, Twitter, etc) 35% Locational/geospatial data 27% Clickstream 27% Image (large video/photographic) data 13% Scientific/genomic data 12% Other 7% Dont know 5% Base: 60 IT Professionals who are Forrester clientsSource: How Forrester Clients Are Using Big Data, September 2011 (Upcoming) 18 © 2011 Forrester Research, Inc. Reproduction Prohibited
  19. 19. BI, ERP and CRM applications are the biggest recipients of big data insights “Do your Big Data applications stand on their own?” Business intelligence (BI), analytics 55% Enterprise applications (ERP, CRM) 28% Dont know 17% Lower responses for BPM and BRE Business rules (BRE) 17% indicate that automation of processes using Business processes (BPM) 17% insights from big data is not as common as it might eventually be. Other 12% (multiple responses accepted) Base: 60 IT Professionals who are Forrester clientsSource: How Forrester Clients Are Using Big Data, September 2011 (Upcoming) 19 © 2011 Forrester Research, Inc. Reproduction Prohibited
  20. 20. A significant number of Forrester clients are still using commercial technology for Big Data Commercial source Big Data tools 47% Open source Big Data technology (Hadoop, MapReduce, Cassandra, and the other Apache 37% open source specs) Dont know 23% Other 17% (multiple responses accepted) Base: 60 IT Professionals who are Forrester clientsSource: How Forrester Clients Are Using Big Data, September 2011 (Upcoming) 20 © 2011 Forrester Research, Inc. Reproduction Prohibited
  21. 21. What are the Technology Concerns with Big Data?21 21 © 2010 Forrester Research, Inc. Reproduction Prohibited © 2011 Forrester Research, Inc. Reproduction Prohibited 2
  22. 22. Customer challenges for securing Big Data • Customers are not actively talking about Awareness and security concerns. understanding are • Customers need help understanding threats lacking in a Big Data environment • Main considerations: Synchronizing retention Country policies and disposition policies across jurisdictions, moving data across countries. and laws add complexity • Customers need help navigating frameworks and changes. Key takeaway Customers are in need of education and guidance. 22 © 2011 Forrester Research, Inc. Reproduction Prohibited
  23. 23. Storage Efficiency Challenges for Big Data • Challenge: In most instances, data is random and inconsistent, not duplicated • Opportunity: There is a need for more De-duplication intelligent identification of data • Challenge: Compression normally happens instead of de-duplication, yet, will compress duplicate data regardless Compression • Opportunity: There is a need for an automated manner in doing both de- duplicating, and then compressing 23 © 2011 Forrester Research, Inc. Reproduction Prohibited
  24. 24. A Conservative Perspective of Storage $ Increase How do you expect your firm’s spending on the following IT infrastructure expenses to change over the next 12 months? Stay about the same Increase 5%-10% Increase more than 10% Storage technologies 36% 40% 15% IT infrastructure/operations staff salaries 52% 34% 4% Systems management 59% 25% 4% Data center and IT facilities 49% 30% 9% Servers and server operating systems 43% 35% 9% Infrastructure systems integration and consulting services 57% 21% 6% PCs and PC operating systems 48% 29% 6% Infrastructure outsourcing and managed services (based 57% 18% 5% on fixed-price, long-term contracts)Pay-per-use online hosted infrastructure services, such as 54% 16% 5% cloud Base: 1725 North American and European IT executives and technology decision-makers Source: Forrester’s Forrsights Hardware Survey, Q3 201024 © 2011 Forrester Research, Inc. Reproduction Prohibited
  25. 25. Commodity vs Specialized Hardware: Pros and Cons • Opportunity: Cost efficient, scalable, open Commodity • Challenge: reliability, availability • Opportunity: performance, scalable Specialized • Challenge: Cost, proprietary Source: Forrester Research 25 © 2011 Forrester Research, Inc. Reproduction Prohibited
  26. 26. Potential future state architecture: ANY DATA Analytics Tools Hadoop •Compression •De-duplication •Encryption Intelligent Management •Provisioning •Virtualization DAS SAN NAS Network Network Network Source: Forrester Research26 © 2011 Forrester Research, Inc. Reproduction Prohibited
  27. 27. Moving Ahead27 27 © 2010 Forrester Research, Inc. Reproduction Prohibited © 2011 Forrester Research, Inc. Reproduction Prohibited 2
  28. 28. On-Premise vs Cloud: Pros and Cons • Challenge: Capex, space On-Premise • Opportunity: Security, ownership • Challenge: Security, manageability Cloud • Opportunity: Cost efficient, scalable, opex Source: Forrester Research 28 © 2011 Forrester Research, Inc. Reproduction Prohibited
  29. 29. The market is rapidly evolving to fill gaps  Other file storage systems that capture some Hadoop market share.  Other computational frameworks, beyond MapReduce that improve operational analytics.  Advanced streaming and transactional processing that further support real-time.  Business-friendly features that empower your workforce.  Storage efficiency and security issues will be solved by somebody…(?)29 © 2011 Forrester Research, Inc. Reproduction Prohibited
  30. 30. Gaps in the Big Data market mean opportunities  Intelligent Management – Storage efficiency – Security  Interactive analytics – – Hadoop is batch only at present – More real time analytics  Analytics, Big Data as a Service – Leveraging cloud30 © 2011 Forrester Research, Inc. Reproduction Prohibited
  31. 31. How Does This Role Evolve?  Data ownership is everyone’s responsibility  Important to bring together a team within IT as well as a business analyst  Understand your peers in applications and database  Train or hire a data analyst that can handle data lifecycle management31 © 2011 Forrester Research, Inc. Reproduction Prohibited
  32. 32. Questions, comments, discussion? 32 © 2011 Forrester Research, Inc. Reproduction Prohibited
  33. 33. Thank youVanessa Alvarez+1 617.613.6259valvarez@forrester.comTwitter: @vanessaalvarez1Blog: blog.forrester.comwww.forrester.com © 2009 Forrester Research, Inc. Reproduction Prohibited
  34. 34. Big Data Is Disruptive Not Incremental Big data is not incremental solutions to old problems where data has grown bigger  Big data is – New techniques and technologies – To handle that we do not deal with today – Using scalable and parallel technologies – That trade off consistency for high availability and node failures.  Forrester definition excludes some solutions that other might call “big data”. Examples: – Massively parallel data warehouses – In-memory databases34 © 2011 Forrester Research, Inc. Reproduction Prohibited
  35. 35. …but the “why” is about speed to value35 © 2011 Forrester Research, Inc. Reproduction Prohibited
  36. 36. Security considerations for Big Data are nascent Current customer focus: – What is Big Data, what can we do with it? – Concerns over stability, scale, cost, availability, unified management – Not security (not yet) Current storage and security vendor focus: – What do we need to know about Big Data? – What can be done from a security perspective? – Where do we fit? Another type of discussion emerging: – Big Data for security – Example – Zettaset Security Data Warehouse, using Big Data to identify security threats36 © 2011 Forrester Research, Inc. Reproduction Prohibited
  37. 37. Key Pieces to the Big Data Puzzle  The role of SSD – Prices coming down – Commoditization of hardware happens more at the SAN, NAS than SSD – Analytics require speed and performance, in order to achieve near real-time  Storage Virtualization – Just like servers were virtualized, so will storage be, and a “storage hypervisor” will determine where storage capacity will come from, whether it be SAN, NAS or SSD (DAS) – Enterprises are looking to still leverage their existing infrastructure, instead of ripping and replacing with “commodity infrastructure”37 © 2011 Forrester Research, Inc. Reproduction Prohibited

×