2014 Big Data Research by IDG Enterprise


Published on

The 2014 IDG Enterprise Big Data research was completed with the goal of gaining a better understanding of organizations’ big data initiatives, investments and strategies. Additional information on the research can be found here: http://bit.ly/1cW0wWR.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Processor or Virtual Storage· 1 Bit = Binary Digit· 8 Bits = 1 Byte· 1024 Bytes = 1 Kilobyte · 1024 Kilobytes = 1 Megabyte · 1024 Megabytes = 1 Gigabyte · 1024 Gigabytes = 1 Terabyte · 1024 Terabytes = 1 Petabyte · 1024 Petabytes = 1 Exabyte· 1024 Exabytes = 1 Zettabyte · 1024 Zettabytes = 1 Yottabyte · 1024 Yottabytes = 1 Brontobyte· 1024 Brontobytes = 1 Geopbyte Disk Storage· 1 Bit = Binary Digit· 8 Bits = 1 Byte· 1000 Bytes = 1 Kilobyte · 1000 Kilobytes = 1 Megabyte · 1000 Megabytes = 1 Gigabyte · 1000 Gigabytes = 1 Terabyte · 1000 Terabytes = 1 Petabyte · 1000 Petabytes = 1 Exabyte· 1000 Exabytes = 1 Zettabyte · 1000 Zettabytes = 1 Yottabyte · 1000 Yottabytes = 1 Brontobyte· 1000 Brontobytes = 1 Geopbyte   This is based on the IBM Dictionary of computing method to describe disk storage - the simplest. Bit: A Bit is the smallest unit of data that a computer uses. It can be used to represent two states of information, such as Yes or No.Byte: A Byte is equal to 8 Bits. A Byte can represent 256 states of information, for example, numbers or a combination of numbers and letters. 1 Byte could be equal to one character. 10 Bytes could be equal to a word. 100 Bytes would equal an average sentence.Kilobyte: A Kilobyte is approximately 1,000 Bytes, actually 1,024 Bytes depending on which definition is used. 1 Kilobyte would be equal to this paragraph you are reading, whereas 100 Kilobytes would equal an entire page.Megabyte: A Megabyte is approximately 1,000 Kilobytes. In the early days of computing, a Megabyte was considered to be a large amount of data. These days with a 500 Gigabyte hard drive on a computer being common, a Megabyte doesn't seem like much anymore. One of those old 3-1/2 inch floppy disks can hold 1.44 Megabytes or the equivalent of a small book. 100 Megabytes might hold a couple volumes of Encyclopedias. 600 Megabytes is about the amount of data that will fit on a CD-ROM disk.Gigabyte: A Gigabyte is approximately 1,000 Megabytes. A Gigabyte is still a very common term used these days when referring to disk space or drive storage. 1 Gigabyte of data is almost twice the amount of data that a CD-ROM can hold. But it's about one thousand times the capacity of a 3-1/2 floppy disk. 1 Gigabyte could hold the contents of about 10 yards of books on a shelf. 100 Gigabytes could hold the entire library floor of academic journals.Terabyte: A Terabyte is approximately one trillion bytes, or 1,000 Gigabytes. There was a time that I never thought I would see a 1 Terabyte hard drive, now one and two terabyte drives are the normal specs for many new computers.  To put it in some perspective, a Terabyte could hold about 3.6 million 300 Kilobyte images or maybe about 300 hours of good quality video. A Terabyte could hold 1,000 copies of the Encyclopedia Britannica. Ten Terabytes could hold the printed collection of the Library of Congress. That's a lot of data.Petabyte: A Petabyte is approximately 1,000 Terabytes or one million Gigabytes. It's hard to visualize what a Petabyte could hold. 1 Petabyte could hold approximately 20 million 4-door filing cabinets full of text. It could hold 500 billion pages of standard printed text. It would take about 500 million floppy disks to store the same amount of data.Exabyte: An Exabyte is approximately 1,000 Petabytes. Another way to look at it is that an Exabyte is approximately one quintillion bytes or one billion Gigabytes. There is not much to compare an Exabyte to. It has been said that 5 Exabytes would be equal to all of the words ever spoken by mankind.Zettabyte: A Zettabyte is approximately 1,000 Exabytes. There is nothing to compare a Zettabyte to but to say that it would take a whole lot of ones and zeroes to fill it up.Yottabyte: A Yottabyte is approximately 1,000 Zettabytes. It would take approximately 11 trillion years to download a Yottabyte file from the Internet using high-power broadband. You can compare it to the World Wide Web as the entire Internet almost takes up about a Yottabyte.Brontobyte: A Brontobyte is (you guessed it) approximately 1,000 Yottabytes. The only thing there is to say about a Brontobyte is that it is a 1 followed by 27 zeroes!Geopbyte: A Geopbyte is about 1000 Brontobytes! Not sure why this term was created. I'm doubting that anyone alive today will ever see a Geopbyte hard drive. One way of looking at a geopbyte is 15267 6504600 2283229 4012496 7031205 376 bytes!
  • Question: All_10 Given the rate of data growth at most companies, findings such as 65 percent of respondents feeling overwhelmed by incoming data makes sense. On average, respondents currently manage about 164 terabytes of data, although the number varies considerably by size--larger companies report an average of 291 terabytes, while smaller companies average about 57 terabytes. However, respondents expect that average to reach nearly 290 terabytes of data within the next year to 18 months, and 31 percent of larger companies--companies with $1 billion or more in annual revenues--expect to manage more than one petabyte of data. In 2012, respondents indicated an increase from194 terabytes managed to 297 terabytes by the end of 2013. The decrease in difference in the amount of data currently and expected to manage from 2012 to 2013 signifies overestimated expectations moving to a more realistic view.
  • Question: All_1 As game-changing technology trends such as mobile device usage, social media and sensor-derived data further feed the explosive growth of data, companies are intensifying their efforts to derive economic value through big data initiatives. Interest in big data continues to rise, as nearly half (49 percent) of respondents are implementing or plan to implement big data projects within their organizations in the future -- a 5 percentage point increase from 2012.
  • Question: CIO1Further underscoring big data’s compelling business case and the need for IT/business partnership, organizationssurveyed indicated that their big data efforts are being supported and sponsored at the CEO (47 percent) or Line of Business level (34 percent).
  • Question: All_20
  • Question: All_9Companies are concentrating on building out internal infrastructure and building up their analytics expertise. When it comes to technology, 38 percent in developing or buying software, 37 percent are investing in server hardware, and 34 percent are hiring staff with analytics skills. Not sure 11%
  • Question: All_6Unsurprisingly, smaller companies (49 percent) were more likely to run into fiscal constraints than enterprise-size companies (40 percent.) However, while last year’s survey respondents cited legacy issues such as integration of existing tools and security issues as top challenges, those hurdles have been supplanted in 2013 by the much-vaunted scarcity of skilled data scientists. In fact, respondents cited the skill shortage as part of the top challenges to their big data projects, an issue that remained consistent regardless of company size.With graduates of top collegiate math programs deluged with job offers, many companies find themselves scrambling to find skilled employees capable of managing and analyzing big data.  Other <1,000: 2 percent1,000+: 1 percentNone <1,000: 2 percent1,000+: 1 percentNot sure <1,000: 2 percent1,000+: 4 percent
  • Question: All_16Organizations are planning staff increases to support Big Data initiatives. Only 18 percent of respondents currently employ a data scientist, but about a quarter more plan to hire for the position in the next 12-18 months.
  • 2014 Big Data Research by IDG Enterprise

    1. 1. Big Data Growing Trends and Emerging Opportunities Conducted across the IDG Enterprise brands: CIO, Computerworld, CSO, InfoWorld, ITworld and Network World
    2. 2. Purpose and Methodology Survey Sample Survey Method Total Respondents 751 Collection Online Questionnaire Margin of Error +/- 3.6% Audience Base CIO, Computerworld, CSO, InfoWorld, ITworld, and Network World sites, email, and LinkedIn forums. Number of Questions 46 (incl. demographics) Survey Goal To gain a better understanding of organizations' big data initiatives, investments and strategies. Source: IDG Enterprise Big Data Study, 2014 1
    3. 3. Big Data Definition Large volumes of a wide variety of data collected from various sources across the enterprise including transactional data from enterprise applications/databases, social media data, mobile device data, unstructured data/documents, machine-generated data and more. Big data technologies are a new generation of technologies and architectures designed to extract value economically from these data sets by enabling high-velocity capture, discovery and/or analysis. Data in high volume, high velocity and a high variety of information assets can deliver enhanced insights and decision making. Source: IDG Enterprise Big Data Study, 2014 2
    4. 4. Data Terms UNIT SIZE WHAT IT MEANS Bit (b) 1 or 0 Short for “binary digit,” after the binary code (1 or 0) computers use to store and process data. Byte (B) 8 bits Enough information to create an English letter or number in computer code. It is the basic unit of computing. Kilobyte (KB) 1,000 or 210 bytes From “thousand” in Greek. One page of typed text is 2KB. Megabyte (MB) 1,000KB; 220 bytes From “large” in Greek. The complete works of Shakespeare total 5MB. A typical pop song is about 4MB. Gigabyte (GB) 1,000MB; 230 bytes From “giant” in Greek. A two-hour film can be compressed into 1-2GB. Terabyte (TB) 1,000GB; 240 bytes From “monster” in Greek. All the catalogued books in America’s Library of Congress total 15TB. Petabyte (PB) 1,000TB; 250 bytes All letters delivered by America’s postal service this year will amount to around 5PB. Google processes around 1PB every hour. Exabyte (EB) 1,000PB; 260 bytes Equivalent to 10 billion copies of The Economist. Zettabyte (ZB) 1,000EB; 270 bytes Approximately 1,000 Exabytes. Data in 2010 cracked the zettabyte barrier. Yottabyte (YB) 1,000ZB; 280 bytes Currently too big to imagine. Source: The Economist 3 The prefixes are set by an intergovernmental group, the International Bureau of weights and Measures. Yotta and Zetta were added in 1991; terms for larger amounts have yet to be established.
    5. 5. Amount of Data Climbing Sharply Q. Approximately, how much data is your organization currently managing? AND Q. How much data do you expect to be managing 12-18 months from now? BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 4
    6. 6. Interest in Big Data Continues to Rise 39% 15% 12% 14% 10% 10% eady deployed/implemented big datain the process… We have plans likely to implement big data projects inhavefuture but to deployspecific timeframedata projects We are projects considering deploying to… We are We are or implementing big data projectsWe the no plans have no or implement big in place within the next 13-24 months Q. Is your company currently implementing, planning or considering projects (i.e. devising strategies and projects to generate more value from existing data)? Source: IDG Enterprise Big Data Study, 2014 5
    7. 7. CEO Largest Supporter of Big Data Efforts CEO 47% 34% Line of Business (as head) 29% Board of Directors 27% Marketing 24% Finance/Accounting 21% Strategy 20% Engineering 19% Sales 18% Research & Development 14% Customer Service 10% None (this is an IT-only initiative) Not sure 4% Q. Which of the following business groups are supporting or sponsoring your big data efforts? BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 6
    8. 8. Exec. IT Heavily Involved in Big Data Initiatives 36% Capturing 23% 18% 21% 38% Storing 24% 32% 10% 39% Searching/ Sorting 28% 25% 21% 31% Sharing 21% 14% 20% 48% Analyzing 34% 34% 45% 33% Visualizing 22% 17% 21% IT Exec Mid-Level IT IT Pro LOB Q What role do you play personally in your organization’s big data initiatives? I am involved in… BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 7
    9. 9. Laying the Groundwork for Big Data Developing or buying software applications 38% Investing in additional server or storage hardware/software 37% 34% Hiring staff with analytics skills Utilizing an open source software framework (e.g., Apache Hadoop) Migrating to cloud-based services for storage Increasing network bandwidth 30% 28% 27% Migrating to cloud-based services for analytics 26% Contracting outside big data expertise 26% Q. Which of the following is your organization doing to manage big data initiatives? BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 8
    10. 10. Need for Skilled Employees Ranks High 40% 41% Limited availability of skilled employees to analyze data 35% Limited availability of skilled employees to manage big data Limited budget 40% 30% Legacy issues (integration of exisiting tools) 36% 31% 33% 29% 32% 39% 31% 28% 30% 27% 29% 27% 26% Poor data quality Difficulty integrating or analyzing real-time data Development time Growing demand on storage capacity/infrastructure Scalability issues Ability to demonstrate ROI from investments 18% Ownership/control issues 23% 25% 22% 24% 21% 21% 21% Access and usability for end users Ensuring adequate bandwidth to support data capture and analysis Difficulty uncovering actionable insights/inadequate reporting 14% Escalating infrastructure and maintence costs due to data growth Lack of management support 49% 34% 37% Security issues Compliance/regulatory issues 41% 13% 21% 22% 19% 17% <1,000 1,000+ Q. What are the challenges that your organization is facing or expects to experience with its big data initiatives? BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 9
    11. 11. Staffing Up for Big Data Initiatives Q. With regard to big data initiatives, what skill sets does your organization currently have? AND Q. Which skill sets is your organization planning to hire within the next 12-18 months? BASE: Plans to deploy/implement big data projects. Source: IDG Enterprise Big Data Study, 2014 10
    12. 12. Conclusion • Organizations are seeing exponential growth in the amount of data managed with an expected increase of 76% within the next 12-18 months. • Companies are intensifying their efforts to derive value through big data initiatives with nearly half (49%) of respondents already implementing big data projects or in the process of doing so in the future; however, enterprise organizations are ahead of the curve in implementation plans compared to SMB organizations. • CEOs are focused on the value of big data and are partnering with IT executives who will purchase/manage/execute on the strategies. • Organizations are investing in developing or buying software applications, additional sever hardware, and hiring staff with analytics skills in preparation for big data initiatives. • Organizations are facing numerous challenges with big data initiatives and limited availability of skilled employees to analyze and manage data tops the list. • In the next 12-18 months, organizations plan to invest in skill sets necessary for big data deployments, including data scientists (27%), data architects (24%), data analysts (24%), data visualizers (23%), research analysts (21%), and business analysts (21%). Source: IDG Enterprise Big Data Study, 2014 11
    13. 13. Learn More • To view the full results, please contact Sue Yanovitch, VP, Marketing of IDG Enterprise at syanovitch@idgenterprise.com. 12