Big Data: A HorizonWatching Trend Report

Contact: Bill Chamberlin, HorizonWatching Community Leader January 17, 2012 Big Data A HorizonWatching Trend ReportNote: This presentation represents my thoughts and ideas….not those of my employer. - Bill Chamberlin, HorizonWatching Community Leader - whchamb@gmail.com

HorizonWatching CommunityAbout This Trend Report  Purpose: The slides provide an overview on the Big Data trend for HorizonWatching Community members  Content: Summary information about the Big Data trend is provided along with many links to additional resources. The slides are meant to be read/studied and the links are there for you to continue your learning. You may want to view the slides in slideshow mode so you can easily follow the links  Other HorizonWatching Trend Reports: Are announced/discussed on the HorizonWatching blog and slide decks are posted on the Horizonwatching channel on Slideshare  HorizonWatching Community: For more on the HorizonWatching community please see the appendix Note: This presentation represents my thoughts and ideas….not those of my employer. - Bill Chamberlin, whchamb@gmail.com2 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityBig Data – Trend Overview“Data sets whose size is beyond the ability of commonly used software tools to capture, manage, andprocess the data within a tolerable elapsed time. Big data sizes are a constantly moving target currentlyranging from a few dozen terabytes to many petabytes of data in a single data set.” – Wikipedia (Big Data)Drivers• Volume, variety, velocity, and complexity of incoming data streams• Growth of “Internet of Things” results in explosion of new data• Commoditization of inexpensive terabyte-scale storage hardware is making storage less costly ….so why not store it?• Increasingly enterprises are needing to store non-traditional and unstructured data in a way that is easily queried• Desire to integrate all the data into a single sourceChallenges• Data comes from many different sources (enterprise apps, web, search, video, mobile, social conversations and sensors)• All of this information has been getting increasingly difficult to store in traditional relational databases and even datawarehouses• Unstructured or semi-structured text is difficult to query. How does one query a table with a billion rows• Culture, skills, and business processesImplications• Emerging capabilities to process vast quantities of structured and unstructured data are bringing about changes in technologyand business landscapes• As data sets get bigger and the time allotted to their processing shrinks, look for ever more innovative technology to helporganizations glean the insights theyll need to face an increasingly data-driven future3 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityBig Data refers to how we collect, store, and manage information thatcomes into an enterprise so that it can be harvested for decision makingData Growth “The amount of information created and replicated will surpass 1.9 zettabytes in 2011, growing by almost nine times in just five years.” – IDC (link) “Worldwide information volume is growing annually at a minimum rate of 59 percent annually.” – Gartner (link)Big Data Defined “Everyday, we create 2.5 quintillion bytes of data. This data comes from everywhere: from sensors used to gather climate information, posts to social media sites, digital pictures and videos posted online, transaction records of online purchases, and from cell phone GPS signals to name a few. This data is big data.” – IBM (link) Source: MIT Sloan Management Review “While volume is a significant challenge in managing big data, business and IT leaders must focus on information volume, variety and velocity” – Gartner (link) “The bringing together of a vast amount of data from public and private sources, combined with the intuition of business and thought leaders and the speed and affordability of todays computers, is what Big Data is all about. This is the inflection point in which artificial intelligence becomes actual intelligence” – IDC (link) 4 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityThe Big Data trend presents a huge challenge (and opportunity) forInformation Management professionalsCharacteristics of Big Data IDC Webinar (registration required) “Variety – Big data extends beyond structured data, including unstructured data of all varieties: text, audio, video, click streams, log files and more. Velocity – Often time-sensitive, big data must be used as it is streaming in to the enterprise in order to maximize its value to the business. Volume – Big data comes in one size: large. Enterprises are awash with data, easily amassing terabytes and even petabytes of information.” – IBM (link) IDC Predictions 2012: A Big Year for Big DataA New Era of Information Management “Clearly, the big data revolution is fostering a powerful new type of data science. Having more comprehensive data sets at our disposal will enable more fine-grained long-tail analysis, microsegmentation, next best action, customer experience optimization, and digital marketing applications” – Forrester (link) “The ideal enterprise data warehouse has been envisaged as a centralized repository for 25 years, but the time has come for a new type of warehouse to handle "big data." This "logical data warehouse" demands radical realignment of practices and a hybrid architecture of repositories and services” – Gartner (link) 5 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityBig Data – Additional Information and Resources • BusinessWeek: CEO Guide to Hadoop • Forbes: To Manage Big Data, You Need Smarter Storage “Most organizations are ill prepared to address both the technical and • Forrester : Blog post – Big Data Technoloy management challenges posed by • IBM: Enterprise Big Data and eBook: IBM big data; as a direct result, few will Customer Success Stories be able to effectively exploit this • IBM Software: Big Data and Data trend for competitive advantage.” – Gartner (link) Management • IBM Systems: Big Data • InfoWorld: Data Explosion and “The adoption barriers that • MIT Sloan Management Review: Big organizations face are managerial Data, Analytics and the Path From Insights to and cultural rather than related to Value data and technology.” – MIT (link to • Technorati: Blog posts tagged Big Data PDF file) • Wikipedia: Big Data6 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityBig Data – IBM Resources and Links ibm.com links IBM ebook: Understanding Big Data • IBM Software: Big Data and Data Management • IBM Systems: Big Data • IBM Big data Education Social Media • Blog: Smarter Computing Big Data posts • Smarter Planet Blog Post on Big Data • Twitter: @IBMbigdata and @ibmdb2 and @ibm_bao • Facebook: IBM-Big-Data and IBM DB2 • YouTube: IBM Big Data - Designing for Data • For more, see Connect to IBM Data Management Communities 7 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

HorizonWatching CommunityAppendix: About The HorizonWatching Community  Launched in 2006  An overview of HorizonWatching is located here: About HorizonWatching  Blogs – Typepad = Horizonwatching – Thought Leadership Blog – Tumblr = HorizonWatching – Quote Blog  Twitter = HorizonWatching  Slideshare = HorizonWatching (an external version of this deck will be posted here)  Facebook = HorizonWatching  LinkedIn Community = HorizonWatching8 January 17, 2012 Big Data: A HorizonWatching Trend Report © 2012 HorizonWatching Creative Commons Attribution License

http://horizonwatching.typepad.com	1218
http://www.horizonwatching.typepad.com	150
http://translate.googleusercontent.com	6

Big Data: A HorizonWatching Trend Report

by Bill Chamberlin on Jan 25, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 1,374

Statistics

Big Data: A HorizonWatching Trend Report — Presentation Transcript