Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big Data and Hadoop - key drivers, ecosystem and use cases

1,311 views

Published on

Overview of the Big Data market.

Published in: Technology
  • Be the first to comment

Big Data and Hadoop - key drivers, ecosystem and use cases

  1. 1. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org [[The Wikibon Project]] Big Data and Hadoop: Key Drivers, Ecosystem and Use Cases November 2011
  2. 2. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org What is Big Data? 2 Big Data n Data sets whose size, type and/or speed make them impractical to process and analyze with traditional database technologies and related data management tools.
  3. 3. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Why is Big Data Important? 3 Big  Data  is  the  new  de.initive  source   of  competitive  advantage  across   industries  … …  For  those  organizations  that   embrace  Big  Data,  the  possibilities   for  innovation,  improved  agility,  and   increased  pro.itability  are  nearly   endless.
  4. 4. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Three Key Big Data Drivers 4 1.  Volume, Variety, Velocity 2.  Hardware Commoditization 3.  Cloud Computing
  5. 5. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Characteristics of Big Data 5
  6. 6. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Sources of Big Data 6
  7. 7. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop 7 Open source framework for processing, storing and analyzing Big Data. Fundamental concept: Rather than banging away at one, huge block of data with a single machine, Hadoop breaks up Big Data into multiple parts so each part can be processed and analyzed in parallel.
  8. 8. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop: The Pros and Cons 8 First the pros … Hadoop is a time- and cost-effective approach to store, process and analyze large volumes of unstructured data allowing for new and unprecedented types of analytics. Now the cons … Hadoop is complex and difficult to deploy and manage; there’s a dearth of Hadoop-savvy engineers and Data Scientists on the job market; the risk of forking and vendor lock-in remains.
  9. 9. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop: The Pros and Cons cont. 9 More pros … Many bright minds contributing to Hadoop resulting in rapid development and an ecosystem of vendors emerging to make Hadoop enterprise-ready.
  10. 10. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org The Big Data Ecosystem 10
  11. 11. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers 11 •  Largest Hadoop instance on the planet … 40,000 nodes handling 200+ PB of data. •  Used to support research for ad systems and Web search. •  Match ads with users, detect spam in Yahoo! Mail, pick relevant top stories.
  12. 12. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers cont. 12 •  Two major clusters processing and storing over 30 PB of data. •  Uses HDFS to store copies of internal log and dimension data. •  Developed Hive to perform large-scale analytics on user data. •  Using HBase to store, manage and retrieve Facebook Messenger data.
  13. 13. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers cont. 13 •  Uses Hadoop to support “People You May Know” feature. •  Tailors its search engine to return most relevant results for recruiters, employers and job seekers. •  Created a visualization tool to allow users to explore their professional network to discover hidden patterns.
  14. 14. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data in Financial Services 14 •  Over 30,000 databases and 15,000 applications spread across 7 business units. •  Using Hadoop as the basis of its Common Data Platform. •  Looking to establish 360 degree view of customer for upsell and cross-sell opportunities.
  15. 15. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data in Financial Services cont. 15 •  Risk management and analysis to understand financial exposure. •  Detecting fraudulent transactions and potentially criminal activity. •  Conduct sentiment analysis on social media data.
  16. 16. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Thank You 16 Jeffrey F. Kelly Principal Research Contributor The Wikibon Project jeff.kelly@wikibon.org @jeffreyfkelly www.wikibon.org www.siliconangle.com

×