Your SlideShare is downloading. ×
Big Data and Hadoop - key drivers, ecosystem and use cases
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Big Data and Hadoop - key drivers, ecosystem and use cases

581
views

Published on

Overview of the Big Data market.

Overview of the Big Data market.

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
581
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org [[The Wikibon Project]] Big Data and Hadoop: Key Drivers, Ecosystem and Use Cases November 2011
  • 2. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org What is Big Data? 2 Big Data n Data sets whose size, type and/or speed make them impractical to process and analyze with traditional database technologies and related data management tools.
  • 3. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Why is Big Data Important? 3 Big  Data  is  the  new  de.initive  source   of  competitive  advantage  across   industries  … …  For  those  organizations  that   embrace  Big  Data,  the  possibilities   for  innovation,  improved  agility,  and   increased  pro.itability  are  nearly   endless.
  • 4. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Three Key Big Data Drivers 4 1.  Volume, Variety, Velocity 2.  Hardware Commoditization 3.  Cloud Computing
  • 5. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Characteristics of Big Data 5
  • 6. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Sources of Big Data 6
  • 7. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop 7 Open source framework for processing, storing and analyzing Big Data. Fundamental concept: Rather than banging away at one, huge block of data with a single machine, Hadoop breaks up Big Data into multiple parts so each part can be processed and analyzed in parallel.
  • 8. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop: The Pros and Cons 8 First the pros … Hadoop is a time- and cost-effective approach to store, process and analyze large volumes of unstructured data allowing for new and unprecedented types of analytics. Now the cons … Hadoop is complex and difficult to deploy and manage; there’s a dearth of Hadoop-savvy engineers and Data Scientists on the job market; the risk of forking and vendor lock-in remains.
  • 9. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Hadoop: The Pros and Cons cont. 9 More pros … Many bright minds contributing to Hadoop resulting in rapid development and an ecosystem of vendors emerging to make Hadoop enterprise-ready.
  • 10. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org The Big Data Ecosystem 10
  • 11. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers 11 •  Largest Hadoop instance on the planet … 40,000 nodes handling 200+ PB of data. •  Used to support research for ad systems and Web search. •  Match ads with users, detect spam in Yahoo! Mail, pick relevant top stories.
  • 12. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers cont. 12 •  Two major clusters processing and storing over 30 PB of data. •  Uses HDFS to store copies of internal log and dimension data. •  Developed Hive to perform large-scale analytics on user data. •  Using HBase to store, manage and retrieve Facebook Messenger data.
  • 13. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data Pioneers cont. 13 •  Uses Hadoop to support “People You May Know” feature. •  Tailors its search engine to return most relevant results for recruiters, employers and job seekers. •  Created a visualization tool to allow users to explore their professional network to discover hidden patterns.
  • 14. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data in Financial Services 14 •  Over 30,000 databases and 15,000 applications spread across 7 business units. •  Using Hadoop as the basis of its Common Data Platform. •  Looking to establish 360 degree view of customer for upsell and cross-sell opportunities.
  • 15. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Big Data in Financial Services cont. 15 •  Risk management and analysis to understand financial exposure. •  Detecting fraudulent transactions and potentially criminal activity. •  Conduct sentiment analysis on social media data.
  • 16. © Wikibon 2008© Wikibon 2011 | Confidential www.wikibon.org Thank You 16 Jeffrey F. Kelly Principal Research Contributor The Wikibon Project jeff.kelly@wikibon.org @jeffreyfkelly www.wikibon.org www.siliconangle.com