Hadoop and WANdisco: The Future of Big Data


Published on

View the webinar recording here... http://youtu.be/O1pgMMyoJg0

Who: WANdisco CEO, David Richards, and core creaters of Apache Hadoop, Dr. Konstantin Shvachko and Jagane Sundare.

What: WANdisco recently acquired AltoStor, a pioneering firm with deep expertise in the multi-billion dollar Big Data market.

New to the WANdisco team are the Hadoop core creaters, Dr. Konstantin Shvachko and Jagane Sundare. They will cover the the acquisition and reveal how WANdisco's active-active replication technology will change the game of Big Data for the enterprise in 2013.

Hadoop, a proven open source Big Data technolgoy, is the backbone of Yahoo, Facebook, Netflix, Amazon, Ebay and many of the world's largest databases.

When: Tuesday, December 11th at 10am PST (1pm EST).

Why: In this 30-minute webinar you’ll learn:

The staggering, cross-industry growth of Hadoop in the enterprise

How Hadoop's limitations, including HDFS's single-point of failure, are impacting the productivity of the enterprise

How WANdisco's active-active replication technology will alleviate these issues by adding high-availability to Hadoop, taking a fundamentally different approach to Big Data

View the webinar Q&A on the WANdisco blog here...http://blogs.wandisco.com/2012/12/14/answers-to-questions-from-the-webinar-of-dec-11-2012/

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • SPOFs – Name Node, Hbase, YARN
  • SPOFs – Name Node, Hbase, YARN
  • Hadoop and WANdisco: The Future of Big Data

    1. WANdisco and Hadoop:The Future of Big Data December 11, 2012
    2. • WANdisco: Wide Area Network Distributed Computing• Patented technology for active-active replication• Leader in tools for software engineers (Subversion)• No venture capital, angel investors or private equity funding• Listed on the London Stock Exchange on June 1, 2012 in a highly successful IPO (LSE:WAND)• Offices in San Ramon (CA), Boston (MA), Sheffield (UK), Belfast (UK), Chengdu (China), Tokyo (Japan) 2
    3. WANdisco Technology Traditional Approach 3
    4. "unlike conventional solutions, the multi-sitecomputing system architecture does not rely on a centraltransaction coordinator that is known to be a single-point-of-failure." 4
    5. “Big Data is the new definitive source of competitive advantage across all industries. For those organizations thatunderstand and embrace the new reality of Big Data, the possibilities fornew innovation, improved agility, and increased profitability are nearly endless” - Wikibon 5
    6. • Fixing specific problems: • Easy to use appliance • High availability • Disaster recovery / zero time to recovery (over a WAN)• Highly differentiated • Nobody else can do active-active replication over a WAN 6
    7. Dr. Konstantin Shvachko• Co-founder of AltorStor, acquired by WANdisco• Was part of the team that invented Hadoop at Yahoo in 2006 and went on to become the Principal Big Data architect at eBay• Credited with the creation and maintenance of the Hadoop Distributed File System (HDFS), which is at the very core of both Hadoop and any replication solution for HadoopJagane Sundar• Co-founder of AltorStor, acquired by WANdisco• Was responsible for conceiving, architecting and managing the development of AltoScale’s Hadoop As A Service platform before selling it to VertiCloud• Visionary behind AltoStor’s Cloud and Big Data Storage Appliance• Former Director of Hadoop Engineering at Yahoo! and managed the development of Hadoop 0.20.204 with Disk Fail In PlaceComplementary IP and skills• Ideal fit with our patented active-active replication technology• Altostor founders faced problems (scaling, performance and high availability) we are planning to solve 7
    8. • 24-by-7 Reliability, Availability, Scalability and Performance• Planned as well as unplanned outages are extremely expensive• Steep learning curve and dearth of trained specialists• Many enterprises forced to rely on public cloud options such as Amazon • Expensive hourly billing models • Vendor lock-in with difficult migration paths • Periodic availability and performance problems • Data security concerns with cloud-based deployments• Moving from batch model to real-time transaction model• Our product suite will be designed to meet all of these challenges 8
    9. • Plug-and-play pre-packaged software eliminates the need for specialized Hadoop skills• Wizard based deployment, monitoring and management• Supports migration from Amazon to private in-house clouds• S3-enabled filesystem unique to WANdisco’s AltoStor appliance • Allows searches for any kind of data (images, videos, etc.) based on descriptive characteristics• HBase support for real-time transaction processing 9
    10. NameNode NameNode NameNode HDFS Data 10
    11. • Works over a LAN within a single data center NameNode NameNode• Works over a WAN across data centers thousands of miles apart• Supports simultaneous read and write access on every server HDFS Data 11
    12. Public Cloud S3 Apps for the private cloud – e.g. JungleDisk, SmugMug, Senduit, Zmanda, etc. Traditional Hadoop M/R HBase Apps AppsStep 1 WANdisco AltoStor Appliance Hadoop Mgmt Server S3 API HBase API HDFS API JobTracker• Deploy Hadoop(s)• Manage Hadoop AltoStor Hadoop• Monitor Hadoop Step 3 Enterprise Physical (e.g. rack of Dell servers) or Active Virtual Infrastructure (e.g. VMware VI) Directory Step 2 for WANdisco AltoStor to use in building Hadoop 12
    13. • WANdisco AltoStor appliance has the capability to deploy on virtualized infrastructure such as VMware• Advantages: • Extra level of reliability • Elasticity – live cluster shrinking and expansion • Extremely high hardware resource utilization • Resource isolation • Ease of management – preconfigured VMs 13
    14. HDFS ClientsHDFS Cluster DataNodes Active Standby NameNode NameNode Shared Storage 14
    15. Client Client Client Active Active Active NameNode NameNode … NameNode Proposal Handler Proposal Handler Proposal Handler Dispatch DispatchDispatch WANdisco PAXOS 15
    16. Questions and Answers
    17. Thank Youhttp://blogs.wandisco.com/author/jagane-sundarVisit us www.wandisc.com