• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Ambari Meetup: NameNode HA
 

Ambari Meetup: NameNode HA

on

  • 1,367 views

 

Statistics

Views

Total Views
1,367
Views on SlideShare
1,364
Embed Views
3

Actions

Likes
7
Downloads
66
Comments
0

3 Embeds 3

http://172.16.9.53 1
http://www.linkedin.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Ambari Meetup: NameNode HA Ambari Meetup: NameNode HA Presentation Transcript

    • © Hortonworks Inc. 2013 HDFS NameNode HA Jing Zhao Hortonworks Page 1
    • © Hortonworks Inc. 2013 Background and Motivation • Prior to Hadoop 2.0 – One NameNode for metadata management – Single point of failure • HDFS High Availability – Two NameNodes in the same cluster – Active NameNode: responsive for all client operations – Standby NameNode: maintain enough state to provide a fast failover • Shared storage – Active NN writes edit log – Standby NN reads edit log and applies to its own namespace – During failover, Standby NN reads all the edits and transitions to Active state Page 2 Architecting the Future of Big Data
    • © Hortonworks Inc. 2013 Quorum Journal Management • Before QJM – NFS directory as shared storage – Limitation: custom hardware, complicated deployment… • QJM – Quorum commits of edit log to separate daemons (JournalNodes) – Paxos-like protocol to guarantee correctness • JournalNodes – A successful edit log commit must write to a majority of JNs – Lightweight, can be co-located on machines for NN, JT, or RM – Recommended number of JNs: 3 or 5 Page 3 Architecting the Future of Big Data
    • © Hortonworks Inc. 2013 Page 4 Architecting the Future of Big Data DataNode DataNode DataNode . . . . . Standby NameNode Active NameNode Shared Edits JournalNode JournalNode JournalNode Block Reports to Active & Standby Write Read
    • © Hortonworks Inc. 2013 Automatic Failover • Manual failover – Administrative commands • Use ZKFailoverController for Automatic Failover – ZKFC: a ZooKeeper client that monitors and manages the NN state – Health monitoring – ZooKeeper session management – ZooKeeper-base election Page 5 Architecting the Future of Big Data
    • © Hortonworks Inc. 2013 Page 6 Architecting the Future of Big Data DataNode DataNode DataNode . . . . . Standby NameNode Write Active NameNode Shared Edits JournalNode JournalNode JournalNode ZKFC ZKFC ZooKeeper Service Cluster Monitor and try to take active lock Monitor liveness and health. Manage HA state. Write Read Monitor liveness and health. Manage HA state. Monitor and maintain active lock