(c) Infinity

1
Preparation
Phase

• Access Grants
• Assessment of Current Infrastructure
• Inventory of Current Setup
• Meta-Data Backup
...


Upgrade a cluster by configuring a new cluster with same capacity and
newer Hadoop version and then migrate the files/d...
CopyToLocal /CopyFromLocal


The process flows, the files are copied to local drive using Hadoop command
CopyToLocal and ...
 Upgrading

existing cluster from V1 to V2 inline by
Installing/Configuring new cluster and updating
metadata.
Hadoop V2
...
Shutdown of
Cluster/Node
s

Validation of
Upgrade
Activity

Software
Stack Stage

Upgrade
Phase

(c) Infinity

Production
...
 http://wiki.apache.org/hadoop/Hadoop_Upgrade

(c) Infinity

7
 Drop

me mail @
 dwivedishashwat@gmail.com
 Visit

my blog
 http://helpmetocode.blogspot.com

(c) Infinity

8
Upcoming SlideShare
Loading in …5
×

Hadoop migration and upgradation

454 views
317 views

Published on

migrate hadoop, hadoop migration, hadoop upgradation, upgrade hadoop

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
454
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Hadoop migration and upgradation

  1. 1. (c) Infinity 1
  2. 2. Preparation Phase • Access Grants • Assessment of Current Infrastructure • Inventory of Current Setup • Meta-Data Backup • New Cluster Setup with Hadoop 2.x . OR Inline Migration to Hadoop 2.x from Hadoop 1.x Methodology Staging Environment Production Rollout • Replica of Current Setup in Staging Environment • Upgrade in Staging Environment • Validation of Migration in Staging Environment • Upgrade of Production Environment • Production Environment validation. (c) Infinity 2
  3. 3.  Upgrade a cluster by configuring a new cluster with same capacity and newer Hadoop version and then migrate the files/data from old cluster to new one. (c) Infinity 3
  4. 4. CopyToLocal /CopyFromLocal  The process flows, the files are copied to local drive using Hadoop command CopyToLocal and the files are then pushed to the new cluster using CopyFromLocal, and the older cluster can be decommissioned. Using Hadoop CP command  This is a kind of cluster to cluster copy, using Hadoop ‘cp’ command the files are transferred from one HDFS to other HDFS. As the version are different we need a mechanism called copy from HFTP where the command is executed from the target cluster by defining source as old cluster with HFTP protocol and target as HDFS protocol. Using Hadoop DISTCP command  This is a kind of cluster to cluster copy, using Hadoop ‘distcp’ command the files are transferred from one HDFS to other HDFS. As the version are different we need a mechanism called copy from HFTP where the command is executed from the target cluster by defining source as old cluster with HFTP protocol and target as HDFS protocol. (c) Infinity 4
  5. 5.  Upgrading existing cluster from V1 to V2 inline by Installing/Configuring new cluster and updating metadata. Hadoop V2 Hadoop V1 HDFS  Upgraded Metadata HDFS (c) Infinity 5
  6. 6. Shutdown of Cluster/Node s Validation of Upgrade Activity Software Stack Stage Upgrade Phase (c) Infinity Production Rollout 6
  7. 7.  http://wiki.apache.org/hadoop/Hadoop_Upgrade (c) Infinity 7
  8. 8.  Drop me mail @  dwivedishashwat@gmail.com  Visit my blog  http://helpmetocode.blogspot.com (c) Infinity 8

×