DRP (Stretch Cluster) for HDP - Future of Data : Paris
•
1 like•923 views
How to design a Disaster Recovery Plan for HDP (Hortonworks Data Platform) Clusters? Mohamed Mehdi BEN AISSA, Big Data Practice Manager at FINAXYS and Big Data ITO at CACIB For HDP Clusters, we suggest, in a first phase, different Disaster Recovery Plan solutions depending on the SLA (Service-level agreement): RPO (Recovery Point Objective), RTO (Recovery Time Objective). In a second phase, we focus more on the stretch cluster solution: the advantages, the drawbacks and the impact of this choice on the global architecture. Finally, we explain in detail how to configure and deploy this solution and how to integrate each layer (storage layer, processing layer ...) into the architecture.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to DRP (Stretch Cluster) for HDP - Future of Data : Paris
Similar to DRP (Stretch Cluster) for HDP - Future of Data : Paris (20)
Recently uploaded
Recently uploaded (20)
DRP (Stretch Cluster) for HDP - Future of Data : Paris
- 2. NOTRE SINGULARITÉ ? NOUS SOMMES PLURIELS ! Mohamed Mehdi BEN AISSA Big Data Practice Manager – Finaxys Big Data ITO - CACIB linkedin.com/in/mehdi-ben-aissa/ @Ben_Aissa_mehdi
- 3. HOW TO DESIGN A DISASTER RECOVERY PLAN FOR HDP CLUSTERS? 2017-09
- 4. PLAN I. INTRODUCTION II. BIG DATA DRP ARCHITECTURES III. STRETCH CLUSTER ARCHITECTURE IV. HDP ARCHITECTURE V. HDFS : STRETCH CLUSTER CONFIGURATION VI. YARN : STRETCH CLUSTER CONFIGURATION VII. CONCLUSION 4
- 5. INTRODUCTION
- 6. INTRODUCTION 6 • SLA (Service-Level Agreement) : Particular aspects of the service (quality, availability, responsibilities) : • RTO (Recovery Time Objective) : The targeted duration of time and a service level within which a business process must be restored after a disaster • RPO (Recovery Point Objective) : The maximum targeted period in which data might be lost • Goals : 24/7 RPO € RTO=0 RPO=0 Cost=0 Consistency Performance
- 7. BIG DATA DRP ARCHITECTURES
- 8. BIG DATA DRP ARCHITECTURES : MULTI-CLUSTER ARCHITECTURE VS STRETCH CLUSTER 8 Cluster 1 Cluster 2 Data Center 1 Data Center 2 Data Center 3 Data Replication Replication (1) Multi-cluster Architecture (2) Stretch Cluster
- 9. STRETCH CLUSTER : ARCHITECTURE 9 Control NodesControl Nodes Gateway Node Witness Nodes Master Nodes Worker Nodes Gateway Node DC1 DC2 DC3
- 10. HORTONWORKS DATA PLATFORM ARCHITECTURE
- 11. HORTONWORKS DATA PLATFORM : ARCHITECTURE (1) 11
- 12. HORTONWORKS DATA PLATFORM : ARCHITECTURE (2) 12 Data Collect & Storage Data Processing Serving Layer Apache Kafka NoSQL Database Indexed Data Search
- 13. HORTONWORKS DATA PLATFORM : ARCHITECTURE (3) 13 HDFS Kafka YARN HBase Solr Flume Spark Tez MapReduce Hive Oozie Sqoop
- 14. HDFS : STRETCH CLUSTER CONFIGURATION
- 15. STRETCH CLUSTER : HDFS ARCHITECTURE 15 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4
- 16. STRETCH CLUSTER : HDFS ARCHITECTURE - DEFAULT CONFIGURATION 16 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 Cluster Config: dfs.replication = 4 B B B B
- 17. STRETCH CLUSTER : HDFS ARCHITECTURE – RACK AWARENESS (1) 17 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 Cluster Config: dfs.replication = 4 Rack.awareness B B B B
- 18. STRETCH CLUSTER : HDFS ARCHITECTURE – RACK AWARENESS (2) 18 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 B B B B B Cluster Config: dfs.replication = 4 Rack.awareness
- 19. STRETCH CLUSTER : HDFS ARCHITECTURE – HVE (1) 19 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) B B B B
- 20. STRETCH CLUSTER : HDFS ARCHITECTURE – HVE (2) 20 DC3 Zookeeper 1 Journalnode 1 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Rack3 Rack4 Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) B B DC1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Namenode Rack1 Rack2 B B Timeout: 10mn
- 21. STRETCH CLUSTER : HDFS ARCHITECTURE – TIMEOUT 21 DC3 Zookeeper 1 Journalnode 1 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Rack3 Rack4 Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) Heartbeat.recheck-interval: 5mn-> 5s B B DC1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Namenode Rack1 Rack2 B B Timeout: 10s
- 22. STRETCH CLUSTER : HDFS ARCHITECTURE – SPLIT BRAIN (1) 22 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 B B B B Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) Heartbeat.recheck-interval: 5mn-> 5s
- 23. STRETCH CLUSTER : HDFS ARCHITECTURE – SPLIT BRAIN (2) 23 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 B B Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) Heartbeat.recheck-interval: 5mn-> 5s X B B
- 24. STRETCH CLUSTER : HDFS ARCHITECTURE – SPLIT BRAIN (3) 24 DC1 DC3 Zookeeper 1 Journalnode 1 Zookeeper 2 Journalnode 2 Zookeeper 3 Journalnode 3 DC2 Zookeeper 4 Journalnode 4 Zookeeper 5 Journalnode 5 Datanode 1 Datanode 2 Datanode 3 Datanode 4 Datanode 5 Datanode 6 Datanode 7 Datanode 8 Namenode Namenode Rack1 Rack2 Rack3 Rack4 B B X B B Cluster Config: dfs.replication = 4 Rack.awareness HVE (Hadoop Virtualization Extensions) Heartbeat.recheck-interval: 5mn-> 5s
- 25. YARN : STRETCH CLUSTER CONFIGURATION
- 26. STRETCH CLUSTER : YARN ARCHITECTURE (1) 26 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) A AA A
- 27. STRETCH CLUSTER : YARN ARCHITECTURE (2) 27 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) Node Labels Node.label: dc1 Node.label: dc2 Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 A A A A A A A A
- 28. STRETCH CLUSTER : YARN ARCHITECTURE - DC FAILURE 28 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) Node Labels Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 A A A A A A A A A A A A
- 29. STRETCH CLUSTER : YARN ARCHITECTURE - SPLIT BRAIN (1) 29 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) Node Labels Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 A A A A A A A A
- 30. STRETCH CLUSTER : YARN ARCHITECTURE – SPLIT BRAIN (2) 30 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) Node Labels Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 XA A A A A A A A A A A A
- 31. STRETCH CLUSTER : YARN ARCHITECTURE – SPLIT BRAIN (2) 31 DC1 DC3 Zookeeper 1 Zookeeper 2 Zookeeper 3 DC2 Zookeeper 4 Zookeeper 5 Node Manager 1 Node Manager 2 Node Manager 7 Node Manager 8 Resource Manager Resource Manager Rack1 Rack2 Rack3 Rack4 Cluster Config: Rack.awareness HVE (Hadoop Virtualization Extensions) Node Labels Node Manager 3 Node Manager 4 Node Manager 5 Node Manager 6 XA A A A A A A A
- 32. CONCLUSION
- 33. CONCLUSION 33 • There is no one ideal architecture that can respond to all needs: o RPO = 0 o RTO = 0 o Performance o Consistency • You can combine many architectures in the same Cluster : Hybrid Architectures • Monitoring Tools are required to keep track of your replication process and have a global visibility about your cluster status • Resiliency and Performance Tests are required to validate your DRP Architecture
- 34. MERCI 34