Hadoop MapReduce Performance
Study on ARM cluster
Yanjun Wang
yanjun@cse.unl.edu
Outlines
•
•
•
•
•

Motivation
Introduction
Evaluation
Conclusion
Questions
Motivation
• A credit card size Raspberry Pi can run general
Linux with very low power consumption
Motivation
• ARM cluster vs. x86_64 cluster
Introduction
• Hadoop MapReduce
• Cubieboard2
Evaluation
• Environment

– ARM cluster: 4 cubieboard2, 1 head node, 3 worker
nodes; lubuntu for ARM, java-1.7 for ARM
– X...
Evaluation
• Equivalent Performance
– Run program on current device and get
turnaround time
– To achive the same turnaroun...
Evaluation
• loadgen
Evaluation
• MDAD
Evaluation
• Loadgen Energy
Evaluation
• MDAD energy
Conclusion
• We build a ARM cluster which is composed of
4 cubieboard2 cards.
• We setup Hadoop cluster on the ARM cluster...
Upcoming SlideShare
Loading in …5
×

Hadoop mapreduce performance study on arm cluster

6,768 views

Published on

This presentation demonstrates a performance study of Hadoop MapReduce based on ARM cluster. It compared MapReduce applications performance and energy consumption between ARM cluster and general x86_64 cluster.

Published in: Technology, Business

Hadoop mapreduce performance study on arm cluster

  1. 1. Hadoop MapReduce Performance Study on ARM cluster Yanjun Wang yanjun@cse.unl.edu
  2. 2. Outlines • • • • • Motivation Introduction Evaluation Conclusion Questions
  3. 3. Motivation • A credit card size Raspberry Pi can run general Linux with very low power consumption
  4. 4. Motivation • ARM cluster vs. x86_64 cluster
  5. 5. Introduction • Hadoop MapReduce • Cubieboard2
  6. 6. Evaluation • Environment – ARM cluster: 4 cubieboard2, 1 head node, 3 worker nodes; lubuntu for ARM, java-1.7 for ARM – X86_64 cluster: 2 firefly nodes. 1 head node, 1 worker node; CentOS 6.3, java-1.7 • Hadoop 1.2 [1] • Testcases – Loadgen – MDAD (Molecular Dynamics Simulation based on Hadoop MapReduce [2]) • • [1]Apache Hadoop [2]Chen He, “Molecular Dynamics Simulation based on Hadoop MapReduce” , Master thesis, 2011
  7. 7. Evaluation • Equivalent Performance – Run program on current device and get turnaround time – To achive the same turnaround time, how many new devices we need, or could we this? • Energy consumption – Kill-a-Watt device to collect ARM cluster energy; – ServerTech PDU for gathering x86_64 cluster energy consumption
  8. 8. Evaluation • loadgen
  9. 9. Evaluation • MDAD
  10. 10. Evaluation • Loadgen Energy
  11. 11. Evaluation • MDAD energy
  12. 12. Conclusion • We build a ARM cluster which is composed of 4 cubieboard2 cards. • We setup Hadoop cluster on the ARM cluster • We compared the performance and energy consumption between two clusters • Based on our current data, we conclude that ARM cluster is not an alternative choice to replace X86_64.

×