Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Hadoop MapReduce Performance
Study on ARM cluster
Yanjun Wang
yanjun@cse.unl.edu
Outlines
•
•
•
•
•

Motivation
Introduction
Evaluation
Conclusion
Questions
Motivation
• A credit card size Raspberry Pi can run general
Linux with very low power consumption
Motivation
• ARM cluster vs. x86_64 cluster
Introduction
• Hadoop MapReduce
• Cubieboard2
Evaluation
• Environment

– ARM cluster: 4 cubieboard2, 1 head node, 3 worker
nodes; lubuntu for ARM, java-1.7 for ARM
– X...
Evaluation
• Equivalent Performance
– Run program on current device and get
turnaround time
– To achive the same turnaroun...
Evaluation
• loadgen
Evaluation
• MDAD
Evaluation
• Loadgen Energy
Evaluation
• MDAD energy
Conclusion
• We build a ARM cluster which is composed of
4 cubieboard2 cards.
• We setup Hadoop cluster on the ARM cluster...
Upcoming SlideShare
Loading in …5
×

Hadoop mapreduce performance study on arm cluster

7,195 views

Published on

This presentation demonstrates a performance study of Hadoop MapReduce based on ARM cluster. It compared MapReduce applications performance and energy consumption between ARM cluster and general x86_64 cluster.

Published in: Technology, Business

Hadoop mapreduce performance study on arm cluster

  1. 1. Hadoop MapReduce Performance Study on ARM cluster Yanjun Wang yanjun@cse.unl.edu
  2. 2. Outlines • • • • • Motivation Introduction Evaluation Conclusion Questions
  3. 3. Motivation • A credit card size Raspberry Pi can run general Linux with very low power consumption
  4. 4. Motivation • ARM cluster vs. x86_64 cluster
  5. 5. Introduction • Hadoop MapReduce • Cubieboard2
  6. 6. Evaluation • Environment – ARM cluster: 4 cubieboard2, 1 head node, 3 worker nodes; lubuntu for ARM, java-1.7 for ARM – X86_64 cluster: 2 firefly nodes. 1 head node, 1 worker node; CentOS 6.3, java-1.7 • Hadoop 1.2 [1] • Testcases – Loadgen – MDAD (Molecular Dynamics Simulation based on Hadoop MapReduce [2]) • • [1]Apache Hadoop [2]Chen He, “Molecular Dynamics Simulation based on Hadoop MapReduce” , Master thesis, 2011
  7. 7. Evaluation • Equivalent Performance – Run program on current device and get turnaround time – To achive the same turnaround time, how many new devices we need, or could we this? • Energy consumption – Kill-a-Watt device to collect ARM cluster energy; – ServerTech PDU for gathering x86_64 cluster energy consumption
  8. 8. Evaluation • loadgen
  9. 9. Evaluation • MDAD
  10. 10. Evaluation • Loadgen Energy
  11. 11. Evaluation • MDAD energy
  12. 12. Conclusion • We build a ARM cluster which is composed of 4 cubieboard2 cards. • We setup Hadoop cluster on the ARM cluster • We compared the performance and energy consumption between two clusters • Based on our current data, we conclude that ARM cluster is not an alternative choice to replace X86_64.

×