Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Introduction of Apache Hama            Edward J. Yoon, October 11, 2011                  <edwardyoon@apache.org>
About Me   Founder of Apache Hama.   Committer of Apache Bigtop.   Employee for KT.   http://twitter.com/eddieyoon
What Is Hama?   Apache Incubator Project.   BSP (Bulk Synchronous Parallel) for massive scientific    computations.   W...
Hama Characteristics   Provides a Pure BSP model .       Job submission and management interface.       Multiple tasks ...
Bulk Synchronous Parallel?   Parallel programming model introduced by Valiant.   Consist of a sequence of supersteps.  ...
Schematic diagram of a superstepLocal Computation   ……….             Idle  Communication     ……….             Idle        ...
Internals   Proactive Task Scheduling.   Hadoop RPC is used for BSP tasks to communicate each    other.   Collection an...
Pi Calculation   Each task executes locally its portion of the loop a    number of times.   One task acts as master and ...
Structural Analysis of Network Traffic Flows   Traffic flows in KT clouds.       traffic engineering, anomaly detection,...
Random Communication Benchmarks   Benchmarked on 16 1U servers using 10 tasks per server.   X axis is the time (sec.)of ...
What’s Next?   Support Input/Output Formatter like MapReduce.   Message Compression for High Performance.   Add some fr...
More Information   http://incubator.apache.org/hama   http://wiki.apache.org/hama
Upcoming SlideShare
Loading in …5
×

Introduction of Apache Hama - 2011

22,692 views

Published on

for the lecture.

Published in: Technology
  • Be the first to comment

Introduction of Apache Hama - 2011

  1. 1. Introduction of Apache Hama Edward J. Yoon, October 11, 2011 <edwardyoon@apache.org>
  2. 2. About Me Founder of Apache Hama. Committer of Apache Bigtop. Employee for KT. http://twitter.com/eddieyoon
  3. 3. What Is Hama? Apache Incubator Project. BSP (Bulk Synchronous Parallel) for massive scientific computations. Written In Java. Currently 2 releases, 3 main committers.
  4. 4. Hama Characteristics Provides a Pure BSP model .  Job submission and management interface.  Multiple tasks per node.  Checkpoint recovery. Supports to run in the Clouds using Apache Whirr. Supports to run with Hadoop nextGen.
  5. 5. Bulk Synchronous Parallel? Parallel programming model introduced by Valiant. Consist of a sequence of supersteps. Conceptually simple and intuitive from a programming standpoint. Used for a variety of applications e.g., scientific computing, genetic programming, …
  6. 6. Schematic diagram of a superstepLocal Computation ………. Idle Communication ………. Idle Barrier Synchronization
  7. 7. Internals Proactive Task Scheduling. Hadoop RPC is used for BSP tasks to communicate each other. Collection and bundling of messages as a technique to reduce network overheads and contentions. Zookeeper is used for Barrier Synchronization.
  8. 8. Pi Calculation Each task executes locally its portion of the loop a number of times. One task acts as master and collects the results through the BSP communication interface.
  9. 9. Structural Analysis of Network Traffic Flows Traffic flows in KT clouds.  traffic engineering, anomaly detection, traffic forecasting and capacity planning Currently BSP jobs are experimentally running on 512 multi-cores machines.
  10. 10. Random Communication Benchmarks Benchmarked on 16 1U servers using 10 tasks per server. X axis is the time (sec.)of BSP job execution (32 supersteps). Y axis is the number of messages to be sent to random BSP tasks in each superstep.
  11. 11. What’s Next? Support Input/Output Formatter like MapReduce. Message Compression for High Performance. Add some frameworks on top of Hama.
  12. 12. More Information http://incubator.apache.org/hama http://wiki.apache.org/hama

×