Apache Hadoop India Summit 2011 talk "Adaptive Parallel Computing over Distributed Military Computing Infrastructures" by Rituraj Kumar

Adaptive parallel computing over distributed military computing infrastructures
RiturajKumar
Center of Artificial Intelligence and Robotics – DRDO Lab

2
Contents
Introduction
Adaptive Parallel Computing
Approaches for APC
MPI + DDS
Hadoop
Conclusions

3
Introduction
Net-Centric Paradigm of warfare needs highly compute intensive operations to be performed.
Parallel Computing is the only feasible solution that would guarantee reduced response time.
Tactical deployment would not amenable for –
Large Clusters in the field
Back-hauling
Intelligent use of the existing spare compute capacity of the computing devices within the cloud is essential.

4
Cloud of Heterogeneous Computing Devices
MILCOM

5
Amdahls Law
The Amdahl's law is concerned with the speedup achievable from an improvement to a computation that affects a proportion P of that computation where the improvement has a speedup of S.
Where,
S = Speedup
P = Parallel fraction of Program
N = Number of Processors
If 95% of the program can be parallelized, the theoretical maximum speedup using parallel computing would be 20×, no matter how many processors are used.

6
Cost for parallel Computation = Computation Cost + Serialization Cost

7
APC Approaches
MPI + DDS
MPI Code:
Parallel implementation of
complex mathematical models
MPI Controller:
Execution of parallel MPI code
over distributed network.
DDS:
Reliable Communication over
challenged network.
DMI:
Identification of computing
nodes in the distributed
network.
MPI Code
MPI
Controller
Dynamic Membership Identifier
DDS

8
APC Approaches
MPI + DDS
Advantage:
Well-known MPI API Framework.
DDS provides assurance of reliable communication between nodes.
Disadvantage:
Needs a wrapper for converting MPI calls to DDS framework.
Performance degradation because of the wrapper.

9
APC Approaches
HADOOP
Map/Reduce can be effectively used for parallelization
Used extensively in varieties of information systems.
Java Based
Designed to handle large data sets

10
APC Approaches
Hadoop
Advantage:
Highly Scalable
Excellent fault tolerance capabilities
Good hardware interoperability
Current challenges:
Requirement of slight architectural changes
Currently not suitable for resource constraint hardware

11
Conclusion
Military domain requires reliable parallel computing infrastructure over disadvantaged communication network.
Dynamic topology poses great challenge for computing infrastructure.
MPI framework has few disadvantages
Hadoop is a promising candidate in these conditions.

12
Thank You

Apache Hadoop India Summit 2011 talk "Adaptive Parallel Computing over Distributed Military Computing Infrastructures" by Rituraj Kumar

by Yahoo! Developer Network on Feb 24, 2011

Accessibility

Upload Details

Usage Rights

1 Embed 151

Statistics

Apache Hadoop India Summit 2011 talk "Adaptive Parallel Computing over Distributed Military Computing Infrastructures" by Rituraj Kumar Presentation Transcript