INTER-TASK COMMUNICATION ON VOLATILE NODES Masters in Science (Computer Science) Thesis Presented by NAGARAJAN KANNA Advis...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
INTRODUCTION <ul><li>Parallel & Distributed computing </li></ul><ul><ul><li>Multi-processor, Multi-core, Multi-computer (C...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
MOTIVATION <ul><li>Under 1% of available PCs are being harnessed (BOINC generates > 1PetaFLOPS) </li></ul><ul><li>Ordinary...
RELATED WORK <ul><li>BOINC </li></ul><ul><ul><li>Middleware for Volunteer computing </li></ul></ul><ul><ul><li>Lack of sup...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
DATASPACE <ul><li>New ‘DATASPACE’ modeled  ~LINDA [1]  tuplespace </li></ul><ul><li>A robust communication layer that faci...
COMMUNICATION MODEL Client machine running User application Dataspace Server (DSS) 1. Initialize application with DSS info...
DATASPACE - ‘PUT’ Dataspace Server Client – Replica 1 Client – Replica 2 PUT Meta info from process 1 Tag & Data Status fo...
DATASPACE - ‘READ’ and ‘GET’ Dataspace Server GET (Meta info & Tag)  from process 1 Data matching the Tag from Dataspace G...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
PROGRAMMING MODEL <ul><li>Library supports C/C++/Fortran applications </li></ul><ul><li>One instance of dataspace server p...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
IMPLEMENTATION <ul><li>Implemented on C/C++ with UNIX TCP Sockets </li></ul><ul><li>All communications with DSS initiated ...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
EXPERIMENTS <ul><li>APPLICATIONS </li></ul><ul><ul><li>Replica Exchange Molecular Dynamics (REMD) </li></ul></ul><ul><ul><...
REMD <ul><li>A approach for studying the folding thermodynamics of small to modest size proteins in explicit solvent </li>...
PSRS 27 23 22 16 13 10
SIEVE OF ERATOSTHENES <ul><li>Simple algorithm for finding all prime numbers up to specified integer </li></ul><ul><li>Usi...
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
REMD – Temperature swapping between replicas <ul><li>Application run with 8 replicas (8 temperatures) </li></ul><ul><li>Pa...
REMD: Using Dataspace vs. PERL <ul><li>Disk IO needs to reduced for improving performance </li></ul>
ROUND TRIP TIME MEASUREMENT <ul><li>Case 1 </li></ul><ul><li>Case 2 </li></ul>Dataspace Server Start time Dataspace operat...
RTT: DATASPACE OPERATIONS <ul><li>PUT only marginally faster </li></ul>
RTT: WITH 2 PROCESSES
BANDWIDTH: DATASPACE OPERATIONS
BANDWIDTH / HOP: WITH 2 PROCESSES
RTT: ‘PUT’ WITH REPLICAS <ul><li>No impact of replicas on Dataspace PUT </li></ul>
RTT: ‘READ’ WITH REPLICAS
RTT: ‘GET’ WITH REPLICAS
BANDWIDTH: ‘PUT’ WITH REPLICAS
BANDWIDTH: ‘READ’ WITH REPLICAS
BANDWIDTH: ‘GET’ WITH REPLICAS
SCALABILITY: PSRS
SCALABILITY: SoE
REDUNDANCY: PSRS
REDUNDANCY: SoE <ul><li>~25% increase in execution time with each replica </li></ul>
FAILURE: PSRS
FAILURE: SoE <ul><li>Drop in execution time for  2 node failures: ~3% </li></ul>
PSRS: WITH BOINC & WITHOUT BOINC <ul><li>Variations due to BOINC scheduler </li></ul>
OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL ...
SUMMARY OF RESULTS <ul><li>REMD: Use for Dataspace API in real world apps </li></ul><ul><li>Micro benchmarks : On adding r...
OUR CONTIBUTION <ul><li>Task parallelism: Dataspace API facilitates data exchange between processes </li></ul><ul><li>Robu...
THANK YOU
Upcoming SlideShare
Loading in...5
×

Inter Task Communication On Volatile Nodes

736

Published on

Idle desktop computers are already used for high performance computing. But there is a lack of wider use for parallel computing due to the limitations of the programming models available. We have built a new communication library that facilitates execution of parallel scientific applications on virtual clusters composed of volatile ordinary PC nodes.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
736
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Inter Task Communication On Volatile Nodes"

  1. 1. INTER-TASK COMMUNICATION ON VOLATILE NODES Masters in Science (Computer Science) Thesis Presented by NAGARAJAN KANNA Advisor: Dr. Jaspal Subhlok Committee: Dr. Edgar Gabriel Dr. Margaret S. Cheung
  2. 2. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  3. 3. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  4. 4. INTRODUCTION <ul><li>Parallel & Distributed computing </li></ul><ul><ul><li>Multi-processor, Multi-core, Multi-computer (Cluster, Grid, Volunteer) </li></ul></ul><ul><li>Idle desktop computers </li></ul><ul><ul><li>Immense pool of unused resources </li></ul></ul><ul><ul><li>Utilized by: BOINC, Condor </li></ul></ul><ul><li>Volatile nodes </li></ul><ul><ul><li>Varying computation, communication, storage </li></ul></ul><ul><ul><li>Availability is unpredictable </li></ul></ul><ul><li>Handling Volatility </li></ul><ul><ul><li>Check point/ Restart </li></ul></ul><ul><ul><li>Process Migration </li></ul></ul><ul><ul><li>Process Replication </li></ul></ul>
  5. 5. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  6. 6. MOTIVATION <ul><li>Under 1% of available PCs are being harnessed (BOINC generates > 1PetaFLOPS) </li></ul><ul><li>Ordinary desktop PCs not utilized for running parallel applications </li></ul><ul><li>Many scientific applications have low degree of communication between processes </li></ul><ul><li>Lack of a good ‘programming model’ to utilize volatile resources for parallel application </li></ul>
  7. 7. RELATED WORK <ul><li>BOINC </li></ul><ul><ul><li>Middleware for Volunteer computing </li></ul></ul><ul><ul><li>Lack of support for communication parallel programs </li></ul></ul><ul><li>CONDOR </li></ul><ul><ul><li>Job scheduler, Support parallel programs </li></ul></ul><ul><ul><li>Uses check pointing: Not suitable for volatile systems </li></ul></ul><ul><li>CLUSTERS </li></ul><ul><ul><li>Expensive: Resources, Maintenance </li></ul></ul><ul><li>Our Goal: Execution of communicating parallel programs on volatile ordinary desktops </li></ul>
  8. 8. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  9. 9. DATASPACE <ul><li>New ‘DATASPACE’ modeled ~LINDA [1] tuplespace </li></ul><ul><li>A robust communication layer that facilitates data exchanges between tasks </li></ul><ul><ul><li>Abstract dataspace that acts as (intermediary) for data exchange between tasks </li></ul></ul><ul><ul><li>Asynchronous one-way communication: Tasks that exchange data need not be alive at the same time </li></ul></ul><ul><ul><li>Use of process replication to handle volatile nature of nodes </li></ul></ul><ul><li>Integration with existing volunteer computing systems </li></ul>[1] N. Carriero and D. Gelernter. The s/net’s linda kernel. ACM Trans. Comput. Syst., 4(2):110–129, 1986
  10. 10. COMMUNICATION MODEL Client machine running User application Dataspace Server (DSS) 1. Initialize application with DSS info 2. Establish connection with DSS 3. Put data (101, ABCD) 4. Read data (101) 5. Get data (101) 6. Terminate connection with DSS ABCD 101 DATA TAG DATA TAG
  11. 11. DATASPACE - ‘PUT’ Dataspace Server Client – Replica 1 Client – Replica 2 PUT Meta info from process 1 Tag & Data Status for PUT operation PUT Meta info from process 1 Status - SUCCESS DATA TAG PutCount ProcessId 1 1 PutCount ProcessId
  12. 12. DATASPACE - ‘READ’ and ‘GET’ Dataspace Server GET (Meta info & Tag) from process 1 Data matching the Tag from Dataspace GET (Meta info & Tag) from process 1 Corresponding Data Client – Replica 1 Client – Replica 2 ABCD 101 DATA TAG GetCount ProcessId 1 1 GetCount ProcessId DATA TAG 2 1 READ Buffer 2 ABCD 1 READ Buffer
  13. 13. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  14. 14. PROGRAMMING MODEL <ul><li>Library supports C/C++/Fortran applications </li></ul><ul><li>One instance of dataspace server per application </li></ul>DATASPACE API int volpex_put ( const char* tag, int tagSize, const void* data, int dataSize) int volpex_get ( const char* tag, int tagSize, void* data, int dataSize) int volpex_read ( const char* tag, int tagSize, void* data, int dataSize) SUPPORTING API int volpex_init ( int argc, char * argv[]) int volpex_getProcId ( void ) int volpex_getNumProc ( void ) void volpex_finish ( void )
  15. 15. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  16. 16. IMPLEMENTATION <ul><li>Implemented on C/C++ with UNIX TCP Sockets </li></ul><ul><li>All communications with DSS initiated by client </li></ul><ul><li>DSS: Single threaded; Uses TCP select to switch </li></ul><ul><li>Dataspace </li></ul><ul><ul><li>In memory storage of data </li></ul></ul><ul><ul><li>Hash table indexed on tag </li></ul></ul><ul><ul><li>Fixed #entry READ BUFFER to serve processes replicas </li></ul></ul><ul><ul><li>PUT: Request from replica check only for data size > 2KB </li></ul></ul><ul><li>On connectivity failure, fixed number of retries with exponentially increasing time interval </li></ul><ul><li>Dataspace does not differentiate process replicas </li></ul><ul><li>BOINC : Linking with dataspace library @ compilation </li></ul>
  17. 17. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  18. 18. EXPERIMENTS <ul><li>APPLICATIONS </li></ul><ul><ul><li>Replica Exchange Molecular Dynamics (REMD) </li></ul></ul><ul><ul><li>Parallel Sorting by Regular Sampling (PSRS) </li></ul></ul><ul><ul><li>Sieve of Eratosthenes </li></ul></ul><ul><li>TEST CASES </li></ul><ul><ul><li>Micro benchmarks </li></ul></ul><ul><ul><li>Scalability </li></ul></ul><ul><ul><li>Redundancy </li></ul></ul><ul><ul><li>Failure </li></ul></ul><ul><li>TEST BED </li></ul><ul><ul><li>Clients: Atlantis Itanium2 1.3GHz dual core with 4GB RAM </li></ul></ul><ul><ul><li>DSS: AMD Athlon 2.4GHz dual core with 2GB RAM </li></ul></ul>
  19. 19. REMD <ul><li>A approach for studying the folding thermodynamics of small to modest size proteins in explicit solvent </li></ul><ul><li>Simulations alternate between phases of conformational sampling and swapping </li></ul><ul><li>Relatively small communication requirements – Ideally suited for Dataspace API </li></ul><ul><li>Use of Dataspace </li></ul><ul><ul><li>Synchronization of processes </li></ul></ul><ul><ul><li>Store/Read energy values between neighbors </li></ul></ul><ul><ul><li>Exchange temperate values for next simulation run </li></ul></ul>
  20. 20. PSRS 27 23 22 16 13 10
  21. 21. SIEVE OF ERATOSTHENES <ul><li>Simple algorithm for finding all prime numbers up to specified integer </li></ul><ul><li>Using Dataspace </li></ul><ul><ul><li>Total set (till specified integer) is divided equally between all the processes </li></ul></ul><ul><ul><li>MASTER processes identifies candidate prime number and writes it onto the dataspace </li></ul></ul><ul><ul><li>All other processes reads candidate primer numbers and knocks of all multiples from its list </li></ul></ul><ul><ul><li>MASTER process collects local sum from all processes </li></ul></ul>
  22. 22. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  23. 23. REMD – Temperature swapping between replicas <ul><li>Application run with 8 replicas (8 temperatures) </li></ul><ul><li>Parameters: nstlimit=4000; nsteps=5 </li></ul><ul><li>Processes that swap temperatures at a step have same background color </li></ul>
  24. 24. REMD: Using Dataspace vs. PERL <ul><li>Disk IO needs to reduced for improving performance </li></ul>
  25. 25. ROUND TRIP TIME MEASUREMENT <ul><li>Case 1 </li></ul><ul><li>Case 2 </li></ul>Dataspace Server Start time Dataspace operation Status End time Client process Start time End time PUT (101, ABC) GET (102) GET (101) PUT (102, XYZ) Process 1 Process 2 Dataspace Server ABC 101 DATA TAG DATA TAG XYZ 102 DATA TAG
  26. 26. RTT: DATASPACE OPERATIONS <ul><li>PUT only marginally faster </li></ul>
  27. 27. RTT: WITH 2 PROCESSES
  28. 28. BANDWIDTH: DATASPACE OPERATIONS
  29. 29. BANDWIDTH / HOP: WITH 2 PROCESSES
  30. 30. RTT: ‘PUT’ WITH REPLICAS <ul><li>No impact of replicas on Dataspace PUT </li></ul>
  31. 31. RTT: ‘READ’ WITH REPLICAS
  32. 32. RTT: ‘GET’ WITH REPLICAS
  33. 33. BANDWIDTH: ‘PUT’ WITH REPLICAS
  34. 34. BANDWIDTH: ‘READ’ WITH REPLICAS
  35. 35. BANDWIDTH: ‘GET’ WITH REPLICAS
  36. 36. SCALABILITY: PSRS
  37. 37. SCALABILITY: SoE
  38. 38. REDUNDANCY: PSRS
  39. 39. REDUNDANCY: SoE <ul><li>~25% increase in execution time with each replica </li></ul>
  40. 40. FAILURE: PSRS
  41. 41. FAILURE: SoE <ul><li>Drop in execution time for 2 node failures: ~3% </li></ul>
  42. 42. PSRS: WITH BOINC & WITHOUT BOINC <ul><li>Variations due to BOINC scheduler </li></ul>
  43. 43. OUTLINE <ul><li>INTRODUCTION </li></ul><ul><li>MOTIVATION </li></ul><ul><li>DATASPACE </li></ul><ul><li>PROGRAMMING MODEL </li></ul><ul><li>IMPLEMENTATION </li></ul><ul><li>EXPERIMENTS </li></ul><ul><li>RESULTS </li></ul><ul><li>CONCLUSION </li></ul>
  44. 44. SUMMARY OF RESULTS <ul><li>REMD: Use for Dataspace API in real world apps </li></ul><ul><li>Micro benchmarks : On adding replicas </li></ul><ul><ul><li>PUT: Execution time remains constant </li></ul></ul><ul><ul><li>READ, GET: Execution time increases with data size </li></ul></ul><ul><li>Scalability: PSRS & SoE </li></ul><ul><ul><li>PSRS scales till 32 processes, SoE only till 16processes </li></ul></ul><ul><li>Redundancy : PSRS & SoE </li></ul><ul><ul><li>Increase in execution time < 25% with each replica </li></ul></ul><ul><ul><li>Centralized single threaded server: BOTTLENECK </li></ul></ul><ul><li>Failure: PSRS & SoE </li></ul><ul><ul><li>Linear decrease in execution time with increasing failures </li></ul></ul><ul><li>Integration with BOINC </li></ul><ul><ul><li>Some impact on the overall application performance </li></ul></ul>
  45. 45. OUR CONTIBUTION <ul><li>Task parallelism: Dataspace API facilitates data exchange between processes </li></ul><ul><li>Robustness: Dataspace API supports use of process replication to handle volatility </li></ul><ul><li>Pluggable: Dataspace API works well with BOINC </li></ul><ul><li>FUTURE WORK </li></ul><ul><ul><li>Testing on ordinary desktops </li></ul></ul><ul><ul><li>Dataspace Implementation: Combining In-memory & Disk </li></ul></ul><ul><ul><li>Distributed Dataspace </li></ul></ul><ul><ul><li>Non-blocking dataspace API </li></ul></ul>
  46. 46. THANK YOU

×