4 HPA Examples Of Vampir Usage

1,351 views

Published on

Published in: Education, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,351
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

4 HPA Examples Of Vampir Usage

  1. 1. What Is It Good For?! Real World Examples of Vampir Usage at IU Robert Henschel rhensche@indiana.edu Thanks to Scott Teige, Don Berry, Scott Michael, Huian Li, Judy Qiu for providing traces. May 2009
  2. 2. Contents • IPP – Repeated Execution of an Application • Runtime Anomaly on the iDataPlex (Tachyon) • Cell B.E. Tracing • Particle Swarm Optimizer – Memory Tracing • Finding an I/O Bottleneck (EMAN) • Cluster Challenge – WPP, GAMESS • Tracing on Windows • Swarp – Pthread Tracing • MD Simulation Robert Henschel
  3. 3. What is this all about • Provide a feeling for what Vampir can be used for • Show a few of Vampirs features • Raise the awareness that tracing software is a complex undertaking Robert Henschel
  4. 4. IPP - Repeated Execution of an Application • Image analysis pipeline • Every module/binary is called a few hundred times during a single run • Setting VT_FILE_UNIQUE=yes allows for normal execution of the pipeline, while traces are produced • We ended up with 640 traces, that we did not want to look at • otfprofile was used in bash script to retrieve the profile of every trace, and then combine to a profile of the entire run Robert Henschel
  5. 5. IPP - Repeated Execution of an Application ppImage_103.0.def.z ppImage_103.0.marker.z ppImage_103.1.events.z ppImage_103.otf ppImage_103.otf_collop.csv ppImage_103.otf_data.csv ppImage_103.otf_func.csv ppImage_103.otf_p2p.csv ppImage_103.otf_result.aux ppImage_103.otf_result.dvi ppImage_103.otf_result.log ppImage_103.otf_result.ps ppImage_103.otf_result.tex Robert Henschel
  6. 6. IPP - Repeated Execution of an Application Robert Henschel
  7. 7. IPP - Repeated Execution of an Application Robert Henschel
  8. 8. IPP - Repeated Execution of an Application Robert Henschel
  9. 9. IPP - Repeated Execution of an Application Robert Henschel
  10. 10. IPP - Repeated Execution of an Application Robert Henschel
  11. 11. IPP - Repeated Execution of an Application Robert Henschel
  12. 12. IPP - Repeated Execution of an Application Robert Henschel
  13. 13. IPP - Repeated Execution of an Application Robert Henschel
  14. 14. IPP - Repeated Execution of an Application Robert Henschel
  15. 15. IPP - Repeated Execution of an Application Robert Henschel
  16. 16. IPP - Repeated Execution of an Application Robert Henschel
  17. 17. IPP - Repeated Execution of an Application Robert Henschel
  18. 18. IBM iDataPlex Testing • We are testing an IBM iDataPlex system – 84 dual quad-core node cluster – Infiniband interconnect • Tachyon, a part of the SPEC MPI2007 benchmark suite had a strange runtime behavior – 100 vs. 200 seconds • Tracing the application, made the problem go away, stable runtime at 110 seconds • There will be a deeper interaction of OpenMPI and VampirTrace in the future, OpenMPI 1.3 now contains VampirTrace Robert Henschel
  19. 19. IBM iDataPlex Testing Robert Henschel
  20. 20. IBM iDataPlex Testing Robert Henschel
  21. 21. IBM iDataPlex Testing Robert Henschel
  22. 22. IBM iDataPlex Testing Robert Henschel
  23. 23. IBM iDataPlex Testing Robert Henschel
  24. 24. Cell B.E. Tracing • Code exists as serial version, MPI version and as Cell B.E. version • Great to be able to use the same tracing tool on both platforms and for different parallelization strategies • We used a beta version of VampirTrace for Cell B.E. Robert Henschel
  25. 25. Cell B.E. Tracing Robert Henschel
  26. 26. Cell B.E. Tracing Robert Henschel
  27. 27. Cell B.E. Tracing Robert Henschel
  28. 28. Cell B.E. Tracing Robert Henschel
  29. 29. Cell B.E. Tracing Robert Henschel
  30. 30. Cell B.E. Tracing Robert Henschel
  31. 31. Cell B.E. Tracing Robert Henschel
  32. 32. Cell B.E. Tracing Robert Henschel
  33. 33. Cell B.E. Tracing Robert Henschel
  34. 34. Cell B.E. Tracing Robert Henschel
  35. 35. Cell B.E. Tracing Robert Henschel
  36. 36. Cell B.E. Tracing Robert Henschel
  37. 37. Cell B.E. Tracing Robert Henschel
  38. 38. PSO – Memory Footprint • Determining the memory footprint of a particle swarm optimizer Robert Henschel
  39. 39. PSO – Memory Footprint Robert Henschel
  40. 40. PSO – Memory Footprint Robert Henschel
  41. 41. EMAN Tracing • Application ran very slow on regular hardware, NFS mounted file system • Application was faster on Lustre • Application was really fast on shared memory filesystem • Suspected I/O problem • However, the actual I/O problem was not visible in Vampir, as this had to do with file locking Robert Henschel
  42. 42. EMAN Tracing Robert Henschel
  43. 43. EMAN Tracing Robert Henschel
  44. 44. EMAN Tracing Robert Henschel
  45. 45. Cluster Challenge • WPP (Wave Propagation Program) • Vampir helped to understand the code (few comments, not sure what the code does, and how it does it) • PAPI counter helped to show that on X86, peak floating point performance was already achieved, thus, not much room for tuning, besides rewriting the algorithm Robert Henschel
  46. 46. Cluster Challenge Robert Henschel
  47. 47. Cluster Challenge Robert Henschel
  48. 48. Cluster Challenge Robert Henschel
  49. 49. Cluster Challenge Robert Henschel
  50. 50. Cluster Challenge Robert Henschel
  51. 51. Cluster Challenge Robert Henschel
  52. 52. Cluster Challenge Robert Henschel
  53. 53. Cluster Challenge Robert Henschel
  54. 54. Cluster Challenge • GAMESS • Problem with MPI communication, while socket communication worked just fine Robert Henschel
  55. 55. Tracing on Windows • On Windows, the trace creation is handled by MS MPI • Unfortunately, not a lot of details are provided, thus ,the traces look a bit different to traces that are created on Linux – MPI collective operations are not marked as such! • However, on Windows, you are able to trace C# application in addition to C/C++/Fortran applications – And possibly other applications, that are build on top of MS MPI Robert Henschel
  56. 56. Tracing on Windows Robert Henschel
  57. 57. Tracing on Windows Robert Henschel
  58. 58. Tracing on Windows Robert Henschel
  59. 59. Tracing on Windows Robert Henschel
  60. 60. Tracing on Windows Robert Henschel
  61. 61. Tracing on Windows Robert Henschel
  62. 62. Tracing on Windows Robert Henschel
  63. 63. Tracing on Windows Robert Henschel
  64. 64. Tracing on Windows Robert Henschel
  65. 65. Swarp - Pthread Tracing • SWARP can be run as a Pthread parallel application • Tracing of Pthread parallel applications is a new feature of the latest VampirTrace version • Enabled by default, if Pthread flag is detected on the compile and link commands, but can be forced as well • Compiling with -DVTRACE_PTHREAD and including vt_user.h in the appropriate file – Will trace the overhead of Pthread functions Robert Henschel
  66. 66. Swarp - Pthread Tracing Robert Henschel
  67. 67. Swarp - Pthread Tracing Robert Henschel
  68. 68. Swarp - Pthread Tracing Robert Henschel
  69. 69. Swarp - Pthread Tracing Robert Henschel
  70. 70. Swarp - Pthread Tracing Robert Henschel
  71. 71. MD Application, Serial, OpenMP, MPI, Hybrid • MD application, part of the research of physics professor Chuck Horowitz • Used for studying dense nuclear matter in supernovae, white dwarf and neutron stars • In production use on a daily basis • Exists as serial, MPI, OpenMP, Hybrid and MDGRAPE-2 application • MDGRAPE-2 tracing is not supported out of the box, but could be implemented using manual instrumentation Robert Henschel
  72. 72. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  73. 73. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  74. 74. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  75. 75. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  76. 76. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  77. 77. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  78. 78. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  79. 79. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  80. 80. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  81. 81. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  82. 82. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  83. 83. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  84. 84. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  85. 85. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  86. 86. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel
  87. 87. MD Application, Serial, OpenMP, MPI, Hybrid Robert Henschel

×