Performance Analysis

573 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
573
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
24
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Performance Analysis

    1. 1. Performance Analysis Necessity or Add-on in Grid Computing Michael Gerndt Technische Universität München [email_address]
    2. 2. LRR at Technische Universität München <ul><li>Chair for Computer Hardware & Organisation / Parallel Computer Architecture (Prof. A. Bode) </li></ul><ul><li>Three groups in parallel & distributed architectures </li></ul><ul><ul><li>Architectures </li></ul></ul><ul><ul><ul><li>SCI Smile project </li></ul></ul></ul><ul><ul><ul><li>DAB </li></ul></ul></ul><ul><ul><ul><li>Hotswap </li></ul></ul></ul><ul><ul><li>Tools </li></ul></ul><ul><ul><ul><li>CrossGrid </li></ul></ul></ul><ul><ul><ul><li>APART </li></ul></ul></ul><ul><ul><li>Applications </li></ul></ul><ul><ul><ul><li>CFD </li></ul></ul></ul><ul><ul><ul><li>Medicine </li></ul></ul></ul><ul><ul><ul><li>Bioinformatics </li></ul></ul></ul>
    3. 3. New Campus at Garching
    4. 4. Outline <ul><li>PA on parallel systems </li></ul><ul><li>Scenarios for PA in Grids </li></ul><ul><li>PA support in Grid projects </li></ul><ul><li>APART </li></ul>
    5. 5. Performance Analysis for Parallel Systems <ul><li>Development cycle </li></ul><ul><ul><li>Assumption: Reproducibility </li></ul></ul><ul><li>Instrumentation </li></ul><ul><ul><li>Static vs Dynamic </li></ul></ul><ul><ul><li>Source-level vs object-level </li></ul></ul><ul><li>Monitoring </li></ul><ul><ul><li>Software vs Hardware </li></ul></ul><ul><ul><li>Statistical profiles vs Event traces </li></ul></ul><ul><li>Analysis </li></ul><ul><ul><li>Source-based tools </li></ul></ul><ul><ul><li>Visualization tools </li></ul></ul><ul><ul><li>Automatic analysis tools </li></ul></ul>Coding Performance Monitoring and Analysis Production Program Tuning
    6. 6. Grid Computing <ul><li>Grids </li></ul><ul><ul><li>enable communities (“virtual organizations”) to share geographically distributed resources as they pursue common goals -- assuming the absence of… </li></ul></ul><ul><ul><ul><li>central location, </li></ul></ul></ul><ul><ul><ul><li>central control, </li></ul></ul></ul><ul><ul><ul><li>omniscience, </li></ul></ul></ul><ul><ul><ul><li>existing trust relationships. </li></ul></ul></ul><ul><ul><ul><ul><li>[Globus Tutorial] </li></ul></ul></ul></ul><ul><li>Major differences to parallel systems </li></ul><ul><ul><li>Dynamic system of resources </li></ul></ul><ul><ul><li>Large number of diverse systems </li></ul></ul><ul><ul><li>Sharing of resources </li></ul></ul><ul><ul><li>Transparent resource allocation </li></ul></ul>
    7. 7. Scenarios for Performance Monitoring and Analysis <ul><li>Post-mortem application analysis </li></ul><ul><li>Self-tuning applications </li></ul><ul><li>Grid scheduling </li></ul><ul><li>Grid management </li></ul><ul><ul><li>[GGF performance working group, DataGrid, CrossGrid] </li></ul></ul>
    8. 8. Post-Mortem Application Analysis <ul><li>Requires </li></ul><ul><ul><li>either resources with known performance characteristics (QoS) </li></ul></ul><ul><ul><li>or system-level information to assess performance data </li></ul></ul><ul><ul><li>scalability of performance tools </li></ul></ul><ul><li>Focus will be on interacting components </li></ul><ul><li>George submits job to the Grid </li></ul><ul><li>Job is executed on some resources </li></ul><ul><li>George receives performance data </li></ul><ul><li>George analyzes performance </li></ul>
    9. 9. Self-Tuning Applications <ul><li>Requires </li></ul><ul><ul><li>Integration of system and application monitoring </li></ul></ul><ul><ul><li>On-the-fly performance analysis </li></ul></ul><ul><ul><li>API for accessing monitor data (if PA by application) </li></ul></ul><ul><ul><li>Performance model and interface to steer adaptation (If PA and tuning decision by external component.) </li></ul></ul><ul><li>Chris submits job </li></ul><ul><li>Application adapts to assigned resources </li></ul><ul><li>Application starts </li></ul><ul><li>Application monitors performance and adapts to resource changes </li></ul>
    10. 10. Grid-Scheduling <ul><li>Requires </li></ul><ul><ul><li>PA of the grid application </li></ul></ul><ul><ul><li>Possibly benchmarking the application </li></ul></ul><ul><ul><li>Access to current performance capabilities of resources </li></ul></ul><ul><ul><li>Even better to predicted capabilities </li></ul></ul><ul><li>Gloria determines performance critical application properties </li></ul><ul><li>She specifies a performance model </li></ul><ul><li>Grid scheduler selects resources </li></ul><ul><li>Application is started </li></ul>
    11. 11. Grid-Management <ul><li>Requires </li></ul><ul><ul><li>PA of historical system information </li></ul></ul><ul><ul><li>Need to be done in a distributed fashion </li></ul></ul><ul><li>George claims to see bad performance since one week. </li></ul><ul><li>The helpdesk runs the Grid performance analysis software. </li></ul><ul><li>Periodical saturation of connections is detected. </li></ul>
    12. 12. New Aspect of Performance Analysis <ul><li>Transparent resource allocation </li></ul><ul><li>Dynamism in resource availability </li></ul><ul><li>Approaches in the following projects: </li></ul><ul><ul><li>Damien </li></ul></ul><ul><ul><li>Datagrid </li></ul></ul><ul><ul><li>Crossgrid </li></ul></ul><ul><ul><li>GrADS </li></ul></ul>
    13. 13. Analyzing Meta-Computing Applications <ul><li>DAMIEN (IST-25406), 5 partners </li></ul><ul><li>www.hlrs.de/organization/pds/projects/damien/ </li></ul><ul><li>Goals </li></ul><ul><ul><li>Analysis of GRID-enabled applications </li></ul></ul><ul><ul><ul><li>using MpCCI (www.mpcci.org) </li></ul></ul></ul><ul><ul><ul><li>using PACX-MPI (www.hlrs.de/organization/pds/projects/pacx-mpi) </li></ul></ul></ul><ul><ul><li>Analysis of GRID components </li></ul></ul><ul><ul><ul><li>PACX-MPI and MpCCI </li></ul></ul></ul><ul><ul><li>Extend Vampir/Vampirtrace technology </li></ul></ul>
    14. 14. MetaVampirtrace for Application Analysis GRID-MPI profiling routine ( PPACX_Send ) Native MPI GRID communication layer Compiled code ( PACX_Send ) Routine call Tracefile MetaVT wrapper ( PACX_Send ) Routine call Name shift (CPP) Application code ( MPI_Send )
    15. 15. MetaVampirtrace for GRID Component Analysis Name shift (CPP) Application code ( MPI_Send ) Tracefile MetaVT wrapper ( MPI_Send ) MPI profiling routine ( PMPI_Send ) Compiled code ( PACX_Send ) Routine call GRID-MPI layer ( PACX_Send ) Routine call TCP/IP GRID-MPI communication layer
    16. 16. MetaVampir <ul><li>General counter support </li></ul><ul><ul><li>Grid component metrics </li></ul></ul><ul><li>Hierarchical analysis </li></ul><ul><ul><li>Analysis at each level </li></ul></ul><ul><ul><li>Aggregate data for groups </li></ul></ul><ul><ul><li>Improves scalability </li></ul></ul><ul><li>Structured tracefiles </li></ul><ul><ul><li>Subdivided into frames </li></ul></ul><ul><ul><li>Stripe data across multiple files </li></ul></ul>Metacomputer Node 2 Node 1 SMP node 1 P_1 GRID–Daemons MPI processes Send Recv SMP node 2 P_n All MPI Processes P_1 P_n
    17. 17. Process Level
    18. 18. System Level
    19. 19. Grid Monitoring Architecture <ul><li>Developed by GGF Performance working group </li></ul><ul><li>Separation of data discovery and data transfer </li></ul><ul><ul><li>Data discovery via (possibly distributed) directory service </li></ul></ul><ul><ul><li>Data transfer among producer – consumer </li></ul></ul><ul><li>GMA interactions </li></ul><ul><ul><li>Publish/subscribe </li></ul></ul><ul><ul><li>Query/response </li></ul></ul><ul><ul><li>Notification </li></ul></ul><ul><li>Directory includes </li></ul><ul><ul><li>Types of events </li></ul></ul><ul><ul><li>Accepted protocols </li></ul></ul><ul><ul><li>Security mechanisms </li></ul></ul>Consumer Producer Directory Service event publication information event publication information
    20. 20. R-GMA in DataGrid <ul><li>DataGrid www.eu-datagrid.org </li></ul><ul><li>R-GMA www.cs.nwu.edu/~rgis </li></ul><ul><li>DataGrid WP3 hepunx.rl.ac.uk/edg/wp3 </li></ul><ul><li>Relational approach to GMA </li></ul><ul><ul><li>Producers announce: SQL “CREATE TABLE” publish: SQL “INSERT” </li></ul></ul><ul><ul><li>Consumers collect: SQL “SELECT” </li></ul></ul><ul><ul><li>Approach to use the relational model in a distributed environment </li></ul></ul><ul><ul><li>It can be used for information service as well as system and application monitoring. </li></ul></ul>
    21. 21. P-Grade and R-GMA <ul><li>P-GRADE Environment developed at MTA SZTAKI </li></ul><ul><ul><li>GRM (Distributed monitor) </li></ul></ul><ul><ul><li>Prove (Visualization tool) </li></ul></ul><ul><li>GRM creates two tables in R-GMA </li></ul><ul><ul><li>GRMTrace (String appName, String event): all events </li></ul></ul><ul><ul><li>GRMHeader (String appName, String event): important header events only </li></ul></ul><ul><li>GRM Main Monitor </li></ul><ul><ul><li>SELECT “*” FROM GRMHeader WHERE appName=“...” </li></ul></ul><ul><ul><li>SELECT “*” FROM GRMTrace WHERE appName=“...” </li></ul></ul>
    22. 22. Main Monitor Site User’s Host Host 1 Host 2 Application Process Appl. Process Appl. Process R-GMA PROVE Connection to R-GMA
    23. 23. Analyzing Interactive Applications in CrossGrid <ul><li>CrossGrid funded by EU: 03/2002 – 02/2005 </li></ul><ul><li>www.eu-crossgrid.org </li></ul><ul><li>Simulation of vascular blood flow </li></ul><ul><ul><li>Interactive visualization and simulation </li></ul></ul><ul><ul><ul><li>response times are critical </li></ul></ul></ul><ul><ul><ul><li>0.1 sec (head movement) to 5 min (change in simulation) </li></ul></ul></ul><ul><ul><li>Performance analysis </li></ul></ul><ul><ul><ul><li>response time and its breakdown </li></ul></ul></ul><ul><ul><ul><li>performance data for specific interactions </li></ul></ul></ul>
    24. 24. CrossGrid Application Monitoring Architecture <ul><li>OCM-G = Grid-enabled OMIS-Compliant Monitor </li></ul><ul><li>OMIS = On-line Monitoring Interface Specification </li></ul><ul><li>Application-oriented </li></ul><ul><ul><li>Information about running applications </li></ul></ul><ul><li>On-line </li></ul><ul><ul><li>Information collected at runtime </li></ul></ul><ul><ul><li>Immediately delivered to consumers </li></ul></ul><ul><li>Information collected via instrumentation </li></ul><ul><ul><li>Activated / deactivated on demand </li></ul></ul><ul><ul><li>Information of interest defined at runtime (lower overhead) </li></ul></ul>
    25. 25. OMIS Performance Tool Service Manager LM LM LM P3 th_stop(Sim) P1 P2 P4 P5 th_stop(P1,P2) th_stop(P4,P5) th_stop(P3) Stop Stop Stop Stop Stop
    26. 26. G-PM
    27. 27. Application Specific Measurement <ul><li>G-PM offers standard metrics </li></ul><ul><ul><li>CPU time, communication time, disk I/O, ... </li></ul></ul><ul><li>Application programmer provides </li></ul><ul><ul><li>Relevant events inside application ( probes ) </li></ul></ul><ul><ul><li>Relevant data computed by the application </li></ul></ul><ul><ul><li>Association between events in different processes </li></ul></ul><ul><li>G-PM allows to define new metrics </li></ul><ul><ul><li>Based on existing ones and application specific information </li></ul></ul><ul><ul><li>Metric Definition Language under development </li></ul></ul><ul><ul><li>Compilation or interpretation will be done by High-Level Analysis Component . </li></ul></ul>
    28. 28. Managing Dynamism: The GrADS Approach <ul><li>GrADS (Grid Application Development Software) </li></ul><ul><ul><li>Funded by National Science Foundation, started 2000 </li></ul></ul><ul><li>Goal: </li></ul><ul><ul><li>Provide application development technologies that make it easy to construct and execute applications with reliable [and often high] performance in the constantly-changing environment of the Grid. </li></ul></ul><ul><li>Major techniques to handle transparency and dynamism: </li></ul><ul><ul><li>Dynamic configuration to available resources (configurable object programs) </li></ul></ul><ul><ul><li>Performance contracts and dynamic reconfiguration </li></ul></ul>
    29. 29. GrADS Software Architecture P S E Config. object program whole program compiler Source appli- cation libraries Realtime perf monitor Dynamic optimizer Grid runtime System (Globus) negotiation Software Components Scheduler/ Service Negotiator Performance feedback Program Preparation System Execution Environment
    30. 30. Configurable Object Programs <ul><li>Integrated mapping strategy and cost model </li></ul><ul><li>Performance enhanced by context-depend. variants </li></ul><ul><li>Context includes potential execution platforms </li></ul><ul><li>Dynamic Optimizer performs final binding </li></ul><ul><ul><li>Implements mapping strategy </li></ul></ul><ul><ul><li>Chooses machine-specific variants </li></ul></ul><ul><ul><li>Inserts sensors and actuators </li></ul></ul><ul><ul><li>Perform final compilation and optimization </li></ul></ul>
    31. 31. Performance Contracts <ul><li>A performance contract specifies the measurable performance of a grid application. </li></ul><ul><li>Given </li></ul><ul><ul><li>set of resources, </li></ul></ul><ul><ul><li>capabilities of resources, </li></ul></ul><ul><ul><li>problem parameters </li></ul></ul><ul><li>the application will </li></ul><ul><ul><li>achieve a specified, measurable performance </li></ul></ul>
    32. 32. Creation of Performance Contracts Program Performance Model Resource Broker Resource Assignment Performance Contract <ul><li>Developer </li></ul><ul><li>Compiler </li></ul><ul><li>Measurements </li></ul>MDS NWS
    33. 33. History-Based Contracts <ul><li>Resources given by broker </li></ul><ul><li>Capabilities of resources given by </li></ul><ul><ul><li>Measurements of this code on those resources </li></ul></ul><ul><ul><li>Possibly scaled by the Network Weather Service </li></ul></ul><ul><ul><li>e.g. Flops/second and Bytes/second </li></ul></ul><ul><li>Problem parameters </li></ul><ul><ul><li>Given by the input data set </li></ul></ul><ul><li>Application intrinsic parameters </li></ul><ul><ul><li>Independent of execution platform </li></ul></ul><ul><ul><li>Measurements of this code with same problem parameters </li></ul></ul><ul><ul><li>e.g. floating point operation count, message count, message bytes count </li></ul></ul><ul><li>Measurable Performance Prediction </li></ul><ul><ul><li>Combining application parameters and resource capabilities </li></ul></ul>
    34. 34. Application and System Space Signature <ul><li>Application Signature </li></ul><ul><ul><li>trajectory of values through N-dimensional metric space </li></ul></ul><ul><ul><li>one trajectory per process </li></ul></ul><ul><ul><li>e.g. one point per iteration </li></ul></ul><ul><ul><li>e.g. metric: iterations/flop </li></ul></ul><ul><li>System Signature </li></ul><ul><ul><li>trajectory of values through N-dimensional metric space </li></ul></ul><ul><ul><li>will vary across application executions, even on the same resources </li></ul></ul><ul><ul><li>e.g. metric iterations/second </li></ul></ul>resource capabilities
    35. 35. Verification of Performance Contracts Execution Contract Monitor Rescheduling Sensor Data Steer Dynamic Optimizer <ul><li>Violation detection </li></ul><ul><li>Fault detection </li></ul>
    36. 36. APART <ul><li>ESPRIT IV Working Group, 01/1999 – 12/2000 </li></ul><ul><li>IST Working Group, 08/2001 – 07/2004 </li></ul><ul><li>www.fz-juelich.de/apart </li></ul><ul><li>Focus: </li></ul><ul><ul><li>Network European development projects for automatic performance analysis tools </li></ul></ul><ul><ul><ul><li>Testsuite for automatic analysis tools </li></ul></ul></ul><ul><ul><li>Automatic Performance Analysis and Grid Computing (WP3 – Peter Kacsuk) </li></ul></ul>
    37. 37. Summary <ul><li>Scenarios </li></ul><ul><ul><li>Post-mortem Application Tuning </li></ul></ul><ul><ul><li>Self-tuning applications </li></ul></ul><ul><ul><li>Grid scheduling </li></ul></ul><ul><ul><li>Grid management </li></ul></ul><ul><li>How to handle transparency and dynamism? </li></ul><ul><li>Approaches here: </li></ul><ul><ul><li>Damien: Provide static environment. </li></ul></ul><ul><ul><li>Datagrid: Combining system and application monitoring </li></ul></ul><ul><ul><li>Crossgrid: On-line analysis </li></ul></ul><ul><ul><li>GrADS: Performance models and contracts </li></ul></ul>

    ×