0
Session ID: BTOT-TU-1700/6 <br />Twitter hashtag #HPSWU<br />
Speaker Name: Martin Moser, PhD<br />Date: 30.11.2010<br />Session ID: BTOT-TU-1700/6<br />SAP ® Best Practices for Perfor...
3<br />HP Confidential<br />Software Performance & Speed Bumps<br />
4<br />HP Confidential<br />Software Performance & Speed Bumps<br />Expectation and observation regarding response time<br...
Removing the Speed Bumps<br />How Agile Methods Help <br />
4 week sprint run without (management) interference<br />Sprints must deliver software that satisfy quality requirements<b...
SAP Best Practices<br />Measurement Methodology and Process <br />
8<br />HP Confidential<br />SAP Performance Map<br />Full picture of system performance<br /><ul><li>KPIs for all potentia...
Software architecture dependant
Measured in two-stage process</li></ul>Full performance map includes front-end<br />Server-only performance map excludes f...
9<br />HP Confidential<br />SAP Performance Test Process<br />Analysis<br />Reportgeneration<br />Test execution <br />Sys...
Average resource consumption of the scenario
For multiple users accessing the server in parallel
Focus on scalability and stability of the system</li></ul> Detect ~20% of all performance issues (mostly difficult issues...
Resource consumption per dialog step
For one single user using the server exclusively
Focus on end-to-end response time</li></ul> Detect ~80% of all performance issues (mostly simple issues)<br />
10<br />HP Confidential<br />Stage 1: Single-Users Tests<br />The SUPA tool<br /><ul><li>Generic UI
Plug-in based server
Plug-ins collect full performance map:
Upcoming SlideShare
Loading in...5
×

SAP Best Practices for Performance Testing

7,538

Published on

How SAP uses HP LoadRunner and HP QuickTime Professional for performance measurements

1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total Views
7,538
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
261
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "SAP Best Practices for Performance Testing"

  1. 1. Session ID: BTOT-TU-1700/6 <br />Twitter hashtag #HPSWU<br />
  2. 2. Speaker Name: Martin Moser, PhD<br />Date: 30.11.2010<br />Session ID: BTOT-TU-1700/6<br />SAP ® Best Practices for Performance Testing<br />How SAP ® uses HP LoadRunner ® and HP QuickTime Professional ® for performance measurements<br />
  3. 3. 3<br />HP Confidential<br />Software Performance & Speed Bumps<br />
  4. 4. 4<br />HP Confidential<br />Software Performance & Speed Bumps<br />Expectation and observation regarding response time<br />Response time<br />Months/Release<br />
  5. 5. Removing the Speed Bumps<br />How Agile Methods Help <br />
  6. 6. 4 week sprint run without (management) interference<br />Sprints must deliver software that satisfy quality requirements<br />6<br />HP Confidential<br />Agile development @ SAP<br />Cross-Functional Team<br />Daily Scrum Meeting and Artifacts Update<br />4 week sprint<br />Sprint Backlog of Tasks<br />Product <br />Owner<br />Potentially shippable product increment<br />
  7. 7. SAP Best Practices<br />Measurement Methodology and Process <br />
  8. 8. 8<br />HP Confidential<br />SAP Performance Map<br />Full picture of system performance<br /><ul><li>KPIs for all potential bottlenecks
  9. 9. Software architecture dependant
  10. 10. Measured in two-stage process</li></ul>Full performance map includes front-end<br />Server-only performance map excludes front-end<br />ABAP<br />server<br />Java server<br />Java DB<br />ABAP DB<br />
  11. 11. 9<br />HP Confidential<br />SAP Performance Test Process<br />Analysis<br />Reportgeneration<br />Test execution <br />System restart<br />LoadRunner <br />script recording<br />System check<br /> Test execution<br />System restart<br />Tool config<br />Reporting<br />System check<br />Stage 2<br />Stage 1<br />SAP performance test process<br />Stage 2: Multi-user tests<br /><ul><li>Measure server-only performance map of a scenario
  12. 12. Average resource consumption of the scenario
  13. 13. For multiple users accessing the server in parallel
  14. 14. Focus on scalability and stability of the system</li></ul> Detect ~20% of all performance issues (mostly difficult issues)<br />Stage 1: Single-user tests<br /><ul><li>Measure full performance map of a scenario
  15. 15. Resource consumption per dialog step
  16. 16. For one single user using the server exclusively
  17. 17. Focus on end-to-end response time</li></ul> Detect ~80% of all performance issues (mostly simple issues)<br />
  18. 18. 10<br />HP Confidential<br />Stage 1: Single-Users Tests<br />The SUPA tool<br /><ul><li>Generic UI
  19. 19. Plug-in based server
  20. 20. Plug-ins collect full performance map:
  21. 21. End-to-end response time
  22. 22. Garbage collection data
  23. 23. CPU consumption
  24. 24. Network traffic
  25. 25. DB load</li></ul>UI<br />Server<br />E-t-E<br />ABAP<br />server<br />Java server<br />GC<br />CPU<br />Java DB<br />ABAP DB<br />...<br />
  26. 26. 11<br />HP Confidential<br />Single-Users Tests: Follow-up<br />The SAP Performance Standard defines KPIs that are a prerequisite for good performance<br />Violations of the KPIs are identified using the SUPA tool<br />Specific tools are available to investigate the root-cause of KPI violations, among others:<br /><ul><li>SAP Java Virtual Machine Profiler
  27. 27. Memory Analyzer Tool (Open source tool sponsored by SAP)
  28. 28. ABAP transactions such as ST05, STAD, ST02</li></ul>The findings are reported back to development for fixing<br />
  29. 29. 12<br />HP Confidential<br />Stage 2: Multi-User Tests<br />HP LoadRunner used to check<br /><ul><li>Server response time for fixed number of users
  30. 30. Linear scaling of throughput with number of users
  31. 31. Stability over longer periods of time</li></ul>Follow-up as for single-user tests.<br />ABAP<br />server<br />Java server<br />Java DB<br />ABAP DB<br />
  32. 32. 13<br />HP Confidential<br />HP LoadRunner strengths and weaknesses<br />Strengths:<br /><ul><li>High load generation with off-the-shelf hardware
  33. 33. Supports most well-known technologies, among others: HTTP, Flash, Silverlight, SOAP, TCP/IP
  34. 34. Scriptable in various languages, such as: C, Java, Visual Basic
  35. 35. Extendable monitoring</li></ul>Weaknesses:<br /><ul><li>Correlation is complex (see TruClient below)
  36. 36. No monitoring for: Java garbage collection, CPU break-down & HTTP message lengths</li></li></ul><li>14<br />HP Confidential<br />SAP internal extensions to HP LoadRunner <br />Scripted extensions:<br /><ul><li>Java garbage collection
  37. 37. CPU break-down
  38. 38. HTTP Request and response lengths</li></ul>Controller<br />Monitors<br />ABAP<br />server<br />Java server<br />LR<br />Script<br />SAP monitoringroutines<br />Java DB<br />ABAP DB<br />
  39. 39. 15<br />HP Confidential<br />SAP Performance Test Process Automation<br />Analysis<br />Reportgeneration<br />Test execution <br />System restart<br />LoadRunner <br />script recording<br />System check<br /> Test execution<br />System restart<br />Tool config<br />Reporting<br />System check<br />ARIES suite (full automation)<br />NetWeaver<br />TestSuite<br />NetWeaver<br />TestSuite<br />HP QTP<br />HP QTP<br />LoadRunner Analysis<br />SUPA<br />LoadRunner Controller<br />LoadRunner VUGen<br />Stage 2<br />Stage 1<br />SAP Performance Test Process<br />Semi-automated<br />Semi-automated<br />NetWeaver<br />
  40. 40. Getting Even Better<br />Budgets and Price Tags <br />
  41. 41. 17<br />HP Confidential<br />The Way Forward to Predicting Performance<br />Typical waterfall model<br /><ul><li>Design the UI
  42. 42. Design the architecture
  43. 43. Build the system
  44. 44. Measure the performance</li></ul>Long and expensive feedback cycle.<br />
  45. 45. 18<br />HP Confidential<br />The Way Forward to Predicting Performance<br />Budget and price tags<br /><ul><li>Define resource budget for dialog steps
  46. 46. Measure resource consumption of
  47. 47. Major UI elements
  48. 48. Major backend operations
  49. 49. Attach price tags to them</li></ul>UI designers and architects can estimate performance of designs<br />
  50. 50. 19<br />HP Confidential<br />LoadRunner TruClient: First experiences @ SAP<br />Strengths:<br /><ul><li>Runs fully embedded in the browser
  51. 51. No correlation required</li></ul>Weaknesses:<br /><ul><li>Significantly higher hardware requirements
  52. 52. Instrumentation extension possible, but cumbersome</li></li></ul><li>Continue the conversation with your peers at the HP Software Community hp.com/go/swcommunity<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×