SAP Best Practices for Performance Testing

  • 6,461 views
Uploaded on

How SAP uses HP LoadRunner and HP QuickTime Professional for performance measurements

How SAP uses HP LoadRunner and HP QuickTime Professional for performance measurements

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
6,461
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
193
Comments
1
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Session ID: BTOT-TU-1700/6
    Twitter hashtag #HPSWU
  • 2. Speaker Name: Martin Moser, PhD
    Date: 30.11.2010
    Session ID: BTOT-TU-1700/6
    SAP ® Best Practices for Performance Testing
    How SAP ® uses HP LoadRunner ® and HP QuickTime Professional ® for performance measurements
  • 3. 3
    HP Confidential
    Software Performance & Speed Bumps
  • 4. 4
    HP Confidential
    Software Performance & Speed Bumps
    Expectation and observation regarding response time
    Response time
    Months/Release
  • 5. Removing the Speed Bumps
    How Agile Methods Help
  • 6. 4 week sprint run without (management) interference
    Sprints must deliver software that satisfy quality requirements
    6
    HP Confidential
    Agile development @ SAP
    Cross-Functional Team
    Daily Scrum Meeting and Artifacts Update
    4 week sprint
    Sprint Backlog of Tasks
    Product
    Owner
    Potentially shippable product increment
  • 7. SAP Best Practices
    Measurement Methodology and Process
  • 8. 8
    HP Confidential
    SAP Performance Map
    Full picture of system performance
    • KPIs for all potential bottlenecks
    • 9. Software architecture dependant
    • 10. Measured in two-stage process
    Full performance map includes front-end
    Server-only performance map excludes front-end
    ABAP
    server
    Java server
    Java DB
    ABAP DB
  • 11. 9
    HP Confidential
    SAP Performance Test Process
    Analysis
    Reportgeneration
    Test execution
    System restart
    LoadRunner
    script recording
    System check
    Test execution
    System restart
    Tool config
    Reporting
    System check
    Stage 2
    Stage 1
    SAP performance test process
    Stage 2: Multi-user tests
    • Measure server-only performance map of a scenario
    • 12. Average resource consumption of the scenario
    • 13. For multiple users accessing the server in parallel
    • 14. Focus on scalability and stability of the system
     Detect ~20% of all performance issues (mostly difficult issues)
    Stage 1: Single-user tests
    • Measure full performance map of a scenario
    • 15. Resource consumption per dialog step
    • 16. For one single user using the server exclusively
    • 17. Focus on end-to-end response time
     Detect ~80% of all performance issues (mostly simple issues)
  • 18. 10
    HP Confidential
    Stage 1: Single-Users Tests
    The SUPA tool
    • Generic UI
    • 19. Plug-in based server
    • 20. Plug-ins collect full performance map:
    • 21. End-to-end response time
    • 22. Garbage collection data
    • 23. CPU consumption
    • 24. Network traffic
    • 25. DB load
    UI
    Server
    E-t-E
    ABAP
    server
    Java server
    GC
    CPU
    Java DB
    ABAP DB
    ...
  • 26. 11
    HP Confidential
    Single-Users Tests: Follow-up
    The SAP Performance Standard defines KPIs that are a prerequisite for good performance
    Violations of the KPIs are identified using the SUPA tool
    Specific tools are available to investigate the root-cause of KPI violations, among others:
    • SAP Java Virtual Machine Profiler
    • 27. Memory Analyzer Tool (Open source tool sponsored by SAP)
    • 28. ABAP transactions such as ST05, STAD, ST02
    The findings are reported back to development for fixing
  • 29. 12
    HP Confidential
    Stage 2: Multi-User Tests
    HP LoadRunner used to check
    • Server response time for fixed number of users
    • 30. Linear scaling of throughput with number of users
    • 31. Stability over longer periods of time
    Follow-up as for single-user tests.
    ABAP
    server
    Java server
    Java DB
    ABAP DB
  • 32. 13
    HP Confidential
    HP LoadRunner strengths and weaknesses
    Strengths:
    • High load generation with off-the-shelf hardware
    • 33. Supports most well-known technologies, among others: HTTP, Flash, Silverlight, SOAP, TCP/IP
    • 34. Scriptable in various languages, such as: C, Java, Visual Basic
    • 35. Extendable monitoring
    Weaknesses:
    • Correlation is complex (see TruClient below)
    • 36. No monitoring for: Java garbage collection, CPU break-down & HTTP message lengths
  • 14
    HP Confidential
    SAP internal extensions to HP LoadRunner
    Scripted extensions:
    • Java garbage collection
    • 37. CPU break-down
    • 38. HTTP Request and response lengths
    Controller
    Monitors
    ABAP
    server
    Java server
    LR
    Script
    SAP monitoringroutines
    Java DB
    ABAP DB
  • 39. 15
    HP Confidential
    SAP Performance Test Process Automation
    Analysis
    Reportgeneration
    Test execution
    System restart
    LoadRunner
    script recording
    System check
    Test execution
    System restart
    Tool config
    Reporting
    System check
    ARIES suite (full automation)
    NetWeaver
    TestSuite
    NetWeaver
    TestSuite
    HP QTP
    HP QTP
    LoadRunner Analysis
    SUPA
    LoadRunner Controller
    LoadRunner VUGen
    Stage 2
    Stage 1
    SAP Performance Test Process
    Semi-automated
    Semi-automated
    NetWeaver
  • 40. Getting Even Better
    Budgets and Price Tags
  • 41. 17
    HP Confidential
    The Way Forward to Predicting Performance
    Typical waterfall model
    • Design the UI
    • 42. Design the architecture
    • 43. Build the system
    • 44. Measure the performance
    Long and expensive feedback cycle.
  • 45. 18
    HP Confidential
    The Way Forward to Predicting Performance
    Budget and price tags
    • Define resource budget for dialog steps
    • 46. Measure resource consumption of
    • 47. Major UI elements
    • 48. Major backend operations
    • 49. Attach price tags to them
    UI designers and architects can estimate performance of designs
  • 50. 19
    HP Confidential
    LoadRunner TruClient: First experiences @ SAP
    Strengths:
    • Runs fully embedded in the browser
    • 51. No correlation required
    Weaknesses:
    • Significantly higher hardware requirements
    • 52. Instrumentation extension possible, but cumbersome
  • Continue the conversation with your peers at the HP Software Community hp.com/go/swcommunity