Blackboard xpLor Performance Test Automation

916 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
916
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • In case you want to use a specifi
  • In case you want to use a specifi
  • Notification sent for manually triggered test, too?
  • Notification sent for manually triggered test, too?
  • Vuserworkrate: The number of HTTP requests each virtual user processed per minute during the test.
  • The Cumulative Response Time's Rate of Change charts the rate that HTTP requests are accumulating response time. When the rate increases, it's an indication that overall system performance got worse. Therefore, we want the chart to look flat and low.The takeaway from this chart is that something happened in the 31-37 minute mark to make system performance become really bad.
  • Scatter plot:Number of active virtual usersResponse times
  • Blackboard xpLor Performance Test Automation

    1. 1. Automation: xpLor Performance Test
    2. 2. 2 Performance Test Automation Objectives • Enable developers to run performance tests against any specified xpLor instances to proactively create performance aware culture • Provide comprehensive reports, analysis, and visualization for xpLor performance test results that are persisted forever • Create continuous feedback of performance and script breakage against the latest xpLor builds
    3. 3. 3 Performance Test Automation Automation Types • Continuous: Runs nightly at 6pm EST against the latest successful build in Bamboo • Ad hoc: Triggered by developers against a desired xpLor instance
    4. 4. 4 Architecture Galileo + Fusion + Ansible + JMeter
    5. 5. 5 xpLor
    6. 6. 6 Ansible http://www.ansibleworks.com/ • GPL licensed software for configuration management, application deployment, and continuous delivery • Written in Python • Manages servers over SSH – doesn’t require any agents • YAML for reusable system descriptions https://github.com/blackboard/perf-test- tools/tree/develop/ansible-playbooks/xplor
    7. 7. 7 JMeter Data + Load Generation Server pool r6x64o11-pv087, r6x64o11-pv088, r6x64o11-pv089, r6x64o11-pv090, r6x64o11-pv091, r6x64o11-pv092 Data model https://github.com/blackboard/perf-test- tools/tree/develop/jmeter/xplor/config/datamodels Test scenario https://github.com/blackboard/perf-test- tools/tree/develop/jmeter/xplor/config/scenarios
    8. 8. 8 Test Results Reports and Notifications • Automated analysis + notifications • *Overview statistics • *Cumulative response time rate of change • *Vusers vs. response time – top 5,000 slowest transactions • Transactions details statistics • Continuous trend * These reports have the ability to view results in categories of all, realtime, or long-running transaction types
    9. 9. 9 Test Results Automated Analysis + Email Notifications • High transaction response time: 90th percentile = 3 seconds • Low script quality: HTTP 500 = 50 requests; HTTP 400 = 75 requests; failure rate > 1% • Top 5 Transaction Response Time Degradation with 20% threshold • 100% failed transaction(s) found • Uneven app load distribution • Crashed JMeter generator
    10. 10. 10 Test Results Automated Analysis + Email Notifications
    11. 11. 11 Test Results Overview Statistics
    12. 12. 12 Test Results Cumulative response time rate of change
    13. 13. 13 Test Results Vusers vs. Response Time – Top 5,000 Slowest Transactions
    14. 14. 14 Test Results Transaction Details Statistics
    15. 15. 15 Test Results Navigating to Your Test Results • From process ID (PID) link in email notification • Galileo “Browse” tab
    16. 16. 16 Test Results Customizing Your Test Results Dashboard
    17. 17. 17 Test Results Continuous Trend
    18. 18. 18 Fusion – Task Scheduler Template - http://tinyurl.com/perftest-xplor
    19. 19. 19 Fusion – Task Scheduler How to run test against your server 1. Go to Fusion tab on Galileo (http://galileo.pd.local) 2. Create “New Fusion Process” 1. Project: jmeter 2. Process: My Test 3. Click on the star icon to create a Fusion task from a template 4. Choose the group icon to select “xpLor – Ad hoc Test” template 5. Modify PID variables accordingly (app.server.hosts, app.server.ports, file.server.hosts, file.server.ports, loadtest.duration)
    20. 20. 20 Fusion – Task Scheduler How to run test against your server xpLor Configuration Requirements: • Set isTesting to true • Set exports.development to false; • Set exports.timestampExpirySeconds to 99999 (We don’t run ping requests in JMeter to keep sessions live)
    21. 21. 21 Zabbix Monitoring Templates
    22. 22. 22 Next Steps What’s coming? • Instructional videos for running ad hoc testing • Institute performance aware culture • Shared ownership of JMeter scripts • JMeter training

    ×