The document outlines an agile performance test methodology and tools developed by PushToTest, focusing on open source test automation for modern software development. It discusses the benefits of repurposing tests as business assets and highlights the importance of scalability and collaboration in testing environments. Additionally, it provides insights into various testing techniques, tools, and resources available for developers and organizations.

1. The Agile Performance Test Methodology and Tools
Agile Open Source Performance Testing
A Workshop For Developers, Testers, IT Ops
Frank Cohen, +01 (408) 871-0122, fcohen@pushtotest.com
January 5, 2012

2. Open Source Test Automation
About PushToTest
‣ Our Mission: Test, To Make A More Reliable World
‣ Make It Easier To Move From Manual To Automated Testing
‣ Support Advanced Testing, including Agile, Load, Integration
‣ Open Source Testing (OST) For Everyone
2

3. Open Source Test Automation
When OST Is Your Best Choice
‣ Modern Agile Software Development Techniques
‣ Highly-Usable, Highly-Functional Applications for Web Deployment
‣ Complex Data, Sophisticated User Interaction,
Application Business Logic (Workflow)
3
http://www.pushtotest.com/10-ways-to-build-and-test-rich-internet-applications-ria.html

4. Open Source Test Automation
Agile Repurposing
‣ Agile Delivers More Projects Faster
‣ Requires Repurposing Between Teams
‣ Treats Tests As Business Asset (Version Control, Sharing, Reuse)
Business Requirements to Launch
Biz Integration Test and
SLA Compliance
IT
Dev Functional Tests Repurposed As
Test
Load and Performance Tests
4

5. Open Source Test Automation
OST Architecture
Application Under Test PTTMonitor
Web, RIA, BPM, SOA CPU, Net, Memory
Functional
Test Injector Test Authoring
Test
Bundles ScriptRunners Data Production Libraries SoapUI (Pro)
MySQL, HtmlUnit Sahi, Selenium, SoapUI, Java csv files, rdbms queries Sahi (Pro)
JUnit Class Framework
Load Test Eclipse Java IDE
Fitness
Test Operation Console Languages: Java, Python
Wiki Collaboration
Production Designer Grid and Cloud Repository Results Analysis
Record/Playback Test Deployment MySQL RDBMS Business Intelligence Tool
Monitor
Continuous Test
Repository Bug Tracker Monitoring
Integration Management
5

6. Open Source Test Automation
Component Approach
TestScenario Component Application
Log-In
Load Test
100 Users 500 Users 1000 Users setUp
Data runTest
Test Use Case
Database Production
tearDown
Development RDBMS DPL Log-In
Database CSV DPL Product Search Component
QA Regression Order Product
Custom DPL
Database Confirm Order Component
Load Test
Log-Out
Database
Production
6

7. Open Source Test Automation
Repurposing
7

8. Open Source Test Automation
Test Authoring and Scale
Approach Advantages and Disadvantages Scale
1000+ VUs per TestNode
Protocol Level Testing Script-level coding 1000+ VUs
Extra code for JavaScript
1000+ VUs per TestNode
HTTP Archive (HAR) Script-less Test Authoring 1000+ VUs
Automatic for JavaScript
Record/Playback
SeleniumHtmlUnit 20-200 VUs
Operates client-side logic
Same Test Script
Browser Functional Testing Visual Test Debugging 2-4 VUs
Slow Test Operation
8

9. Open Source Test Automation
TestMaker Demonstration
‣ Fun And Easy Test Script Creation
‣ Selenium and TestMaker
‣ Repurposing A Single Test
‣ Functional Test, Load and Performance Test, Production Monitor
‣ Results Analysis
‣ Chart Engine
9

10. Open Source Test Automation
Next let’s talk about
Calibration Testing
‣ Test Environment Scalability
‣ Early Look At Results
10

11. Open Source Test Automation
Calibration Testing
11

12. Open Source Test Automation
Expensive For Time
12

13. Open Source Test Automation
Start In The Middle
13

14. Open Source Test Automation
Work Down or Up
14

15. Open Source Test Automation
Load Testing Timing
‣ TestScenario Options Tab
‣ Ramp up speed
‣ Sleep Time
15

16. Open Source Test Automation
Individuals and Organizations
For the individual tester needing a For organizations running large scale
no-cost test tool to test Web and business-critical Web, RIA, SOA, BPM
RIA for functionality, load and and needing world-class tools,
performance, and monitoring. methodology, training, and support.
‣ Functional Testing with TestMaker Object Designer ‣ Load Testing to Millions of Virtual Users
‣ Repurposing Selenium, soapUI Sahi and unit tests into ‣ Cloud and Grid Test Deployment
‣
data-driven functional tests, load and performance
tests, and business service monitors Collaborative Test Object Repository
‣ Load Testing To 50 Virtual Users ‣ Data-driven tests, RDBMS Data Production Libraries
‣ Data-driven tests using CSV files. ‣ Collaborative Results Analysis Repository
‣ 25 Charts and Reports ‣ Flex/Flash 3.0, 4.0, AMF
‣ Community Support and access to all documentation ‣ 1000+ Charts and Reports
16
‣ Support, QuickStart, Training, Consulting

17. Open Source Test Automation
Avoid Retreats
‣ Load Test Shows: 3,230 Success, 12 Failures
‣ Avoid The Urge For Perfection
‣ Even TheTo Functional Testing
Go Back
Best Managers Sometimes
‣ Even Failed Transactions Generate Load
‣ Sometimes the whole point is...
17

18. Open Source Test Automation
18

19. Open Source Test Automation
Alex: load test is running fine
Inga: Coolio!
Alex: Uh oh
Alex: Success: 3210, Failure: 15
Alex: Success: 3210, Failure: 204
Alex: Success: 3210, Failure: 483
Inga: What’s up?
Alex: The whole application failed
Inga: Yep, avail threads = 0
Alex: Back to the drawing board
19

20. Open Source Test Automation
Fear Not
We get paid to break things.
When you break it you understand it.
When you understand it you can fix it.
20

21. Open Source Test Automation
Slides on Twitter
‣ http://twitter.com/fcohen
‣ Please retweet
‣ Please follow me
‣ #ost
21

22. Open Source Test Automation
Where To Go From Here
‣ Watch A Screencast and Tutorial
‣ http://www.pushtotest.com/screencastcentral
‣ Attend An Open Source Test Workshop
‣ http://workshop.pushtotest.com
‣ Ask For A Proposal on Licenses, Training, Support, Consulting
‣ Call +01 408 871 0122, sales@pushtotest.com
22