The Agile Performance Test Methodology and ToolsAgile Open Source Performance Testing    A Workshop For CIOs, CTOs, Senior...
Open Source Test AutomationSlides on Twitter‣ http://twitter.com/fcohen‣ Please retweet‣ Please follow me‣ #ost2
Open Source Test AutomationAbout PushToTest‣ Our Mission: Test, To Make A More Reliable World‣ Make It Easier To Move From...
Open Source Test AutomationWeb 2.0 Success Driving Business‣ Rich Internet Applications     •   Fast Time-To-Market     • ...
Open Source Test AutomationWhen OST Is Your Best Choice‣ Modern Agile Software Development Techniques‣ Highly-Usable, High...
Open Source Test AutomationAgile Repurposing‣ Agile Delivers More Projects Faster‣ Requires Repurposing Between Teams‣ Tre...
Open Source Test AutomationTesting Costs (TCO Model)‣ Proprietary Test Tools Offer:                               Function...
Open Source Test AutomationWhat We Have For You‣ Implement effective tests‣ Deploy tests to desktop, grid, and cloud  envi...
Open Source Test AutomationOST Architecture                                Application Under Test                         ...
Open Source Test AutomationAn OST Cloud                 Thousands of virtual users                                        ...
Open Source Test AutomationAn OST Cloud                 Thousands of virtual users                                        ...
Open Source Test AutomationComponent Approach                     TestScenario                              Component     ...
Open Source Test AutomationReady For Load Testing?‣ Selenium Core and RC Require A Browser‣ SeleniumEnablement In Firefox ...
Open Source Test AutomationTest Authoring and Scale     Approach                     Advantages and Disadvantages   Scale ...
Open Source Test AutomationRepurposing15
Open Source Test AutomationTestMaker Demonstration‣ Fun And Easy Test Script Creation  ‣ Selenium and TestMaker‣ Repurposi...
Open Source Test AutomationCalibration Testing17
Open Source Test AutomationExpensive For Time18
Open Source Test AutomationStart In The Middle19
Open Source Test AutomationWork Down or Up20
Open Source Test Automation     Next let’s talk about     PushToTest Options     ‣ OnDemand     ‣ QuickStart     ‣ POC Opt...
Open Source Test AutomationImmediate Benefits‣ Product Solutions  ‣ Better, Lower-Cost, More Efficient Test Tool  ‣ Busine...
Open Source Test AutomationIndividuals and OrganizationsFor the individual tester needing a                          For o...
Open Source Test AutomationPricing‣ Annual Subscription Model Based On:  ‣ Seats for TestMaker Console Installation  ‣ Sea...
Open Source Test Automation     ‣ As-Needed, On-Demand Basis     ‣ Test In Your Environment,       Or Our Cloud Test Envir...
Open Source Test AutomationThe QuickStart Option‣ Your First Test, Built and Operated by PushToTest Experts     ‣ Ideal Fo...
Open Source Test AutomationThe POC Option‣ PushToTest Experts Help Your TestMaker Evaluation     ‣ Ensures TestMaker Works...
Open Source Test AutomationPushToTest Global Services‣ Put Our Experts To Work For Your Projects‣ Methodology, Solutions, ...
Open Source Test AutomationAvoid Retreats‣ Load Test Shows: 3,230 Success, 12 Failures‣ Avoid The Urge For Perfection  ‣ E...
Open Source Test Automation30
Open Source Test Automation     Alex:   load test is running fine     Inga:   Coolio!     Alex:   Uh oh     Alex:   Succes...
Open Source Test Automation     Fear Not         We get paid to break things.      When you break it you understand it.   ...
Open Source Test AutomationSlides on Twitter‣ http://twitter.com/fcohen‣ Please retweet‣ Please follow me‣ #ost33
Open Source Test AutomationWhere To Go From Here‣ Evaluate Selenium, soapUI, TestMaker  ‣ http://www.pushtotest.com/produc...
Upcoming SlideShare
Loading in …5
×

Agile Open Source Performance Testing Workshop for Business Managers

1,184 views

Published on

Use Selenium, soapUI, Sahi, TestMaker Performance Testing In Your Organization. These are slides from a free Webinar titled "The Agile Open Source Way To Load Test, Scalability Test, and Stress Test In Your Organization". Your organization may have adopted Agile Software Development Methodology and forgot about load and performance testing! Learn the Open Source Testing way to load and performance test your Web applications, Rich Internet Applications (RIA, using Ajax, Flex, Flash, Oracle Forms, Applets,) and SOAP and REST Web services. This free Webinar delivers a testing methodology, tools, and best/worst practices.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,184
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agile Open Source Performance Testing Workshop for Business Managers

  1. 1. The Agile Performance Test Methodology and ToolsAgile Open Source Performance Testing A Workshop For CIOs, CTOs, Senior Managers Frank Cohen, +01 (408) 871-0122, fcohen@pushtotest.com January 4, 2011
  2. 2. Open Source Test AutomationSlides on Twitter‣ http://twitter.com/fcohen‣ Please retweet‣ Please follow me‣ #ost2
  3. 3. Open Source Test AutomationAbout PushToTest‣ Our Mission: Test, To Make A More Reliable World‣ Make It Easier To Move From Manual To Automated Testing‣ Support Advanced Testing, including Agile, Load, Integration‣ Open Source Testing (OST) For Everyone3
  4. 4. Open Source Test AutomationWeb 2.0 Success Driving Business‣ Rich Internet Applications • Fast Time-To-Market • Complex Business Workflows • Client-less Access‣ SOA Composite Applications • Faster Time-To-Market • Increase Revenue • Cost Savings‣ Service Enabled Applications • Deliver More Integration Faster4 • Less Cost
  5. 5. Open Source Test AutomationWhen OST Is Your Best Choice‣ Modern Agile Software Development Techniques‣ Highly-Usable, Highly-Functional Applications for Web Deployment‣ Complex Data, Sophisticated User Interaction, Application Business Logic (Workflow)5 http://www.pushtotest.com/10-ways-to-build-and-test-rich-internet-applications-ria.html
  6. 6. Open Source Test AutomationAgile Repurposing‣ Agile Delivers More Projects Faster‣ Requires Repurposing Between Teams‣ Treats Tests As Business Asset (Version Control, Sharing, Reuse) Business Requirements to Launch Biz Integration Test and SLA Compliance IT Dev Functional Tests Repurposed As Test Load and Performance Tests6
  7. 7. Open Source Test AutomationTesting Costs (TCO Model)‣ Proprietary Test Tools Offer: Functional Test Load Test Functional Test Platform $19,000 Load Test Platform $20,000 Functional Tests $8,000 Simulated User Licenses $120,000 Protocols: AJAX, SOAP $12,000 Recoding The Test $35,000 Support $5,000 Re-qualify The Test $12,000 $40,000 $187,000‣ QA Labs for 10,000 Users = $35,000 Per Year‣ DoesCause Analysis Consultants Root Not Include: Cost of Test Employees,‣ Does Not Guarantee Success7
  8. 8. Open Source Test AutomationWhat We Have For You‣ Implement effective tests‣ Deploy tests to desktop, grid, and cloud environments‣ A clear tutorial approach to the PushToTest methodology‣ How It All Fits Together8
  9. 9. Open Source Test AutomationOST Architecture Application Under Test PTTMonitor Web, RIA, BPM, SOA CPU, Net, MemoryFunctional Test Injector Test Authoring Test Bundles ScriptRunners Data Production Libraries SoapUI (Pro) MySQL, HtmlUnit Sahi, Selenium, SoapUI, Java csv files, rdbms queries Sahi (Pro) JUnit Class FrameworkLoad Test Eclipse Java IDE Fitness Test Operation Console Languages: Java, Python Wiki CollaborationProduction Designer Grid and Cloud Repository Results Analysis Record/Playback Test Deployment MySQL RDBMS Business Intelligence Tool Monitor Continuous Test Repository Bug Tracker Monitoring Integration Management9
  10. 10. Open Source Test AutomationAn OST Cloud Thousands of virtual users Stores results of every TestNode perform real-life scenarios on Results DB transaction, step, and the Application Under Test monitored resource PTTMonitor Application Under Test Load Injectors 400+ charts identify the root Analysis cause to performance bottlenecks, functional issues Central location from which a TestMaker tester, developer, IT manager, CI Console service runs and monitors tests10
  11. 11. Open Source Test AutomationAn OST Cloud Thousands of virtual users Stores results of every TestNode perform real-life scenarios on Results DB transaction, step, and the Application Under Test monitored resource PTTMonitor Application Under Test Load Injectors 400+ charts identify the root Analysis cause to performance bottlenecks, functional issues Central location from which a TestMaker tester, developer, IT manager, CI Console service runs and monitors tests11
  12. 12. Open Source Test AutomationComponent Approach TestScenario Component Application Log-In Load Test 100 Users 500 Users 1000 Users setUp Data runTest Test Use Case Database Production tearDown Development RDBMS DPL Log-In Database CSV DPL Product Search Component QA Regression Order Product Custom DPL Database Confirm Order Component Load Test Log-Out Database Production12
  13. 13. Open Source Test AutomationReady For Load Testing?‣ Selenium Core and RC Require A Browser‣ SeleniumEnablement In Firefox Only, Coding Required To Be Useful, No Data IDE Record ‣ TestMaker Object Designer for recording in IE‣ Selenium Grid for Functional Testing Only‣ No Reports or Results Analysis‣ No Cloud or Grid Deployment‣ No CI or QC Integration‣ RIA Testing using Ajax, Flex, Flash13
  14. 14. Open Source Test AutomationTest Authoring and Scale Approach Advantages and Disadvantages Scale 1000+ VUs per TestNode Protocol Level Testing Script-level coding 1000+ VUs Extra code for JavaScript 1000+ VUs per TestNode HTTP Archive (HAR) Script-less Test Authoring 1000+ VUs Automatic for JavaScript Record/Playback SeleniumHtmlUnit 20-200 VUs Operates client-side logic Same Test Script Browser Functional Testing Visual Test Debugging 2-4 VUs Slow Test Operation14
  15. 15. Open Source Test AutomationRepurposing15
  16. 16. Open Source Test AutomationTestMaker Demonstration‣ Fun And Easy Test Script Creation ‣ Selenium and TestMaker‣ Repurposing A Single Test ‣ Functional Test, Load and Performance Test, Production Monitor‣ Results Analysis ‣ Chart Engine16
  17. 17. Open Source Test AutomationCalibration Testing17
  18. 18. Open Source Test AutomationExpensive For Time18
  19. 19. Open Source Test AutomationStart In The Middle19
  20. 20. Open Source Test AutomationWork Down or Up20
  21. 21. Open Source Test Automation Next let’s talk about PushToTest Options ‣ OnDemand ‣ QuickStart ‣ POC Option21
  22. 22. Open Source Test AutomationImmediate Benefits‣ Product Solutions ‣ Better, Lower-Cost, More Efficient Test Tool ‣ Business Service Monitoring at the Application Tier (SLA Compliance)‣ Methodology Solutions ‣ Improved Test Methodology for my Organization and Team‣ Services ‣ Turnkey Load Testing ‣ Integrate TestMaker Into My Build Environment ‣ Add A Protocol, Chart, Task to TestMaker To Meet My Needs22
  23. 23. Open Source Test AutomationIndividuals and OrganizationsFor the individual tester needing a For organizations running large scaleno-cost test tool to test Web and business-critical Web, RIA, SOA, BPMRIA for functionality, load and and needing world-class tools,performance, and monitoring. methodology, training, and support.‣ Functional Testing with TestMaker Object Designer ‣ Load Testing to Millions of Virtual Users‣ Repurposing Selenium, soapUI Sahi and unit tests into ‣ Cloud and Grid Test Deployment ‣ data-driven functional tests, load and performance tests, and business service monitors Collaborative Test Object Repository‣ Load Testing To 50 Virtual Users ‣ Data-driven tests, RDBMS Data Production Libraries‣ Data-driven tests using CSV files. ‣ Collaborative Results Analysis Repository‣ 25 Charts and Reports ‣ Flex/Flash 3.0, 4.0, AMF‣ Community Support and access to all documentation ‣ 1000+ Charts and Reports23 ‣ Support, QuickStart, Training, Consulting
  24. 24. Open Source Test AutomationPricing‣ Annual Subscription Model Based On: ‣ Seats for TestMaker Console Installation ‣ Seats for SOAP and REST Test Authoring ‣ Virtual User Levels for Load Testing ‣ Count of Business Service Monitor ‣ Technical support: 1messagesday or faster response to emails, phone calls, instant business24
  25. 25. Open Source Test Automation ‣ As-Needed, On-Demand Basis ‣ Test In Your Environment, Or Our Cloud Test Environment, Or Both ‣ Ideal For Consulting Organizations Where Test Needs Are Unknown ‣ AMI Distribution ‣ Pay Only For What You Use ‣ Annual Cap On Total Costs25
  26. 26. Open Source Test AutomationThe QuickStart Option‣ Your First Test, Built and Operated by PushToTest Experts ‣ Ideal For Organizations With Little or No Load Testing Experience ‣ PushToTest Experts Build Your First Test ‣ PushToTest Experts Operate The Test, Provide Root Cause Analysis and Mitigation ‣ PushToTest Trains Your Team To Build Their Own Tests26 ‣ $10,000 USD
  27. 27. Open Source Test AutomationThe POC Option‣ PushToTest Experts Help Your TestMaker Evaluation ‣ Ensures TestMaker Works In Your Application/System ‣ PushToTest Issues Price Proposal ‣ Applies Engineers To Build and Operate Your Test ‣ PushToTest Trains Your Team To Use TestMaker ‣ Success of POC Confirms Intention To Purchase27 ‣ Cost: $0
  28. 28. Open Source Test AutomationPushToTest Global Services‣ Put Our Experts To Work For Your Projects‣ Methodology, Solutions, Support, Enthusiasm‣ Offices: USA (Texas and California,) Bangalore, London‣ The Near Shore Option ‣ 150 Test Experts ‣ Costa Rica: CST, English, Inexpensive, On Site in 3-5 Hours‣ http://www.pushtotest.com/services “PushToTest Uniquely Built Load Tests For Our Customers, Including Pepsi, And Worked With Our Engineers To Surface and Solve Performance Bottlenecks.” Alan Rodriguez, Director of Testing,28 Tribal DDB Dallas
  29. 29. Open Source Test AutomationAvoid Retreats‣ Load Test Shows: 3,230 Success, 12 Failures‣ Avoid The Urge For Perfection ‣ Even TheTo Functional Testing Go Back Best Managers Sometimes ‣ Even Failed Transactions Generate Load‣ Sometimes the whole point is...29
  30. 30. Open Source Test Automation30
  31. 31. Open Source Test Automation Alex: load test is running fine Inga: Coolio! Alex: Uh oh Alex: Success: 3210, Failure: 15 Alex: Success: 3210, Failure: 204 Alex: Success: 3210, Failure: 483 Inga: What’s up? Alex: The whole application failed Inga: Yep, avail threads = 0 Alex: Back to the drawing board31
  32. 32. Open Source Test Automation Fear Not We get paid to break things. When you break it you understand it. When you understand it you can fix it.32
  33. 33. Open Source Test AutomationSlides on Twitter‣ http://twitter.com/fcohen‣ Please retweet‣ Please follow me‣ #ost33
  34. 34. Open Source Test AutomationWhere To Go From Here‣ Evaluate Selenium, soapUI, TestMaker ‣ http://www.pushtotest.com/products-comparison‣ Watch A Screencast and Tutorial ‣ http://www.pushtotest.com/screencastcentral‣ Attend An Open Source Test Workshop ‣ http://workshop.pushtotest.com‣ Ask For A Proposal on Licenses, Training, Support, Consulting ‣ Call +01 408 871 0122, sales@pushtotest.com34

×