Testwarez 2009 Use Proper Tool

107 views

Published on

How testing and testers evolved within the time and benefits of testing tools during mature testing lifecycle. Presentation by NESS Technologies in Krakow, Poland in 2009

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
107
On SlideShare
0
From Embeds
0
Number of Embeds
81
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Testwarez 2009 Use Proper Tool

  1. 1. ©2009Ness Technologies – Proprietary and Confidential How testing and testers evolved within the time and benefits of testing tools during mature testing lifecycle. Testwarez 2009, Krakow Testing Lifecycle: Don’t be a fool, use a proper tool! Zdeněk Grössl & Lucie Riedlová 20. 10. 2009
  2. 2. www.ness.com Our aim: to use the mature Testing Lifecycle Testing is strategic for us Over 2.500 testers (18 countries worldwide) sharing their corporate knowledge Different modes – test labs, consulting, custom development, SAAS Proof? Satisfied customers around the world • Telco • IT • Public and Military • In CZ customers from TOP 100 Let’s see that “Lifecycle” Testing LifecycleTesting Lifecycle
  3. 3. www.ness.com What do we mean by “Lifecycle” :-)What do we mean by “Lifecycle” :-)
  4. 4. www.ness.com From random clicks to Testing LifecycleFrom random clicks to Testing Lifecycle test coordinadion progress of testing connection to management test cases RQ coverage test scripts and test data management automatization managing test team reporting about status test functionality and requirements before release bug report inefficient testing difficult to reproduce tests uncoordinatefd action feedback on test process imputs for next iteration Tools can help a lot ! How to handle this ?
  5. 5. www.ness.com Test ManagementTest Management
  6. 6. www.ness.com Testing tools – Test managementTesting tools – Test management Dynamic changes on smaller projectDynamic changes on smaller project Project: Financial sector Testing implemented from scratch, about 12 team members Dynamic development process Testing documentation in Excel sheets Sudden change: Increase of RQs 40% new people hired Issues with versions of Excel sheets Information about changes was not reliable A need to formalize testing process Solution: Tool for test Management Easy to setup Online access Suitable for smaller teams Keeps history of records SPIRA TEAM was selected as pilot Benefits: Testing was formalized History and more detailed information about testing process SPIRA extended to other projects
  7. 7. www.ness.com Testing tools – Test management – screenshotsTesting tools – Test management – screenshots Example: SPIRA TEAM Project Home My Page
  8. 8. www.ness.com Testing tools – Test managementTesting tools – Test management Too much extra work at world’s HW leaderToo much extra work at world’s HW leader Project: UAT testing of world-wide application for Sales (12.000 users) Two successive releases were tested, about 1 year between releases Role: Regional Test Coordinator (40 testers), reporting about progress Complete Worldwide team: 140 members First release: Shared Excel files Too much extra work Three different locations to login into before testing each day Reporting for management: combine excel sheets and reports from Bugzilla Take care about testers (motivate, coach, inform about bug fixes…) Extremely challenging! Second release: HP Mercury QC A pro tool: all in one; no multiple logins Simple administration of all modules User-friendly testing Easy overview of testing progress, Bug tracking and traceability Sophisticated reports for management. Benefits: More effective testing on such a large project. Clear overview for management Focus on testing, not on extra activities!
  9. 9. www.ness.com Testing tools – Test management – screenshotsTesting tools – Test management – screenshots Example: Evolution from Excel to Mercury QC
  10. 10. www.ness.com Testing tools – Test managementTesting tools – Test management SummarySummary
  11. 11. www.ness.com Test AnalysisTest Analysis
  12. 12. www.ness.com Testing tools – Test Analysis and DesignTesting tools – Test Analysis and Design We got lost in Test CasesWe got lost in Test Cases Project: Financial sector (saving bank) Deliver complete QA for new module of client’s internal system One customer, three dislocated suppliers Totally about 70 new RQs (about ¼ of the whole system) All in Excel sheets (RQ, TC, test data) Excel sheet too complex Difficult to review test coverage Poor reporting options Issues with access to files Several errors occurred Requirements on the tool Achieve and review RQ coverage easily Supports evaluation of TCs User-friendly environment Quick and easy setup Accessible worldwide SPIRA TEAM selected Benefits: Administrative / human errors avoided Great overview for test analyst Module delivered with remaining 6 minor bugs (totally over 140) Transparent for client and increase of our credit
  13. 13. www.ness.com Testing tools – Test Analysis – screenshotsTesting tools – Test Analysis – screenshots Example: SPIRA TEAM Map TC to RQ RQ coverage overview
  14. 14. www.ness.com Testing tools – Test Analysis & DesignTesting tools – Test Analysis & Design SummarySummary
  15. 15. www.ness.com Test DesignTest Design 15
  16. 16. www.ness.com Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis Regression testing under stressRegression testing under stress Project: SPIRA TEAM as testing tool on the project (Finance) Releases 8x per year, hot-fixes need to be released within 8 hours Much time spent on regression testing Need of change: Regression testing not 100% effective under stress An error was delivered to the customer More formalized regression tests were not good solution We need tool for automated testing Options: Test Complete, Win Runner Situation on project did not enable it. Good luck: our SW-coder recommended Selenium – add-on for Firefox Benefits: Future errors avoided Connection with SPIRA to automatically record test runs It is useful to discuss testing issues with developers as well
  17. 17. www.ness.com Testing tools – TestTesting tools – Test DesignDesign –– screenshotsscreenshots Example: SPIRA TEAM together with Selenium ► Selenium IDE – write and run tests in Firefox ► Selenium Remote Control – runs tests in multiple browsers and platforms (IE7, Safari, Opera, Firefox, Windows, Linux, Solaris, others)
  18. 18. www.ness.com Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis More than was requested (by SilviaMore than was requested (by Silvia ValientovValientováá)) Project: Biggest insurance company in CZ 33 methods, 6 web services were tested separately What was requested? Test the methods like a process Customer: Buy us a new testing tool! SoapUI was known tool for customer Why SoapUI The old tool has requested functionality Property Transfer and Xpath Assertion Wizard Link responses with requests Support of E2E testing Benefits: Client can now easily perform this testing No extra costs required to purchase licenses Easy upgrade from Integration testing to E2E testing We found what customer really NEEDS, not what he just REQUESTS
  19. 19. www.ness.com Testing tools – TestTesting tools – Test DesignDesign –– screenshotsscreenshots Example: The SOAP UI Story ► Used for Web Service Testing
  20. 20. www.ness.com Testing tools – Test Design and AnalysisTesting tools – Test Design and Analysis Sometimes just look around for a tool!Sometimes just look around for a tool! Project: Internetbanking system (Lead Czech saving bank) Load testing was newly required Tested system was developed by NESS Czech Requirements on the tool Simulate simultaneous access of 300 users per 10 minutes Minimum time and costs to set up a tool Restriction on client’s side Why Visual Studio 2005 (for Testers)? Tested application was programmed in this environment Developers were familiar with the tool Quick implementation and results in one week Benefits: Developers can be very good test designers Scope of our services for the client broadened
  21. 21. www.ness.com Testing tools – Test Design & AnalysisTesting tools – Test Design & Analysis SummarySummary automated testing language & environment independent connection to other tools
  22. 22. www.ness.com Test ExecutionTest Execution 22
  23. 23. www.ness.com Testing tools – Test ExecutionTesting tools – Test Execution Testing scope broadened (by Jaroslav StrharskTesting scope broadened (by Jaroslav Strharskýý)) Project: Major Telco company in CZ Excel for RQs and TCs Originally only 2 testers planned Need of change: 25 testers involved Excel files too complicated Several bugs got misrouted Testing with HP Mercury Quality Center Testers complained about difficulty of the tool and level of formalism Monitoring of users was necessary Formalized and free tests were put in place Benefits: Inform business testers about fixed bugs they discovered: they will be satisfied Training for testers takes approx. the same time regardless of a tool Testing with this tool is as quick as without a tool. But consider the benefits!
  24. 24. www.ness.com Testing tools – TestTesting tools – Test ExecutionExecution –– screenshotsscreenshots HP Mercury Quality Center
  25. 25. www.ness.com Testing tools – Test ExecutionTesting tools – Test Execution No time for new toolsNo time for new tools Project: TCs and bugs maintained in Word and Excel Communication to developers was face-to-face Bugs were fixed almost immediately Need of change: Model does not work under stress Big volume of reported bugs (+300%) Management required reporting Only 3 weeks to deliver the testing! Why JIRA from Atlassian? JIRA is used in NESS as internal tracking tool (over 750 users) Every employee is familiar with it Skillful administrators Benefits : No more misunderstandings Reports for management All set up in less than 1 day
  26. 26. www.ness.com Testing tools – TestTesting tools – Test ExecutionExecution –– screenshotsscreenshots JIRA by Atlassian
  27. 27. www.ness.com Testing tools – Test ExecutionTesting tools – Test Execution SummarySummary
  28. 28. www.ness.com Summary? There is no perfect tool! ► Every tool suits something ► If the tool is TOO complex and TOO perfect: Is it still easy to maintain? Is there anything else we should be aware of? Answer these questions about testing tools: ► Easy to install? ► Friendly to use? ► Even for non-IT users? ► Any additional features? • Notifications, • Attachments, screenshots • Reporting options • Upgrades • … ► Could you customize the tool? ► When you tried it, does it serve you well? Overall summaryOverall summary
  29. 29. www.ness.com What do we use? ► We – Lucie and Zdenek – like SPIRA (and several our customers too  ) We use tools from our partners (Atlassian, HP, IBM, Inflectra, Microsoft…) Use tool best for the purpose and for the client ► Excel? OK, but educate clients! And remember their maturity! Which tool should you use? See questions on previous slide! ► Summarize your needs Don’t you know your needs? ► Talk to someone who helps you define them Most powerful tool: OPEN MINDS! And what about testing at NESS Czech?And what about testing at NESS Czech?
  30. 30. www.ness.com Complete QA solution in one package Server and thin-client-based application Very user-friendly environment Rich reporting options Document repository Option to connect to LDAP Positive feedback from clients Even from business (non-IT) people After one year over 30 projects and over 60 external users SPIRA TEAM by InflectraSPIRA TEAM by Inflectra http://www.inflectra.com
  31. 31. www.ness.com bug tracking project management agile software development technical support and help-desk used by more than 10 000 organizations create any kind of issue in seconds keep your projects and issues organized track all activities in your project designed with both business and technical users in mind easy to administer and customize Because you’ve got issuesBecause you’ve got issues
  32. 32. www.ness.com Our question: What did you take from this presentation? ► There is no perfect tool! ► Discuss and ask questions before you decide to use certain tool! ► Ask those who know about this stuff… We are ready for your questions… … and for the chat about testing – later :-) Questions and AnswersQuestions and Answers It looks like end... but it is rather start 

×