Successfully reported this slideshow.

Software Testing

1,039 views

Published on

Software Testing

  1. 1. IBM RATIONAL TOOLS FOR TESTING Presentation: V ítek Linduška Demo: Václav Novotný
  2. 2. Agenda <ul><li>Manual Tester </li></ul><ul><li>Functional Tester </li></ul><ul><li>Performance Tester </li></ul><ul><li>ClearQuest </li></ul>
  3. 3. Manual Tester
  4. 4. Why Teams use Manual Testing? <ul><ul><li>Recruitment of subject-matter experts and end-users to test </li></ul></ul><ul><ul><li>Poor or non-existent test automation-ready interfaces </li></ul></ul><ul><ul><li>Strict regulatory or process requirements aren’t met with automation tools </li></ul></ul>
  5. 5. Challenges of Manual Testing <ul><li>Labor Intensive </li></ul><ul><ul><li>Too much time is spent creating and maintaining manual test documentation </li></ul></ul>“ More time is spent maintaining test documentation than on executing tests.” “ We struggle to document even a minimum set of tests.”
  6. 6. Challenges of Manual Testing <ul><li>Labor Intensive </li></ul><ul><ul><li>Too much time is spent creating and maintaining manual test documentation </li></ul></ul><ul><li>Unreliable </li></ul><ul><ul><li>Inadvertent human-error often skews test results </li></ul></ul>“ Our results are unreliable due to human error.” “ Our tests aren’t detailed enough for the testers to consistently execute them.”
  7. 7. Challenge 1: Labor Intensive <ul><li>Challenge </li></ul><ul><ul><li>Creating and maintaining manual tests is labor intensive and difficult with tools such as Microsoft Word or Excel. </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Implement a solution that promotes the best practice of modular tests which can share common content </li></ul></ul><ul><ul><li>Leverage existing Word or Excel test documents to quick-start test development </li></ul></ul>Share test content
  8. 8. <ul><li>Challenge </li></ul><ul><ul><li>Human error often skews tests results </li></ul></ul>Challenge 2: Unreliable <ul><li>Resolution </li></ul><ul><ul><li>Assist testers with the tasks most prone to error – data entry and data verification </li></ul></ul><ul><ul><li>Build manual tests that are clear, concise and easy for the tester to follow </li></ul></ul>
  9. 9. Leverage Existing Test Documents Import from Word or Excel <ul><ul><li>Jump-start adoption of Rational Manual Tester by leveraging existing assets </li></ul></ul><ul><ul><li>Bulk import manual tests documented in Word or Excel </li></ul></ul>Microsoft Word documents Microsoft Excel spreadsheets Rational Manual Tester test
  10. 10. Assisted Data Entry and Data Verification Reduces human error <ul><ul><li>Store critical data used during the test as part of the test instructions </li></ul></ul>Test execution window
  11. 11. Assisted Data Entry and Data Verification Reduces human error <ul><ul><li>Store critical data used during the test as part of the test instructions </li></ul></ul><ul><ul><li>Automate data entry during test execution to speed entry and reduce errors </li></ul></ul>Test execution window Application under test
  12. 12. Assisted Data Entry and Data Verification Reduces human error <ul><ul><li>Store critical data used during the test as part of the test instructions </li></ul></ul><ul><ul><li>Automate data entry during test execution to speed entry and reduce errors </li></ul></ul><ul><ul><li>Automate data validation during test execution to ensure accurate results </li></ul></ul>
  13. 13. Enables Shared Content Across Multiple Tests Reduces test maintenance and ensure consistency <ul><ul><li>Create commonly used test statements once </li></ul></ul><ul><ul><li>Reuse statements across multiple tests through simple drag and drop </li></ul></ul><ul><ul><li>Maintenance of shared content is efficient with single-point updating of shared content </li></ul></ul><ul><ul><li>Ensures consistency and reduces test maintenance </li></ul></ul>
  14. 14. On-Screen Prompted Test Execution Immediate and accurate results collection <ul><ul><li>On-screen prompting of test steps </li></ul></ul><ul><ul><li>Immediately record test results and attach images or files for clarity </li></ul></ul><ul><ul><li>Tool will assist with data entry and data verification </li></ul></ul><ul><ul><li>Results recorded in permanent log </li></ul></ul>
  15. 15. Spread-Sheet Ready Test Results Use with any 3 rd party reporting tool <ul><ul><li>Results easily exported to common file format </li></ul></ul><ul><ul><li>Import into any 3 rd party tool for evaluation and consolidation with other test data </li></ul></ul>
  16. 16. Functional Tester
  17. 17. Challenge of Manual Testing with Short Test Cycles <ul><li>Manual testing alone can’t keep pace with the rapid development of applications </li></ul><ul><ul><li>Quality will be sacrificed to keep on schedule </li></ul></ul>Rerun the first tests and... Time Tests Manual testing can’t keep pace! ...take more time ...the new tests... Build 1 Build 2 Build 3, 4, 5 …
  18. 18. Challenges of Functional Testing <ul><li>Controlling cost of test automation </li></ul><ul><ul><li>Inexperienced testers face steep learning curve for tools </li></ul></ul><ul><ul><li>Test maintenance outpaces testing </li></ul></ul>“ Our testers are not experienced in building automated tests.” “ Application UI changes are always breaking our test scripts.”
  19. 19. Challenges of Functional Testing <ul><li>Control cost of test automation </li></ul><ul><ul><li>Inexperienced testers face steep learning curve for tools </li></ul></ul><ul><ul><li>Test maintenance grows with project </li></ul></ul><ul><li>Testing Tools Lack Flexibility </li></ul><ul><ul><li>Power users hands are tied by tool limitations </li></ul></ul>“ I need one tool for both my expert and novice testers.” “ Our script language limits our ability to overcome some automation obstacles.”
  20. 20. Challenges of Functional Testing <ul><li>Control cost of test automation </li></ul><ul><ul><li>Inexperienced testers face steep learning curve for tools </li></ul></ul><ul><ul><li>Test maintenance grows with project </li></ul></ul><ul><li>Testing Tools Lack Flexibility </li></ul><ul><ul><li>Power users hands are tied by tool limitations </li></ul></ul><ul><li>Test Team Integration </li></ul><ul><ul><li>Managing script changes with parallel development </li></ul></ul><ul><ul><li>Sharing test assets with the larger software team </li></ul></ul>“ How do I keep my team in connected with the project team?” “ We’re geographically distributed and we’re worried about parallel script development .”
  21. 21. <ul><li>Challenge </li></ul><ul><ul><li>Overcome the complexity of test automation to ensure teams spend more time on high value quality assessment rather than on low return test maintenance. </li></ul></ul>Challenge 1: Controlling the cost of Automation <ul><li>Resolution </li></ul><ul><ul><li>Provide technologies to enable novice testers to craft complex tests with no programming </li></ul></ul><ul><ul><ul><li>Data driven wizard </li></ul></ul></ul><ul><ul><ul><li>Dynamic data verification </li></ul></ul></ul><ul><ul><li>Reduce script maintenance </li></ul></ul><ul><ul><ul><li>Wizard driven global updates to centralized object map </li></ul></ul></ul>
  22. 22. <ul><li>Challenge </li></ul><ul><ul><li>Fulfill both standard test needs as well as expert requirements to ensure usage limited only by the imagination of the project team. </li></ul></ul>Challenge 2: Tools Lack Flexibility <ul><li>Resolution </li></ul><ul><ul><li>Provide two test script development languages </li></ul></ul><ul><ul><ul><li>Java </li></ul></ul></ul><ul><ul><ul><li>Visual Basic .NET </li></ul></ul></ul><ul><ul><li>Give professional testers the flexibility to overcome testing challenges in the language of their choice </li></ul></ul>
  23. 23. Challenge 3: Test Team Integration <ul><li>Challenge </li></ul><ul><ul><li>Accommodate the requirements of both dedicated, independent QA teams as well as integrated project teams to ensure high operational integrity. </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Provide version control support in the box with ClearCase LT </li></ul></ul><ul><ul><li>Provide integrations for standard version control features </li></ul></ul><ul><ul><ul><li>Check In/Out </li></ul></ul></ul><ul><ul><ul><li>Compare with Previous Versions </li></ul></ul></ul><ul><ul><ul><li>Show change history </li></ul></ul></ul>
  24. 24. Reduce Test Script Maintenance Reliable Playback with Script Assure Version 1.0 Version 2.0 No User Intervention Required With ScriptAssure TM DeterminesMatch Tester Sees Tool Sees
  25. 25. Performance Tester
  26. 26. Challenges of Performance Testing <ul><li>Complex Tools & Complex Task </li></ul><ul><ul><li>Tool complexity and lack of experience intimidates many first time users </li></ul></ul>“ We don’t have a lot of experience building good load tests.” “ Most load test tools are much more complicated than we require.”
  27. 27. Challenges of Performance Testing <ul><li>Complex Tools & Complex Task </li></ul><ul><ul><li>Tool complexity and lack of experience intimidates many first time users </li></ul></ul><ul><li>Testing Tools Lack Flexibility </li></ul><ul><ul><li>Power users hands are tied by tool limitations </li></ul></ul>“ We don’t have a good understanding of what exactly is occurring during a load test.” “ We have highly technical performance testers that need to customize their tests.”
  28. 28. Challenge 1: Complex Tools & Complex Task <ul><li>Challenge </li></ul><ul><ul><li>Tool complexity and lack of experience intimidates many first time users </li></ul></ul><ul><li>Resolution </li></ul><ul><ul><li>Remove the need for programming </li></ul></ul><ul><ul><li>Tests are represented in a tree view as a sequential flow web pages </li></ul></ul><ul><ul><li>User interface enables </li></ul></ul><ul><ul><ul><li>Data pooling </li></ul></ul></ul><ul><ul><ul><li>Data correlation </li></ul></ul></ul><ul><ul><ul><li>Looping </li></ul></ul></ul><ul><ul><ul><li>Conditional events </li></ul></ul></ul>
  29. 29. <ul><li>Challenge </li></ul><ul><li>Power users hands are tied by tool limitations </li></ul>Challenge 2: Tools Lack Flexibility <ul><li>Resolution </li></ul><ul><ul><li>Tests may call custom Java code provided by the user </li></ul></ul><ul><ul><ul><li>Offers flexibility for power users to perform complex tasks </li></ul></ul></ul><ul><ul><li>Expose behind the scenes details which are hidden from novice users by default </li></ul></ul><ul><ul><ul><li>Provide the flexibility to overcome any testing obstacles </li></ul></ul></ul>
  30. 30. Performance Problem Identification During Test <ul><li>Performance Testing finds bottlenecks </li></ul><ul><ul><li>Next logical question is Why? </li></ul></ul><ul><ul><li>Root Cause Analysis provides to tools to answer this question </li></ul></ul>Page Performance Report Shows average response time per page. Highest bar = Performance Problem
  31. 31. Performance Problem Identification During Test <ul><li>Performance Testing finds bottlenecks </li></ul><ul><ul><li>Next logical question is Why? </li></ul></ul><ul><ul><li>Root Cause Analysis provides to tools to answer this question </li></ul></ul>
  32. 32. Response Time Breakdown <ul><li>Feature : </li></ul><ul><ul><li>Breakdown page response times into composite element response time </li></ul></ul><ul><li>Benefit : </li></ul><ul><ul><li>Breakdown data drills down to highlight slowest components of a given page. Breakdown into tiers and components is a unique competitive advantage for RPT. </li></ul></ul>Response data broken down by tier and by transaction component (JDBC, JSP, Servlet, etc…)
  33. 33. Deep Diagnostic Data Providing the details you need to solve the performance problem mystery UML Sequence diagram showing method calls between classes with timing information. User can drill down to source code. Execution Statistics Shows response time for every method
  34. 34. ClearQuest
  35. 35. <ul><li>Eclipse Based Test Management </li></ul><ul><ul><li>Joins Functional and Performance Tester to complete transition of testing tools to eclipse platform </li></ul></ul><ul><ul><li>Native and web clients also available </li></ul></ul><ul><li>Key Functional Areas </li></ul><ul><ul><li>Test Planning </li></ul></ul><ul><ul><ul><li>Create test plans, test cases and configured test cases </li></ul></ul></ul><ul><ul><li>Test Execution </li></ul></ul><ul><ul><ul><li>Create suites or run individual test cases </li></ul></ul></ul><ul><ul><li>Test Reporting </li></ul></ul><ul><ul><ul><li>Extensive query and reporting facilities to measure test progress </li></ul></ul></ul>Test Management with ClearQuest Eclipse, Windows and Web Clients for test project tracking
  36. 36. Lifecycle Traceability Query one source for all aspects of project progress <ul><li>Query to find </li></ul><ul><ul><li>Test results associated to requirements </li></ul></ul><ul><ul><li>Defects associated to test results </li></ul></ul><ul><ul><li>Defects associated to tests associated to requirements </li></ul></ul><ul><li>Traceability from requirements to tests and defects closes the loop on the software development cycle </li></ul>Requirements Tests Defects
  37. 37. Lifecycle Traceability Query one source for all aspects of project progress <ul><li>Leverage existing reports </li></ul><ul><li>or Create new reports with Crystal Reports </li></ul><ul><li>To provide the information you need for decision making </li></ul>Test Case Trending Test Verdicts
  38. 38. ClearQuest Web Interface Distributed access for test planning and reporting <ul><li>Instant access for remote users </li></ul><ul><li>Access for alternative platforms </li></ul><ul><li>No additional licenses required </li></ul><ul><li>Perfect for test planning and reporting </li></ul>
  39. 39. ClearQuest MultiSite Replication and synchronization for geographically distributed teams <ul><li>Automatic replication and synchronization of ClearQuest repositories </li></ul><ul><ul><li>Access current information, regardless of location </li></ul></ul><ul><ul><li>Automatic recovery of repositories in the event of system failure maintains data integrity </li></ul></ul><ul><ul><li>Transmission of only incremental changes for improved network performance </li></ul></ul>`
  40. 40. Centralized User Management LDAP authentication for common user id support <ul><li>Reduce administration with centralized user management – maintain only one list of user ids LDAP authentication supports common user ids for ClearQuest and RequisitePro </li></ul>Rational RequisitePro Authorization Rational RequisitePro Rational ClearQuest LDAP 3.0 Compliant Server Rational ClearQuest Authorization Rational ClearQuest User DB Logon Dialog LDAP
  41. 41. Compliance and Control Using Audit Trails and eSignatures to track change <ul><li>Audit Trails can track changes to individual records </li></ul><ul><ul><li>Provides immutable change log of all record change activity </li></ul></ul><ul><li>e-Signature requirement can be added to any event on any artifact </li></ul><ul><ul><li>Will record who changed a record, when and how the record was changed </li></ul></ul><ul><ul><li>e-Signature provides immutable evidence of process completion </li></ul></ul>Validate Results Test Execution Test Case Requirement Approve e-Signature Approval
  42. 42. Configure and Enforce Your Process Customize appearance and behavior of all forms and processes <ul><li>Customize everything </li></ul><ul><ul><li>Modify forms </li></ul></ul><ul><ul><li>Configure State Transition </li></ul></ul><ul><ul><li>Add fields and control behavior </li></ul></ul><ul><ul><li>Extend your system with custom coded hooks </li></ul></ul><ul><li>Extend functionality using pre-defined Packages </li></ul><ul><ul><li>eSignatures </li></ul></ul><ul><ul><li>Audit Trail </li></ul></ul><ul><ul><li>Many, many more… </li></ul></ul>
  43. 43. Extensible Test Ecosystem Integrated with Rational, open source and 3rd party test automation tools <ul><li>Collect and report on results from Rational test execution engines </li></ul><ul><ul><li>Rational Manual Tester </li></ul></ul><ul><ul><li>Rational Functional Tester </li></ul></ul><ul><ul><li>Rational Performance Tester </li></ul></ul><ul><li>Collect and report on results from Eclipse Test and Performance Tools Project </li></ul><ul><ul><li>jUnit tests </li></ul></ul><ul><ul><li>Manual tests </li></ul></ul><ul><ul><li>URL tests </li></ul></ul><ul><li>Open API Extensibility </li></ul><ul><ul><li>SPI Dynamics QAInspect </li></ul></ul>
  44. 44. Rational Software Quality Solutions Developer Test Functional Test Automated Manual Rational RequisitePro Rational ClearQuest Rational ClearQuest Defects Project Dashboards Detailed Test Results Quality Reports Performance Test SOFTWARE QUALITY SOLUTIONS Test and Change Management Test Automation Quality Metrics DEVELOPMENT OPERATOINS BUSINESS Rational ClearQuest Requirements Test Change Rational PurifyPlus Rational Test RealTime Rational Functional Tester Plus Rational Functional Tester Rational Robot Rational Manual Tester Rational Performance Tester
  45. 45. WAKE UP, IT’S DEMO TIME!

×