Run and Analyze Software Tests with Rational ClearQuest Test ...

6,059 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
6,059
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
60
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Run and Analyze Software Tests with Rational ClearQuest Test ...

  1. 1. Run and Analyze Software Tests with Rational ClearQuest Test Management Test Management with IBM Rational ClearQuest Skill Level: Intermediate Brian Bryson (bbryson@ca.ibm.com) Technology Evangelist IBM 17 Oct 2006 Learn how to handle test management with the new functionality and features in IBM® Rational® ClearQuest® V 7.0 TestManager, which now handles the test management functions previously provided by IBM Rational TestManager. This tutorial gives you a basic understanding of installation, configuration, test planning, and running different types of tests with this new tool. It also covers how to use ClearQuest query and reporting capabilities for test analysis. Section 1. Introduction Test Management with IBM Rational ClearQuest The Version 7.0 release of IBM Rational ClearQuest heralds a significant strategy shift in the IBM Rational approach to test management. Before this release, test management functionality was provided by IBM Rational TestManager, which included test planning, test execution, and test results analysis for quality assurance (QA) organizations. Starting with the V 7.0 release, ClearQuest handles test management, instead. ClearQuest provides the same core functionality available in Rational TestManager but extends it with features that address the challenges of the modern QA organization. Among the benefits are these new features: • Web interface Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 1 of 32
  2. 2. developerWorks® ibm.com/developerWorks • Eclipse integration • Tighter defect integration • Enterprise-level database support • More powerful reporting • Query capabilities for regulatory compliance through eSignatures and AuditTrails • Support for geographically distributed test teams, using ClearQuest MultiSite Tutorial overview The purpose of this tutorial is to provide a basic understanding of the new test management functionality introduced in the 7.0 version of IBM Rational ClearQuest. This tutorial takes you through the following steps: • Installation and configuration of ClearQuest and associated packages • Test planning, including creating a test asset registry, test plan, and test cases • Execution of different types of tests • Test analysis, focusing on the ClearQuest query and reporting capabilities Intended audience This tutorial is intended for first-time users of the ClearQuest test management capabilities. This includes users who are evaluating ClearQuest for the first time, as well as those who already use ClearQuest for change management and would like to take advantage of the new test management capabilities. Time to complete tutorial After you have the software installed, this tutorial should take approximately 30-60 minutes to complete. Section 2. Tutorial Setup There are no pre-requisites for this tutorial, other than having the IBM Rational ClearQuest software and time to complete the tutorial. To help you understand the Run and Analyze Software Tests with Rational ClearQuest Test Management Page 2 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  3. 3. ibm.com/developerWorks developerWorks® full capabilities of ClearQuest, this tutotrial includes a .zip file of sample tests, which you will use to better understand various concepts. (See Figure 1.) Follow these steps to install the files: 1. Using a tool such as WinZip, expand the included .zip file to your C drive (C:). The files will be expanded to a directory called CQTM Test Scripts. 2. ClearQuest requires that this directory be accessible through a Microsoft® Windows® network shared location. To enable this directory as a shared resource, open Windows Explorer 3. Within Windows Explorer, right-click the C:CQTM Test Script File Locations directory, and then select Sharing and Security. 4. If you have never enabled file sharing on your machine, proceed to the next step. If you have previously enabled file sharing, skip ahead to Step 7. 5. In the resulting CQTM Test Scripts Properties dialog Sharing tab, click this statement: If you understand the security risks but want to share files without running the wizard, click here. 6. In the resulting Enable File Sharing dialog, select Just enable file sharing, and then click OK. 7. In the CQTM Test Scripts Properties dialog, select Share this folder on the network. Accept the default name of CQTM Test Scripts 8. Although not necessary for this tutorial, check Allow network users to change my files, because ClearQuest writes test results to this shared folder. 9. Click OK to close the CQTM Test Scripts Properties dialog. 10. Finally, click Yes to acknowledge the warning about shared folder names that are longer than 12 characters. Figure 1. CQTM test scripts dialog Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 3 of 32
  4. 4. developerWorks® ibm.com/developerWorks Section 3. Install ClearQuest and Create Your Database This tutorial assumes no previous knowledge of IBM Rational ClearQuest. It walks you through every step, from installation through reporting. In this first section, you will complete your first two steps. First, you'll install ClearQuest, and then you'll create the databases necessary to store your ClearQuest data. Install IBM Rational ClearQuest Get ClearQuest software Run and Analyze Software Tests with Rational ClearQuest Test Management Page 4 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  5. 5. ibm.com/developerWorks developerWorks® At the time of publication, IBM Rational ClearQuest was not available for trial download. To get the software, contact your local IBM Rational representative, and request an evaluation copy and license key. Follow these steps to install IBM Rational ClearQuest: 1. Run setup.exe, and then proceed through the wizard to perform a desktop installation. 2. Accept all defaults. Feel free to install to any location or to add components, such as the ClearQuest Web interface. This tutorial doesn't cover them, but you can install them now for future use. 3. In the configuration wizard panel, click Done. You do not need to configure anything at this point. 4. If you have received trial software with an evaluation key, upon completion of the installation, you will be taken to the IBM Rational License Key Administrator to enter your evaluation license key. Current customers can use the wizard to point to their current license server. Create a schema repository and sample database Follow these steps to create a sample database: 1. Start the ClearQuest Maintenance Tool. 2. In the ClearQuest Maintenance Tool, select Schema Repository > Create. 3. Select MS_ACCESS as the vendor, and then choose a location on your hard drive to store the schema repository. This tutorial uses C:CQTM TutorialCQTM_SchemaRepo.mdb. 4. In the Existing Connections field, change the connection name to CQTM 5. Click Next, and then acknowledge the warning about placing the repository in a shared location. 6. Click Next to accept the ClearQuest Data Code Page. This will differ, depending on the language of your OS. In Canada and the U.S., the default will be: 1252 (MS Windows Latin 1). 7. On the next screen, check the Create sample database box, and then select the Enterprise Schema. 8. Enter the database name: CQTST. You have the option of providing a Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 5 of 32
  6. 6. developerWorks® ibm.com/developerWorks description. 9. Click Next. 10. Again, select MS_ACCESS as the database type for the sample data,, and then provide a Physical Database Name of C:CQTM TutorialCQTM_Data.mdb. 11. Click Finish to complete the process, again acknowledging the warning about storing data in a shared location. 12. After you have reviewed the status messages, click Done, and then click Exit the ClearQuest Maintenance Tool. Launch ClearQuest and log in IBM Rational ClearQuest clients There are many interfaces or clients for IBM Rational ClearQuest. Looking into the program group that was created when you installed ClearQuest reveals two clients for ClearQuest. The first is the ClearQuest Eclipse client, and the second is the ClearQuest for Windows client. Note: There is also a third ClearQuest Eclipse plug-in and, of course, the ClearQuest Web client. There are no icons in the program group for either of those. The difference between the Eclipse client and the Eclipse plug-in is that the Eclipse client provides its own Eclipse shell. The plug-in is for users who have already installed Eclipse and merely want to include ClearQuest functionality in their pre-existing shells. Users who do not wish to install a full ClearQuest client or those who are using Apple hardware can access ClearQuest data through the ClearQuest Web interface. For Linux and Unix users, there are also native Unix and Linux clients for ClearQuest. For this tutorial, you'll use the ClearQuest Eclipse client, which is an Eclipse-based stand-alone tool. Everything you do in this tutorial can be done from the Eclipse plug-in, but some of the menu navigation differs slightly between the two clients. Much of what you'll do in this tutorial will be based on the ClearQuest Test Manager view, which is available only in the Eclipse clients. Therefore, do not attempt to complete this tutorial using the ClearQuest Windows or other native OS clients. Follow these steps (also see Figure 2): 1. Click Start > All Program > IBM Rational > IBM Rational ClearQuest > ClearQuest to start ClearQuest 2. Optional: Click the Overview icon on the Welcome screen. From there, Run and Analyze Software Tests with Rational ClearQuest Test Management Page 6 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  7. 7. ibm.com/developerWorks developerWorks® you can watch a 15-minute Rational ClearQuest TestManager video tour. Then close the Welcome screen. You can return here later if you wish by going to Help > Welcome. 3. Select File > Database > Manage connections. 4. Select the CQTM database set, and then click Add Connection.... 5. Accept the default Schema Repository of CQTM, and then click Next. 6. Enter the user ID: admin. There is no password on this account. 7. In the Connect dialog, select Remember the password, and then click OK to log in. 8. The Test Manager view should be visible, although it will not contain data. Figure 2. IBM Rational ClearQuest You are now connected to the CQTST database that you created previously, and you are ready to start this tutorial. Section 4. Test Project Configuration Before you begin to create your test plans and test cases, you need to complete a one-time setup and configuration. You must provide core information to ClearQuest, such as the location where you'd like to store your test assets. The asset registry Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 7 of 32
  8. 8. developerWorks® ibm.com/developerWorks The asset registry will hold all of your assets. It will contain all of your test plans, which contain all of your test cases. It will contain all of your test suites, as well as file locations for all of your IBM Rational Manual Tester, IBM Rational Functional Tester, and IBM Rational Performance Tester tests. It will also contain all of your test results. See Figure 3 and Figure 4. Complete these steps to create your asset registry: 1. Right-click the Asset Registries branch in the Test Manager view, and then select New Asset Registry. 2. Type the name CQTM Tutorial, and then your sample test assets description. 3. Click OK to create your asset registry Figure 3. Asset registry creation dialog Figure 4. Newly created asset registry with test plans, test suites, iterations, and file locations Run and Analyze Software Tests with Rational ClearQuest Test Management Page 8 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  9. 9. ibm.com/developerWorks developerWorks® Configurations and configuration attributes What if I don't use configurations? Although configurations are a powerful feature of IBM Rational ClearQuest, not every team tests multiple configurations. ClearQuest requires at least one base configuration and one base configuration attribute to run tests. If you need only one configuration, create only a single attribute and a single configuration. ClearQuest TestManager recognizes that most test teams run tests across a variety of configurations. As an example, QA teams often run the same tests on various operating systems and versions, such as Microsoft® Windows XP®, Microsoft® Windows Server®, or Linux® operating systems. Alternatively, teams might need to test different hardware configurations, such as a machine with 512 MB of RAM and another machine with 2 GB of RAM. Whatever the configuration, ClearQuest TestManager can record various configuration attributes (such as operating system, memory, disk space, and so forth) so that you can group attributes into common configurations. Configurations and configuration attributes are not part of your asset registry. Therefore, they can be shared across asset registries. The benefit of this is that you need to enter this information only one time, rather than having to recreate configurations for every asset registry. Create configuration attributes Follow these steps to create basic configuration attributes (also see Figure 5): 1. Expand the Configurations folder to reveal the Configuration Attributes folder. 2. Right-click the Configuration Attributes folder, and then select New Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 9 of 32
  10. 10. developerWorks® ibm.com/developerWorks Configuration Attribute. 3. Type Operating System in the Name field, and then click OK to create the attribute. 4. Right-click the newly created attribute, and then select New Configuration Value. 5. Enter Linux in the Name field, and then click OK to create the attribute value. 6. Repeat the last two steps, entering Windows XP in the Name field. 7. Right-click the Configuration Attributes folder, and then select New Configuration Attribute. 8. Type Memory in the Name field, and then click OK. 9. Right-click the newly created attribute, and then select New Configuration Value. 10. Type 2 GB in the Name field, and then click OK to create the attribute value. 11. Repeat the last two steps, this time typing 512 MB in the Name field. Figure 5. Configuration attributes Create configurations 1. Right-click the Configurations folder, and then select New Configuration. 2. Provide the Name: Minimum Memory Windows XP. 3. Next to the Configuration Attributes List, click Add. 4. In the Browse Record Type dialog, click Search to reveal all of the configuration attributes that you just created. 5. Hold down the CTRL key while you click to select both Windows XP Run and Analyze Software Tests with Rational ClearQuest Test Management Page 10 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  11. 11. ibm.com/developerWorks developerWorks® operating system and 2 GB Memory. 6. Click OK to close the Browse dialog. 7. Click OK to close the Create (TMConfiguration)dialog. 8. Repeat the process to create a configuration called Recommended Windows XP Configuration. Again, select the Windows XP operating system and 512 MB of memory simultaneously. (See Figure 6). Figure 6. Completed configurations and configuration attributes Section 5. Test Planning Asset registry and configuration are simple, one-time steps that you typically do early in a test project, and then forget. However, test planning is process that continues throughout a test project. In the next section, you will create a multi-tiered test plan. A test plan is an organized structure of test cases. Test cases are the specific tests or validations that you want to perform on your system. Thus, after you create your test plans, you will insert your test cases into those test plans. Finally, you will associate the configurations that you created in the previous section with your test cases. A simple test case is an abstract concept, in that it merely defines something that you want to test. By associating a configuration with a test case, you create a configured test case, which identifies not only what you want to test, but where you want to run your tests (on which platform or configuration). Create test plans Sample application: Classics Online Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 11 of 32
  12. 12. developerWorks® ibm.com/developerWorks From this point forward in this tutorial, you will base all of your test plans, test cases, and suites on the >Classics Online sample application. Classics Online is a simple, online music CD-ordering system. It's a lightweight Java™ application that serves as your application under test. You can download the sample application from the downloads section of this tutorial. To run the sample application, expand the file to any location on your hard drive, and then double-click the ClassicsJavaA.jar application. Note: You must have a Java Runtime Engine installed to run this application. The first step in your test planning process is to create a test plan to hold all of your test cases. There are many ways to organize your test plans and test cases. Rather than reading through a long discussion of the pros and cons of test plan organization here, you will simply use the very common functional decomposition structure to create child test plans for various functional areas of the application. By using this structure, your test plan architecture matches your application architecture. Follow these steps to create your basic test plan structure: 1. Right-click the Test Plans branch of the CQTM Tutorial asset registry that you created earlier, and then select New Test Plan. 2. Type Master Functional Test Plan as the headline. 3. Click OK to create your test plan. Now that you have created your main test plan, you can create test plans for some of the functional areas of the Classics Online application (see sidebar for details on the sample application). 1. Right-click the Master Functional Test Plan branch that you just created, and then select New Test Plan. 2. Enter Order Placement Tests as the headline. 3. Click OK to create your test plan. 4. Repeat the process to create a test plan for Administrative Function Tests. 5. Repeat the process to create a test plan for Order Review Tests. Figure 7. ClearQuest supports hierarchical test plans for better test plan and test case organization Run and Analyze Software Tests with Rational ClearQuest Test Management Page 12 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  13. 13. ibm.com/developerWorks developerWorks® Create test cases Feel free to wander about The goal of this tutorial is to walk though the process of enterprise test management. To get through the entire process within the alloted time, we're not going delve deeply into some subjects. Nonetheless, there are interesting side attractions for you. As an example, in the next exercise we'll create some very basic test cases, but if you flip through the various tabs of a test case, you'll see that there's lots of information that you could be entering, such as: Test Motivator: Here you can attach any supporting file or documentation that might provide more information about the test case. You can attach any type of file, such as a spreadsheet or screen captures, to add more context to your test case. Legacy data: These fields map to ClearQuest TestManager fields for backward compatibility. Notes: These are freeform text fields where you can make any general notes about the test case. Requirements: Here is where ClearQuest can integrate with IBM Rational RequisitePro®, the Rational requirements management tool. If you are running RequisitePro, you can associate your requirements with your tests, which later enables you to associate your test verdicts with your requirements. This gives you a full lifecycle view of the progress of your application, from requirements to tests and defects. After you have established your test plan structure, the next step is to populate the test plan with test cases. The test cases define each of the validations that you plan to perform on the system. (See Figure 8.) Follow these steps to create a test case for each of your test plans: 1. Right-click Order Placement Tests, and then select New Test Case. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 13 of 32
  14. 14. developerWorks® ibm.com/developerWorks 2. Type Place Valid Order as the headline. 3. Under Priority, select 1 - Critical. 4. Now right-click Administrative Function Tests, and then select New Test Case. 5. Enter View Customer List as the headline. 6. For Priority, select 3 - Medium. 7. Right-click Administrative Function Tests, and then select New Test Case again. 8. Type View Existing Order Status as the headline. 9. For Priority, select 2 - High. Figure 8. Test plans with associated test cases Associate test scripts with test cases The test plan and test cases represent your logical test plan. You can think of the test plan as a list of test cases to run. But how will you run these tests? And how will you know if a test passed or failed? Answer: The Execution tab. You use the Execution tab to link your test case to an actual test script. The test script then issues the pass or fail verdict when it runs. The ClearQuest TestManager supports execution of test scripts that were created in IBM Rational Functional Tester, IBM Rational Manual Tester, and IBM Rational Performance Tester test automation tools. It also supports the Test and Run and Analyze Software Tests with Rational ClearQuest Test Management Page 14 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  15. 15. ibm.com/developerWorks developerWorks® Performance Tool Platform, or TPTP-compatible tests, such as TPTP JUnit tests. • Rational Functional Tester is a script-based regression testing tool that captures user actions against an application in script form for subsequent playback and system validation. • Rational Manual Tester is a manual test authoring and execution tool. You can use it to capture test script steps and actions that you want to run on an application, and then run the tests. It then automatically enters and validates data as the you run the test. • Rational Performance Tester is a tool to help eveluate system response times for Web-based and enterprise resource planning (ERP) applications. It simulates simultaneous user activity on your system and measures response times to help you poinpoint performance issues and bottlenecks before you release your system to production. Establish file locations Where did these file locations come from? In this section, you use the tutorial working assets that you established during the Tutorial Setup section. If you skipped that step, go back now to create these sample assets. Before you can associate your test scripts with your test cases, you need to tell ClearQuest the file locations of your tests. This is a one-time setup for each asset registry. After that setup, all test cases will be able to access the scripts in the file locations that you specified. (See Figure 9.) Follow these steps to establish file locations for your Rational Manual Tester and Rational Functional Tester scripts: 1. Right-click the File Locations folder in the asset registry, and then select New File Location. 2. Click Next to accept CQTM Tutorial as the default asset registry. 3. Type Functional Tester Tests in the Name field. 4. Provide this file location: C:CQTM Test ScriptsFunctional TesterFunctional Tests. 5. Provide the test log location: C:CQTM Test ScriptsFunctional TesterFunctional Tests_logs. 6. Click Finish to create the file locations. 7. Repeat the process to create a file location with the following properties: Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 15 of 32
  16. 16. developerWorks® ibm.com/developerWorks • File location name: Manual Tester Tests • File location: C:CQTM Test ScriptsManual TesterManual Tester Scripts • Test log location: C:CQTM Test ScriptsManual TesterManual Tester Logs Figure 9. File locations tell ClearQuest where to find test automation scripts Establish associations between test scripts and test cases Now that ClearQuest knows where your scripts are, follow these steps to associate Rational Functional Tester and Rational Manual Tester scripts with your test cases (also see Figure 10): 1. Right-click the Place Valid Order test case, and then select Modify. 2. Navigate to the Execution tab. 3. Click the Browse button. 4. Select Rational Functional Tester as the test type. 5. Expand the Functional Tester Tests file location branch. 6. Select the Place_Valid_Order script. 7. Click Finish to complete the association. 8. Click OK to close the test case. 9. Note the icon change on your test case. There is now a small script icon on the top-right corner of your test case icon. 10. Repeat the process to associate the Functional Tester Test called View_Existing_Order with the View Existing Order Status test case. Run and Analyze Software Tests with Rational ClearQuest Test Management Page 16 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  17. 17. ibm.com/developerWorks developerWorks® 11. Switch the test type to Rational Manual Tester. 12. Repeat the process to associate the Manual Tester Test called View_Customer_List with the View Customer List test case. Figure 10. Test script associated with test case The View Existing Order test script is associated with the View Existing Order Status test case. (Your record ID, in this case CQTST00000047, may be different.) Section 6. Test Execution More to come... Customers who are migrating from Rational TestManager will notice that ClearQuest's execution model is less mature than that of ClearQuest TestManager -- for now. Currently, ClearQuest supports basic sequential test execution. Many customers have requested additional functionality, and the IBM Rational development teams are increasing capabilities in this area, which will be released in the near future. To make this tutorial as generally applicable as possible, we did not want to assume that you had installed any test script execution tools, such as Rational Manual Tester or Rational Functional Tester. That is why what you do in the next section goes only up to the point of actually executing a test. First, you run an individual test case, and then you create a suite of test cases. Your first step, regardless of how you execute tests, is to associate configurations with test cases. (See Figure 11.) Associate configurations with test cases Test cases are an abstract artifact. A test case represents something that needs to be tested, but a test case itself cannot be executed. A test case becomes executable when you associate it with a specific configuration. You have already created two configurations during this tutorial, so your next step is to tie your test cases to your Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 17 of 32
  18. 18. developerWorks® ibm.com/developerWorks configurations. Follow these steps to associate your configurations with your test cases: 1. While holding down the control key to select all three test cases simultaneously, click all three of the test cases you created in the previous step: Place Valid Order, View Customer List, and View Existing Order Status. 2. Right-click the Place Valid Order test case (or any other test case), and then select Add Configured Test Cases. 3. In the Add Configured Test Cases dialog, select Recommended Windows XP Configuration. 4. Click OK to add the configuration to the three test cases. 5. Repeat the process, adding the Minimum Memory XP configuration to the Place Valid Order test case only. Figure 11. Your test planning process is complete, and your configured test cases are ready to run Schedule test iterations You can think of configurations as defining where to run a test case. In ClearQuest, you also define when to run a test case. The when of a test case is called an iteration. (See Figure 12.) If you are familiar with the IBM Rational Unified Process®, or RUP®, then the concept of iterations will not be new. Rational TestManager users will also be familiar with the concept. The idea of an iteration is to specify at what point in time, Run and Analyze Software Tests with Rational ClearQuest Test Management Page 18 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  19. 19. ibm.com/developerWorks developerWorks® or at what stage of development, a test case can be run. Iterations are specific to your organization and your development process. To keep things simple, use the following steps to create three simple stages of development, or iterations: 1. Right-click the Iterations folder of your asset registry, and then select New Iteration. 2. Provide the Name: Inception. 3. Provide the Start Date: January 1, 2007. 4. Provide the End Date: January 31, 2007. 5. Click OK to create your iteration. 6. Repeat the process to create an iteration called Elaboration that runs from February 1, 2007 to February 28, 2007. 7. Repeat the process again to create an iteration called Construction that runs from March 1, 2007 to March 31, 2007. Figure 12. Iterations Run a single configured test case For such an important step, the actual process of running or executing a test is surprisingly simple. You just right-click a configured test case, and then select Execute. To execute a configured test case, the automation tool must be installed. If it is not installed, the Execute option will be disabled. Note: Test execution is not possible from all ClearQuest clients. The Windows and Web clients cannot execute any scripts. The Linux client can execute only Rational Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 19 of 32
  20. 20. developerWorks® ibm.com/developerWorks Functional Tester Java scripts. The ClearQuest client that you are using in this tutorial can execute only Rational Manual Tester scripts. Only the Eclipse plug-in can execute all types of scripts. Therefore, for any customers who will be launching scripts directly from ClearQuest, the recommendation is to use the Eclipse plug-in as your primary ClearQuest client. Create a suite of configured test cases Suites are configuration-specific When you create a test suite, you need to specify a configuration for the suite. This selection is important, because the suite will be able to execute only configured test cases that are associated with this configuration. Although possible, you will not often execute individual configured test cases. It is far more likely that you will assemble a series of configured test cases for sequential execution. In ClearQuest terminology, this is called a test suite. (See Figure 13.) Follow these steps to create a test suite of configured test cases: 1. Right-click the Test Suites folder of your asset registry, and then select New Test Suite. 2. Within the Main tab, provide this headline: Smoke Test Suite. 3. Within the Execution tab, select Recommended Windows XP Configuration. 4. Click OK to create the test suite. Take the following steps to add configured test cases to you test suite: 1. Right-click your newly created Smoke Test Suite, and then select Configure.... 2. Click Add.... 3. A list of all configured test cases that configurations that match the suites configuration will display. 4. Select all of the test cases by clicking the first test case, and then holding down the Shift key until you have selected the last test case. 5. Click OK to add the test cases. 6. Click the last configured test case, View Existing Order Status. Run and Analyze Software Tests with Rational ClearQuest Test Management Page 20 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  21. 21. ibm.com/developerWorks developerWorks® 7. Click Move Up to move this test case into the second position. Test scripts are executed in the that order they are presented in this dialog. 8. Click Finish to complete the test suite configuration, Figure 13. The configured test cases will be executed in the order that they are presented in this dialog Run a test suite Much like running an individual configured test case, all you need to do to execute a suite is to right-click the test suite, and then select Execute. Again, for your present purposes, you will not be executing the tests. In the upcoming step, you will simulate an execution by importing the results from these tests that the IBM Rational test labs already ran for you. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 21 of 32
  22. 22. developerWorks® ibm.com/developerWorks Import test results Import test log format The format for a raw test log file comes from the Test and Performance Tools Platform (TPTP) specification. The file itself is a zip-compressed XML file. You can use a file-compression utility, such as Winzip, to open the file and look at the XML that it contains. Why? Because this is a very powerful feature for getting results into ClearQuest. If you are using tool that are not supported by ClearQuest for test execution, this is how you can export results from those tools into ClearQuest for consolidated test reporting. At this point, you would normally run your configured test cases or your test suite, and your results would be populated automatically in ClearQuest as test log records. However, because we're working on the assumption that you are not using these tools, you will use an alternate method to populate ClearQuest with the test log records. ClearQuest has an import function that enables you to associate a test log record with a configured test case. A test log record contains two key pieces of information: a time and date stamp and a verdict. The verdict is Pass, Fail, or Inconclusive. The test log record is the key asset. When assimilated with all other test log records, it provides you with the most information about your testing project. Take the following steps to import test log records for your configured test cases: 1. Right-click the Place Valid Order - Recommended Windows XP Configuration configured test case, and then select import Test Log.... 2. For Test Type, select Rational Functional Tester. 3. Select the test log: Functional Tests_logs > Place_Valid_Order > null. 4. Enter the name: Build 1. 5. Click Finish to import the record. 6. In your Test Results view, right-click the newly imported record, and then select Commit. Where did that build come from? You've seen configurations and iterations, but when you import your test log record, you're also asked for build information. A build is a further level of granularity that you can use to subdivide and classify your test results. Although not really necessary in a small example such as this one, it is an invaluable feature for large projects, where new builds of the application under test are a daily occurrence. Run and Analyze Software Tests with Rational ClearQuest Test Management Page 22 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  23. 23. ibm.com/developerWorks developerWorks® Section 7. Test Analysis Finally, you reach the payoff point. Here is where you see the benefit of your hard work of capturing and executing all of your test cases. At this stage, you will be able to run queries and reports to give you a definitive diagnosis of the health of your test project. You can run these reports as often as you like to get up to get an up-to-the-minute status report. You can also share the reports and query results with other members of your team. Users who have configured ClearQuest Web can provide Web access to all reports. Users who do not set up this free feature of ClearQuest can simply export and e-mail results to other team members. Regardless of how you convey the information to your colleagues, ClearQuest gives you all of the information you need to assess your application under test. Switch to Sample database Rather than having to import hundreds of test logs, for this tutorial, you're going to do your analysis against a pre-populated database of results to save time. This sample database also includes several custom-created queries from IBM Rational labs that you can use for your own projects. (See Figure 14.) Follow these steps to connect to the sample database. 1. Select File > Database > Manage connections. 2. Click Add Database Set.... 3. In the Name field, type Tutorial, and then select MS_ACCESS as the Vendor. 4. Click Next, and then browse to C:CQTM Test ScriptsCQTM Sample DataSampleMaster.mdb. 5. Click Add Connection.... 6. Select the Schema Repository called Tutorial, and then click Next. 7. Provide the user ID: jan (which is case-sensitive), and then click Finish. 8. In the Connect dialog, provide the password: jan. Select Remember the password, and then click OK. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 23 of 32
  24. 24. developerWorks® ibm.com/developerWorks 9. Click Close to close the ClearQuest Connection Management dialog. Figure 14. Newly established connection to the tutorial Sample Data database ClearQuest provides charts, queries, and reports ClearQuest provides three formats to analyze your data: charts, queries, and rReports. (See Figure 15.) You will navigate through examples of all three of these. Take the following steps to navigate to the analysis tools: 1. Switch to the ClearQuest Navigator tab. 2. Collapse the admin, CQTM@CQTST connection that you have been working with. 3. Expand the jan, Tutorial@AUCT connection. 4. Open the public Queries folder, and then find the TM Charts, TM Queries, and TM Reports folders. All of your next steps will be working within these folders. Figure 15. TM Charts, TM Queries, and TM Reports folders contain analytical tools that come with ClearQuest Run and Analyze Software Tests with Rational ClearQuest Test Management Page 24 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  25. 25. ibm.com/developerWorks developerWorks® Run ClearQuest TestManager Charts Take the following steps to examine some of the out of the box charts that ClearQuest includes (see Figure 17 and Figure 1 for examples): 1. Expand the TM Charts folder. 2. Double-click the Execution - Verdict Distribution chart. 3. Highlight the Auction asset registry. 4. Click OK to run the report 5. Double-click the Clear Quest Chart: Execution - Verdict Distributiontab to expand the report to full size. Figure 16. This chart gives a graphical representation of all pass and fail verdicts in the asset registry selected. This is a high-level, global summary of all tests. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 25 of 32
  26. 26. developerWorks® ibm.com/developerWorks 1. Double-click the Clear Quest Chart: Execution - Verdict Distribution tab to return the tab to normal size. 2. Close the Clear Quest Chart: Execution - Verdict Distribution tab. 3. Run the CTC - Iteration Distribution chart by double-clicking on the chart name. Expand the chart as necessary. Figure 17. This chart gives a graphical representation of all of the configured test cases that are tagged with an iteration number. You can use this report to see at which stage you will perform most of your testing. Run and Analyze Software Tests with Rational ClearQuest Test Management Page 26 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  27. 27. ibm.com/developerWorks developerWorks® Run ClearQuest TestManager Reports Report customization ClearQuest's reports are created with the Crystal Reports reporting tool from Business Objects. To learn more about creating and modifying ClearQuest reports, see the IBM Technote on How to create reports in ClearQuest using Crystal Reports. Whereas charts contain purely graphical data, reports can contain both graphics and text. To examine the reports that ship with ClearQuest, follow these steps: 1. Collapse the TM Charts folder 2. Expand the TM Reports folder 3. Double-click Verdict Summary by Build report to run that report. 4. Select Asset Registry Auction. 5. Click OK to run the report 6. Double-click the Verdict Summary by Build tab to expand the report to Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 27 of 32
  28. 28. developerWorks® ibm.com/developerWorks full size. Figure 18. Verdict Summary by Build report ClearQuest Reports can contain both graphic and text data. In Figure 19, the Verdict Summary by Build report breaks down the pass-or-fail verdict summary by builds. You also have the option of editing ClearQuest reports with the Crystal Reports report editing software, which is available from Business Objects. Run ClearQuest TestManager Queries Queries do not offer the visual representation that Charts and Reports can offer, but they are probably the most powerful tool in the data analysis toolkit. You can extract any data you want with a query. They're easy to build, quick to run, and can provide information at any level. Take the following steps to run some of the queries included with ClearQuest TestManager: 1. Collapse the TM Reports folder Run and Analyze Software Tests with Rational ClearQuest Test Management Page 28 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  29. 29. ibm.com/developerWorks developerWorks® 2. Expand the TM Queries folder 3. Double-click Configured Test Cases - Verdict History to run that query. 4. Select Asset Registry Auction. 5. Click OK to run the report. Figure 19. Configured Test Cases - Verdict History query The Configured Test Cases - Verdict History query provides us with a listing of every execution verdict of every configured test case. You can use this query to investigate whether a given configured test case has a history of failing. Create your own query It is quite likely that you are going to want and need to create your own queries to analyze your data. Imagine a scenario where your project lead comes to you and asks for a list of all of the test cases that are currently failing. How would you respond? ClearQuest has all the information, but how would you get it out? Take the following steps to create a custom query to show you which configured test cases are failing. 1. Right-click the Personal Queries folder of the jan, Tutorial@AUCT connection, and then select New Query. 2. Provide the query name: Failing Tests. 3. Select the record type TMConfiguredTestCase to run a query containing information on your configured test cases. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 29 of 32
  30. 30. developerWorks® ibm.com/developerWorks 4. You must first provide your query filter to select only configured test cases that failed the most recent execution. To do this, you'll need to filter the Test Log Verdict and the Test Log Latest fields. First, expand TestLogs branch of the TMConfiguredTestCase folder, and then double-click to select both the Latest and Verdict fields. 5. Click Next to specify the values to filter for these fields. 6. Expand the And filter on the right, and then select the TestLogs Verdict field. In the Define Filters box, make sure that the operator is Equal. 7. Click Values, and then select Fail. 8. Click OK to close the dialog. 9. Select the TestLogs.Latest filter, and again ensure that the operator is set to Equal. 10. In the Values field, type True. Setting these two filters means that you will see only records for configured test cases that failed the most recent test. 11. Click Next to select the fields to display as part of the query. 12. Double-click the ID field to add it to the display list. In the sorting column, select Ascending. 13. Double-click the Headline field to add it to the display list. 14. Drill down, and then double-click the TestLogs > Verdict field to add it to the display list. 15. Click Finish. 16. Your query is now complete and will display in your personal queries folder. 17. Double-click the query to see the results (also Figure 20). Figure 20. Query for Failing Configured Test Cases Run and Analyze Software Tests with Rational ClearQuest Test Management Page 30 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.
  31. 31. ibm.com/developerWorks developerWorks® Your custom query shows a list of all of the configured test cases with fail as the most recent verdictl. You can double-click any of these records to read the details of the record. Examine custom queries included with ClearQuest You will notice that there are several queries in the Personal Queries folder. These are all custom-designed queries that testers in the IBM Rational lab built for you to examine and try. Many of these queries contain information from different types of records. In the Integrated Requirement-Defect-TestCase Queries folder, as an example, you will find records that relate test cases to requirements and then to defects. Use, examine, and learn from these queries to see how ClearQuest can provide you with all of the information that you need to test the quality of your software and track your testing projects. Section 8. Recap and Next Steps You covered a lot of ground in this tutorial. Starting from nothing, you created a test plan and test cases. You created configurations and iterations, learned how to run a ClearQuest test, and imported test case results. The goal was to get you to a point where you can turn all of the data about your testing project into useful information. You did this in the analysis phase by examining the queries, charts, and reports that ClearQuest provides. Then you created queries to extract information specific to your needs. You have just completed ClearQuest TestManager basic training, yet you've only scratched the surface. You have not touched on eSignatures, audit trails, nor sharing data with remote sites by using ClearQuest MultiSite and ClearQuest Web. You haven't customized any fields, forms, or actions. Whenever you're ready or need more information, there is more to learn. That's the beauty of IBM Rational ClearQuest. For the novice, you can use the configuration provided to get up and running in an hour. For those who want to customize the many options of ClearQuest, the flexibility is unlimited. Run and Analyze Software Tests with Rational ClearQuest Test Management © Copyright IBM Corporation 1994, 2006. All rights reserved. Page 31 of 32
  32. 32. developerWorks® ibm.com/developerWorks Downloads Description Name Size Download method Classics Online Sample Application ClassicsOnlineSampleApp.zipFTP 274 KB Sample tests and results CQTMTestScripts.zip MB 5.9 FTP Information about download methods About the author Brian Bryson Brian Bryson joined Rational Software in 1995 with a strong background in Quality Assurance. Currently, he is a member of the product team that plans and delivers the IBM Rational automated toolset. A former instructor at Carleton University in Ottawa, Canada, Mr. Bryson has taught the Rational approach to quality automation to college students as well as employees of organizations such as Microsoft, Sun Microsystems, and Oracle. Run and Analyze Software Tests with Rational ClearQuest Test Management Page 32 of 32 © Copyright IBM Corporation 1994, 2006. All rights reserved.

×