http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615Five Fundamentals of a Successful Application Test StrategyMany IT managers face a similar struggle when it comes to application test planning. On one hand,they understand that vigorous testing is necessary to validate system performance and functionality.On the other hand, the push to meet strict development deadlines often leaves little, if any, time tocomplete-testing.This research note addresses the need to develop application test strategy as an integral first step of anapplication testing process. Specifically, this note: Defines an application test strategy. Explains the goals of an application test strategy. Describes the steps required to craft and maintain a test strategy.A test strategy is required to achieve thorough and successful application testing. Consider testingscope, schedule, resources, defect tracking methods, and functional metrics when developing anapplication test strategy.Implementation PointMany IT managers face a similar struggle when it comes to application test planning. On one hand,they understand that vigorous testing is necessary to validate system performance and functionality.On the other hand, the push to meet strict development deadlines often leaves little, if any, time tocomplete testing.Developing an application test strategy is an integral first step of an application testing process. Ifcreated early in the development lifecycle, a test strategy will pave the way to effective test execution.Defining a detailed test strategy requires input from testing, development, and business analystresources. The result of this effort is a detailed document that outlines the scope, schedule, resources,defect tracking methods, and functional metrics of the testing effort.Key ConsiderationsAn application test strategy is a critical input into the application testing phase of the systemdevelopment lifecycle. A test strategy is a document that defines the scope, approach, and schedule ofthe planned testing activities. It identifies, at a high level, the items that require testing along with theresources who will execute the testing.A test strategy should be developed early in the development cycle when the IT manager can obtainconsensus and commitment from the entire project team including development, testing, operations,and project stakeholders. A test strategy is a development aid and should be promoted as such;shaping the test strategy prior to beginning application development allows the project team time todigest the strategy and incorporate its finite details into related test plans.
http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615Why Create a Test Strategy?The IT manager should note there are several advantages to defining an application test strategy.These include the following: A test strategy eliminates confusion among test team members. It defines the processes and procedures for application testing in addition to who is testing specific features and functionality. A test strategy outlines all test items deemed in scope, reducing the risk of forgetting to test certain aspects of the application. A test strategy established early in the development lifecycle includes a schedule that coordinates with the overall project plan. This allows for flexibility to modify the project schedule to accommodate the test strategy if necessary. A test strategy ensures consistency throughout the testing effort. A test strategy will clearly indicate how testing should be designed, executed, and reported. Employ the ITA Premium “Application Test Strategy Template” to document a comprehensive testing plan in the early stages of the application development lifecycle.Implementation & IntegrationEnterprises must adopt a formal strategy for creating and maintaining an application test strategy. Ifcrafted well, a test strategy can be used for repeated cycles of application development. Follow Info-Tech’s approach depicted in Figure 1 to develop an application test strategy.Figure 1. Test Strategy Development StepsSource: Info-Tech Research GroupStep 1: Define Testing Scope, Objectives and ApproachDefining the testing scope and objectives requires the inclusion of several pieces of information: Objectives. Test strategy objectives give a summary of the products to be tested, schedule overview including major milestones, and resource overview. Scope. The test strategy scope defines the high-level items that will be tested and also details items excluded from testing. The scope should include applications and systems that are in scope and out of scope. If major functions or process (e.g. accounts receivable, contact management, service requests) are known, those should also be included. Approach. The testing approach specifies the testing phases that will be integrated in the overall testing effort and estimates the required effort and duration for each. Each type of
http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615 testing requires its own test plan. Refer to the ITA Premium research note, “Set Your Sights on the Six Software Testing Targets” to understand the six types of testing recommended to achieve a high quality end product: Unit Testing, Functional Testing, System Testing, Regression Testing, System Integration Testing, and Acceptance Testing. The testing approach should also include a list of all deliverables required for each type of testing. The deliverable list will include items such as test plans, test scenarios, test cases, and test signoff procedures.Step 2: Establish Test ScheduleThe test schedule outlines the dates during which each phase of testing is planned to take place.Based on the testing scope and approach the test manager should estimate the planned start andduration required for each test phase. Specifically, the test schedule should include: An overall start and end date. These dates denote the first day that the first phase of testing will commence and the last phase of testing will be completed. All phases of testing must be planned between these two dates. Start and end date of each test phase. These dates denote the start and end dates of the subtesting phases that are executed within the overall test strategy. Test entry criteria. Test entry criteria indicate what is required to initiate a specific testing phase. For example, to initiate unit testing, entry criteria may state that each module be coded to the specifications outlined in the detailed design document. Test exit criteria. Test exit criteria indicate what is required to declare a testing phase complete. For example, to complete functional testing, the exit criteria may indicate that each piece of functionality pass through a test cycle without raising any high severity defects. Test milestones. The test schedule should include any significant milestones that will be reached throughout testing efforts.While planning the schedule, the test manager is often asked to accommodate dates provided by theproject management team; the test manager will be given a date when testing must be complete. Insome situations the dates are unachievable based on the amount of testing required to ensure quality.It is the responsibility of the test manager to alert project management when this occurs. To rectifythis situation, the project management team should look for alternate solutions such as revising thetesting end date, removing application functionality or reducing the testing effort.Step 3: Assign Testing ResourcesTesting resources requirements are based on the scope of the testing effort and the testing schedule.Resources should be planned to complete the testing described in the scope within the time allotted inthe schedule.The test strategy should include a matrix that states which testing resources are assigned to particulartesting assignments. This information should be available to the testing team at the onset of theproject. Providing this information early on will allow the testing team members to form a workingrelationship with the developers of the code.
http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615The testers should be aware of the developers who coded the areas they are responsible for testing,allowing them to raise potential questions and concerns with them. Table 2 shows a sample high-levelmatrix that assigns resources to specific testing areas.Table 2. Sample Testing Resource AssignmentsSource: Info-Tech Research GroupResource Test AssignmentResource 1 GUI layout, font consistency, color usage, style, sizeResource 2 Stress test, load test, volume limitsResource 3 Installation, on-line user documentationResource 4 Communications: TCP/IP, APPC, LANResource 5 Storage control, file interfaces, Web servicesResource 6 Control systems, fatal errors, start up, shut downStep 4: Define Test Environment RequirementsTest environment requirements should be specified in the application test strategy: Hardware. Identify the hardware required to build a testing environment. Computers, networks, printers, and mainframe access are examples of physical machinery that may be required to conduct application testing. Software. Identify the software required to complete testing activities. This includes software in addition to what is being tested. Test management software, automated testing software, and databases are examples of software that may be required to complete testing. Location. Identify where the application testers will be located. Determine what office space, seating assignments, and office furniture are required to suit the testing effort. Consider if the testers require a special environment to perform testing such as a “clean room,” a disaster recovery site, a testing lab or a remote-site. Publications. Identify all documents, manuals, and user guides that are required to support the testing team. The publications may include use cases, application requirements, out-of- the-box application guides, and application design specifications.Step 5: Ascertain Defect Tracking ProceduresThe test strategy should define a procedure to raise, track, and resolve defects found throughouttesting. The procedure can be as involved as using defect tracking software or can be as simple asmaintaining a list of defects in a spreadsheet shared among the testers and developers. Regardless, itis imperative that the following information be documented: Date Raised. Specify the date the defect was raised. Raised By. Specify the resource who raised the defect. Issue Description. A description of the defect. Where possible, include detailed steps on how to reproduce the issue.
http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615 Testing Phase. Specify the phase in which the defect was raised, i.e. functional testing, system testing, etc. Assigned To. Specify the developer to whom the defect is assigned for resolution. Sign-Off Date. Specify the date the developer fixes and signs-off on the defect resolution back to the tester. Sign-Off Description. The developer provides a description of what was done to fix the issue. Severity. Assign a severity to the defect. There are several options available to help define defect severity. The following scale provides a manageable range of severity ranking and is based on information found on www.stickyminds.com. o High: A major issue where a large piece of functionality or major system component is completely broken. There is no workaround and testing must halt. o Medium: A significant issue where a large piece of functionality or major system component is not working properly. There is a workaround, however, and testing can proceed. o Low: A minor issue that imposes some loss of functionality, but for which there is an acceptable and easily reproducible workaround. Testing can proceed without interruption. Priority. Assign a priority to the defect. A defect priority level can be used along with severity to help determine the urgency in fixing the defect. The following three point priority scale can be used in common testing practices. The levels are: o High: This has a major impact on the customer. This must be fixed immediately. o Medium: This has a major impact on the customer. The problem should be fixed before release of the current version in development, or a patch must be issued if possible. o Low: This has a minor impact on the customer. The flaw should be fixed if there is time, but it can be deferred until the next releaseStep 6: Define Constructive Testing MetricsTest metrics provide a vehicle to communicate testing progress to the project team. Metrics detailwhat has been tested, when and how often it has been tested, and whether or not the testing wassuccessful. The test strategy should include all metrics required by the IT Manager needs to report onprogress and make informed decisions about the direction of the project. Some common metricsincluded in test strategies are: Functional Metrics. Number of requirements verified (may be broken down by testing phase, component, testing resource or all of the above). Code Metrics. Percent of code executed (may be broken down by test and/or component). Problem Metrics. Problems found per day, problems found per component. Schedule Metrics. Percent of tests completed. Estimated Days to Completion. Time to Complete Testing by Component.Testing metrics can be used to generate charts and graphs to help report progress to projectmanagement. Although useful, creating visual aids can be time consuming and might never used to
http://www.h2kinfosys.com | USA: 770 777 1269 | UK: (020) 3371 7615make business decisions. Involve project management when producing reporting metrics to helpdetermine which data will be most valuable.Bottom LineA test strategy is required to achieve thorough and successful application testing. Consider testingscope, schedule, resources, defect tracking methods, and functional metrics when developing anapplication test strategy. IT Training and PlacementLIVE ONLINE JOB ORIENTED TRAINING Register for FREE DEMO http://www.h2kinfosys.com USA: 770 777 1269 UK: 020 33717615Why H2Kinfosys Training..? Instructor taught LIVE training on a specific day and time. The online training courses are designed to meet the jobs and in tune with technologies oftomorrow. LIVE virtual classroom training with the same feel as you see and hear the course instructor in realtime. Instructor led - Face2Face - True Live Online. The online training courses are self-paced with interactive exercises On Demand online course can be obtained right from the drawing room of your home saving youtravel costs and cut down travel time. Live - Green - Everywhere