Your SlideShare is downloading. ×
0
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
TEST EXECUTION AND REPORTING
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

TEST EXECUTION AND REPORTING

5,914

Published on

Published in: Education, Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,914
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
246
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Test ExecutionTest Execution & Reporting& Reporting
  • 2. AgendaAgenda  General Test procedureGeneral Test procedure  Executing testsExecuting tests Set up systems and softwareSet up systems and software Execute test casesExecute test cases  Verifying and analyzing resultsVerifying and analyzing results  ReportingReporting
  • 3. General Test ProcedureGeneral Test Procedure A Single test normally includes three steps:A Single test normally includes three steps:  PlanningPlanning  ExecutionExecution  EvaluationEvaluation Tester Plan testing Execute testing Evaluate testing
  • 4. General Test ProcedureGeneral Test Procedure Plan testing Execute testing Evaluate testing If I click OK, without input into text field, the application will probably crash. Let’s see if it does It sure did!
  • 5. PlanningPlanning  It is the first step to determine what should be tested. AlsoIt is the first step to determine what should be tested. Also referred as test case design.referred as test case design.  Design the test cases (inputs and outputs) used to test theDesign the test cases (inputs and outputs) used to test the system:system:  Identify the test case (Identify the test case (What has to be testedWhat has to be tested).).  Specify the test activities(Specify the test activities( How to proceedHow to proceed).).  Indicate how test results should be defined (Indicate how test results should be defined (Correct outcomeCorrect outcome).).
  • 6. ExecutionExecution It’s Show Time!It’s Show Time! Execution of a test is a three stepExecution of a test is a three step Process:Process:  Preparing the Test InfrastructurePreparing the Test Infrastructure  Conduct Test.Conduct Test.  Verify Results and Report Results.Verify Results and Report Results.
  • 7. Prepare Test InfrastructurePrepare Test Infrastructure  The prepare Test Infrastructure taskThe prepare Test Infrastructure task ensures that all the components such asensures that all the components such as technical resources, human resources,technical resources, human resources, configuration, data, and test designconfiguration, data, and test design specification are in place for successfulspecification are in place for successful execution of the test.execution of the test.  The key to setting up the testingThe key to setting up the testing environment is to establish a set ofenvironment is to establish a set of standards that everyone in the project canstandards that everyone in the project can adhere to.adhere to.
  • 8. Prepare Test InfrastructurePrepare Test Infrastructure Establish a set of standards that theEstablish a set of standards that the organization follows.organization follows.  The key to setting up the testingThe key to setting up the testing environment is to establish a set ofenvironment is to establish a set of standards that everyone in thestandards that everyone in the project can adhere to.project can adhere to.
  • 9. PreparePrepare Test InfrastructureTest Infrastructure  Components of the testingComponents of the testing environment include:environment include: Supporting software An inventory of hardware The code under test A workbook or execution logs
  • 10. PreparePrepare Test InfrastructureTest Infrastructure Supporting softwareSupporting software  Includes operating systems, middleware,Includes operating systems, middleware, system utilities, application, patches and sosystem utilities, application, patches and so on.on.  Supporting software is stored on CD orSupporting software is stored on CD or available on a shared file system.available on a shared file system.  All supporting software will need to beAll supporting software will need to be installed in the environment in the exactinstalled in the environment in the exact sequence required by the components.sequence required by the components.
  • 11. PreparePrepare Test InfrastructureTest Infrastructure Supporting softwareSupporting software  Detailed installation steps will need to be providedDetailed installation steps will need to be provided to you as part of the implementation of the testto you as part of the implementation of the test cases.cases.  When installing multiple supporting softwareWhen installing multiple supporting software elements, the order of installation is critical to theelements, the order of installation is critical to the proper functioning of the environment.proper functioning of the environment.
  • 12. PreparePrepare Test InfrastructureTest Infrastructure An inventory of hardware:An inventory of hardware:  Properly setting up the test-execution environmentProperly setting up the test-execution environment requires to have access the inventory of hardwarerequires to have access the inventory of hardware that is needed, along with any special instructionsthat is needed, along with any special instructions for configuring the hardware.for configuring the hardware.  It is important to ensure that you execute the testIt is important to ensure that you execute the test case in the exact hardware configuration identifiedcase in the exact hardware configuration identified by the test cases.by the test cases.  Tester must confirm that the hardware meets allTester must confirm that the hardware meets all requirements and specifications such as those forrequirements and specifications such as those for memory, disk space, processor speeds and so on.memory, disk space, processor speeds and so on.
  • 13. PreparePrepare Test InfrastructureTest Infrastructure Code under test:Code under test:  Code under test, also referred to as a driver or aCode under test, also referred to as a driver or a build is the packaged new code that is ready to bebuild is the packaged new code that is ready to be installed and tested.installed and tested.  Properly preparing the environment for test-caseProperly preparing the environment for test-case execution also requires that you have access to theexecution also requires that you have access to the code under test.code under test.  Code is stored in a well-known place with namingCode is stored in a well-known place with naming conventions that indicate the exact driver or buildconventions that indicate the exact driver or build levels.levels.  Detailed procedures need to be in place to enableDetailed procedures need to be in place to enable you to install the code accurately and efficiently.you to install the code accurately and efficiently.
  • 14. PreparePrepare Test InfrastructureTest Infrastructure Workbook or execution logs:Workbook or execution logs:  Workbooks and execution logs are used to recordWorkbooks and execution logs are used to record the test-execution activity.the test-execution activity.  Workbooks are used when manually running tests,Workbooks are used when manually running tests, while execution logs are used to capture executionwhile execution logs are used to capture execution of tests run programmatically.of tests run programmatically.  Store the log files in a well-known place withStore the log files in a well-known place with naming conventions to indicate the date capturednaming conventions to indicate the date captured and the build level being tested.and the build level being tested.
  • 15. Conduct TestConduct Test Execute Test caseExecute Test case  The purpose of the task is toThe purpose of the task is to implement the test executionimplement the test execution plan, produce test results, andplan, produce test results, and compare between the expectedcompare between the expected test results and actual testtest results and actual test results.results.  After set up the environment,After set up the environment, ready to execute the tests.ready to execute the tests.  Execute each test exactly asExecute each test exactly as defined in the test-casedefined in the test-case implementation.implementation.
  • 16. Execute Test case – The processExecute Test case – The process Software to be tested Test Case Output Verifier Test Case Generation
  • 17. Testing processTesting process Design test cases Prepar e test data Run pr ogram with test da ta Compar e results to test cases Test cases Test data Test results Test repor ts InputInput Actual OutputActual Output Comparison ProcessComparison Process Test case : Specifies • Inputs + pre-test state of the software • Expected results (outputs and state) Test case : Specifies • Inputs + pre-test state of the software • Expected results (outputs and state)
  • 18. Testing policesTesting polices  Only exhaustive testing can show a programOnly exhaustive testing can show a program is free from defects. However, exhaustiveis free from defects. However, exhaustive testing is impossible,testing is impossible,  Testing policies define the approach to beTesting policies define the approach to be used in selecting system tests:used in selecting system tests: • All functions accessed through menusAll functions accessed through menus should be tested.should be tested. • Where user input is required, all functionsWhere user input is required, all functions must be tested with correct and incorrectmust be tested with correct and incorrect input.input.
  • 19. Testing guidelinesTesting guidelines  Testing guidelines are hints for the testingTesting guidelines are hints for the testing team to help them choose tests that willteam to help them choose tests that will reveal defects in the systemreveal defects in the system • Choose inputs that force the system toChoose inputs that force the system to generate all error messages.generate all error messages. • Design inputs that cause buffers to overflow.Design inputs that cause buffers to overflow. • Repeat the same input or input series severalRepeat the same input or input series several times.times. • Force invalid outputs to be generated.Force invalid outputs to be generated. • Force computation results to be too large orForce computation results to be too large or too small.too small.
  • 20. Verify the existence of the DefectVerify the existence of the Defect  Verify that the element under test,Verify that the element under test, the execution environment and thethe execution environment and the test case-implementation aretest case-implementation are consistent with the test case designconsistent with the test case design  Verify and confirm the discrepancyVerify and confirm the discrepancy between the expected test resultsbetween the expected test results and actual test results.and actual test results. Verify and Analyze ResultsVerify and Analyze Results
  • 21. Analyze the DefectAnalyze the Defect  The purpose of the task is toThe purpose of the task is to analyze and resolve variancesanalyze and resolve variances encountered during test execution.encountered during test execution.  Each variance is analyzed toEach variance is analyzed to determine the cause.determine the cause.  Analysis may indicate that the causeAnalysis may indicate that the cause of the variance is a defect, or that aof the variance is a defect, or that a change request is required.change request is required. Verify and Analyze ResultsVerify and Analyze Results
  • 22. Analyze the DefectAnalyze the Defect  Investigate further andInvestigate further and gather as muchgather as much information as possibleinformation as possible about the symptoms.about the symptoms.  Analyze failures by varyingAnalyze failures by varying conditions, options andconditions, options and settings to gauge thesettings to gauge the severity of the defectseverity of the defect Verify and Analyze ResultsVerify and Analyze Results
  • 23. Analyze failures by varying conditions, options and settingsAnalyze failures by varying conditions, options and settings  Change theChange the conditionsconditions under which an error occurred byunder which an error occurred by changing what you do. As an example, if the problem seemschanging what you do. As an example, if the problem seems related to entering data very quickly, try slowing your rate ofrelated to entering data very quickly, try slowing your rate of input.input.  Change theChange the options and settingsoptions and settings of the program underof the program under test. As an example, if the problem seems related to desktoptest. As an example, if the problem seems related to desktop screen-resolution settings, try varying the settings.screen-resolution settings, try varying the settings.  Change theChange the software and hardware environmentsoftware and hardware environment. For. For example, if the defect seems related to a low-end hardwareexample, if the defect seems related to a low-end hardware configuration, try using the lowest configuration specified inconfiguration, try using the lowest configuration specified in the requirements.the requirements.  Change theChange the datadata used by the program. As an example, if theused by the program. As an example, if the defect seems related to strings exceeding a certain numberdefect seems related to strings exceeding a certain number of characters in length, create a second set of test data toof characters in length, create a second set of test data to check the hypothesis.check the hypothesis. Verify and Analyze ResultsVerify and Analyze Results
  • 24. Documenting yourDocumenting your execution resultsexecution results  Test logs to recordTest logs to record test executiontest execution  Create a testCreate a test incident report forincident report for problemsproblems ReportingReporting Run Test Test Logs ERROR REPORT TEST CASES Verify & Analyze Result Bug Tracking
  • 25. Effective defectEffective defect reportsreports  Convey the properConvey the proper message, gets themessage, gets the job done andjob done and simplifies thesimplifies the process forprocess for everyone.everyone.  Can Pig RideCan Pig Ride technique to writetechnique to write effective incidenteffective incident reports.reports. ReportingReporting C Condense: Say it clearly but briefly A Accurate: Is it a user error or set-up problem? System problem? Application problem? Test-case problem? N Neutral: Just the facts; no zingers, no humor, no emotion, no sarcasm P Precise: Explicitly, what is the problem? I Isolate: What was done to isolate the problem? G Generalize: What was done to understand how general the problem is? R Re-create: What is essential to re-create this problem? I Impact: What is the impact if the bug were to surface in a customer environment D Debug: What will the developer need to debug this problem? E Evidence: What will prove the existence of the error?
  • 26. Test-Defect reportTest-Defect report  The IEEE Std. for Software Test Documentation forThe IEEE Std. for Software Test Documentation for Test Defect template offers the following fields.Test Defect template offers the following fields. ReportingReporting • Incident Summary report Identifier Specify the unique identifier assigned to this test-defect report. •Incident Summary Summarize the incident. Identify the test items involved, indicating their version and revision level. Supply references to the appropriate test-procedure specification, test-case specification and test log.
  • 27.  Incident Description Provide a description of the incident that includes the following items:  Inputs  Expected results  Actual results  Date and time  Procedure steps  Environment  Attempts to repeat  Testers  Observers  Related activities and observations Include any of these that may help to isolate the correct the cause of the incident. Test-Defect reportTest-Defect report  The IEEE Std. for Software Test Documentation forThe IEEE Std. for Software Test Documentation for Test Defect template offers the following fields.Test Defect template offers the following fields. ReportingReporting
  • 28. ResolveResolve  True defects - referred to the DefectTrue defects - referred to the Defect Management process.Management process.  Rated for severity and a priorityRated for severity and a priority  Change Requests - referred to the ChangeChange Requests - referred to the Change Management processManagement process Prioritizes the revision of requirements,Prioritizes the revision of requirements, the specification, design and constructionthe specification, design and construction of each item.of each item. Changed items reenter the test processChanged items reenter the test process ReportingReporting

×