Software testing basic concepts
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Software testing basic concepts

on

  • 5,396 views

 

Statistics

Views

Total Views
5,396
Views on SlideShare
5,396
Embed Views
0

Actions

Likes
4
Downloads
220
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Instill: lam cho tham nhuan dan, truyen dan
  • This slide lists some incorrect understandings about software testing
  • Teacher should talk about the required conditions for efficient testing margin: du tru
  • Instill: lam cho tham nhuan dan, truyen dan
  • Instill: lam cho tham nhuan dan, truyen dan
  • Pfleeger Page 313 Before running a test, review the test cases to verify that the cases are correct, feasible, and provide the desired degree of coverage, and demonstrate the desired functionality

Software testing basic concepts Presentation Transcript

  • 1. © Copyright 2006 FPT Software 1© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Software TestingSoftware Testing Basic ConceptsBasic Concepts Instructor: Dang Tu LinhInstructor: Dang Tu Linh
  • 2. © Copyright 2006 FPT Software 2© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 AgendaAgenda  Duration: 1.5 Hours  Purpose: Understand basic Test concepts  Audience: All Members  Content: 1. Introduction 2. Defect concepts 3. Test process 4. Test plan & test design 5. Testing strategy
  • 3. © Copyright 2006 FPT Software 3© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 IntroductionIntroduction Objectives: To arrive at a common understanding of the scope of testing activity To familiarize the audience with the terminology used in the testing work Understand the required test skills of developers
  • 4. © Copyright 2006 FPT Software 4© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Typical Project Blues…Typical Project Blues…
  • 5. © Copyright 2006 FPT Software 5© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing definitionTesting definition Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements - (IEEE 83a) Process:  Sequence of steps performed for a given purpose. (IEEE) Software Process  A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products.(SEI-CMM)
  • 6. © Copyright 2006 FPT Software 6© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing ObjectivesTesting Objectives Primary: Execute a program with the intent of finding errors to  Determine whether system meets specifications  Determine whether system meets user’s needs Secondary  Continuously improve the testing process
  • 7. © Copyright 2006 FPT Software 7© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 MisunderstandingsMisunderstandings about testingabout testing Testing is debugging Testing is not the job of a programmer If programmers were more careful testing would be unnecessary Testing never ends Testing activities start only after the coding is complete Testing is not a creative task
  • 8. © Copyright 2006 FPT Software 8© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Theory & RealityTheory & Reality Theory  Need sufficient time  Baselined & freezed requirements  Automation testing  Have trained testers: test process, business flow,… Reality  Time Pressures to keep delivery date  Continuously updated requirements  Don’t know when to stop: Update & update Requirement continuously (CRs)  Tester updates TC and re-tests  Manual testing  Lack of dedicated software test environment: share, steal  Inexperienced testers without appropriate training
  • 9. © Copyright 2006 FPT Software 9© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect conceptsDefect concepts Objectives: Understand the definition of defect  Understand the cost of a bug to software development  Understand the common types of defects and their perceived severity What is defect?What is defect? A defect is any error found by testing and reviewing activities (All errors found by internal reviewer, external reviewer and customer).
  • 10. © Copyright 2006 FPT Software 10© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Where defects come from?Where defects come from? Products  SRS, DD, Source code, Test cases, etc. Quality Control  Review, Test, Audit, Inspection Processes  Requirements, Design, Coding, Test, etc. Other sources:  Change requests, misunderstanding, carelessness, planning, etc.
  • 11. © Copyright 2006 FPT Software 11© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Cost of defectsCost of defects
  • 12. © Copyright 2006 FPT Software 12© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect still exists in the product, change status to ERROR FSoft Defect lifecycleFSoft Defect lifecycle Defect classification:  Open defects  Closed defects  Leakage Log defects Analyze defects Fix defects Re-test Step 1 Step 2 Step 3 Step 4 What: To log defect into DMS with clear information: project, module, stage, QC activity, severity, type, due date, tester, created date ... Who: Tester Defect’s status: ERROR What: -To analyze the causes of the defect, find- out solution and assign to person who is responsible for correcting it. - To accept the defect if it’s acceptable (by concession of authority or customer). Who: Project Leader Defect’s status: ASSIGNED / ACCEPTED What: To correct the defect and perform unit testing to make sure the defect has been removed. Who: Developer Defect’s status: CORRECTED What: To retest the function(s) containing the defect and make sure that the defect has been corrected completely Who: Tester Defect’s status: CLOSED
  • 13. © Copyright 2006 FPT Software 13© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect status: • Error • Assigned • Corrected • Cancelled • Accepted: by PM/PTL/Customer • Closed: by tester/PM/PTL Leakage Defects: • Acceptance test • After Release test • After Release review Fsoft Defect classificationFsoft Defect classification
  • 14. © Copyright 2006 FPT Software 14© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Fsoft Defect classificationFsoft Defect classification Defect type • Functionality o Requirement misunderstanding o Feature missing o Coding logic o Business logic • User Interface • Performance • Design issue • Coding standard Severity: How bad the problem is (some time disagreement btw. Dev & Tester)  Fatal: the system is broken  Serious: can not work around  Medium: impact the function  Cosmetic: not impact the function
  • 15. © Copyright 2006 FPT Software 15© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Fsoft Defect classificationFsoft Defect classification Defect origin  Requirement  Design  Coding  Test  CM  Document control Priority  Immediately  High priority  Normal priority  Low priority Work product  Software module  Software Package  ADD  DDD  Test Cases (UT/IT/ST) Quality control activity  Code review  Unit test  Integration test  System test  Document review  Inspection  Audit
  • 16. © Copyright 2006 FPT Software 16© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test ProcessTest Process Objectives:  Understand the steps followed for a test process  Understand test planning process  FSoft Test Process
  • 17. © Copyright 2006 FPT Software 17© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Tester CompetencyTester Competency Soft skills (non-technical attributes)  Discipline and Perseverance • Testing is repetitive and requests a lot of manual effort • Have the ability to withstand pressures and workload • Say ‘no’ to managers when quality is insufficient  Reading skills: study many docs of specs, design  Communication & Interpreter skills • Both Verbal and Written communication • Diplomatic skill • Be able to communicate with technical and non-technical people, engineers, managers, customers  Negative thinking: foresee things that can go wrong, evaluate risks  Attitude: ‘test to break’ attitude  Time management and Effort prioritization
  • 18. © Copyright 2006 FPT Software 18© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Tester CompetencyTester Competency Technical skills  Familiarity with software development architectures, processes  Familiarity with testing methodologies  Very clearly articulate specs, designs, business rules, inspection reports, configurations, code changes, TP, TC, bug reports, user manuals,…  Know how and where to look for bugs  Know how to report the defects effectively for quickly reproduce
  • 19. © Copyright 2006 FPT Software 19© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 The Test ProcessThe Test Process Data Analysis - Analyze root Causes of defects - Identify actions Test Execution Perform testing Bug Reporting - Bug Fixing - TC Updating Bug Tracking (Re-test) Test Report Test Preparation SRS study Create Test Design Review & Approval - Develop test cases, test scripts - Prepare test data Test Planning Create Test Plan (by Test leader) Definition Solution Construction Transition Termination Initiation Project life cycle Test process
  • 20. © Copyright 2006 FPT Software 20© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Input/Output for testingInput/Output for testing Input: • Customer requirements and Acceptance criteria • Change requests • Software Requirement Specification (SRS) • Design documents (ADD, DDD) • Programs (Modules) Output: • Test documents: Test plan, Test cases and procedures, Test script, Test data • Defect list • Test report (Pass/Fail of each TC, Summary report)
  • 21. © Copyright 2006 FPT Software 21© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 FSoft Test resourcesFSoft Test resources Guideline: Test process & Test guideline Templates for test documents  Test Plan  Test case specification  Test report  Defect analysis report FSOFT tools  Defect tracking tool: DMS  Test Effort tracking tool: Timesheet  Test schedule: FSoft Insight  Test automation tools – Rational Robot (Functional & Performance test) – OpenSTA (Open source) – Witir (Open source) FATA: Test Council
  • 22. © Copyright 2006 FPT Software 22© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test strategyTest strategy Objectives  Understand Unit Testing concept  Understand Integration Testing concept  Understand System Testing concept  Understand Acceptance Testing concept  Overview of Test types
  • 23. © Copyright 2006 FPT Software 23© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 V-ModelV-Model
  • 24. © Copyright 2006 FPT Software 24© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing stagesTesting stages There are 4 stages of Testing:  Unit Test  Integration Test  System Test  Acceptance Test
  • 25. © Copyright 2006 FPT Software 25© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Unit TestUnit Test  Purpose: to verify that the component/module functions properly  Check: – internal data structures – Logic – boundary conditions for input/output data  Normally: White box oriented  Doer: Development team  Metrics: Test coverage: – Statement – Branch – Path – Condition
  • 26. © Copyright 2006 FPT Software 26© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Integration TestIntegration Test  Purpose: • To ensure that code is implemented and designed properly • to take unit tested modules and build a program structure that has been dictated by design  IT only after UT  Combining the individual components to uncover errors associated with interfacing  Normally: Black box oriented  Doer: independent Test team
  • 27. © Copyright 2006 FPT Software 27© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 System TestSystem Test  Purpose: To ensure that the system does what the requirement specifies  Types – Function Testing – Performance Testing – Installation Testing  Normally: Black box oriented  Doer: independent Test team
  • 28. © Copyright 2006 FPT Software 28© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Acceptance TestAcceptance Test  Customer’s way to verify that what was wanted is what is built  Uncovers more than requirements discrepancies  Allows the customers to determine what they really want, whether specified in the document or not.  New problems may arise  Customers may decide that the problem as changed and a different solution is needed  Normally: Black box oriented  Doer: Customer/independent Test team
  • 29. © Copyright 2006 FPT Software 29© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing typesTesting types Functional test  Function testing  User Interface testing  Data and database integrity testing  Business cycle testing User Interface: Layout Performance test  Performance profiling  Load testing  Stress testing  Volume testing Access control & Security test Regression test
  • 30. © Copyright 2006 FPT Software 30© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test techniques/methodsTest techniques/methods White Box Testing  It guarantees that all independent paths within a module have been exercised at least once  Exercise all logical decisions on their true an false sides  Execute all loops their boundaries and within their operational bounds  Exercise internal data structures to assure their validity Black Box Testing  Focuses on functional requirement of the software  Derive sets of input conditions that will fully exercise all functional requirements for a program  Not an alternative to white box
  • 31. © Copyright 2006 FPT Software 31© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test Plan, Test DesignTest Plan, Test Design & Test Cases& Test Cases Objectives  Understand the principles of test plan & test design  Understand the concepts for test case design  Understand the structure of test cases
  • 32. © Copyright 2006 FPT Software 32© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test Planning: ObjectivesTest Planning: Objectives Test planning is to:  Define Test strategy: – What to test: list of requirements to test – How to test (Test techniques) – Test types – Test tool, ….  Define the responsibilities  Decide test metrics & manage through metrics  “When to Stop Testing?” – Not good Unit Test – Low quality of code,… – Meet requested Test coverage – All Test cases are tested and passed
  • 33. © Copyright 2006 FPT Software 33© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test PlanTest Plan  What to do  Who does it: Test team, Development team  How to do it: Techniques  When to do it
  • 34. © Copyright 2006 FPT Software 34© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test PlanTest Plan Test Plan to define:  Scope of test: stages and types of test  Risks may affect testing: quantify the loss of the risk  Training needs for testers  Requirements to test: what will be tested  Testing strategy: how will the test be performed  Completion criteria (Pass/Fail)  Criteria to stop testing  Tools to be used for testing  Testing resource and environment (Responsibilities of testers)  Test Milestone: schedule of testing activities  Test deliverables: outputs of testing
  • 35. © Copyright 2006 FPT Software 35© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 When to Stop Testing?When to Stop Testing? Not good UT: Defects of UT do not meet Defect plan Low quality of code: too many defects (over the number identified in planning phase/kick-off meeting) Does not meet requested Test coverage (by customer/PM). It must be identified at planning phase.
  • 36. © Copyright 2006 FPT Software 36© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test design structureTest design structure Specifications  Basic flow  Alternative flow Test Design  All test cases should be traceable to requirements  Design framework and rules for test cases (large/average items)  Plan number of test cases for normal, abnormal & boundary data  Assign testers for module/function/large items
  • 37. © Copyright 2006 FPT Software 37© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test DesignTest Design Structure/Framework for Test cases (TC): • Modules or functions • Large items • GUI • Functions • Performance Number of TC:  Total  For each module  For each type: GUI/Function  For normal/abnormal/boundary cases
  • 38. © Copyright 2006 FPT Software 38© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test CaseTest Case Test environment Test condition A test case includes: • Action (sample: click on Add button, on a link, input birth day…) • Input data (valid/invalid/boundary data) • Expected result (message, …)  Good test case:  Structure of TC is clear and reasonable.  Follow requirement closely.  Cover all of cases that can occur.  Description and test conditions is brief and easy to understand  Procedure and expected result is a step by step process
  • 39. © Copyright 2006 FPT Software 39© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing MeasurementTesting Measurement Objectives  Understand the need for metrics and measurement in Testing  Understand the minimal metrics required for the measurement of the testing activity
  • 40. © Copyright 2006 FPT Software 40© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Examples of testing measurement?Examples of testing measurement?  Product: We’ve tested 80% of the lines of code  Plan: We’ve run 80% of the test cases  Results: We’ve discovered 593 defects  Effort: We’ve worked 80 hours a week on this for 4 months. We’ve run 7,243 tests  Quality of Testing: Beta testers have found 30 defects that we missed. Our regression tests seems ineffective  History across projects: At this milestone on previous projects, we had fewer than 12% of the defects found still open. We should be at that percentage on this product too
  • 41. © Copyright 2006 FPT Software 41© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 How do we measure Testing inHow do we measure Testing in FSOFTFSOFT?? Measurements:  Defects in DMS  Test effort in Timesheet  Defect rate in Fsoft Insight: Weighted defects/ project size (in UCP)  Test coverage: number of executed test cases/ Total number of test cases  Test successful coverage: number of test cases executed successfully/total number of test cases  Defect removal efficiency Metrics to assess tester performance:  Test effectiveness • Weighted defects/ Test execution effort  Leakage • Weighted defects found after release/ project size
  • 42. © Copyright 2006 FPT Software 42© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Q & AQ & A