• Like
  • Save
Software testing basic concepts
Upcoming SlideShare
Loading in...5
×
 

Software testing basic concepts

on

  • 3,604 views

 

Statistics

Views

Total Views
3,604
Views on SlideShare
3,604
Embed Views
0

Actions

Likes
2
Downloads
125
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Instill: lam cho tham nhuan dan, truyen dan
  • This slide lists some incorrect understandings about software testing
  • Teacher should talk about the required conditions for efficient testing margin: du tru
  • Instill: lam cho tham nhuan dan, truyen dan
  • Instill: lam cho tham nhuan dan, truyen dan
  • Pfleeger Page 313 Before running a test, review the test cases to verify that the cases are correct, feasible, and provide the desired degree of coverage, and demonstrate the desired functionality

Software testing basic concepts Software testing basic concepts Presentation Transcript

  • © Copyright 2006 FPT Software 1© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Software TestingSoftware Testing Basic ConceptsBasic Concepts Instructor: Dang Tu LinhInstructor: Dang Tu Linh
  • © Copyright 2006 FPT Software 2© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 AgendaAgenda  Duration: 1.5 Hours  Purpose: Understand basic Test concepts  Audience: All Members  Content: 1. Introduction 2. Defect concepts 3. Test process 4. Test plan & test design 5. Testing strategy
  • © Copyright 2006 FPT Software 3© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 IntroductionIntroduction Objectives: To arrive at a common understanding of the scope of testing activity To familiarize the audience with the terminology used in the testing work Understand the required test skills of developers
  • © Copyright 2006 FPT Software 4© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Typical Project Blues…Typical Project Blues…
  • © Copyright 2006 FPT Software 5© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing definitionTesting definition Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements - (IEEE 83a) Process:  Sequence of steps performed for a given purpose. (IEEE) Software Process  A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products.(SEI-CMM)
  • © Copyright 2006 FPT Software 6© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing ObjectivesTesting Objectives Primary: Execute a program with the intent of finding errors to  Determine whether system meets specifications  Determine whether system meets user’s needs Secondary  Continuously improve the testing process
  • © Copyright 2006 FPT Software 7© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 MisunderstandingsMisunderstandings about testingabout testing Testing is debugging Testing is not the job of a programmer If programmers were more careful testing would be unnecessary Testing never ends Testing activities start only after the coding is complete Testing is not a creative task
  • © Copyright 2006 FPT Software 8© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Theory & RealityTheory & Reality Theory  Need sufficient time  Baselined & freezed requirements  Automation testing  Have trained testers: test process, business flow,… Reality  Time Pressures to keep delivery date  Continuously updated requirements  Don’t know when to stop: Update & update Requirement continuously (CRs)  Tester updates TC and re-tests  Manual testing  Lack of dedicated software test environment: share, steal  Inexperienced testers without appropriate training
  • © Copyright 2006 FPT Software 9© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect conceptsDefect concepts Objectives: Understand the definition of defect  Understand the cost of a bug to software development  Understand the common types of defects and their perceived severity What is defect?What is defect? A defect is any error found by testing and reviewing activities (All errors found by internal reviewer, external reviewer and customer).
  • © Copyright 2006 FPT Software 10© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Where defects come from?Where defects come from? Products  SRS, DD, Source code, Test cases, etc. Quality Control  Review, Test, Audit, Inspection Processes  Requirements, Design, Coding, Test, etc. Other sources:  Change requests, misunderstanding, carelessness, planning, etc.
  • © Copyright 2006 FPT Software 11© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Cost of defectsCost of defects
  • © Copyright 2006 FPT Software 12© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect still exists in the product, change status to ERROR FSoft Defect lifecycleFSoft Defect lifecycle Defect classification:  Open defects  Closed defects  Leakage Log defects Analyze defects Fix defects Re-test Step 1 Step 2 Step 3 Step 4 What: To log defect into DMS with clear information: project, module, stage, QC activity, severity, type, due date, tester, created date ... Who: Tester Defect’s status: ERROR What: -To analyze the causes of the defect, find- out solution and assign to person who is responsible for correcting it. - To accept the defect if it’s acceptable (by concession of authority or customer). Who: Project Leader Defect’s status: ASSIGNED / ACCEPTED What: To correct the defect and perform unit testing to make sure the defect has been removed. Who: Developer Defect’s status: CORRECTED What: To retest the function(s) containing the defect and make sure that the defect has been corrected completely Who: Tester Defect’s status: CLOSED
  • © Copyright 2006 FPT Software 13© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Defect status: • Error • Assigned • Corrected • Cancelled • Accepted: by PM/PTL/Customer • Closed: by tester/PM/PTL Leakage Defects: • Acceptance test • After Release test • After Release review Fsoft Defect classificationFsoft Defect classification
  • © Copyright 2006 FPT Software 14© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Fsoft Defect classificationFsoft Defect classification Defect type • Functionality o Requirement misunderstanding o Feature missing o Coding logic o Business logic • User Interface • Performance • Design issue • Coding standard Severity: How bad the problem is (some time disagreement btw. Dev & Tester)  Fatal: the system is broken  Serious: can not work around  Medium: impact the function  Cosmetic: not impact the function
  • © Copyright 2006 FPT Software 15© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Fsoft Defect classificationFsoft Defect classification Defect origin  Requirement  Design  Coding  Test  CM  Document control Priority  Immediately  High priority  Normal priority  Low priority Work product  Software module  Software Package  ADD  DDD  Test Cases (UT/IT/ST) Quality control activity  Code review  Unit test  Integration test  System test  Document review  Inspection  Audit
  • © Copyright 2006 FPT Software 16© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test ProcessTest Process Objectives:  Understand the steps followed for a test process  Understand test planning process  FSoft Test Process
  • © Copyright 2006 FPT Software 17© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Tester CompetencyTester Competency Soft skills (non-technical attributes)  Discipline and Perseverance • Testing is repetitive and requests a lot of manual effort • Have the ability to withstand pressures and workload • Say ‘no’ to managers when quality is insufficient  Reading skills: study many docs of specs, design  Communication & Interpreter skills • Both Verbal and Written communication • Diplomatic skill • Be able to communicate with technical and non-technical people, engineers, managers, customers  Negative thinking: foresee things that can go wrong, evaluate risks  Attitude: ‘test to break’ attitude  Time management and Effort prioritization
  • © Copyright 2006 FPT Software 18© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Tester CompetencyTester Competency Technical skills  Familiarity with software development architectures, processes  Familiarity with testing methodologies  Very clearly articulate specs, designs, business rules, inspection reports, configurations, code changes, TP, TC, bug reports, user manuals,…  Know how and where to look for bugs  Know how to report the defects effectively for quickly reproduce
  • © Copyright 2006 FPT Software 19© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 The Test ProcessThe Test Process Data Analysis - Analyze root Causes of defects - Identify actions Test Execution Perform testing Bug Reporting - Bug Fixing - TC Updating Bug Tracking (Re-test) Test Report Test Preparation SRS study Create Test Design Review & Approval - Develop test cases, test scripts - Prepare test data Test Planning Create Test Plan (by Test leader) Definition Solution Construction Transition Termination Initiation Project life cycle Test process
  • © Copyright 2006 FPT Software 20© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Input/Output for testingInput/Output for testing Input: • Customer requirements and Acceptance criteria • Change requests • Software Requirement Specification (SRS) • Design documents (ADD, DDD) • Programs (Modules) Output: • Test documents: Test plan, Test cases and procedures, Test script, Test data • Defect list • Test report (Pass/Fail of each TC, Summary report)
  • © Copyright 2006 FPT Software 21© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 FSoft Test resourcesFSoft Test resources Guideline: Test process & Test guideline Templates for test documents  Test Plan  Test case specification  Test report  Defect analysis report FSOFT tools  Defect tracking tool: DMS  Test Effort tracking tool: Timesheet  Test schedule: FSoft Insight  Test automation tools – Rational Robot (Functional & Performance test) – OpenSTA (Open source) – Witir (Open source) FATA: Test Council
  • © Copyright 2006 FPT Software 22© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test strategyTest strategy Objectives  Understand Unit Testing concept  Understand Integration Testing concept  Understand System Testing concept  Understand Acceptance Testing concept  Overview of Test types
  • © Copyright 2006 FPT Software 23© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 V-ModelV-Model
  • © Copyright 2006 FPT Software 24© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing stagesTesting stages There are 4 stages of Testing:  Unit Test  Integration Test  System Test  Acceptance Test
  • © Copyright 2006 FPT Software 25© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Unit TestUnit Test  Purpose: to verify that the component/module functions properly  Check: – internal data structures – Logic – boundary conditions for input/output data  Normally: White box oriented  Doer: Development team  Metrics: Test coverage: – Statement – Branch – Path – Condition
  • © Copyright 2006 FPT Software 26© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Integration TestIntegration Test  Purpose: • To ensure that code is implemented and designed properly • to take unit tested modules and build a program structure that has been dictated by design  IT only after UT  Combining the individual components to uncover errors associated with interfacing  Normally: Black box oriented  Doer: independent Test team
  • © Copyright 2006 FPT Software 27© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 System TestSystem Test  Purpose: To ensure that the system does what the requirement specifies  Types – Function Testing – Performance Testing – Installation Testing  Normally: Black box oriented  Doer: independent Test team
  • © Copyright 2006 FPT Software 28© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Acceptance TestAcceptance Test  Customer’s way to verify that what was wanted is what is built  Uncovers more than requirements discrepancies  Allows the customers to determine what they really want, whether specified in the document or not.  New problems may arise  Customers may decide that the problem as changed and a different solution is needed  Normally: Black box oriented  Doer: Customer/independent Test team
  • © Copyright 2006 FPT Software 29© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing typesTesting types Functional test  Function testing  User Interface testing  Data and database integrity testing  Business cycle testing User Interface: Layout Performance test  Performance profiling  Load testing  Stress testing  Volume testing Access control & Security test Regression test
  • © Copyright 2006 FPT Software 30© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test techniques/methodsTest techniques/methods White Box Testing  It guarantees that all independent paths within a module have been exercised at least once  Exercise all logical decisions on their true an false sides  Execute all loops their boundaries and within their operational bounds  Exercise internal data structures to assure their validity Black Box Testing  Focuses on functional requirement of the software  Derive sets of input conditions that will fully exercise all functional requirements for a program  Not an alternative to white box
  • © Copyright 2006 FPT Software 31© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test Plan, Test DesignTest Plan, Test Design & Test Cases& Test Cases Objectives  Understand the principles of test plan & test design  Understand the concepts for test case design  Understand the structure of test cases
  • © Copyright 2006 FPT Software 32© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test Planning: ObjectivesTest Planning: Objectives Test planning is to:  Define Test strategy: – What to test: list of requirements to test – How to test (Test techniques) – Test types – Test tool, ….  Define the responsibilities  Decide test metrics & manage through metrics  “When to Stop Testing?” – Not good Unit Test – Low quality of code,… – Meet requested Test coverage – All Test cases are tested and passed
  • © Copyright 2006 FPT Software 33© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test PlanTest Plan  What to do  Who does it: Test team, Development team  How to do it: Techniques  When to do it
  • © Copyright 2006 FPT Software 34© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test PlanTest Plan Test Plan to define:  Scope of test: stages and types of test  Risks may affect testing: quantify the loss of the risk  Training needs for testers  Requirements to test: what will be tested  Testing strategy: how will the test be performed  Completion criteria (Pass/Fail)  Criteria to stop testing  Tools to be used for testing  Testing resource and environment (Responsibilities of testers)  Test Milestone: schedule of testing activities  Test deliverables: outputs of testing
  • © Copyright 2006 FPT Software 35© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 When to Stop Testing?When to Stop Testing? Not good UT: Defects of UT do not meet Defect plan Low quality of code: too many defects (over the number identified in planning phase/kick-off meeting) Does not meet requested Test coverage (by customer/PM). It must be identified at planning phase.
  • © Copyright 2006 FPT Software 36© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test design structureTest design structure Specifications  Basic flow  Alternative flow Test Design  All test cases should be traceable to requirements  Design framework and rules for test cases (large/average items)  Plan number of test cases for normal, abnormal & boundary data  Assign testers for module/function/large items
  • © Copyright 2006 FPT Software 37© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test DesignTest Design Structure/Framework for Test cases (TC): • Modules or functions • Large items • GUI • Functions • Performance Number of TC:  Total  For each module  For each type: GUI/Function  For normal/abnormal/boundary cases
  • © Copyright 2006 FPT Software 38© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Test CaseTest Case Test environment Test condition A test case includes: • Action (sample: click on Add button, on a link, input birth day…) • Input data (valid/invalid/boundary data) • Expected result (message, …)  Good test case:  Structure of TC is clear and reasonable.  Follow requirement closely.  Cover all of cases that can occur.  Description and test conditions is brief and easy to understand  Procedure and expected result is a step by step process
  • © Copyright 2006 FPT Software 39© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Testing MeasurementTesting Measurement Objectives  Understand the need for metrics and measurement in Testing  Understand the minimal metrics required for the measurement of the testing activity
  • © Copyright 2006 FPT Software 40© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Examples of testing measurement?Examples of testing measurement?  Product: We’ve tested 80% of the lines of code  Plan: We’ve run 80% of the test cases  Results: We’ve discovered 593 defects  Effort: We’ve worked 80 hours a week on this for 4 months. We’ve run 7,243 tests  Quality of Testing: Beta testers have found 30 defects that we missed. Our regression tests seems ineffective  History across projects: At this milestone on previous projects, we had fewer than 12% of the defects found still open. We should be at that percentage on this product too
  • © Copyright 2006 FPT Software 41© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 How do we measure Testing inHow do we measure Testing in FSOFTFSOFT?? Measurements:  Defects in DMS  Test effort in Timesheet  Defect rate in Fsoft Insight: Weighted defects/ project size (in UCP)  Test coverage: number of executed test cases/ Total number of test cases  Test successful coverage: number of test cases executed successfully/total number of test cases  Defect removal efficiency Metrics to assess tester performance:  Test effectiveness • Weighted defects/ Test execution effort  Leakage • Weighted defects found after release/ project size
  • © Copyright 2006 FPT Software 42© FPT SOFTWARE – TRAINING MATERIAL – Internal use 04e-BM/NS/HDCV/FSOFT v2/2 Q & AQ & A