ROLE OF TESTING INCERTIFICATION PROCESSPresented byRishu Seth(firstname.lastname@example.org)M.Sc High Integrity SystemsUniversity of Applied Sciences - Frankfurt am Main
TABLE OF CONTENTS• Introduction• Certification Process• Role of Testing• Principle Certification Problems• Requirements for Testing based on Standards• Conclusion• References
INTRODUCTION• TESTING:According to Dictionary :‘To ascertain (the worth, capability or endurance)(of a person or thing) by subjection to certainexaminations.’ No universally accepted definition.• CERTIFICATION:A written guarantee that a system or componentcomplies with its specified requirements and isacceptable for operational use.
CERTIFICATION PROCESS• Carried out by government agencies or otherorganizations with a national standing.• Can be applied to either organizations,individuals, tools or methods.• Does not necessarily mean that the system iscorrect.
NEED FOR CERTIFICATION• Required for legal reasons.• Important for commercial reasons like havinga sales advantage.• Competence in specific areas.
PARTS OF CERTIFICATION PROCESS• Verification & Validation (V & V) – Parts oflong certification process.• V & V have become really important especiallyin software development process because ofincreased complexity.• V&V is essential from the very beginning ofthe development life cycle.
VERIFICATION• The process of evaluating a system or component todetermine whether the product of a givendevelopment phase satisfies the conditions imposedat the start of that phase. i.e."Are we building the product right?“• Two categories: Dynamic Testing Static Testing
VALIDATION• The process of evaluating a system or componentduring or at the end of the development process todetermine whether it satisfies specifiedrequirements i.e."Are we building the right product?“• Validation Techniques: Formal Methods Fault injectionHazard analysis
ROLE OF TESTING• Software Testing is a process of verifying andvalidating that a software application or program:Meets the business and technical requirementsthat guided its design and development.Works as expected.• Testing provides an insight into the risks that will beincurred if lower quality is accepted which is also theprinciple objective of Testing.
WHY TO DO TESTING?• Software testing is focused on finding defects in the final product.• Example of important defect that better testing would have found: In June 1996 the first flight of the European Space AgencysAriane 5 rocket failed shortly after launching, resulting in anuninsured loss of $500,000,000. The disaster was traced to thelack of exception handling for a floating-point error when a 64-bit integer was converted to a 16-bit signed integer.
WHAT DO WE TEST?• Firstly test what is important i.e. focus on the corefunctionality - the parts that are critical or popular.• The value of software testing is that it goes farbeyond testing the underlying code.• A comprehensive testing regime examines allcomponents associated with the application.
WHAT DO WE TEST?• Testing can involve some or all of the following factors:Business requirementsFunctional design requirementsTechnical design requirementsRegulatory requirementsProgrammer codeSystems administration standards and restrictionsCorporate standardsProfessional or trade association best practicesHardware configurationCultural issues and language differences
WHO DOES THE TESTING?• It is not a one person job.• It takes a team, size of which depends on theapplication.• The developer of the application should play areduced role.
TEST SPECIFICATION TECHNIQUES• White Box Testing: Based on the program code or technical design. Knowledge about the internal structure of the system plays animportant role. Other terms used for this kind of techniques are Glass-box orStructural Testing.• Black Box Testing: Based on functional specifications and quality requirements. Knowledge about the structure of the system is not used, butthe judgement is made merely from a functional point of viewof the system. Other terms used for this kind of techniques are Functional orBehavioural Testing.
V-MODEL OF SOFTWARE TESTING• V-Model of testing incorporates testing into theentire software development life cycle.• It illustrates that testing can and should start at thevery beginning of the project.• It also illustrates how each subsequent phase shouldverify and validate work done in the previous phaseand how work done during development is used toguide the individual testing phases.
V-MODEL OF SOFTWARE TESTING• Unit Testing - Tests the individual units of code thatcomprise the application.• System testing - Validates and verifies the functionaldesign specification and sees how all the moduleswork together.• Integration testing - Tests not only all thecomponents that are new or changed and areneeded to form a complete system, but it alsorequires involvement of other systems.
V-MODEL OF SOFTWARE TESTING• Integration Testing – Has sub-types: Compatibility Testing Load TestingStress Testing• User Acceptance Testing (Beta testing or end usertesting) – It is where testing moves from the handsof the IT department into those of the businessusers, who then perform the real world testing.• Production verification testing - It identifiesunexpected changes to existing processes introducedby the new applications.
V-MODEL OF SOFTWARE TESTING• The V-Model of testing identifies five software testing phases,each with a certain type of test associated with it.
PRINCIPLE CERTIFICATION PROBLEMS• Reliability Certification Problem: Object-oriented technology is good to producereusable modules.Components are often not reused if theirreliability cannot be guaranteed. Object-oriented technology itself does not specifyany particular testing methods.
PRINCIPLE CERTIFICATION PROBLEMS• Verification Methods Drawbacks: Every existing standard uses two softwareverification approaches to verify software.But they are not advanced enough in relation to thesafety integrity levels needed for the software. They are not entirely practical. Only practical demonstrations can validate theusability of some of the verification methods.
REQUIREMENTS FOR TESTING BASEDON STANDARDS• Aerospace: RTCA/DO-178B (EUROCAE ED12B):Software Verification: Verification is the mostimportant in DO-178B which accounts to over twothirds of the total process. It has differentcriticality levels.Level D: Software verification requires testcoverage of high-level requirement only.Level C: Low-level requirement testing isrequired.
REQUIREMENTS FOR TESTING BASEDON STANDARDSLevel B: Decision coverage is required.Level A: Code requires Modified ConditionDecision Coverage (MCDC).
REQUIREMENTS FOR TESTING BASEDON STANDARDS• Biomedical Engineering: IEC 60601-1-4Verification – A plan shall be produced to showhow the Safety requirements for eachDevelopment Life-cycle phase will be verified.Plan includes –Documentation of verification strategiesSelection and utilization of toolsCoverage criteria for Verification.
REQUIREMENTS FOR TESTING BASEDON STANDARDSValidation -A Validation plan shall be produced toshow that correct safety requirements have beenimplemented. The Validation shall be performedaccording to the Validation plan. The results ofValidation activities shall be documented,analyzed and assessed.
CONCLUSION• Software Testing has the potential to save time andmoney by identifying problems early and to improvecustomer satisfaction and safety by delivering a moreerror free product.• To make optimum use of testing, it should bepracticed throughout the development life cycle ofthe product.• The requirements for verification and validationshould be proactively modernized for advancementof the product with introduction of newtechnologies
REFERENCES• Verification/Validation/Certification, Carnegie Mellon University,18-849b Dependable Embedded Systems, Spring 1999, Author:Eushiuan Tran• Software Testing Fundamentals—Concepts, Roles, and Terminology,John E. Bentley, Wachovia Bank, Charlotte NC• Software Testing, A guide to the TMAP Approach, Martin Pol, RuudTeunissen, Erik van Veenendaal• Reliability Certification of Software Components, Claes Wohlin andBjörn Regnel, Department of Communication Systems, LundInstitute of Technology, Lund University, Box 118, SE-221 00 Lund,Sweden.• Software Safety Certification: A Multi-domain Problem, PatriciaRodriguez-Dapena, European Space Agency, Copyrighted IEEE
REFERENCES• Applying DO178B for IV & V of Safety criticalSoftware, White Paper, Sreekumar Panicker, WiproTechnologies• Testing Medical Devices, Written by GaryPowalisz, GE Healthcare Available:http://www.evaluationengineering.com/index.• php/solutions/instrumentation/testing-medical-devices.html. Last access• on:11/01/2011• IPL Testing Tools and IEC 61508, IPL InformationProcessing Ltd.,Eveleigh House, Grove Street, UK