This paper shows the result of applying a new testing model which provides the know-how for performing the different activities covered in the test process for functional testing. It was noticed that the customer risks experienced during examining the accuracy of software used in different business sectors are not the main focus of the Quality Control team members. Moreover there are no standard testing techniques used by the team members during creating the test conditions and test cases, the result is a lot of reworks.

1. THE RESULT OF APPLYING A NEW TESTING MODEL FOR
IMPROVING THE QUALITY OF SOFTWARE PRODUCTS
Dr.Mona Ahmed Kadry
Vice Dean Graduate School of Business at AASTMT
Amira Soliman Ahmed
Master of Quality Management at AASTMT
ABSTRACT
This paper shows the result of applying a new testing model which provides the know-how for
performing the different activities covered in the test process for functional testing. It was noticed that
the customer risks experienced during examining the accuracy of software used in different business
sectors are not the main focus of the Quality Control team members. Moreover there are no standard
testing techniques used by the team members during creating the test conditions and test cases, the
result is a lot of reworks.
1. INTRODUCTION
1.1. About the TPIG
A new testing model named Testing Process Improvement Guide (TPIG) is designed by Software
Engineering Competence Center (SECC) to help software houses in Egypt to build modern and well-
founded testing capabilities and gain competitive edge with reference to product quality. An important
objective was to design a guide that helps companies align testing efforts with business objectives and
product risk. TPIG achieves this by providing companies with an out-of-the-box and ready-to-
implement process that is based on internationally proven standards and techniques. Another
important benefit of the TPIG is that it also helps companies maximize the return on investment made
in training and certifying their testing teams by having a process definition that is well aligned with
the common certification schemes. Similarly it helps testers to directly utilize the knowledge they
gained in training and certification in applying mature and detailed testing processes with strong focus
on business quality objectives. Thus TPIG closes the bridge between theory and implementation and
between personal career interests and organizational business objectives.
1.2. Challenges
The way the TPIG is built has to deal with the following challenges:
 Identifying the target context (application domain, technical domain, organization type and
size).
 Balancing prescription and description.
 Proper mapping to the prescriptive reference model.
 Project versus organization level descriptions.
 Setting the proper model weight and implementation overhead.
 Calibrating model’s data and techniques.
 Achieving scalability.

2. 1.3. The need for a new testing model
There are a number of reasons why a new testing model is needed. Process improvement models
are prescriptive rather than descriptive. These models set goals and expected practices but do not give
detailed procedural steps. In other words these models provide "what" testers should do but do not
provide any "how's".
They do not provide methods for specific activities such as risk assessment, size and effort
estimation, systematic test case determination, test management…etc. There is a need to have a
descriptive model that can act, with little customization, as the operational manual of the testing
organization. TPIG provides such description and includes lots of answers to the "how" question.
Other models are not created in a procedural format and require further effort to be incorporated
into company processes. This leads us to another reason why a new model is needed.
1.4. Implementation Context
The target context was identified at the beginning of the TPIG project through joint analysis
performed by SECC and SQS Europe. The identified context is described as follows:
 The model targets small and medium size software development organizations. As the ratio of
testers to developers is usually between 1:3 to 1:5, a small development organization with
around 10 to 15 developers typically has a testing team of 3 to 5. A medium organization
might have from 5 to 15 testers which is around 20 to 60 developers. Very small
organizations with one or two testers might have trouble trying to implement the model. A
testing team consisting of one test lead, one senior tester and one junior is the minimum to
start implementing the TPIG.
 The model targets co-located project team members or at least team members with the same
organizational affiliation. Organizations with distributed teams or with testing outsourced
might have special communication and coordination issues that are not addressed by the
model.
 The model targets applications with not-too-high level of product risk. The very high risk of
mission-critical applications such as aviation and health equipments' control software is not
the main focus. Mission-critical applications have special issues related to planning, tracking
and measurement, test analysis and design, and regression testing. TPIG product risk levels
start from risks that harm users' business (not including user life or health) to lower levels of
risk that impact other users' interests.
 Typical application domains include business applications such as Banking, Insurance, e-
Business, ERP…etc. Other application domains such as embedded software and controls
systems, graphics and multimedia applications are not the main focus. The main issue would
be the test analysis and design techniques which vary based on the application domain.
However, different versions of the model can be built following the same approach to target
different application domains.
 The model currently focuses on functional testing types and techniques. However it is built to
be scalable to accommodate non-functional and technical testing. Future versions of the
model should include detailed processes related to test automation and performance testing.
1.5. Approach
The following is the approach followed in designing the TPIG model:
1. Focusing on Business Value – Risk-driven testing
2. Reference Models

3. 3. Mixing the What's and the How's
4. Process Map and Modeling Dimensions
5. Procedural Structure
6. Traceability
7. The Toolkit approach
8. Usability
1.6. Methods & Techniques
1. Generic Risk Assessment
2. Detailed Risk Assessment
3. Effort Estimation
4. Test Analysis & Design
5. Incident Management (Ghanim, Osman and Aly, 2010:19).
2. METHODOLOGY
2.1. Type of Research
The type of research that used in this study is qualitative research and quantitative research.
Qualitative researcher aims to gather an in-depth understanding from the quality control managers
about applying the TPIG in their organizations. Moreover, the results will be examined by using
observations in numerical representations and through statistical analysis. Along with questionnaires
that have been given out to respondents for the statistical representation of the findings in the study,
interviews with the respondents and a few experts in this field will also be conducted.
(http://explorable.com/writing-methodology).
2.2. Sampling Method
The research sampling method that used in this study is to target almost all Egyptian IT
companies applying the TPIG, in order to obtain more specific and accurate result reflecting the
population
2.3. Interview
Unstructured interviews were conducted with the quality control managers in order to get
their feedback and comments about using the TPIG during the testing life cycle.
2.4. Questionnaire
The questionnaire is divided into four sections for each category of the TPIG('Test Policy and
Strategy', 'Test Planning', 'Test Monitor and Control’ and ' Test Design and Execution') using either
likert scale or true and false.

4. 2.5. Respondents
The respondents in this research are coming from multiple companies which applied the
TPIG. The respondents include two types, the first one is the Quality Control managers who managed
the TPIG implementation, and they have been communicated through unstructured interviews. The
second type is the Quality Control team members who implemented the TPIG, and they have been
communicated by answering the survey questions
3. THE RESULT
1. The Relation between applying the TPIG and company years of establishment:
Explanation: The study shows that no relation between applying the TPIG and company years of
establishment. As the above graph shows that there are some new companies in market but are
applying the TPIG better than old ones.
2. The Relation Between Applying The TPIG And The Company Size:
Explanation: The study shows that no relation between applying the TPIG and company size. As
shown from the above graph, companies with small size are applying the TPIG better that ones with
larger size.

5. 3. The Relation Between Applying The TPIG And Applying Quality Management
System (QMS):
Explanation: The study shows that no relation between applying the TPIG and applying QMS or not.
There are companies have no QMS and are applying the TPIG better than ones that have QMS.
4. Defect Removal Efficiency (DRE): The term defect removal efficiency refers to the
percentage of total defects found and removed before Software applications are delivered to
customers (Rex Black,2008:14).
DRE=[ *100 ]%
Since the latent defects in a software product is unknown at any point in time, it is approximated by
adding the number of defects removed during the phase to the number of defects found later (but that
existed during that phase).
(Westfall,1996:1).
Explanation: This study shows that after applying the TPIG, the DRE has been increased. This
indicates enhancement in the DRE after applying the TPIG for future projects.

6. 5. Defect Acceptance(DA):This metric determine the number of valid defects that
testing team has identified during execution
DA=[ *100]%
(B.Nirpal and V.Kale,2011:8).
Explanation: This study shows that after applying the TPIG, the DA has been increased. This
indicates enhancement in the DA after applying the TPIG for future projects.
6. Defect Rejection (DR): This metric determines the number of defects rejected during
execution.
DR=[ *100]%
(B.Nirpal and V.Kale,2011:8).
Explanation: This study shows that after applying the TPIG, the DR has been increased. This
indicates enhancement in the DR after applying the TPIG for future projects.
7. Test Efficiency (TE): This metric determine the efficiency of the testing team in identifying
the defects, it also identified the defects missed out during testing phase which migrated to the
next phase.
TE=[ *100]%
Where,
DT=Number of valid defects identified during testing

7. DU=Number of valid defects identified by user after release of application. In other words , post-
testing defect (B.Nirpal and V.Kale,2011:8).
Explanation: This study shows that after applying the TPIG, the TE has been increased. This
indicates enhancement in the TE after applying the TPIG for future projects.
8. Defect Severity Index (DSI): This metric determine the quality of the product under test and
at the time of release, based on which one can take decision for releasing of the product i.e. it
indicates the product quality
DFI=[ *100]%
The severity of the defects may take several values (Critical, Major, Medium and Minor).
Explanation: This study shows that after applying the TPIG, the DSI has been increased. This
indicates enhancement in the DSI after applying the TPIG for future projects.
9. Interviews Result: The TPIG is recommended to be applied for companies that didn’t have
their own testing process yet as it includes all the needed procedures starting from test
planning until test closure activities. Also, companies have their own testing process can
improve these processes by taking whatever they need to complete them, and to make them
more efficient, by applying the value added test estimation, test design techniques, measures,
and product risk approach that the TPIG provides. However, some sheets in the TPIG were
not applicable to use, as they either include so much details, or the inputs of these sheets are
maintained using other work products or automation tools. Some changes are needed in the
TPIG to adapt to projects that use Agile methodology.

8. 4. DISCUSSION
The work done in this study has depended on multiple companies that are applying the TPIG, and
results were collected by using questionnaires and conducting interviews with the Quality Control
managers at these companies.
After collecting the data and analyzing the collected measures, the results have shown that the
developed software quality has been improved after applying the TPIG, This quality improvement
appears during the testing life cycle, and after delivering the working software as well.
The limitation of this study is the number of companies that currently have applied the TPIG, the
results of this study could be changed after increasing the number of companies that will apply the
TPIG in the future.
5. CONCLUSION
The TPIG is a suitable model for most of organizations which tend to provide high quality
products and/or services to their customers, in order to meet the customer expectations and to consider
the customer risks along with all testing activities.
The suggested future research is to study the results of applying a new version from the TPIG
model that is based on Agile methodology.
6. REFRENCES
http://explorable.com/writing-methodology.
Linda Westfall (1996) Defect Removal Effectiveness.
Premal B. Nirpal and K. V. Kale (2011) IJCSE11-03-01-141:A Brief Overview Of Software
Testing Metrics.
Rex Black Consulting Services (www.rbcs-us.com) (2008)Measuring Defect Potentials and Defect
Removal Efficiency.
Yasser Ghanim,Hossam Osman and GamalAly(2010) TPIG Paper v1.0:A practical approach to
functional testing for SMEs.