GRIT: A Practical experience of the independent software ...Document Transcript
GRIT: A Practical experience of the independent software testing group at C.E.S.A.R.
Presented at the International Conference on Practical Software Quality and Testing (PSQT 2008 West)
Luiz Carlos da Silva Karina Karla de Oliveira Gomes Melissa Barbosa Pontes
C.E.S.A.R. C.E.S.A.R. C.E.S.A.R.
R. do Apolo, 81, Bairro do Recife R. do Apolo, 81, Bairro do Recife R. do Apolo, 81, Bairro do Recife
Recife – PE ,Brazil, CEP:50030-020 Recife – PE,Brazil, CEP:50030-220 Recife – PE, Brazil,CEP:50030-220
+55 81 3134-5748 +55 81 3134-5628 +55 81 3134-5610
Luiz.firstname.lastname@example.org Karina.email@example.com Melissa.firstname.lastname@example.org
Abstract of its mobile phones are tested. There are about 40
different mobile phone models released by Motorola
each year, and all of them are tested by the BTC.
This article describes the experiment of GRIT –
Grupo Independente de Testes de Software
(Independent Software Testing Group) at CESAR – There are three main C.E.S.A.R. projects
Centro de Estudos e Sistemas Avançados do Recife within BTC: Feature Test, Test Execution e Test
(Recife Center for Advanced Studies and Systems), a Automation. In all of them research investments are
CMMI3 private institute of research, development and made focusing on processes, techniques, strategies
innovation. Here, it will be described the group’s and tools for better performing the activities of
main activities, its objectives, its importance, the software verification and validation. For that purpose,
benefits the group’s creation brought to the there are around 200 people working in BTC projects.
organization, its starting structure, its difficulties, (free translation)
even the need of its structure reformulation.
What does GRIT mean?
Verification, Validation, Software, Testing, Process, By the implementation of Brazil Test Center
Research, Development, Innovation, Skill, program in C.E.S.A.R, it came up the necessity of
Improvement formation and specialization for the role of Test
Engineer within this organization. Therefore, more
Introduction and more professionals with this specific qualification
have been formed by C.E.S.A.R. what makes it a
reference center in software testing worldwide.
C.E.S.A.R. – Centro de Estudos e Sistemas
Avançados do Recife (Recife Center for Advanced
Studies and Systems) is a CMMI3 private institute of However, some projects of software
research, development and innovation, without profit development implemented by C.E.S.A.R. had their
purposes that have created software products, testing services done by other companies, and other
processes, services and enterprising companies since ones have not even had the allocation of test
1996 using information and communication engineers. That is why the software developers
technologies (TIC) . decided to carry out the testing activities themselves.
This way, in many cases, these activities were not
performed in the most appropriate way, since it is
In 2005, C.E.S.A.R. and Motorola, along with required specific technical skills and a most adequate
3 other Brazilian research institutes: Informatics profile for the engineers in charge of them .
Center of Pernambuco’s Federal University
(Cin/UFPE), Eldorado Institute and Santa Catarina ‘s
Federal University, established a partnership for the With the fast growth of the company and its
creation of BTC – Brazil Test Center, a worldwide projects, C.E.S.A.R. starts to face a new reality. By
testing center of Motorola in Recife, Brazil, where all one side there is an innovative software testing
project, producing first class research and expertise, improvement of testing process
and on the other side there are software development implementation in projects.
projects which are being tested by C.E.S.A.R.
developers, or by outsourced software testing • Contribute to the growth of software testing
companies. C.E.S.A.R. used to work based on three area inside the company and throughout the
different approaches concerned to verification and country – stimulating and guiding researches
validation of projects, though there has not been a and scientific productions in the area.
prior guideline to be followed by all the company.
Since the beginning, the GRIT was created as a
combination of a classical independent test group and
integrated test team. The group was independent by
the fact of not being subordinated to any development
project manager. On the other hand, the main purpose
was about allocating test engineers in development
projects and, in this case, these engineers would report
their activities to the same project manager as the
When it was created, the GRIT was carried out by
eight experienced test engineers. Three out of them
Figure 1 – C.E.S.A.R before the GRIT were totally allocated in software development
projects, and the other five ones continued allocated
into the BTC testing projects, and continued doing
Within this context, it was pointed out the
overtime in consulting and support. As the practical
necessity of not only allocating some experienced
results from the GRIT procedures have been known,
people in software testing for the projects, but also the
and the awareness of this test has been spread
necessity of guiding and monitoring the testing
throughout the company, more test engineers have
activities inside the organization. This way, the GRIT
been allocated into new development projects. In
(Independent Test Group), was created by C.E.S.A.R.
some situations it was necessary the hiring of new test
senior test management in October 2006, as a training
engineers to support the team. In many cases, these
project, in order to:
new professionals did not have enough experience
due to fact of this new professional profile not to be
• Allocate at least one test engineer for easily found in Brazil. And by this fact, they have
software development projects – gathering been trained by the company under the GRIT
information related to the project complexity, supervision.
time frame, available budget, experience of
According to Randall W. Rice, there is not a
test engineer to be allocated, and number of
standard relation between the number of tests
developers involved. Besides this
engineers and the number of software developers for a
information, an adequate number of test
project. However, it is possible to have an agreement
engineers were suggested for each project.
about the staffing ratio for developers and testers for
• Stimulate the awareness for the necessity of
software testing activities within the company Kathy Iberle  and Johanna Rothman 
– spreading knowledge in and out of the support the idea that each organization must define its
group through technical weekly presentations. variances taking into account the analysis of its own
• Track the test activities – providing support Focusing on C.E.S.A.R context and based on listed
for solving problems regarding the articles, the GRIT followed the variances below:
The consequences of an inappropriate group
• Related to the product: The extent of the structure
product, the quality of requirements
• Related to the project: The history of previous Although the GRIT actions have been
projects, complexity of projects, customer’s providing many benefits for the company, some
tolerance, service quality problems started to come up after one year of its
• Related to people: Experience level of implementation, making it difficult to carry on with
developers and tests engineers its activities. It would be necessary an urgent
reconfiguration for this unit not to lose power, and to
Based on these variances, and after identifying the continue existing.
project necessities, it is determined the number of
tests engineers necessary for the project, and it is
The first problem we had to cope with was
allocated at least one experienced professional to run
about the disorganized growth of the group, which
the tests activities. In case it is necessary a team
from eight participants went up to fifteen ones,
formation, this professional will be in charge to lead
making it difficult the physical allocation of them as
the group and spread out the awareness about the tests
well as the synchronization of their availability for
process throughout the organization. This way, it is
taking part of meetings. Besides this, the meetings
possible to identify the necessity of training for less
became difficult to be conducted due to the big
experienced professionals, shortening the learning
variety of specific topics to be discussed.
process, and making possible this in-house formation.
The consulting and training areas of GRIT are in
charge of identifying these opportunities. This big variety of specific topics started
making it difficult the formation of effective
specialists because the problem was discussed with all
There have been one-hour weekly meetings in
the participants, and only those who had time
order to provide interaction among the engineers, and
available, and interest, decided to be in charge of
to facilitate the tracking of procedures keeping the
solving the problem. This way, we did not have the
confidential information of each project. Therefore,
guarantee that it was the most appropriate person who
the problems found were discussed, and solutions
was being allocated for some specific activity. We
were suggested by the most experienced engineers. In
were losing the opportunity of solving problems with
these meetings the agenda was about the necessity of
someone who could do it much more efficiently only
training for specific situations, the evaluation of
because this specialist was not part of the group,
testing tools, the presentation of how to research the
either for not having interest, or not being available to
state of art in software testing, re-evaluating the
take part of weekly meetings. We were missing the
institute test process, etc.
control of our own knowledge.
As the group actions became better known
into the company, the people who did not use to be
part of the group realized that it would be important to
contribute, but as there were not clear instructions
about the access of new participants, it resulted in an
overload of activities for those who were already part
of the group.
For these reasons, it was necessary to change
the group structure in order to make it possible to
continue accomplishing our goals in an effective way.
Figure 2 – GRIT’s initial structure
Reconfiguration suggestions of how to solve occasional problems that
may occur during its course.
The new structure of GRIT divided it into seven areas It tracks the C.E.S.A.R. pre-sales in order to
of specific knowledge, and one area of operation, estimate the number of test engineers for each project
which were created by gathering the activities carried and to give support on planning test procedures in
out by the group. The areas are subordinated to general.
It stimulates innovative actions as well as
scientific and technological research, providing
integration between academic and market
perspectives as it guides and follows the submission
of essays, reports of previous experiences and work
that may raise the level of knowledge for the people
involved, also bringing up benefits for the company.
Figure 3 – GRIT’s reconfiguration
Each area has its own focus with specific It collects metrics from projects working with
a test team and does periodical analysis to track the
activities and purposes that are described as it follows:
efforts spent by tests procedures. It evaluates ways of
minimizing these efforts and checks how the
Management: collecting of metrics is being done in order to warrant
Area responsible to maintain the group’s its accuracy.
appropriate structure thought planning knowledge
area activities and to report the achieved results to the Training:
organization’s senior management.
It identifies improvement opportunities for
Operation: tests engineers and promotes their professional
development, providing lectures, training, and
It has as main purpose the maintenance of all workshops for a constant updating of their knowledge
GRIT operational structure and warrant that all the on the state of art of software testing. It does the
specific knowledge areas are accomplishing their mapping of training necessities for tests engineers and
goals. It is the area responsible for structuring an it is in charge of maintaining a schedule of training for
environment for announcing and tracking of activities each specific needs.
from each area, and also for establishing a way of
communication among the participants of the group. Performance:
Consulting: It gives support to development projects that
may have some performance problems within their
It defines, standards, and warrants the applications, and develops projects and studies about
appropriate functioning of the main procedures for engineering performance for these projects, in order to
software tests into a project, tracking it systematically use these studies for new partnerships and/or new
in order to guide, to identify risks, and to make projects for the customers involved. It has also the
purpose to make the local environment of tests the chosen area. The contribution may be occasional
(GRIT) and the C.E.S.A.R. important references in or in a regular basis. It is up to the collaborator, what
tests performance, spreading out their knowledge makes possible for anyone interested to take part of
through writing essays related to the corresponding the group, managing his time the best way he wishes,
area. in order to contribute for an area which he is
It provides the link between the generic GRIT Results
projects of the company with the tests process. It
identifies improvement aspects for the process, and During the first year of its activities, the group
also tracks the request of alteration as well as achieved its goals. The actions listed below show how
investigates ways of reconfiguration for tests process. it was done:
Besides, it is a decisive area for making decisions in
establishing institutional tools for the management of
tests. • The group was institutionalized into the
organization, and nowadays, the projects in
pre-sales are submitted to the approval of
Automation: GRIT in what concerns to the amount and
requirements for the tests that must be
It identifies within the existing projects which allocated/contracted for any new project.
ones may be automatized, and it is also responsible
for allocating people for this activity. It is responsible • Thirteen out of thirty-five development
for managing all automation process. projects which the organization has already
been carrying out are now working with test
The benefits of reconfiguration
• The tests activities have become much better
The new structure of GRIT provides great structured and uniform by processes.
agility for solving problems since that each area is
formed by consultants with specific skills who may
create their own team for the progressive development • According to questionnaires about customers’
of techniques as well as personal skills of each satisfaction which were applied before and
participant. after the actions of the group, they seem to be
much happier about their results.
By the actions performed into each area, it • Increasing of interest by the members of the
was able to make evident the importance of skills organization for the formation of study
development for test engineers in C.E.S.A.R., since groups, and the ways of getting get
each project requires test professionals with some certificates in tests area.
specific characteristics and people were becoming
specialists, more capable to reach specific areas goals.
• Essays accepted by national and international
Based on this new structure, the people who
are interested in contributing to GRIT must only
access the wiki page of the organization, and there
they will find all the actions distributed into areas.
Then, it is only necessary to choose the action wanted
and report how it is going focusing the perspective of
The creation of GRIT provided the organization a
greater testing activities structuring since the
beginning of a development project, creating an
increase in the quality of the client’s delivered product
and making it possible for the testing activities receive
its actual value within the software development
circle of life, promoting likewise a greater visibility of
This group made possible the sharing and
managing of testing knowledge not only inside
C.E.S.A.R. but also in the national market, mainly
through the first EBTS - Encontro Brasileiro de
Testes de Software (Brazilian Meeting of Software
Testers) - which is already in its second edition,
taking place at the city of Recife-Brazil, and the
The qualification and valorization of
people were accomplished goals, contributing for the
rise of a known professional profile: The testing
engineer, more and more necessary at the
The GRIT has the mission of continuing this
project, searching permanent development and
providing the improvement in the software testing
area in Brazil and worldwide.
 Urbano, Clemente C., Cidinha, Costa G., and Ana
Cláudia, D. (2007) Resultados práticos de V&V dos
projetos do C.E.S.A.R, REVVIS.
Craig, Rick D. and Jaskiel Stefan P. (2007),
Systematic Software Testing, Artech House
Publishers, 8th Edition
 Kathy Iberle and Sue Bartlett (2001) Estimating
Tester/Developer Ratios (or Not), Proceedings of the
Pacific Northwest Software Quality Conference.
 It Depends: Deciding on the Correct Ratio of
Developers to Testers,