Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

GRIT: A Practical experience of the independent software ...


Published on

  • Be the first to comment

  • Be the first to like this

GRIT: A Practical experience of the independent software ...

  1. 1. GRIT: A Practical experience of the independent software testing group at C.E.S.A.R. Presented at the International Conference on Practical Software Quality and Testing (PSQT 2008 West) Luiz Carlos da Silva Karina Karla de Oliveira Gomes Melissa Barbosa Pontes C.E.S.A.R. C.E.S.A.R. C.E.S.A.R. R. do Apolo, 81, Bairro do Recife R. do Apolo, 81, Bairro do Recife R. do Apolo, 81, Bairro do Recife Recife – PE ,Brazil, CEP:50030-020 Recife – PE,Brazil, CEP:50030-220 Recife – PE, Brazil,CEP:50030-220 +55 81 3134-5748 +55 81 3134-5628 +55 81 3134-5610 Abstract of its mobile phones are tested. There are about 40 different mobile phone models released by Motorola each year, and all of them are tested by the BTC[1]. This article describes the experiment of GRIT – Grupo Independente de Testes de Software (Independent Software Testing Group) at CESAR – There are three main C.E.S.A.R. projects Centro de Estudos e Sistemas Avançados do Recife within BTC: Feature Test, Test Execution e Test (Recife Center for Advanced Studies and Systems), a Automation. In all of them research investments are CMMI3 private institute of research, development and made focusing on processes, techniques, strategies innovation. Here, it will be described the group’s and tools for better performing the activities of main activities, its objectives, its importance, the software verification and validation. For that purpose, benefits the group’s creation brought to the there are around 200 people working in BTC projects. organization, its starting structure, its difficulties, (free translation)[1] even the need of its structure reformulation. What does GRIT mean? Keywords Verification, Validation, Software, Testing, Process, By the implementation of Brazil Test Center Research, Development, Innovation, Skill, program in C.E.S.A.R, it came up the necessity of Improvement formation and specialization for the role of Test Engineer within this organization. Therefore, more Introduction and more professionals with this specific qualification have been formed by C.E.S.A.R. what makes it a reference center in software testing worldwide. C.E.S.A.R. – Centro de Estudos e Sistemas Avançados do Recife (Recife Center for Advanced Studies and Systems) is a CMMI3 private institute of However, some projects of software research, development and innovation, without profit development implemented by C.E.S.A.R. had their purposes that have created software products, testing services done by other companies, and other processes, services and enterprising companies since ones have not even had the allocation of test 1996 using information and communication engineers. That is why the software developers technologies (TIC) [1]. decided to carry out the testing activities themselves. This way, in many cases, these activities were not performed in the most appropriate way, since it is In 2005, C.E.S.A.R. and Motorola, along with required specific technical skills and a most adequate 3 other Brazilian research institutes: Informatics profile for the engineers in charge of them [2]. Center of Pernambuco’s Federal University (Cin/UFPE), Eldorado Institute and Santa Catarina ‘s Federal University, established a partnership for the With the fast growth of the company and its creation of BTC – Brazil Test Center, a worldwide projects, C.E.S.A.R. starts to face a new reality. By testing center of Motorola in Recife, Brazil, where all one side there is an innovative software testing 1
  2. 2. project, producing first class research and expertise, improvement of testing process and on the other side there are software development implementation in projects. projects which are being tested by C.E.S.A.R. developers, or by outsourced software testing • Contribute to the growth of software testing companies. C.E.S.A.R. used to work based on three area inside the company and throughout the different approaches concerned to verification and country – stimulating and guiding researches validation of projects, though there has not been a and scientific productions in the area. prior guideline to be followed by all the company. Since the beginning, the GRIT was created as a combination of a classical independent test group and integrated test team. The group was independent by the fact of not being subordinated to any development project manager. On the other hand, the main purpose was about allocating test engineers in development projects and, in this case, these engineers would report their activities to the same project manager as the developers do. When it was created, the GRIT was carried out by eight experienced test engineers. Three out of them Figure 1 – C.E.S.A.R before the GRIT were totally allocated in software development projects, and the other five ones continued allocated into the BTC testing projects, and continued doing Within this context, it was pointed out the overtime in consulting and support. As the practical necessity of not only allocating some experienced results from the GRIT procedures have been known, people in software testing for the projects, but also the and the awareness of this test has been spread necessity of guiding and monitoring the testing throughout the company, more test engineers have activities inside the organization. This way, the GRIT been allocated into new development projects. In (Independent Test Group), was created by C.E.S.A.R. some situations it was necessary the hiring of new test senior test management in October 2006, as a training engineers to support the team. In many cases, these project, in order to: new professionals did not have enough experience due to fact of this new professional profile not to be • Allocate at least one test engineer for easily found in Brazil. And by this fact, they have software development projects – gathering been trained by the company under the GRIT information related to the project complexity, supervision. time frame, available budget, experience of According to Randall W. Rice, there is not a test engineer to be allocated, and number of standard relation between the number of tests developers involved. Besides this engineers and the number of software developers for a information, an adequate number of test project. However, it is possible to have an agreement engineers were suggested for each project. about the staffing ratio for developers and testers for each project. • Stimulate the awareness for the necessity of software testing activities within the company Kathy Iberle [2] and Johanna Rothman [3] – spreading knowledge in and out of the support the idea that each organization must define its group through technical weekly presentations. variances taking into account the analysis of its own context. • Track the test activities – providing support Focusing on C.E.S.A.R context and based on listed for solving problems regarding the articles, the GRIT followed the variances below: 2
  3. 3. The consequences of an inappropriate group • Related to the product: The extent of the structure product, the quality of requirements • Related to the project: The history of previous Although the GRIT actions have been projects, complexity of projects, customer’s providing many benefits for the company, some tolerance, service quality problems started to come up after one year of its • Related to people: Experience level of implementation, making it difficult to carry on with developers and tests engineers its activities. It would be necessary an urgent reconfiguration for this unit not to lose power, and to Based on these variances, and after identifying the continue existing. project necessities, it is determined the number of tests engineers necessary for the project, and it is The first problem we had to cope with was allocated at least one experienced professional to run about the disorganized growth of the group, which the tests activities. In case it is necessary a team from eight participants went up to fifteen ones, formation, this professional will be in charge to lead making it difficult the physical allocation of them as the group and spread out the awareness about the tests well as the synchronization of their availability for process throughout the organization. This way, it is taking part of meetings. Besides this, the meetings possible to identify the necessity of training for less became difficult to be conducted due to the big experienced professionals, shortening the learning variety of specific topics to be discussed. process, and making possible this in-house formation. The consulting and training areas of GRIT are in charge of identifying these opportunities. This big variety of specific topics started making it difficult the formation of effective specialists because the problem was discussed with all There have been one-hour weekly meetings in the participants, and only those who had time order to provide interaction among the engineers, and available, and interest, decided to be in charge of to facilitate the tracking of procedures keeping the solving the problem. This way, we did not have the confidential information of each project. Therefore, guarantee that it was the most appropriate person who the problems found were discussed, and solutions was being allocated for some specific activity. We were suggested by the most experienced engineers. In were losing the opportunity of solving problems with these meetings the agenda was about the necessity of someone who could do it much more efficiently only training for specific situations, the evaluation of because this specialist was not part of the group, testing tools, the presentation of how to research the either for not having interest, or not being available to state of art in software testing, re-evaluating the take part of weekly meetings. We were missing the institute test process, etc. control of our own knowledge. As the group actions became better known into the company, the people who did not use to be part of the group realized that it would be important to contribute, but as there were not clear instructions about the access of new participants, it resulted in an overload of activities for those who were already part of the group. For these reasons, it was necessary to change the group structure in order to make it possible to continue accomplishing our goals in an effective way. Figure 2 – GRIT’s initial structure 3
  4. 4. Reconfiguration suggestions of how to solve occasional problems that may occur during its course. The new structure of GRIT divided it into seven areas It tracks the C.E.S.A.R. pre-sales in order to of specific knowledge, and one area of operation, estimate the number of test engineers for each project which were created by gathering the activities carried and to give support on planning test procedures in out by the group. The areas are subordinated to general. manager supervision. Innovation/Research: It stimulates innovative actions as well as scientific and technological research, providing integration between academic and market perspectives as it guides and follows the submission of essays, reports of previous experiences and work that may raise the level of knowledge for the people involved, also bringing up benefits for the company. Tests Metrics: Figure 3 – GRIT’s reconfiguration Each area has its own focus with specific It collects metrics from projects working with a test team and does periodical analysis to track the activities and purposes that are described as it follows: efforts spent by tests procedures. It evaluates ways of minimizing these efforts and checks how the Management: collecting of metrics is being done in order to warrant Area responsible to maintain the group’s its accuracy. appropriate structure thought planning knowledge area activities and to report the achieved results to the Training: organization’s senior management. It identifies improvement opportunities for Operation: tests engineers and promotes their professional development, providing lectures, training, and It has as main purpose the maintenance of all workshops for a constant updating of their knowledge GRIT operational structure and warrant that all the on the state of art of software testing. It does the specific knowledge areas are accomplishing their mapping of training necessities for tests engineers and goals. It is the area responsible for structuring an it is in charge of maintaining a schedule of training for environment for announcing and tracking of activities each specific needs. from each area, and also for establishing a way of communication among the participants of the group. Performance: Consulting: It gives support to development projects that may have some performance problems within their It defines, standards, and warrants the applications, and develops projects and studies about appropriate functioning of the main procedures for engineering performance for these projects, in order to software tests into a project, tracking it systematically use these studies for new partnerships and/or new in order to guide, to identify risks, and to make projects for the customers involved. It has also the 4
  5. 5. purpose to make the local environment of tests the chosen area. The contribution may be occasional (GRIT) and the C.E.S.A.R. important references in or in a regular basis. It is up to the collaborator, what tests performance, spreading out their knowledge makes possible for anyone interested to take part of through writing essays related to the corresponding the group, managing his time the best way he wishes, area. in order to contribute for an area which he is interested in. Process: It provides the link between the generic GRIT Results projects of the company with the tests process. It identifies improvement aspects for the process, and During the first year of its activities, the group also tracks the request of alteration as well as achieved its goals. The actions listed below show how investigates ways of reconfiguration for tests process. it was done: Besides, it is a decisive area for making decisions in establishing institutional tools for the management of tests. • The group was institutionalized into the organization, and nowadays, the projects in pre-sales are submitted to the approval of Automation: GRIT in what concerns to the amount and requirements for the tests that must be It identifies within the existing projects which allocated/contracted for any new project. ones may be automatized, and it is also responsible for allocating people for this activity. It is responsible • Thirteen out of thirty-five development for managing all automation process. projects which the organization has already been carrying out are now working with test engineers. The benefits of reconfiguration • The tests activities have become much better The new structure of GRIT provides great structured and uniform by processes. agility for solving problems since that each area is formed by consultants with specific skills who may create their own team for the progressive development • According to questionnaires about customers’ of techniques as well as personal skills of each satisfaction which were applied before and participant. after the actions of the group, they seem to be much happier about their results. By the actions performed into each area, it • Increasing of interest by the members of the was able to make evident the importance of skills organization for the formation of study development for test engineers in C.E.S.A.R., since groups, and the ways of getting get each project requires test professionals with some certificates in tests area. specific characteristics and people were becoming specialists, more capable to reach specific areas goals. • Essays accepted by national and international academic events. Based on this new structure, the people who are interested in contributing to GRIT must only access the wiki page of the organization, and there they will find all the actions distributed into areas. Then, it is only necessary to choose the action wanted and report how it is going focusing the perspective of 5
  6. 6. Conclusions The creation of GRIT provided the organization a greater testing activities structuring since the beginning of a development project, creating an increase in the quality of the client’s delivered product and making it possible for the testing activities receive its actual value within the software development circle of life, promoting likewise a greater visibility of the area. This group made possible the sharing and managing of testing knowledge not only inside C.E.S.A.R. but also in the national market, mainly through the first EBTS - Encontro Brasileiro de Testes de Software (Brazilian Meeting of Software Testers) - which is already in its second edition, taking place at the city of Recife-Brazil, and the forthcoming partnerships. The qualification and valorization of people were accomplished goals, contributing for the rise of a known professional profile: The testing engineer, more and more necessary at the organizations. The GRIT has the mission of continuing this project, searching permanent development and providing the improvement in the software testing area in Brazil and worldwide. References [1] Urbano, Clemente C., Cidinha, Costa G., and Ana Cláudia, D. (2007) Resultados práticos de V&V dos projetos do C.E.S.A.R, REVVIS. [2]Craig, Rick D. and Jaskiel Stefan P. (2007), Systematic Software Testing, Artech House Publishers, 8th Edition [3] Kathy Iberle and Sue Bartlett (2001) Estimating Tester/Developer Ratios (or Not), Proceedings of the Pacific Northwest Software Quality Conference. [4] It Depends: Deciding on the Correct Ratio of Developers to Testers, ItDepends.html 6