Spago4Q and the QEST nD Model:
               An Open Source Solution
for Software Performance Measurement
        Claudio...
Outline
• QEST nD Model

• Spago4Q Open Source Platform

• Roadmap to build the Integrated Framework

• Case Study

      ...
Introduction
• The availability of detailed and updated information on
  development process is paramount
    – To maintai...
Goals
• Deploy an environment that integrates models for
  software performance evaluation in a more complete
  and cohere...
The QEST nD Model - 1
• Several mono-dimensional performance models are available
  in SE context that integrates individu...
The QEST nD Model - 2
•   The QEST nD model is aimed at measuring software project performances
    addressing multi-dimen...
Mathematical Formalization
• QEST Performance value (P) given by the
  combination
  – Quantitative measurements (RP-Rough...
Mathematical Formalization - 2
•   The basic 3-D model QEST is
    represented as a geometrical regular
    tetrahedron
  ...
Mathematical Formalization - 3
•   Performance P calculated as the distance
    between the center of gravity of
    the o...
Spago4Q
•   Spago4Q (SpagoBI for Quality) is an open source platform for the
    continuous monitoring of software quality...
Spago4Q Structure




            OSS 2010   11
Spago4Q Metamodel




            OSS 2010   12
An Integrated Environment
• The definition of a QEST nD model is a five-step procedure
  fully supported by Spago4Q

• The...
1st Step: KPIs and Model Definition

• Declaration of a complete GQM, with the definition of
   – the analysis dimensions
...
2nd Step: Weights and Thresholds Definition

• Couple each KPI with the
  respective weight
   – Indicates the importance ...
3rd Step: Value Collection
• Measures are taken directly from Spago4Q data
  warehouse
  – The DB is filled by data automa...
4th Step: Global Performance Computation

• Global and dimension-wise performance
  indexes are computed as KPIs that take...
5th Step: Reports
• Sets of reports and dashboards could be
  defined and configured to satisfy reporting
  and managerial...
Case Studies
• Case study based on the monitoring of the complex
  performance of three real projects with several
  measu...
Case Studies: GQM




            OSS 2010   20
Case Studies: KPIs




             OSS 2010   21
Case Studies: Dashboards




• Spago4Q dashboard gives an immediate snapshot of
  the situation, highlighting problems and...
Spago4Q: Reports




• Reports indicate that all projects were concluded with an
  excellent global performance (close to ...
Conclusions
• We provide a complete multi-project multi-process
  performance evaluation environment combining
   – The ma...
Questions?




Thank you for the attention



                 OSS 2010     25
Upcoming SlideShare
Loading in …5
×

Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement

850 views

Published on

The presentation by Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Sergio Oltolina, Mauro Regoli, Gabriele Ruffatti, was shown during the international conference on open source software OSS 2010, which took place at Notre Dame University, Indiana, USA, from 30th May to 2nd June 2010.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement

  1. 1. Spago4Q and the QEST nD Model: An Open Source Solution for Software Performance Measurement Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Mauro Regoli Dipartimento di Tecnologie dell’Informazione Università degli Studi di Milano – Italy Sergio Oltolina, Gabriele Ruffatti Engineering Ingegneria Informatica - Italy
  2. 2. Outline • QEST nD Model • Spago4Q Open Source Platform • Roadmap to build the Integrated Framework • Case Study OSS 2010 2
  3. 3. Introduction • The availability of detailed and updated information on development process is paramount – To maintain the competitiveness level – To operate in new and more challenging markets • The scenario of integrated information is known as Business Intelligence, and enclose all the processes and tools for data acquisition and analysis • A number of structured process models have been adopted depending on the domain and size – To collect specific knowledge about their development processes – To strengthen the know-how in terms of more efficiency and quality OSS 2010 3
  4. 4. Goals • Deploy an environment that integrates models for software performance evaluation in a more complete and coherent framework – Non-invasive – Dynamic – Configurable – Scalable – Based on a formal mathematical model • Provide a solution integrating – The QEST nD model for process performance evaluation – The Open Source Business Intelligence platform Spago4Q OSS 2010 4
  5. 5. The QEST nD Model - 1 • Several mono-dimensional performance models are available in SE context that integrates individual measurements into a single performance index – Models are oversimplified and unable to reflect different performance aspects • Traditional domains (e.g., Business Modeling) exploit multi- dimensional models – Data derived directly from accounting systems, implying multiple analysis viewpoints – Complex viewpoints analyze distinct aspects of the overall process performance OSS 2010 5
  6. 6. The QEST nD Model - 2 • The QEST nD model is aimed at measuring software project performances addressing multi-dimensionality and qualitative-quantitative assessment • In the QEST model the quality can be viewed as the integration of at least three different viewpoints – Economical – Social – Technical • User-specific dimensions could be defined to evaluate ad-hoc analysis dimensions L. Buglione, “Misurare il software. Quantità, Qualità, Standards e Miglioramento di processo nell'Information & Communication Technology,” 3rd Edition, Franco Angeli, 2008 OSS 2010 6
  7. 7. Mathematical Formalization • QEST Performance value (P) given by the combination – Quantitative measurements (RP-Rough Productivity) – Qualitative measurements, calculated as a perception-based measurement of the overall product quality (QF-Quality Factor) OSS 2010 7
  8. 8. Mathematical Formalization - 2 • The basic 3-D model QEST is represented as a geometrical regular tetrahedron – Corners E, S, T and the convergence of the edges to the P vertex describe the performance level – Points Qe, Qs, Qt represent the respective dimension values (Rough Productivity) • Values calculated as the weighted sum of a list of n distinct measures • Represented as the distance between points and corners • If Quality factor is applied, points Q’e, Q’s, Q’t represent an upward or downward translation of Qe, Qs, Qt OSS 2010 8
  9. 9. Mathematical Formalization - 3 • Performance P calculated as the distance between the center of gravity of the original tetrahedron and the center of the plane (Q’e, Q’s, Q’t) section along the tetrahedron height • Through computational geometry, the multi-dimensional case is described with a simple formula for global performance value computation OSS 2010 9
  10. 10. Spago4Q • Spago4Q (SpagoBI for Quality) is an open source platform for the continuous monitoring of software quality – Multi-process multi-project monitoring – Collection of data executed in a fully-transparent way – Equipped with extractors specific of most-common software process environments (IDE, workflow management, text editing, …) • Spago4Q (www.spago4q.org) is a vertical adaptation of SpagoBI, the Open Source Business Intelligence suite (www.spagobi.org) • Recently adopted by European Commission – Directorate General for Regional Policy OSS 2010 10
  11. 11. Spago4Q Structure OSS 2010 11
  12. 12. Spago4Q Metamodel OSS 2010 12
  13. 13. An Integrated Environment • The definition of a QEST nD model is a five-step procedure fully supported by Spago4Q • The procedure is coherent with the PMAI (Plan-Measure- Assess-Improve) cycle: – PLAN, defining a set of KPIs, basing on the GQM approach, and dimensions that characterize the analysis – MEASURE, which includes the collection of data, and the computation of KPI values and global performance value – ASSESS, where results are presented in dashboards and reports – IMPROVE, where every negative or low value is deeply analyzed to find problems in the process OSS 2010 13
  14. 14. 1st Step: KPIs and Model Definition • Declaration of a complete GQM, with the definition of – the analysis dimensions – the concepts to measure – the KPIs to apply to project work-products OSS 2010 14
  15. 15. 2nd Step: Weights and Thresholds Definition • Couple each KPI with the respective weight – Indicates the importance that such a concept plays in the dimension it belongs to • Define the specific thresholds – Evaluates the value with respect to organization policies • Assign (if provided) the QF to each specific dimension OSS 2010 15
  16. 16. 3rd Step: Value Collection • Measures are taken directly from Spago4Q data warehouse – The DB is filled by data automatically collected by extractors accessing process work-products (code package, text documents, project information, …) • KPIs are described in terms of – Default value – Minimum and maximum values (for normalization) – KPI computation algorithm OSS 2010 16
  17. 17. 4th Step: Global Performance Computation • Global and dimension-wise performance indexes are computed as KPIs that take in input configuration data and results of the KPIs • Dimension performance is calculated as the weighted sums of each specific KPI OSS 2010 17
  18. 18. 5th Step: Reports • Sets of reports and dashboards could be defined and configured to satisfy reporting and managerial needs • Spago4Q provides methods and interfaces to directly configure and create new reports using all the facilities provided by the open source reporting tool Eclipse BIRT OSS 2010 18
  19. 19. Case Studies • Case study based on the monitoring of the complex performance of three real projects with several measurements in a three-month time slot • We use a QEST nD model called Business-Service Model and taking into consideration four specific analysis dimensions: 1. QEST-EC: Economic performance indicator 2. QEST-RS: Resource performance indicator 3. QEST-TE: Technical performance indicator 4. QEST-CS: Customer Satisfaction performance indicator OSS 2010 19
  20. 20. Case Studies: GQM OSS 2010 20
  21. 21. Case Studies: KPIs OSS 2010 21
  22. 22. Case Studies: Dashboards • Spago4Q dashboard gives an immediate snapshot of the situation, highlighting problems and suggesting areas that need a process quality improvement – Red squares indicate under-threshold performance values OSS 2010 22
  23. 23. Spago4Q: Reports • Reports indicate that all projects were concluded with an excellent global performance (close to one), showing some issues in the process that is worth analyzing – Project3 experienced bad performances in Technological and Economic dimensions OSS 2010 23
  24. 24. Conclusions • We provide a complete multi-project multi-process performance evaluation environment combining – The mathematical formalization of the QEST nD model – The monitoring and reporting facilities offered by Spago4Q • It gives the possibility to analyze the performance of the development process from different points of view integrating semantically different KPIs in a single indicator • The described QEST nD model will be made available in the future version of Spago4Q (www.spago4q.org) OSS 2010 24
  25. 25. Questions? Thank you for the attention OSS 2010 25

×