• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement

Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement



The presentation by Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Sergio Oltolina, Mauro Regoli, Gabriele Ruffatti, was shown during the international conference on open source software OSS 2010, ...

The presentation by Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Sergio Oltolina, Mauro Regoli, Gabriele Ruffatti, was shown during the international conference on open source software OSS 2010, which took place at Notre Dame University, Indiana, USA, from 30th May to 2nd June 2010.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement Spago4Q and the Quest nD Model: an Open Source Solution for Software Performance Measurement Presentation Transcript

    • Spago4Q and the QEST nD Model: An Open Source Solution for Software Performance Measurement Claudio A. Ardagna, Ernesto Damiani, Fulvio Frati, Mauro Regoli Dipartimento di Tecnologie dell’Informazione Università degli Studi di Milano – Italy Sergio Oltolina, Gabriele Ruffatti Engineering Ingegneria Informatica - Italy
    • Outline • QEST nD Model • Spago4Q Open Source Platform • Roadmap to build the Integrated Framework • Case Study OSS 2010 2
    • Introduction • The availability of detailed and updated information on development process is paramount – To maintain the competitiveness level – To operate in new and more challenging markets • The scenario of integrated information is known as Business Intelligence, and enclose all the processes and tools for data acquisition and analysis • A number of structured process models have been adopted depending on the domain and size – To collect specific knowledge about their development processes – To strengthen the know-how in terms of more efficiency and quality OSS 2010 3
    • Goals • Deploy an environment that integrates models for software performance evaluation in a more complete and coherent framework – Non-invasive – Dynamic – Configurable – Scalable – Based on a formal mathematical model • Provide a solution integrating – The QEST nD model for process performance evaluation – The Open Source Business Intelligence platform Spago4Q OSS 2010 4
    • The QEST nD Model - 1 • Several mono-dimensional performance models are available in SE context that integrates individual measurements into a single performance index – Models are oversimplified and unable to reflect different performance aspects • Traditional domains (e.g., Business Modeling) exploit multi- dimensional models – Data derived directly from accounting systems, implying multiple analysis viewpoints – Complex viewpoints analyze distinct aspects of the overall process performance OSS 2010 5
    • The QEST nD Model - 2 • The QEST nD model is aimed at measuring software project performances addressing multi-dimensionality and qualitative-quantitative assessment • In the QEST model the quality can be viewed as the integration of at least three different viewpoints – Economical – Social – Technical • User-specific dimensions could be defined to evaluate ad-hoc analysis dimensions L. Buglione, “Misurare il software. Quantità, Qualità, Standards e Miglioramento di processo nell'Information & Communication Technology,” 3rd Edition, Franco Angeli, 2008 OSS 2010 6
    • Mathematical Formalization • QEST Performance value (P) given by the combination – Quantitative measurements (RP-Rough Productivity) – Qualitative measurements, calculated as a perception-based measurement of the overall product quality (QF-Quality Factor) OSS 2010 7
    • Mathematical Formalization - 2 • The basic 3-D model QEST is represented as a geometrical regular tetrahedron – Corners E, S, T and the convergence of the edges to the P vertex describe the performance level – Points Qe, Qs, Qt represent the respective dimension values (Rough Productivity) • Values calculated as the weighted sum of a list of n distinct measures • Represented as the distance between points and corners • If Quality factor is applied, points Q’e, Q’s, Q’t represent an upward or downward translation of Qe, Qs, Qt OSS 2010 8
    • Mathematical Formalization - 3 • Performance P calculated as the distance between the center of gravity of the original tetrahedron and the center of the plane (Q’e, Q’s, Q’t) section along the tetrahedron height • Through computational geometry, the multi-dimensional case is described with a simple formula for global performance value computation OSS 2010 9
    • Spago4Q • Spago4Q (SpagoBI for Quality) is an open source platform for the continuous monitoring of software quality – Multi-process multi-project monitoring – Collection of data executed in a fully-transparent way – Equipped with extractors specific of most-common software process environments (IDE, workflow management, text editing, …) • Spago4Q (www.spago4q.org) is a vertical adaptation of SpagoBI, the Open Source Business Intelligence suite (www.spagobi.org) • Recently adopted by European Commission – Directorate General for Regional Policy OSS 2010 10
    • Spago4Q Structure OSS 2010 11
    • Spago4Q Metamodel OSS 2010 12
    • An Integrated Environment • The definition of a QEST nD model is a five-step procedure fully supported by Spago4Q • The procedure is coherent with the PMAI (Plan-Measure- Assess-Improve) cycle: – PLAN, defining a set of KPIs, basing on the GQM approach, and dimensions that characterize the analysis – MEASURE, which includes the collection of data, and the computation of KPI values and global performance value – ASSESS, where results are presented in dashboards and reports – IMPROVE, where every negative or low value is deeply analyzed to find problems in the process OSS 2010 13
    • 1st Step: KPIs and Model Definition • Declaration of a complete GQM, with the definition of – the analysis dimensions – the concepts to measure – the KPIs to apply to project work-products OSS 2010 14
    • 2nd Step: Weights and Thresholds Definition • Couple each KPI with the respective weight – Indicates the importance that such a concept plays in the dimension it belongs to • Define the specific thresholds – Evaluates the value with respect to organization policies • Assign (if provided) the QF to each specific dimension OSS 2010 15
    • 3rd Step: Value Collection • Measures are taken directly from Spago4Q data warehouse – The DB is filled by data automatically collected by extractors accessing process work-products (code package, text documents, project information, …) • KPIs are described in terms of – Default value – Minimum and maximum values (for normalization) – KPI computation algorithm OSS 2010 16
    • 4th Step: Global Performance Computation • Global and dimension-wise performance indexes are computed as KPIs that take in input configuration data and results of the KPIs • Dimension performance is calculated as the weighted sums of each specific KPI OSS 2010 17
    • 5th Step: Reports • Sets of reports and dashboards could be defined and configured to satisfy reporting and managerial needs • Spago4Q provides methods and interfaces to directly configure and create new reports using all the facilities provided by the open source reporting tool Eclipse BIRT OSS 2010 18
    • Case Studies • Case study based on the monitoring of the complex performance of three real projects with several measurements in a three-month time slot • We use a QEST nD model called Business-Service Model and taking into consideration four specific analysis dimensions: 1. QEST-EC: Economic performance indicator 2. QEST-RS: Resource performance indicator 3. QEST-TE: Technical performance indicator 4. QEST-CS: Customer Satisfaction performance indicator OSS 2010 19
    • Case Studies: GQM OSS 2010 20
    • Case Studies: KPIs OSS 2010 21
    • Case Studies: Dashboards • Spago4Q dashboard gives an immediate snapshot of the situation, highlighting problems and suggesting areas that need a process quality improvement – Red squares indicate under-threshold performance values OSS 2010 22
    • Spago4Q: Reports • Reports indicate that all projects were concluded with an excellent global performance (close to one), showing some issues in the process that is worth analyzing – Project3 experienced bad performances in Technological and Economic dimensions OSS 2010 23
    • Conclusions • We provide a complete multi-project multi-process performance evaluation environment combining – The mathematical formalization of the QEST nD model – The monitoring and reporting facilities offered by Spago4Q • It gives the possibility to analyze the performance of the development process from different points of view integrating semantically different KPIs in a single indicator • The described QEST nD model will be made available in the future version of Spago4Q (www.spago4q.org) OSS 2010 24
    • Questions? Thank you for the attention OSS 2010 25