• Like
Improving Quality and Cost-effectiveness in  Enterprise Software Application Development:  an Open, Holistic Approach for Project Monitoring & Control
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Improving Quality and Cost-effectiveness in Enterprise Software Application Development: an Open, Holistic Approach for Project Monitoring & Control

  • 243 views
Published

This presentation describes a more comprehensive approach to project monitoring & control using the QEST/LIME family of models. Such models have been included also in the SPAGO4Q (Spago for …

This presentation describes a more comprehensive approach to project monitoring & control using the QEST/LIME family of models. Such models have been included also in the SPAGO4Q (Spago for Quality) OSS. The lessons learned from two case studies are presented here.

Published in Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
243
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ICSOB 2011 2° Int. Conference on Software Business Brussels (Belgium), June 8-10 2011 Improving Quality and Cost-effectiveness in Enterprise Software Application Development:an Open, Holistic Approach for Project Monitoring & Control Ernesto Damiani, Fulvio Frati Università degli Studi di Milano, Italy Luigi Buglione, Sergio Oltolina, Gabriele Ruffatti Engineering Group, Italy 2nd International Conference on Software Business Brussels, 8-10 June 2011
  • 2. Outline• Introduction• The Puzzle of Project Monitoring & Control• Case study• Conclusions & Future Works 2nd International Conference on Software Business Brussels, 8-10 June 2011 2
  • 3. Need for OS SPM Tools• Open Source Software (OSS) relevance rapidly increased mainly for the OSS capability – Fostering open knowledge sharing across organizations – Reducing Total Cost of Ownership (TCO) – Increasing Return on Investment (ROI)• A large amount of OSS is freely available covering plenty of informative and business goals – However, very few OSS projects deal with goal-oriented measurement gathering data directly from the organization’s information systems 2nd International Conference on Software Business Brussels, 8-10 June 2011 3
  • 4. Our Objective• Describe a new, more comprehensive approach to software project management• Propose a roadmap to set up and manage a reliable and efficient measurement framework• Present Spago4Q, a complete open source platform for process monitoring and automatic data gathering 2nd International Conference on Software Business Brussels, 8-10 June 2011 4
  • 5. The Puzzle of Project Monitoring & Control• Measurement plan set-up often influenced by – Adoption of “standard measures” following an adoption-by-analogy approach – Reduction of budget devoted to measurement & monitoring, leading to a lower level of control on the project – Lack or scarcity of reliable data, making it hard to bind the mean relative error to the phenomena to be predicted• A sound analysis of informative needs, measurements, and metrics represents the main pieces of our Project Monitoring & Control framework 2nd International Conference on Software Business Brussels, 8-10 June 2011 5
  • 6. GQM Approach - 1• Propose a three-tier decomposition, deriving measures (M) from the related questions (Q) to be posed for answering the information goals (G) of the interested stakeholders – www.gqm.nl• GQM-based measurement frameworks are multi-faceted supporting multiple perspectives and analysis dimensions• the Measurement Information Model (MIM) links the information needs to its measurable entities and related attributes – Refines and reinforces the basic GQM idea, stressing the central role of information needs and the instrumental role of measures – Ref: ISO/IEC 15939:2007, Appendix A 2nd International Conference on Software Business Brussels, 8-10 June 2011 6
  • 7. Measurement Information Model (MIM) 2nd International Conference on Software Business Brussels, 8-10 June 2011 7
  • 8. BMP Technique• BMP (Balancing Multiple Perspective) is a technique for determining the right number of measures to be applied within a measurement plan• www.semq.eu/leng/modtechbmp.htm• Characterized by four steps: 1. Determine the dimensions of interest in the project 2. Determine the list of the most representative measures associated with each dimension 3. Identify which other control variables might be impacted negatively (e.g. higher quality often means greater initial costs or longer project duration); 4. Figure out the best combination of indicators to build a measurement plan for the project. 2nd International Conference on Software Business Brussels, 8-10 June 2011 8
  • 9. The QEST-LIME Family• Performance evaluation model designed to tackle decision-making process from a multi-perspective viewpoint – Analyses process Quality under, at least, the Economic, Social, and Technological dimensions – Provides an open, multidimensional shell according to the management objectives of each specific development project – Expresses performance into a unique, single value, as the combination of the specific measures• The QEST is extended by the LIME model to manage dynamic contexts – Collection, monitoring, and control of multidimensional measures are applied at each software life cycle phase – http://www.semq.eu/leng/modtechqlm.htm 2nd International Conference on Software Business Brussels, 8-10 June 2011 9
  • 10. Spago4Q• Spago4Q (SpagoBI for Quality) is – www.spago4q.org – An OS platform for the continuous monitoring of software quality – A vertical adaptation of OS Business Intelligence suite SpagoBI, – Recently adopted by  the European Commission – Directorate General for Regional Policy (DG-REGIO)• Spago4Q provides – Multi-process multi-project monitoring – Automatic data collection executed in a fully transparent way – Equipped with extractors specific of most-common software process environments (IDE, workflow management, text editing, etc.)• Based on a modular metamodel that allows to apply, at the same time, to running projects adopting different development process, the same measurement framework 2nd International Conference on Software Business Brussels, 8-10 June 2011 10
  • 11. Spago4Q Metamodel2nd International Conference on Software Business Brussels, 8-10 June 2011 11
  • 12. Spago4Q Architecture2nd International Conference on Software Business Brussels, 8-10 June 2011 12
  • 13. Putting all Pieces Together• GQM, BMP, QEST-LIME, and tools like Spago4Q provide the pieces needed to solve the process monitoring puzzle• Our approach proposes to join them, keeping the best from each – Define what we need to measure (using a GQM-based technique) – Filter and prioritize what we need to measure (using the BMP technique). – Determine the project/activity performance values with a holistic view (using QEST- LIME), highlighting the improvement goals to manage. – Automate the last step with an OSS tool (e.g. Spago4Q), providing faster and transparent data collecting 2nd International Conference on Software Business Brussels, 8-10 June 2011 13
  • 14. Case Study – 1• The framework was applied to two large proprietary products of Engineering Health Software Factory (HSF) – projects data were analyzed and collected in the period from January 2009 to October 2010 2nd International Conference on Software Business Brussels, 8-10 June 2011 14
  • 15. Case Study – 2• To monitor each phase of the HSF software life cycle, we exploit BMP and LIME• The main business goal for our goal-oriented analysis was to reduce the overall production cost• Such cost is mainly driven by three factors: – Lack of requirement stability, as a source of overhead in design and implementation activities – Working group management overhead, as a source of delays and variance in milestones – Corrective and adaptive maintenance activities• Cost factors allowed to define the measurement goals and the complete list of measures, associated with the respective goal 2nd International Conference on Software Business Brussels, 8-10 June 2011 15
  • 16. Cost FactorsRequirements (analysis)
  • 17. Measurements w.r.t. Cost Factors CMMI-DEVGoal Measure Measure Id. v1.3 PAsREQ-E- 1 Incidence of delays on delivery milestones (deliverables) w.r.t. total # deliverables PMC – MA PR-REQ-E-M1.1 RD – REQM -REQ-E-2 Requirements Variability PR-REQ-E-M1.2 PMCREQ-S-1 #. detected criticalities during the human resources management w.r.t. group size in the phase PPQA – PMC PR-REQ-S-M1.1REQ-S- 2 User satisfaction or % users involvement in the phase all PA (GP2.7 PR-REQ-S-M1.2REQ-T- 1 Document quality: respect of quality standard PPQA PR-REQ-T-M1.1DEV-E- 1 Incidence of delays on delivery milestones (deliverables) w.r.t. total # deliverables PMC – MA PR-DEV-E-M2.1DEV-S- 1 # detected criticalities during the human resources management w.r.t. group size in the phase PPQA – PMC PR-DEV-S-M2.1DEV-T- 1 Document quality: respect of quality standard PPQA PR-DEV-T-M2.1DEV-T- 2 Software quality: complexity, compliance, maintainability PPQA – VAL PR-DEV-T-M2.2DEV-T- 3 Compliance with end-phase checklist PPQA PR-DEV-T-M2.3TES-E- 1 # Defects detected before System Test by PR or verifications w.r.t. code size VER – VA – PMC PR-TES-E-M3.1TES-E- 2 Incidence of delays on delivery milestones (deliverables) w.r.t. total # deliverables PMC – MA PR-TES-E-M3.2TES-S-1 N.o. detected criticalities during the human resources management w.r.t. group size in the phase PPQA – PMC PR-TES-S-M3.1 Incidence of the n.o. reviews (peer reviews or inspection reviews) w.r.t. total # deliverables PPQA – VER PR-TES-T-M3.1TES-T- 1 Percentage defects distribution on phases that produced the defects (consider only analysis and design phase) PP – PMC – MA PR-TES-T-M3.2TES-T- 2 Compliance with end-phase checklist PPQA – PMC PR-TES-T-M3.3 Incidence of defects tested in running and testing phase w.r.t. maintained code size (Lines of Code or Function PMC – MA PR-MAN-E-M4.1MAN-E- 1 Points) Mean defect resolution time w.r.t. severity during running phase IRP PR-MAN-E-M4.2MAN-S- 1 N.o. detected criticalities during the human resources management w.r.t. group size in the phase PPQA – PMC PR-MAN-S-M4.1MAN-S- 2 User satisfaction all PA (GP2.7) PR-MAN-S-M4.2MAN-T-1 Percentage defects distribution on phases that produced the defects (consider only analysis and design phase) PP – PMC - MA PR-MAN-T-M4.1
  • 18. Results analysis - 1• The analysis of measures results suggests different improvement actions to monitor each phase, to improve the overall economic results – Increase document quality produced in the requirements phase (PR-REQ-T-M1.1) – Increase software quality to facilitate its maintainability in the development phase (PR-DEV-T-M2.1 and PR-DEV-T- M2.2) – Increase the number of reviewed deliverables (PR-TES-T- M3.1) • The reviews performed in the analysis and design phases had the goal to discover bugs early in the development cycle (PT-MAN-T- M4.1, PR-TES-T-M3.2) 2nd International Conference on Software Business Brussels, 8-10 June 2011 18
  • 19. Results analysis - 2     Product 1 Product 2 Goals Measure Year 2009 2010 2009 2010 Economic DimensionReduce Defects resolution costs 0.7000 0.7550 0.7870 0.8530 PR-MAN-E PR-MAN-E-M4.1 0.5244 0.5099 0.5793 0.7319 PR-MAN-E-M4.2 0.8173 0.9244 0.9262 0.9333Reduce criticality in working group Social Dimensionmanagement 0.6810 0.7000 0.7100 0.6850 PR-MAN-S PR-MAN-S-M4.1 0.6944 0.5833 0.5000 0.3000 PR-MAN-S-M4.1 0.6750 0.7500 0.8000 0.8500User satisfaction Technical Dimension 0.6304 0.6552 0.7005 0.6842 PR-MAN-TIncreased #. of checks/reviews PR-MAN-T-M4.1 0.6304 0.6552 0.7005 0.6842• Analysis of the goal Reduce Defects Resolution Cost in the Running phase – Improvement of the 7.2% for Product 1 and 7.6% for Product 2 – Improvement of Mean Defect Resolution Time measure (PR-MAN-E- M4.2) – Consequent improvement of Social (S) dimension, and in particular the User Satisfaction metric (PR-MAN-S-M4.2) • Received benefit from the improvements of document quality and the reduction of Mean Defect Resolution Time 2nd International Conference on Software Business Brussels, 8-10 June 2011 19
  • 20. Results Analysis – 3     Product 1 Product 2 Goals Measure Year 2009 2010 2009 2010 Requirement phaseCompliance with pre-defined documents q.l. 0.8333 0.8667 0.8667 0.9000 PR-REQ-T-M1.1 Development phaseCompliance with pre-defined documents q.l. 0.8333 0.8667 0.8667 0.9000 PR-DEV-T-M2.1 Development phaseCompliance with pre-defined software q.l. 0.5000 0.6000 0.7500 0.9000 PR-DEV-T-M2.2 Test phaseIncrease n.o. checks/reviews 0.7500 0.8750 0.5250 0.6250 PR-TES-T-M3.1 • Analysis of measures correlated to the Reduce Defects Resolution Cost goal – The increasing of the number of reviews (PR-TES-T-M3.1) had a positive effect on the improvement of the overall quality of documents (PR-REQ-T-M1.1 and PR-DEV-T-M2.1) – The improvement of software quality (PR-DEV-T-M2.2 w.r.t. PR- MAN-T-M4.1) was mainly due to the decreasing the percentage of defects in initial development phases (analysis and design) 2nd International Conference on Software Business Brussels, 8-10 June 2011 20
  • 21. Results Analysis – 42nd International Conference on Software Business Brussels, 8-10 June 2011 21
  • 22. Conclusions• We present our experience in developing an integrated framework for process monitoring integrating – GQM approach – BPM technique – QEST-LIME framework – Spago4Q platform• The effectiveness of the framework has been tested on two on-going projects, and an analysis of the achieved results has been resented• Our approach represents a good starting point for a full implementation of a Balanced Scorecard (BSC) technique 2nd International Conference on Software Business Brussels, 8-10 June 2011 22
  • 23. Future Works• Future works will focus on – the improvement of Spago4Q reporting features – implementation of a GQM(R) matrix for choosing new possible measures in order to cover a larger plateau of information needs at the same cost• The framework is available as a demo on the Spago4Q web site (www.spago4q.org) 2nd International Conference on Software Business Brussels, 8-10 June 2011 23
  • 24. Questions? THANKS FOR YOUR ATTENTION!Contacts: http://sesar.dti.unimi.it http://www.spagoworld.org fulvio.frati@unimi.it 2nd International Conference on Software Business Brussels, 8-10 June 2011 24