Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
1
Improving Measurement Plans from multiple dimensio...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
2
• Introduction
• Multidimensional Analysis in Proj...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
3
• Introduction
• Multidimensional Analysis in Proj...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
4
Introduction
• Growing interest in Integrated Soft...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
5
• Introduction
• Multidimensional Analysis in Proj...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
6
Multidimensional Analysis in PM
Why is it needed?
...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
7
Multidimensional Analysis in PM
Some examples
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
8
Multidimensional Analysis in PM
What should be mea...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
9
Multidimensional Analysis in PM
Which set of indic...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
10
• Introduction
• Multidimensional Analysis in Pro...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
11
BMP: Balancing Multiple Perspectives
General issu...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
12
BMP: Balancing Multiple Perspectives
The proposed...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
13
BMP: Balancing Multiple Perspectives
A Generic fo...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
14
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
15
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
16
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
17
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
18
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
19
BMP: Balancing Multiple Perspectives
An example w...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
20
BMP: Balancing Multiple Perspectives
Measuring pr...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
21
• Introduction
• Multidimensional Analysis in Pro...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
22
Applying BMP
BMP-Q: the Questionnaire http://www....
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
23
1a 1 Respondents profile by project role (# and %...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
24
Applying BMP
First Results & Feedbacks
Teaching v...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
25
• Introduction
• Multidimensional Analysis in Pro...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
26
Conclusions && Prospects
• Project managers often...
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
27
Q && A
Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
28
luigi.buglione@computer.org , aabran@ele.etsmtl.c...
Upcoming SlideShare
Loading in …5
×

Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP

428 views

Published on

“Tracking & Control” activities in software projects are most often based, in industry, on just two dimensions of analysis: time and cost. Most often, ‘tracking & control’ excludes other dimensions (such as quality, risks & impact on society, stakeholders’ viewpoint in a broader sense) taken into account in Performance Management models such as EFQM or the Malcolm Baldridge model. How can balancing those multiple concurrent control mechanisms across several dimensions of analysis be done? Balancing Multiple Perspective (BMPs) is a procedure designed to help project managers choose a set of project indicators from several concurrent viewpoints. This paper also presents the related questionnaire with a list of 14 candidate measures helping to compare the “as-is” situation and to figure out what will be the desired one, including cost figures to be possibly considered in the budget for next projects.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP

  1. 1. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 1 Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions (BMP) 11stst Workshop onWorkshop on «« MethodsMethods forfor Learning MetricsLearning Metrics »» 19 September 2005, Como (Italy)19 September 2005, Como (Italy) ÉCOLE DE TECHNOLOGIE SUPÉRIEURE – MONTRÉAL - CANADA Luigi Buglione &Luigi Buglione & AlainAlain AbranAbran
  2. 2. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 2 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  3. 3. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 3 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  4. 4. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 4 Introduction • Growing interest in Integrated Software Measurement… ü E.g. BSC, EFQM, MBQA • …but still few documented industrial implementations ü Usually Time and Cost dimensions used • Other possible dimensions of analysis (eg: Quality, Risk, …) are not often taken into account ü Q: how much does it cost project monitoring & control? ü Q: how many measures/indicators are usually tracked during the project lifecycle? And from which perspectives? • Objective: • to optimize the cost for tracking & control projects, balancing the number of measures/indicators used by each perspective of analysis useful to the project
  5. 5. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 5 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  6. 6. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 6 Multidimensional Analysis in PM Why is it needed? • Loss of project control is one of the most frequent causes of failure in Project Management ü Prevention: detailed analysis of content and quality of project tracking • Some basic questions: ü Right number of perspectives? ü Right number of indicators? ü What about hypothesis of relationships among processes? At least, 3 dimensions: ü Management ü Users ü Technical
  7. 7. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 7 Multidimensional Analysis in PM Some examples
  8. 8. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 8 Multidimensional Analysis in PM What should be measured and analysed? STARSTAR taxonomy: a broader view on Measurement & Analysis
  9. 9. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 9 Multidimensional Analysis in PM Which set of indicators to select? • Q: What is the right number of indicators to use? – The Miller’s “magic number” 7 ± 2? • General suggestions to avoid the misbalance in selecting the measures critical to success, whatever the number Ø“Select a small suite of key measures that will help you to understand your group’s work better, and begin collecting them right away, measuring several complementary aspects of your work, such as quality, complexity, and schedule.” (Karl Wiegers)
  10. 10. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 10 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  11. 11. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 11 BMP: Balancing Multiple Perspectives General issue • Q: how can a proper balance of perspectives and indicators be selected when managing a portfolio of projects? PEANUTS © United Feature Syndicate, Inc. The problem is not to reduce the cost of measurement, but optimising it against the informative value provided by the number of measures/indicators balancing them by each perspective of analysis
  12. 12. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 12 BMP: Balancing Multiple Perspectives The proposed Measurement Procedure Œ Determine the dimensions of interest in the projectdimensions of interest in the project • Determine the list of the most representative measuresmeasures associated with each dimensionassociated with each dimension Ž For each of the measures selected, identify which other control variables might be impacted negatively • Figure out the best combination of indicators and thebest combination of indicators and the causal relations between themcausal relations between them in order to build abuild a measurement planmeasurement plan for the project
  13. 13. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 13 BMP: Balancing Multiple Perspectives A Generic four-dimensional BMP
  14. 14. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 14 BMP: Balancing Multiple Perspectives An example with 4 dimensions Œ Determine the dimensions of interest in the projectdimensions of interest in the project A: Time, Cost, Quality & Risk
  15. 15. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 15 BMP: Balancing Multiple Perspectives An example with 4 dimensions • Determine the list of the most representative measures associatedmeasures associated with each dimensionwith each dimension – Note: excerpt from the PSM Guide
  16. 16. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 16 BMP: Balancing Multiple Perspectives An example with 4 dimensions • Determine the list of the most representative measures associatedmeasures associated with each dimensionwith each dimension – Note: excerpt from the PSM Guide
  17. 17. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 17 BMP: Balancing Multiple Perspectives An example with 4 dimensions Ž For each of the measures selected, identify which other control variables might be impacted negatively
  18. 18. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 18 BMP: Balancing Multiple Perspectives An example with 4 dimensions • Figure out the best combination of indicators and the causalbest combination of indicators and the causal relations between themrelations between them in order to build a measurement planbuild a measurement plan for the project
  19. 19. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 19 BMP: Balancing Multiple Perspectives An example with 4 dimensions • Figure out the best combination of indicators and the causalbest combination of indicators and the causal relations between themrelations between them in order to build a measurement planbuild a measurement plan for the project
  20. 20. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 20 BMP: Balancing Multiple Perspectives Measuring projects’ performances from multiple views • Q: What about a possible usage of BMP for measuring the overall project value within a BSC logic? – Pros: BSC helps in managing multiple perspectives – Cons: BSC does not provide the integrated measurement results • A: a joint usage of QEST nD model with BSC framework • A: BMP can help as a tool for considering the counter- productive impacts of a possible control action in a project by each BSC perspective SourceSource: A.Abran & L.Buglione, A Multidimensional Performance Model for Consolidating Balanced Scorecards, International Journal of Advances in Engineering Software, Elsevier Science Publisher, Vol. 34, No. 6, June 2003, pp.339-349
  21. 21. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 21 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  22. 22. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 22 Applying BMP BMP-Q: the Questionnaire http://www.geocities.com/lbu_measure/qestlime/bmp.htm
  23. 23. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 23 1a 1 Respondents profile by project role (# and %) 2 Experience profiles for current project role (# and %) 1 b 3 # analysis viewpoints (OLD) 1c 4 # analysis viewpoints (NEW) 2 1 # selected measures (OLD) 2 # selected metrics (NEW) 3 # affected viewpoints (NEW) 4 Avg of measures by viewpoint (# and %) 5 Ranking of selected measures by: abs value, respondent project role, analysis viewpoint 3a 1 List of causal relationships among measures 2 Ranking of relationships by: abs value, respondent project role, analysis viewpoint 4a 1 % respondents knowing amount of costs for m&c (monitoring & control) activities 4 b 1 Max, Min, Avg and Med for the returned values (%) – OLD 4c 1 Max, Min, Avg and Med for the returned values (%) – NEW Applying BMP BMP-Q: the Metrics http://www.geocities.com/lbu_measure/qestlime/bmp.htm
  24. 24. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 24 Applying BMP First Results & Feedbacks Teaching viewpoint: • simplify the way for explaining the ISO/IEC 15939:2002 Information Model and introducing the cause-effect linkage among development processes, as a as a useful foundation for ICT Balanced Scorecards (BSC) • help in pointing out to practitioners and students that there are plenty of well- proven and established measures within the Software Engineering domain, but also that each Measurement Program must be built having in mind that unique situation and that every measure should be designed and verified to be effectively goal-driven Industrial viewpoint: • increase awareness about cost-benefit analysis in implementing (or not) a certain measure/indicator within their current QMSs. ü Stimulus for industry people to know more and more about the effort/cost splitting of their projects by SDLC phase (in many cases the answer is: don’t know) ü Proposed the Test Selection Algorithm: each measure has its own cost from its definition till the project closure, within a budget constraint ü Key point: balancing the current available amount of budget and not to simply cut controls. And in that eventuality, the point to face was: but which drawbacks of deleting or adding some controls? • Proposed to re-label such measures and indicators as project “controls”, to provide a better match with managers’ responsibilities
  25. 25. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 25 • Introduction • Multidimensional Analysis in Project Management ü Why is it needed? ü Some Multidimensional Models ü What should be measured and analysed? ü Which set of indicators to select? • BMP: Balancing Multiple Perspectives ü The proposed measurement procedure ü An example using 4 dimensions ü Measuring project performances from multiple views • Applying BMP ü BMP-Q: the questionnaire ü BMP-Q: the metrics ü First Results & Feedbacks • Conclusions & Prospects Agenda
  26. 26. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 26 Conclusions && Prospects • Project managers often considers only two dimensions for tracking & control of their projects (Time, Cost) • At least, the Quality perspective should be also taken into account; further perspectives (eg: Risk) could be also be useful if considered from the planning phase on. Even challenging, a multi-perspective approach – as in the BSC – is suggested • It does not exist a “magic number” of indicators to track, but the goal is to optimize costs and informative value derived from that amount of indicators, establishing also the causal relationships among their related goals • BMP (Balancing Multiple Perspectives) proposes a 4-step procedure to select an appropriate balance of indicators from the various perspectives taken into account (e.g. Time, Cost, Risk and Quality) and focus on the core indicators from each of them, thereby helping the project manager in tracking and control activities • Due to its inner multidimensional nature, future joint usages with methods, tools and frameworks taking into account concurrent dimensions (eg: QEST/LIME) will be investigated
  27. 27. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 27 Q && A
  28. 28. Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy) Sep19, 2005 28 luigi.buglione@computer.org , aabran@ele.etsmtl.ca Thank you!

×