Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP

Luigi Buglione & Alain Abran © 2005WLM 2005 – Como(Italy)
Sep19, 2005
1
Improving Measurement Plans from multiple dimensions:
Exercising with Balancing Multiple Dimensions (BMP)
11stst
Workshop onWorkshop on «« MethodsMethods forfor Learning MetricsLearning Metrics »»
19 September 2005, Como (Italy)19 September 2005, Como (Italy)
ÉCOLE DE TECHNOLOGIE SUPÉRIEURE – MONTRÉAL - CANADA
Luigi Buglione &Luigi Buglione & AlainAlain AbranAbran

Sep19, 2005
2
• Introduction
• Multidimensional Analysis in Project Management
ü Why is it needed?
ü Some Multidimensional Models
ü What should be measured and analysed?
ü Which set of indicators to select?
• BMP: Balancing Multiple Perspectives
ü The proposed measurement procedure
ü An example using 4 dimensions
ü Measuring project performances from multiple views
• Applying BMP
ü BMP-Q: the questionnaire
ü BMP-Q: the metrics
ü First Results & Feedbacks
• Conclusions & Prospects
Agenda

Sep19, 2005
3
• Introduction
• Applying BMP
Agenda

Sep19, 2005
4
Introduction
• Growing interest in Integrated Software
Measurement…
ü E.g. BSC, EFQM, MBQA
• …but still few documented industrial implementations
ü Usually Time and Cost dimensions used
• Other possible dimensions of analysis (eg: Quality,
Risk, …) are not often taken into account
ü Q: how much does it cost project monitoring & control?
ü Q: how many measures/indicators are usually tracked during
the project lifecycle? And from which perspectives?
• Objective:
• to optimize the cost for tracking & control projects, balancing
the number of measures/indicators used by each perspective
of analysis useful to the project

Sep19, 2005
5
• Introduction
• Applying BMP
Agenda

Sep19, 2005
6
Multidimensional Analysis in PM
Why is it needed?
• Loss of project control is one of the most frequent
causes of failure in Project Management
ü Prevention: detailed analysis of content and quality of project
tracking
• Some basic questions:
ü Right number of perspectives?
ü Right number of indicators?
ü What about hypothesis of relationships among processes?
At least, 3 dimensions:
ü Management
ü Users
ü Technical

Sep19, 2005
7
Some examples

Sep19, 2005
8
What should be measured and analysed?
STARSTAR taxonomy: a broader view on Measurement & Analysis

Sep19, 2005
9
Which set of indicators to select?
• Q: What is the right number of indicators to use?
– The Miller’s “magic number” 7 ± 2?
• General suggestions to avoid the misbalance in
selecting the measures critical to success, whatever
the number
Ø“Select a small suite of key measures that will help
you to understand your group’s work better, and
begin collecting them right away, measuring
several complementary aspects of your work, such
as quality, complexity, and schedule.”
(Karl Wiegers)

Sep19, 2005
10
• Introduction
• Applying BMP
Agenda

Sep19, 2005
11
BMP: Balancing Multiple Perspectives
General issue
• Q: how can a proper balance of perspectives and
indicators be selected when managing a portfolio of
projects?
PEANUTS © United Feature Syndicate, Inc.
The problem is not to reduce the cost of measurement,
but optimising it against the informative value provided by the number
of measures/indicators balancing them by each perspective of analysis

Sep19, 2005
12
The proposed Measurement Procedure
Œ Determine the dimensions of interest in the projectdimensions of interest in the project
• Determine the list of the most representative measuresmeasures
associated with each dimensionassociated with each dimension
Ž For each of the measures selected, identify which other
control variables might be impacted negatively
• Figure out the best combination of indicators and thebest combination of indicators and the
causal relations between themcausal relations between them in order to build abuild a
measurement planmeasurement plan for the project

Sep19, 2005
13
A Generic four-dimensional BMP

Sep19, 2005
14
An example with 4 dimensions
Œ Determine the dimensions of interest in the projectdimensions of interest in the project
A: Time, Cost, Quality & Risk

Sep19, 2005
15
• Determine the list of the most representative measures associatedmeasures associated
with each dimensionwith each dimension – Note: excerpt from the PSM Guide

Sep19, 2005
16
• Determine the list of the most representative measures associatedmeasures associated
with each dimensionwith each dimension – Note: excerpt from the PSM Guide

Sep19, 2005
17
Ž For each of the measures selected, identify which other control
variables might be impacted negatively

Sep19, 2005
18
• Figure out the best combination of indicators and the causalbest combination of indicators and the causal
relations between themrelations between them in order to build a measurement planbuild a measurement plan for
the project

Sep19, 2005
19
• Figure out the best combination of indicators and the causalbest combination of indicators and the causal
relations between themrelations between them in order to build a measurement planbuild a measurement plan for
the project

Sep19, 2005
20
Measuring projects’ performances from multiple views
• Q: What about a possible usage of BMP for measuring
the overall project value within a BSC logic?
– Pros: BSC helps in managing multiple perspectives
– Cons: BSC does not provide the integrated measurement
results
• A: a joint usage of QEST nD model with BSC
framework
• A: BMP can help as a tool for considering the counter-
productive impacts of a possible control action in a
project by each BSC perspective
SourceSource: A.Abran & L.Buglione, A Multidimensional Performance Model for Consolidating Balanced Scorecards, International Journal of Advances in
Engineering Software, Elsevier Science Publisher, Vol. 34, No. 6, June 2003, pp.339-349

Sep19, 2005
21
• Introduction
• Applying BMP
Agenda

Sep19, 2005
22
Applying BMP
BMP-Q: the Questionnaire http://www.geocities.com/lbu_measure/qestlime/bmp.htm

Sep19, 2005
23
1a 1 Respondents profile by project role (# and %)
2 Experience profiles for current project role (# and %)
1
b
3 # analysis viewpoints (OLD)
1c 4 # analysis viewpoints (NEW)
2 1 # selected measures (OLD)
2 # selected metrics (NEW)
3 # affected viewpoints (NEW)
4 Avg of measures by viewpoint (# and %)
5 Ranking of selected measures by: abs value, respondent project role, analysis viewpoint
3a 1 List of causal relationships among measures
2 Ranking of relationships by: abs value, respondent project role, analysis viewpoint
4a 1 % respondents knowing amount of costs for m&c (monitoring & control) activities
4
b
1 Max, Min, Avg and Med for the returned values (%) – OLD
4c 1 Max, Min, Avg and Med for the returned values (%) – NEW
Applying BMP
BMP-Q: the Metrics http://www.geocities.com/lbu_measure/qestlime/bmp.htm

Sep19, 2005
24
Applying BMP
First Results & Feedbacks
Teaching viewpoint:
• simplify the way for explaining the ISO/IEC 15939:2002 Information Model
and introducing the cause-effect linkage among development processes, as a
as a useful foundation for ICT Balanced Scorecards (BSC)
• help in pointing out to practitioners and students that there are plenty of well-
proven and established measures within the Software Engineering domain,
but also that each Measurement Program must be built having in mind that
unique situation and that every measure should be designed and verified to
be effectively goal-driven
Industrial viewpoint:
• increase awareness about cost-benefit analysis in implementing (or not) a
certain measure/indicator within their current QMSs.
ü Stimulus for industry people to know more and more about the effort/cost splitting of
their projects by SDLC phase (in many cases the answer is: don’t know)
ü Proposed the Test Selection Algorithm: each measure has its own cost from its
definition till the project closure, within a budget constraint
ü Key point: balancing the current available amount of budget and not to simply cut
controls. And in that eventuality, the point to face was: but which drawbacks of deleting
or adding some controls?
• Proposed to re-label such measures and indicators as project “controls”, to
provide a better match with managers’ responsibilities

Sep19, 2005
25
• Introduction
• Applying BMP
Agenda

Sep19, 2005
26
Conclusions && Prospects
• Project managers often considers only two dimensions for tracking & control
of their projects (Time, Cost)
• At least, the Quality perspective should be also taken into account; further
perspectives (eg: Risk) could be also be useful if considered from the
planning phase on. Even challenging, a multi-perspective approach – as in
the BSC – is suggested
• It does not exist a “magic number” of indicators to track, but the goal is to
optimize costs and informative value derived from that amount of indicators,
establishing also the causal relationships among their related goals
• BMP (Balancing Multiple Perspectives) proposes a 4-step procedure to
select an appropriate balance of indicators from the various perspectives
taken into account (e.g. Time, Cost, Risk and Quality) and focus on the core
indicators from each of them, thereby helping the project manager in
tracking and control activities
• Due to its inner multidimensional nature, future joint usages with methods,
tools and frameworks taking into account concurrent dimensions (eg:
QEST/LIME) will be investigated

Sep19, 2005
27
Q && A

Sep19, 2005
28
luigi.buglione@computer.org , aabran@ele.etsmtl.ca
Thank you!

Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP

More Related Content

Similar to Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP

More from Luigi Buglione

Recently uploaded

Improving Measurement Plans from multiple dimensions: Exercising with Balancing Multiple Dimensions - BMP