1) The document presents a process for improving the use of performance assessment instruments to better identify, organize, measure, and utilize performance indicators. 2) It explores ways to improve the identification and construction of performance indicators to provide a greater chance of achieving strategic objectives through the selected tools. 3) The research involved reviewing 35 articles to establish a theoretical framework and identify best practices for a multicriteria model to enhance performance evaluation.

1. Multicriteria Model to Improve the Use of
Performance Evaluation Instruments with Focus
on Indicators
Msc. Edilson Giffhorn, PMP, IPMA - Speaker
PhD. Leonardo Ensslin
PhD. Sandra Rolim Ensslin
Msc. William Barbosa Vianna
ALIO-INFORMS Joint International Meeting
6. – 9.6.2010, Buenos Aires - Argentina

2. Objective:
The aim of this paper is to present a process for improving
the way to use the instruments of performance assessment
on their way to identify, organize, measure and use the
Performance Indicators.
2

3. The use of indicators
as a management tool 20th
Introduction / control
century –
after 1950
20th
century – Evaluate processes
Instrument
until 1950 (Brandão, Guimarães, 2001)
of
Intervention 18th Scientific Management:
century Evaluation of merit (people)
(Guimarães et al. 1998)
Industrial Revolution:
Evaluation
Model Large corporations - production control
1494 (Leão, 1998)
Tratactus de Computis et Scripturis - Summa de
arithmetica, geometrica, proportioni et
proportionalita
Conclusions
(Brudan, 2009; Ensslin; Ensslin, 2009)
3

4. Performance Indicators
Introduction
After 1995
Instrument 3rd Generation Indicators
of
Intervention
1960 - 1995
2nd Generation Indicators
Evaluation
Model
Until
1960
1st Generation Indicators
Conclusions
Ensslin, Ensslin (2009)
4

5. 1st Generation Indicators
Introduction
Instrument
of
Intervention
Emphasis Dimensions Orientation
Evaluation
Financial
Model
Control Efficiency
(cost, profit)
Productivity Accounting
Conclusions (quality) systems
Neely (1999); Ensslin, Ensslin (2009); Oliveira et al (2009)
5

6. 2nd Generation Indicators
Introduction
Instrument
Emphasis Dimensions Orientation
of
Intervention
Multicriteria
Management Financial
(multifunctional)
Productivity
Evaluation
Model
Flexibility
Agility (speed)
Conclusions
Innovation
Bititci, Suwignjo, Carrie; (2001); Ensslin, Ensslin; (2009)
6

7. Characteristics of New requirements
instruments of the context
Introduction
Generic context Specific context
Instrument Generic decision makers Specific decision makers
of
Intervention
Monitoring Continuous Improvement
Evaluation Ordinal and Cardinal
Model Ordinal scales
scales
Does not compare Comparison of
performance indicators performance by integration
Conclusions
Available quantitative Quali-quantitative
properties properties
Ensslin, Ensslin (2009)
7

8. 3rd Generation Indicators
Introduction
Instrument Emphasis Dimensions Orientation
of
Intervention
Customized to the Multicriteria (values
Improve specific
decision maker and preferences)
contexts in a
personalized way
Evaluation
Model
Conclusions
Ensslin, Ensslin; (2009)
8

9. 3rd Generation Indicators
Introduction
Paradigm exchange
Instrument
of
Intervention Isolated measures Integrated measures
Quali-quantitative
Quantitative measures
Evaluation measures
Model
Specific measures for
Generic measures
each context
Conclusions
Measures to improve
Measures to control
performance
Ensslin, Ensslin (2009)
9

10. Even with the recognition of new requirements, many
applications of performance measurement continue to use the
Introduction 1st and 2nd generation.
Traditional historical
and financial
Instrument
of
orientation
Intervention Neely, Powell (2004)
Evaluation
Model
Focus on the
Generic or pre- measure, misaligned
existing with the goals and
Denton (2005), Tangen (2003)
Conclusions unbalanced
Leandri (2001)
10

11. Consequences
Introduction
Instrument Measures not aligned to strategy.
of
Intervention
Schneiderman (1999), Neely (2000), Bourne et al. (2002)
Evaluation
Model
Performance Evaluations poorly developed, misaligned with the
strategy and decision-makers do not understand the impacts of their
decisions and actions.
Conclusions Denton (2005)
11

12. Research Theme
Introduction
Instrument
of Explore ways of improving the identification
Intervention
/ construction of the Performance Indicators.
Evaluation
Model Provide greater chance of achievement the
strategic objectives based on the tools
selected.
Conclusions
12

13. State of the Art
Introduction
Process to identify the theoretical referencial
Instrument
of
Intervention
Keywords
Evaluation
Model
Article
Article
Article
Conclusions
35 Articles
13

14. MCDA-C Phases
Introduction
Instrument
of
Intervention
Evaluation
Model
Conclusions
Ensslin, Dutra, Ensslin (2000)
14

15. MCDA-C selected due:
Introduction
Identify the important aspects in accordance with the decision
maker.
Instrument Expand the understanding of the context of the decision
of
Intervention
maker.
Allow to take into account the interests of the actors involved
with the monitoring of the decision maker.
Evaluation
Model Allow the decision maker to revise his views in the course of
the construction of their knowledge.
Recognize the ordinal and cardinal properties of the
indicators.
Conclusions
Ensure that the decision maker legitimize the process with
tools scientifically valid.
Ensslin et al. (2010)
15

16. Structuring Phase
Introduction
Identification of the Actors
Subsystem
Instrument
of
Intervention
Evaluation
Model
Conclusions
16

17. Structuring Phase
Introduction
Label:
Instrument
of Build a model to improve the mapping of the indicators
Intervention
on the instruments of performance evaluation.
Evaluation
Model
Conclusions
17

18. Structuring Phase
Introduction
Process to transform the Primary
Assessment Elements in Concepts
EPA 61: mathematical operations
Objective underlying the EPA : Capactity to perform mathematical operations between
Instrument scales.
of
Intervention
Best possible Scales are constructed so as to enable mathematical
performance operations between them.
Evaluation
Model
Worst possible
performance Scales used allow only statistical operations.
Conclusions
Intensity: Very Strong
Concept 61: Ensure that mathematical operations can be performed
between the scales... use scales that allow only statistical operations.
18

19. Structuring Phase
Introduction
Label
Instrument
of
Intervention
Areas of
Concern
Evaluation
Model
Concepts
Conclusions

21. General view of the Global Evaluation
final model. Improve the Mapping of Indicators 72 PM Tool 1
in the Performance Assessment 68 PM Tool 2
Introduction
Instruments
40 PM Tool 3
X% Y% Z% W%
Mathematical
Definition Legitimation Process
Foundations
Instrument
of A% B% C% D% E% F% G%
Intervention
PVF 4 – PVF 5 – PVF 6 – PVF 7 –
PVF 1 PVF 2 PVF 3 Value Substitution Ordinal Cardinal
Function Rates Scales Scales
Evaluation Excelence
Model
Good 100 100 100 100 100 100 100
Competitive
Neutral 0 0 0 0 0 0 0
Endanger
Conclusions
Legend: Impact profile of the Performance Measurement tool 1
Impact profile of the Performance Measurement tool 2
Impact profile of the Performance Measurement tool 3
21

22. Conclusion
Introduction
The model will serve as a support instrument to make
Instrument
of Performance Evaluations, in order to have greater accuracy
Intervention
and alignment between the operational level, tactical and
strategic organization.
Evaluation
Model
The model will give ways for improving the identification / construction
of the Performance Indicators.
Conclusions
22

23. Msc. Edilson Giffhorn, PMP, IPMA: edilson.giffhorn@gmail.com
PhD. Leonardo Ensslin: ensslin@deps.ufsc.br
PhD. Sandra Rolim Ensslin: sensslin@deps.ufsc.br
Msc. William Barbosa Vianna: wpwilliam@hotmail.com
23