Presented at the "PMA 2012 Conference: From Strategy to Delivery" in Fitzwilliam College, University of Cambridge, UK, 11.07.2012

1. What is your PMI?
A Model for Assessing the Maturity of
Performance Management in Organizations
Dr. Mika Aho
mika.aho@rongo.fi
PMA 2012 Conference: From Strategy to Delivery
Fitzwilliam College, University of Cambridge, UK
11.07.2012

2. Presentation Outline
Rationale for Performance Management
Maturity Models
Background for the study
Performance Management Index (PM-Index, PMI)
Components
Maturity Levels
Key Capabilities
Example Scores and Results
Summary

3. Rationale for Performance Management (PM)
Central part of business management and the
mechanisms that support decision-making
It broadly encompasses all those measures that an
organization needs to take in order to achieve specific aims
and objectives
The range of methods and procedures that an
organisation uses to manage its personnel effectively
The actions people take are closely related to the
organization’s overall strategy
Effective PM aims to integrate all these procedures,
methods and measures into a single, organization-wide
management system by leveraging modern ICT

4. Rationale for Performance Management
(from Information and Knowledge point of view)
Organizations need reliable, accurate and timely data; these
being the fundamental tools of performance management
Sometimes, the concept of PM means dealing only with the data
for certain pre-defined standards and key performance
indicators (KPI)
This perception of PM leaves the rest of the data and information in
the organization untouched
To implement a really comprehensive PM strategy organizations
should leverage their data by adding analytical functions to their
traditional measurement and monitoring processes
The use of analytics helps organizations to proactively seek out and
identify trends, and to create alternative scenarios for what might
happen in the business

5. Performance Management and Information Technology
Information technology acts as an enabler and integrator to support the overall PM
initiative in an organisation
It integrates all of an organisation’s processes, measures, information and intangible assets so
that they support the different phases involved in implementing a strategy, e.g.
Visualize and monitor the strategy in a form of dashboards, reports, and analytical tools
Create rolling sales forecasts and use those results in the budgeting process
In mature organizations, the performance data is often stored in an organization-wide data warehouse
PM furnish the decision-makers with all the information they need to make really
informed decisions

6. Background for Maturity Models
There is nothing new in the idea that the evolution of an organization, or any
one part of it, can be described by using maturity models
For example the Capability Maturity Model (CMM), have already been in
use for several years to demonstrate the evolution of different areas such as
Corporate Governance (Weill & Ross, 2004)
IT and Strategy Alignment (Luftman & Kempaiah, 2007)
Service Oriented Architectures (Perko, 2008)
Performance Management (Aho, 2009; 2011)

7. Maturity Models
Over the years, organizations have adopted models by which they can
improve their own maturity and capability, and that of their procedures
Maturity = development from some initial stage to an improved, target stage
Capability = what an organization can do and is capable of doing
When moving from a stage to another, the target area, or the whole
organization, improves in terms of its qualitative and quantitative
capabilities over time
The higher the maturity level, the more efficiently the organization is
performing
At the higher maturity levels any procedure is controlled, foreseeable and
the targets that have been set for it are achieved
At the lower maturity levels, however, there might be considerable variation;
the process might be unstable and there are risks associated with it

8. Maturity Models and Performance Management
PM and maturity models have not been studied much academically,
although some attempts had been made in related areas, such as
business intelligence (e.g. Eckerson, 2007; Hostmann, 2006; van Roekel et al., 2009)
analytical competition (Davenport & Harris, 2010)
performance culture (Dresner, 2010)
information technology growth (Nolan, 1979)
IT-business alignment (Luftman & Kempaiah, 2007)
There are some drawbacks in the existing maturity models
They are very seldom based on empirical evidence
There are few indications as to why certain components and maturity levels
were selected
They only concentrate on particular aspects of PM, such as business
intelligence or analytical capability, but fail to examine the concept as a whole

9. How to create a Maturity Model for Performance
Management?
In order to create a maturity model for performance management
1
1. One first needs to identify the components that comprise PM
2
2. Secondly, to define a set of levels that denotes the evolution of PM in organisations
3.
3 Thirdly, to identify the key capabilities for an each component and maturity level
combination
4
4. And finally, to build a metric that can be used to assess the PM maturity, and to
present the results in a convenient and understandable way

10. ”A Construct for Performance Management
Maturity Assessment”
The study was conducted during 2009-2011
A Constructive Research Approach
Empirical evidence was collected from five large
Finnish manufacturing companies
Methods: interviews, questionnaire, participant
observation, literature study
Doctoral dissertation published in November 2011
Tampere University of Technology, Faculty of
Business and Technology Management

11. Overview of the model
(PMI, PM-Index, Performance Management Index)
The focus of this paper
Maturity Model Metric
Main components Web-based tool
User interface
Sub components Maturity
levels
Calculation engine
Supporting components
Questions

12. Main components
Scale and Scope
Business value
Strategy and
Decision support
business
Strategy and objectives
Management and Responsibility
Performance management processes
Performance Measurement and monitoring
Communication
Organizational culture
Data and information analysis
Competencies
Stakeholders
Data and information quality
Information Information production, distribution and availability
Common structures and master data
Intangible
Business and technology alignment
assets
Technology Application architecture
Data warehouse architecture
Methods and Tools

13. Sub components
Scale and Scope
Business value
Strategy and
Decision support
business
Strategy and objectives
Management and Responsibility
Performance management processes
Performance Measurement and monitoring
Communication
Organizational culture
Data and information analysis
Competencies
Stakeholders
Data and information quality
Information Information production, distribution and availability
Common structures and master data
Intangible
Business and technology alignment
assets
Technology Application architecture
Data warehouse architecture
Methods and Tools

14. Supporting components
Scale and Scope
Business value
Strategy and
Decision support
business
Strategy and objectives
Management and Responsibility
Performance management processes
Performance Measurement and monitoring
Communication
Organizational culture
Data and information analysis
Competencies
Stakeholders
Data and information quality
Information Information production, distribution and availability
Common structures and master data
Intangible
Business and technology alignment
assets
Technology Application architecture
Data warehouse architecture
Methods and Tools

15. Maturity Levels
In CMM PMI Maturity Level PMI Maturity Level Description
5 – Optimized Level 5 Performance management has become a strategic tool for management, and
Strategic tool it is a central part of the organization’s control and management system.
Performance management is also strongly connected to the different phases
of strategy implementation in the organization.
4 – Managed Level 4 Business becomes more analytical in key business areas. The metrics and
Analytical business scorecards are closely aligned to the organization’s strategy.
3 – Defined Level 3 Decisions are more often made based on facts, rather than management
Fact-based decision-making instinct. The organization’s data is stored in a centralized data warehouse.
2 – Repeatable Level 2 The organization understands the value of performance management for its
Understanding the value business. Organizational goals and objectives are defined.
1 – Initial Level 1 The solutions are local; there are no common standards, no shared resources
Information silos or management. As a result the management does not get a clear and
consistent picture of the organization as a whole.

16. Maturity Levels
In CMM PMI Maturity Level PMI Maturity Level Description
5 – Optimized Level 5 Performance management has become a strategic tool for management, and
Strategic tool it is a central part of the organization’s control and management system.
Align with
Performance management is also strongly connected to the different phases
of strategy implementation in the organization. strategy process
4 – Managed Level 4 Business becomes more analytical in key business areas. The metrics and
Analytical business scorecards are closely aligned to the organization’s strategy.
Leverage data
3 – Defined Level 3 and information
Decisions are more often made based on facts, rather than management
Fact-based decision-making instinct. The organization’s data is stored in a centralized data warehouse.
2 – Repeatable Level 2 Build foundation
The organization understands the value of performance management for its
Understanding the value business. Organizational goals and objectives are defined.
1 – Initial Level 1 The solutions are local; there are no common standards, no shared resources
Information silos
Create management. As a result the management does not get a clear and
or
understanding picture of the organization as a whole.
consistent

17. Key Capabilities case companies
LEVEL 5
LEVEL 4 Strategic tool
LEVEL 3 Analytical business
LEVEL 2 Fact-based • High quality data
Understanding the decison-making • Competence centre • Strategy feedback and
LEVEL 1 value refinement
• Top management • Balanced metrics
Information silos • Continuous strategic
support • Individual and external
• Operations and stakeholder metrics planning
development plan • Management
• Poor quality data dashboards • Strategy planning and • Strategy scorecard
• Data marts analysis • External data sources
• No defined measures • Common business
• Budgeting vocabulary and master • Rolling forecasts • Integration between
• Temporary ad-hoc
solutions • Creation of strategic data • Enterprise data architectures and
goals • Strategy execution and warehouse (EDW) processes
• Solutions made for
single business cases • Financially focused monitoring • Service-oriented
solution • Planning and forecasting architecture (SOA)
• Reports built on
operational information • Interactive reporting • Key Performance
systems systems Indicators 5,0
• Formal PM method in • Data warehouses
use
4,0
3,22 3,53
3,0
2,0 2,61 2,91 3,07
1,0
0,0
Case 1
For a complete matrix, please visit http://pm-index.com/matrix/ Case 2
Case 3
Case 4
Case 5

18. Descriptions for each maturity level and
component combination
”
At the third maturity level a common organizational culture starts to evolve. The
management, in particular, begins to appreciate the benefits to be gained from
performance management. People in the organization start to adapt fact-based
decision-making, and their decisions and actions are based on their
understanding of performance management systems. People are starting to
understand better the factors that have an impact on the business.
The performance management initiative gets support from top management.
The organization begins to form a performance management competence centre
where specialists from both business and IT are joined together to meet the
needs of the end users.
The organization has a common set of standards and vocabulary. The shared
terms and metrics are standardized across the organization. From the technology
point of view, one characteristic of this level is that there are data warehouses
”
that have more actionable and better quality data…

19. Process for identifying the components
Questionnaire
Questionnaire
Questionnaire
Questionnaire
and interviews
results
Questionnaire
Questionnaire
results
results
results
results
Project plans, minutes
of meetings, internal
Other
documentation etc. sources
Structured
Structured
Structured
results
Structured
Structured
results
Participant observation results
results
results
(~125 days) Field
journal
Summary
Summary
Summary
by Case Analysis of the
Structured Structured Summary
by Case
Summary
by Case empirical evidence
Transcribed bycompany
Case
interview
interview interview bycompany
Case
company
(1) (2) company
company
Interview (15)
Literature
study

20. Process for identifying the components
Literature
Analytical unit (n=475) Grouped Category
analytical
units (n=56)
Empirical • Timeliness of information • Common Data and
evidence helps decision-making business information
• SAP forces to go towards vocabulary
common vocabulary • Master Data
• MDM system in place • Data quality
• Master data has accountable and validity
• Common business vocabulary
from ERP project
• Data quality is challenging
• Data producer is responsible
Methods: for its validity
• Quantifying • …
• Theming
• Typing
• Content analysis

21. Process for identifying the components, cont.
Infrastructure
Information
Information
distribution Information Source
and availability production system
Data and
Information
Data quality
Data input
and validity

22. Process for identifying the maturity levels
and the key capabilities N=759
Level 1 Level 2 Level 3
Technology Application Spreadmarts Data Marts Data
architecture (Eckerson, 2007a) (Eckerson, Warehouses
2007a) (Eckerson,
Traditional back- 2007a)
office applications Applications are
Literature and existing (Luftman & siloed Hybrid
N=374 Kempaiah, 2007) (Hostmann, technologies
maturity models in the area 2007) (Hostmann,
Spreadsheet 2007)
E.g. Brudan, 2009; Balanced Scorecard Institute, applications are
2011; Moncla, 2004; Hostmann, 2007; Luftman & frequently used Improved
Kempaiah, 2007; Eckerson, 2007; Hagerty, 2006; (Hostmann, 2007) reporting
Davenport & Harris, 2007; Davenport & Harris, 2010; systems
Wettstein & Kueng, 2002; Curtis et al., 2009 (Brudan, 2009
… … … …
N=385
Empirical
evidence

23. Validating the Model
Construct validation
Maturity model
Metric validation
validation
Building the Building the Group Changes to the
maturity model Questionnaire Analysis maturity model
metric interviews
Metric calibration
Literature
Observations Prestudy
Interviews
Maturity model validation Metric validation
• No statistical methods were used • Definition of the 283 statements
• Construct validity increased when the construct was • Questionnaire was sent to key employees in the case-
incrementally developed companies in four parts
• The components and their suitability were examined based • Analysis of the data, selection of proper calculation model
on literature, direct observations and interviews • Interview with the case companies
• The selection of the components are justified well • Later the index was further refined and calibrated

24. Validating the Construct, cont.
The dimensions of a weak market test
The extent of usage
Strategic
Team or Division Entire
One person business
department or country organization
unit
Regular use
replacing old Rongo
system(s)
Regular use
The intensity in parallel
of usage with old
system(s)
Ad hoc Case
Weak usage Logica
companies
market test
passed
Case
Used once
companies
Weak
market test
Tried once but not actually used not passed
Reject after unsuccessful implementation trial
Rejected after considering implementation
Rejected before considering implementation
(Labro & Tuomela, 2003)

25. A web-based tool for assessing the PM maturity and to
present the results

26. A web-based tool for assessing the PM maturity and to
present the results
Statement 1 2 3 4 5 0
Performance Management solution supports strategy monitoring x
The performance metrics used in our organization are in balance x
Performance data is used to devise alternative scenarios x
The technological architecture and infrastructure are flexible x
Data and information are high quality x
Data quality problems are openly communicated in our organization x
Top management supports performance management initiative x
The owner of Performance Management comes from the business, not IT x
Through scorecards, the employees participate to strategy execution every day x

27. Example scores and results
Strategy and business
Strategy and business 5
5 4,1
Management and 4 3,8
4 Performance
responsibility 3,0
3
3
2
2
Scale and scope Intangible assets
1 1
0
Business value Strategy and objectives Decision support
Methods and tools Information
Communication Technology
Strategy and business
5
4,1
3,9
4 3,7 3,8
3,4
3,0
3 2,5
2
1
0
Management Business value Methods and Scale and scope Strategy and Communication Decision
and tools objectives support
responsibility

28. Example scores and results
Strategy and business
5
Management and
4 Performance
responsibility
3
Scale and scope
2
Intangible assets
Communication
1 5
3,7 3,9
4 3,2 3,4
3
Methods and tools Information
2 1,7
1
Communication Technology 0
Technology Information Intangible Performance Strategy and
assets business
Communication

29. Summary
The study
presents a novel model for assessing the maturity of PM
further extend PM research by providing a deeper understanding of the
underlying components of PM
attempts to strengthen the existing theoretical framework in this field
increases the understanding of what is a relatively new area of PM research
In the model
the identified nine components are used to describe the key areas of PM
the five maturity levels identify the key characteristics of each development stage
In contrast to existing models, in which PM is usually seen only in
conjunction with strategic management, this model examines PM particularly
from an information and knowledge management point of view

30. For more information, please visit
www.pm-index.com
What is Your PMI?
http://your.pm-index.com
(available in the autumn 2012)
Good at Finnish? Check out my dissertation
http://bit.ly/PMDissertation