Maksym Vyshnivetskyi: PMO KPIs (UA) Lemberg PMO School 2023 Website – https://lembs.com/pmoschool Youtube – https://www.youtube.com/startuplviv FB – https://www.facebook.com/pmdayconference

2. Favorite pet
Ozzy Osbourne. Not that one, but this one with big ears ☺
The quote I like
«A journey of a thousand miles begins with a single step»
Master degree in economical geography
• Delivery Quality Monitoring and Support dept. (it’s not
about testing ☺), Director
• PM Chapter co-lead
Endless talks about
• Why Agile will not help us to survive
• And a bit about classical project management,
quality assurance and Kanban Method
Experience
• 10 year of project management in public sector and outsourcing
• 10 year in process management and quality assurance
Hello, world J

3. When your stakeholders ask you for a simple answer on their complicated question, the following
might take place…
“Answer to the Ultimate Question of Life, the Universe, and Everything”

5. What is KPI
A KPI is a metric measuring how well the organization or an individual performs an operational, tactical or
strategic activity that is critical for the current and future success of the organization.
Wayne W. Eckerson
KPIs are high-level snapshots of how a project is progressing toward predefined targets.
• A major contributor to the success or failure of the project. A KPI metric is therefore only “key” when it
can make or break the project.
KEY
• A metric that can be measured, quantified, adjusted and controlled. The metric must be controllable to
improve performance.
PERFORMANCE
• Reasonable representation of present and future performance.
INDICATOR
Kerzner, Harold. Project Management Metrics, KPIs,
and Dashboards

6. SMART KPI characteristics
Specific: The KPI is clear and focused toward
performance targets or a business purpose.
Measurable: The KPI can be expressed quantitatively.
Attainable: The targets are reasonable and
achievable.
Realistic or relevant: The KPI is directly pertinent to
the work done on the project.
Time-based: The KPI is measurable within a given
time period.

7. • Performance metrics focuses on the outcome you want to achieve.
Strategic
• KPIs should be straightforward and easy to understand, not based on complex indexes that users do not know how to influence directly.
Simple
• Every KPI is “owned” by an individual or group on the business side who is accountable for its outcome.
Owned
• KPIs are populated with timely, actionable data so users can intervene to improve performance before it is too late.
Actionable
• The KPI can be updated frequently so performance can be improved if intervention is needed.
Timely
• The users can relate back to the origins of the use of the metric.
Referenceable
• The performance metric data can be measured and reported with reasonable accuracy.
Accurate
• The KPI can be used to drive the desired business outcome.
Correlated
• Frequent testing and analysis on the KPI can be conducted so that the data is realistic and not fudged or circumvented due to laziness.
Game-proof
• KPIs are always aligned with corporate strategy and objectives
Aligned
• KPIs are based on standard definitions, rules, and calculations so they can be integrated across dashboards throughout the organization.
Standardized
• KPIs gradually lose their impact over time, so they must be periodically reviewed and refreshed.
Relevant
Wayne W. Eckerson, Performance Dashboards: Measuring, Monitoring and Managing Your Business (2nd ed)
SMARTer KPI characteristics

8. KPI Qualitative Types
KPI Type Focus Examples
Diagnostic Current performance
# of deals or # of signed
contracts to date
# of parallel deployes Support team workload
Lagging Past performance # of signed contracts
# of ontime deploys per
period
# of request to support
Leading Drivers for future performance # of leads Time to deploy # of change requests
Leading indicators “represent things that are not facts yet in your business. They have the potential to lead facts later, but at the
moment … they are only suppositions, conjectures, and assumptions ... They are often told as narratives and stories rather than
metrics”
“Give us information (albeit subjective information) about where we might be headed. Leading indicators belong in every company’s
strategic toolkit, but all too often they are completely overlooked”
1. Seeing Around Corners: How to Spot Inflection Points in Business Before They Happen by Rita Gunther McGrath
2. https://www.ritamcgrath.com/

9. “How to measure anything” by Douglas Hubbard: 5Q+4A
Five questions should be asked before KPIs are established for
measurement:
• What is the decision this KPI is supposed to support?
• What really is the thing being measured by the KPI?
• Why does this thing and the KPI matter to the decision being asked?
• What is known about it now?
• What is the value to measuring it further?
Four measurement assumptions that should be considered when
selecting KPIs:
• The problem in selecting a KPI is not as unique as one thinks.
• One has more data than one thinks.
• One needs less data than one thinks.
• There is a useful measurement that is much simpler than one
thinks.

10. Typical PMO ROI KPI
Percentage of projects using/following the enterprise project management system/framework
Ratio of project manager to total project staff (PM density)
Customer satisfaction ratings
Year-over-year throughput
Percentage of projects at risk or in trouble
Number of projects per headcount (staffing tolerance for projects)
Percentage of scope changes per project
Percentage of projects completed on time
Percentage of projects completed within budget

11. METRIC DESCRIPTION WHAT’S PREVENTING IMPLEMENTATION
% of terminated projects This is the number of terminated projects not aligned to strategy
as a % of the total project portfolio
Decision-makers are not brave enough to make hard decisions
Number of project resources with an agreed career path This is the number of project resources with a career
development plan as a % of the total number of project staff
Budgetary restrictions and defining development paths for
project resources is a complex exercise
Effectiveness score of project sponsors and owners
This is a rating of sponsors and owners by the project manager
assessing their contribution to the project
Largely dependent on maturity of organization. There is
reluctance to rate sponsors negatively when there’s likelihood of
them sponsoring future projects.
% of successful projects delivered per delivery approach type
(Managed delivery vs. Staff Augmentation)
This is the total number of successful project completions per
approach type as a % of the total number of projects in the
portfolio
Assessing the effectiveness of a particular delivery approach is
subjective
Number of improvement initiatives implemented based on the
PMO Roadmap
This is the number of improvement initiatives implemented as a
% of the total number of improvement initiatives identified
The perception of Maturity/Organizational Capability
assessments is that it is a time consuming and resource intensive
process
PMO Metrics
https://www.go2ppo.com

12. METRIC DESCRIPTION CONSIDERATIONS
STRATEGIC CONTRIBUTION
% of projects in the portfolio aligned to strategy
This is the number of projects active in the portfolio that are
aligned to strategy as a % of the total number of active projects in
the portfolio
Although easy to calculate, it requires the PMO to have a clear
definition of what qualifies a project to be aligned to strategy
% of strategically aligned projects
This is the number of strategically aligned projects as a % of the
total portfolio delivered in the last 12 months
To facilitate seamless measurement, a formal project closure
process that records the date on which the project was
completed must be in place. In addition, the PMO must ensure
that all projects are formally closed with the closure date
recorded
IMPROVE TIME TO MARKET
Average number of elapsed days between project proposal and
project activation
This is a calculation of the number of elapsed days between
project conception and project start as an average of the total
number of projects activated
The PMO’s management systems must allow for the recording of
project proposal and project approval dates for an effective
measurement.
% Increase/decrease in the number projects completed in the
last 12 months vs. previous period
This is a comparison of the number of projects completed in the
last 12 months against the previous period calculated as a %
Higher volumes of projects delivered does not equate to
increased value but does give an indication of productivity levels
PMO Metrics
https://www.go2ppo.com

13. METRIC DESCRIPTION CONSIDERATIONS
OPERATIONAL
% improvement of projects with approved scope changes
delivered on time in the last 12 months
This is the number of projects delivered on time in the last 12
months as a % of the total number of projects in the portfolio
compared to the previous period
A formal scope change process is required for accurate data
measurement
% of projects delivered within budget within the last 12 months This is the number of project delivered within budget in last 12
months as a % of the number of projects in the portfolio
Too stringent focus on money can lead to a decrease in quality
% of active project portfolio with scope changes This is the number of projects with scope correction or
deviation as a % of the total number of projects in the portfolio
Requires discipline to record and track project scope
Average time taken between identification of risks and
mitigation action
This is a calculation of the number of elapsed days between
identification of risks and mitigation action as an average of the
total number of risks identified
Only measures the responsiveness of project teams to
proactively manage risk without assessing the effectiveness of
the risk management strategy
GOVERNANCE
% adherence to governance requirements to
This is the number of compliant projects as a % of the total
project portfolio
Gathering data can be a time consuming process if governance
tracking and reporting is a manual process
% of projects subject to project health checks at each stage
gate
This is the number of projects with completed health checks as
a % of the total project portfolio
Consumes huge amounts of PMO resource time if done
manually
PMO Metrics
https://www.go2ppo.com

14. METRIC DESCRIPTION CONSIDERATIONS
RESOURCE MANAGEMENT
% of projects appropriately resourced at the time of project start
This is the number of projects that have met the project’s resource
requirements at the time of project start as a % of the total project
portfolio
Without a basic resource request process in place, this can be a
difficult metric to measure
% of active projects with resource challenges
This is the number of active projects with resource issues as a % of
the total project portfolio
To effectively measure resource challenges, the status reporting
template/format must include a mechanism for project managers
to report on resource challenges as part of the reporting cycle
RETURN ON INVESTMENT (ROI)
% of active projects with validated business cases
This is the number of active projects with approved business cases
as a % of the total project portfolio
An accessible document management system is required to limit
the time spent gathering data
% improvement in benefits delivered per year
This is a calculation of the total actual value of business-case
benefits realised as a % of the total value of planned business-case
benefits in a specific year compared to previous years
Tracking business benefit realization must be a coordinated effort
between the PMO and other areas of the business
STAKEHOLDER MANAGEMENT
Average customer/user satisfaction score This is the satisfaction score per project calculated as an average of
the total number of satisfaction assessments conducted
To connect project success with stakeholder satisfaction, the
project closure process must include the satisfaction assessment as
a formal step. It’s important to note that stakeholder ratings tend
to be confined to specific experiences and can be subjective.
PMO Metrics
https://www.go2ppo.com

15. METRIC DESCRIPTION CONSIDERATIONS
PEOPLE AND SKILLS
Average time taken to on-board project staff
This is the number of days taken to on-board a project resource
calculated as an average across the total number of project staff on-
boarded
While quicker on-boarding does not equate to competent project
staff, it forces the PMO to implement a basic on-boarding process for
project resources
% of project staff at the planned competency level
This is the number of project staff as a % of the total project staff
complement at the planned competency level
This metric can be difficult to measure if continuous competency
assessments of project resources and the skill mapping per project
role is not in place
PMO EFFECTIVENESS
% of completed projects versus cancelled or on-hold projects within
the last 12 months
This is the number of completed projects versus cancelled or on-
hold projects as a % of the total project portfolio within the last 12
months
Formal process that facilitates cancelling or placing projects on hold
must be in place
% of project status reports older than X number of days
This is the number of project status reports as a % of the total
project portfolio older than X number of days
This activity has the potential to consumes huge amounts of PMO
resource time if the process of conducting age analysis of project
status reporting is done manually
PMO Metrics
https://www.go2ppo.com

16. KPI management steps
Define Relevant KPIs
Establish Baselines and Targets
Collect and Analyze Data
Monitor KPI Performance
Identify Insights and Patterns
Conduct Root Cause Analysis
Drive Action and Continuous Improvement

17. Quality management gurus help in KPI analysis
William Edwards Deming
1900-1993
Walter Andrew Shewhart
1891 - 1967
Joseph M. Juran
1904 - 2008

18. “A process may be stable, yet turn out faulty items and mistakes. To take action on the process in response to
production of a faulty item or a mistake is to tamper with the process. The result of tampering is only to
increase in the future the production of faulty items and mistakes, and to increase costs – exactly the opposite
of what we wish to accomplish.”
E. Deming
Common (non-assignable) cause
• The natural variation in the process, part of process.
• No corrective actions are required. When using them
for improvement, process distortion (tampering) is
possible.
Special (assignable) cause
• The unexpected variation in the process
• Improvement actions are required
KPI analysis

19. KPI cascading
https://medium.com/tability/how-to-cascade-revenue-goals-into-achievable-team-metrics-e95d8c7d06e3

20. Based on
Igor Philipiev, Kate Makarevich,
Anna Radzikowska

21. The essence of F4P

22. Criteria are context-dependent
• variety of recipes
• delivery within an hour
• must be hot
Working hours:
Ordering pizza for
colleagues to
office
Evenings:
Ordering pizza for
kids
• simple recipes
• delivery within half an hour
• predictability of delivery
• can be reheated if it's cold

23. Functional quality (what?)
Non-functional quality (how?)
Compliance
Main fitness criteria. F4P v.2.0
Timing and predictability

24. Criteria thresholds
For each criteria, the min level for one segment may exceed users’
expectations in another
Min
Criteria
Max

25. Net Promoter Score (NPS)
We are chosen by a
limited set of F4P
criteria!
How we are assessed
average subscription duration attrition
% billable hours
website visits
number of bugs/defects
average time on website per visit
total number of customers
employee satisfaction
return on investment (ROI)
revenue per customer
number of customer complaints

26. Ryanair “Punctuality”

27. Another aircraft company KPI
We're committed to providing you with reliable and on-time baggage
service every time you fly. That's why we're backing your bags with a
guarantee: if your checked bag doesn't arrive at the carousel in 20 minutes
or less after any domestic flight, you are eligible to receive 2,500 bonus
miles. Just complete the below form no later than three days after your
flight's arrival. Requests should only be submitted after your flight.
https://www.delta.com/bags/bagsClaimAutomation.action

28. F4P criteria

29. Health indicators and improvement drivers

30. Vanity metrics

31. Improvement drivers:
• Use on temporary basis
• They inform progress of
improvements
3
Customer Fitness Criteria :
• Make them your KPIs
• Customers choose by them
• Different for each segment
• Establish thresholds of minimal and
exceptional performance
1 Health Indicators :
• Don’t make them your KPIs
• They’re important, but customers
• don’t choose by them
• Monitor within healthy ranges
2
None of the above ?
• Vanity metrics
• But they may fill some emotional
need - not be easy to remove
4
F4P criteria Summary

32. The picture

33. Ferrari

34. How to understand stakeholders’ expectations

35. Flying by in instruments - 1
Q №1:
Would you recommend our product/service to your friends or colleagues?
0 1 2 3 4 5 6 7 8 9 10
I learned a lot from this workshop. I can immediately apply the new knowledge in my
practice. But the vast majority of my colleagues and friends have completely different
interests. I would not recommend this workshop to them.

36. Net Fitness Score (NFS) не Fit for Purpose (F4P)!

37. Flying by in instruments - 2

38. KPI: easy to learn, hard to mastery
https://www.squirrelnorth.com/
Key Performance Indicators (KPIs) are key and, therefore, few
The key indicators aren’t always obvious. You have to know your market, your customers, understand their criteria, by which they find your products or services fit,
instead of your competitors’ or something else entirely, and the thresholds for those criteria
If you identify your KPIs as fitness criteria, here’s what happens when you game them: you only assure the important outcomes you wanted all along
Such KPIs may not be among the multitude of metrics you know in your familiar business domain and are tracking already
Fitness criteria lead to simple decision filters. At each decision point, big or small, strategic, tactical, planning, or operational, ask: does this option make us fitter by this
criterion?
The many metrics you’re already tracking in your business domain may be very good health indicators. It’s important to know their healthy ranges. Going outside the
range may be an early warning indicator that you may be off course to meet your fitness criteria and, consequently, attain your outcomes
As useful as health indicators may be, it’s important not to use them for motivation or rewards
Health indicators that are currently outside their healthy range point to potential improvements. We can set temporary improvement targets, with the understanding
how they impact our performance with respect to our fitness criteria (KPIs) and thresholds. We can see if what we’re trying to do is getting us closer to attaining the
given performance level.

39. Balanced scorecard
Balanced scorecard - a set of measures that gives top
managers a fast but comprehensive view of the business.
It provides answers to four basic questions:
• How do customers see us? (customer perspective)
• What must we excel at? (internal perspective)
• Can we continue to improve and create value? (innovation
and learning perspective)
• How do we look to shareholders? (financial perspective)
Robert S. Kaplan, David P. Norton
Scorecard guards against suboptimization. By forcing senior
managers to consider all the important operational measures
together, the balanced scorecard lets them see whether
improvement in one area may have been achieved at the expense of
another.
Robert S. Kaplan, David P. Norton

40. ECI’s Balanced Scorecard (example)

41. BSC follows strategy. Amazon 2015
Measures Targets
Number of Smartphones Sold
1.5% of Global Smartphone
shipment
Measures Targets
Average sessions to buy NPS score
Maintain the lead in the online
shopping segment and increase the
gap between rivals
Measures Targets
Number of purchases in new
product segments where Amazon
under-index compared to physical
stores
Service the biggest city in every
state by the end of 2015 with
Amazon fresh
Measures Targets
Sold items related to events
predicted by using the new data set
integration
Integrate data of at least one
partner in every under-index area
of products sale
Smartphone
sales
Enhance customer
shopping experience
and Increase (NPS)
Same day
delivery
Integrate third
party data for
better
recommendation

42. Dashboard vs scorecard
Dashboards are visual display mechanisms used in an operationally oriented performance measurement system that
measure performance against targets and thresholds using right-time data.
Scorecards are visual displays used in a strategically oriented performance measurement system that chart progress
towards achieving strategic goals and objectives by comparing performance against targets and thresholds.
Wayne W. Eckerson, Performance Dashboards: Measuring, Monitoring and Managing Your Business
Feature Dashboard Scorecard
Purpose Measures performance Charts progress
Users Supervisors, specialists Executives, managers, and staff
Updates Right-time feeds Periodic snapshots
Data Events Summaries
Display Visual graphs, raw data Visual graphs, comments
Kerzner, Harold. Project Management: A Systems
Approach to Planning, Scheduling, and Controlling

43. Dashboard is a communication tool
The fundamental challenge of dashboard design is to display all the required information on a single screen, clearly and without
distraction, in a manner that can be assimilated quickly.
If this objective is hard to meet in practice, it is because dashboards often require a dense display of information. You must pack a lot of
information into a very limited space, and the entire display must fit on a single screen, without clutter. This is a tall order that requires a
specific set of design principles.
Stephen Few, “Dashboard Design: Beyond Meters, Gauges and Traffic Lights,” Business Intelligence Journal Winter 2005
A project management dashboard is a visual display of a small number of critical
metrics or key performance indicators such that stakeholders and all project
personnel can see the necessary information at a glance in order to make an
informed decision. Raw data is converted into meaningful information. All of the
information should be clearly visible on one computer screen.
Kerzner, Harold. Project Management Metrics, KPIs, and Dashboards

44. Dashboard core attributes
Aesthetic
appearance -
The viewers
want to use
it.
Easily
understood -
The material
is easily
comprehend
ed.
Retention -
The material
will be
remembered.
Kerzner, Harold. Project Management Metrics, KPIs,
and Dashboards
https://www.chartr.co

45. Three Types of Performance Dashboards
Purpose Users Scope Information Updates Emphasis
Operational
Monitor
operations
Supervisors,
specialists
Operational Detailed Intraday Monitoring
Tactical
Measure
progress
Managers,
analysts
Departmental
Detailed /
summary
Daily/weekly Analysis
Strategic Execute strategy
Executives,
managers, staff
Enterprise
Detailed /
summary
Monthly /
quarterly
Management
• Monitor core operational processes
• Primarily used by front-line personel and
their supervisors
• Primarily deliver detailed information
that is only lightly summarized
• Most metrics are updated frequently
• Track departmental processes and
projects that are of interest to a segment
of the organization limited group of
people.
• Compare performance areas/projects, to
plans, forecasts, or last period’s results.
• Monitor the execution of strategic
objectives
• The goal is to align the organization
around strategic objectives and get every
group marching in the same direction
Kerzner, Harold. Project Management Metrics, KPIs,
and Dashboards

46. Building Dashboard: Dashboard Canvas
1. Team 7. Question →Format 2. Task details 9. Maintenance 3. Users
4. Context and data
consumption
6. Data
8. Sketch
5. Question and business goals
Who will develop the dashboard
Roles & Responsibilities
Why do we need this dashboard?
How will it help business?
How does the customer's
department work?
Who will use the dashboard?
What are their roles and tasks?
Who will be in the test group?
Who will be a data steward?
What objective/goals does the customer have?
What are the most important KPIs? What are the thresholds?
What does client do before and after using the dashboard? What is the
business process and what solutions are used?
What is the context of interaction?
How often and under what
conditions will the dashboard be
used?
Devices and conveying ways?
What data do we have?
How do we get it?
What is the data structure?
What graphs and visualizations will
address the questions from section
5?
How metrics can be aggregated?
How will we support the
dashboard and its users?
Adapted from Roman Bunin

47. Dashboard Types

48. Dashboard Types

49. Dashboard Types

50. Dashboard Types

51. KLT: key lesson’s takeaways
KPI key characteristics
• Aligned with stakeholders
• Free of wishful thinking
• Focused on goals
• Include leading and lagging
indicators
• Reviewed regularly
• Consider the environment
• Readable and explainable
• Support decision making
Fit-for-purpose focus
• KPI are indicators driving the
business
• Health indicators and
improvement drivers support
KPI
• Defined with front-line
personnel engagement
Dashboards
• Begin with an understanding
of the user’s needs.
• Should be defined from data
consumer point of view
• One dashboard is not enough
• Meet requirements of
aesthetic appearance, should
be easily understood and
keep retention

52. That’s all, folks