July 31

1. Establishing Key Risk Indicators for IT
July 31, 2012
Maximo Neira Schliemann Ravi Mishra
Founder & Partner at Beyond Economics & Manager Product Marketing - IT GRC Solutions
Former CIO Ros Casares Corporation in Spain & MetricStream
Member of the CIO office at Baxter
© 2012 MetricStream, Inc. All Rights Reserved.

2. Agenda
• What are KRIs and how they differ from KPI and KCI?
• Why is KRIs important to your IT?
• Selecting the right set of KRIs for your IT organization
• Leverage KRIs for effective IT Risk Management and improving
business performance
© 2012 MetricStream, Inc. All Rights Reserved.

3. THE ENDLESS POSSIBILITIES
OF REPUTATION, RISK &
DESIGN IN BUSINESS.
KRIs, KPIs & IT
Maximo Neira Schliemann
maxneira@beyondeconomics.es
@neiraschliemann
July 31st, 2012

4. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Whether you love or hate them, it is hard to
dispute the popularity and mystique of fortune
cookies in their reputed ability to predict the
future…
“Your life will prosper only if you see and
acknowledge your faults, and work to reduce
them...”

5. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
What are KRIs?
How do they differ from KPIs?
Why are KRIs important for IT?
How to select the right KRIs?
How to leverage from KRIs?

6. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
“key risk indicators (KRIs) are
metrics or pieces of data serving
as ‘early warning indicators’ of
increased risk exposure in various
areas of the enterprise.”
COSO, 2010
Algorithmic & Heuristic

7. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
“Key Performance Indicators
(KPIs) are designed to provide a
high-level overview of the past
performance of the organization
and its major operating units,
often focused almost exclusively
on historical data.”
COSO, 2010
Algorithmic

8. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
KPIs KRIs
External
GeoPolitical
External
Social

9. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Algorithmic
simple
COSO, 2010

10. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
“Not everything that can be counted
counts, and not everything that counts
can be counted.”
Albert Einstein
Heuristic & Inferred

11. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Reputation.
A Construct with more than 35 observable variables across 7
domains with proven impact on Performance.
PERSONAL
EXPERIENCE
S
SUPPORTING
ATTITUDES
ATTITUDES
FEELINGS
DOMAINS
CORPORATE REPUTATION RESULTS
ACTIONS
PROSPECTS
6
THIRD PARTY
OPINION 7 4
Heuristic & Inferred

12. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Reputation.
A Process with more than 35 observable variables across 7
domains
with Impact on Performance.
Products
Purchase
Innovation
ATTITUDES
Trust Recommend
FEELINGS
DOMAINS
Workplace Esteem Anti-crisis
Governance Admiration RESULTS
Word of Mouth
Citizenship Reputation Invest in
Leadership
Work at
Performance

13. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Causal analysis and Constructs.
Can’t be directly observed, but it can be inferred.
Cronbach Alfa
Source: Reputation Institute

14. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Reputation KRI and Market Value KPI have a causal
relationship.
Source: Reputation Institute.

15. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Developing effective KRIs is crucial to the
success of any management program.
First, as they assist in predicting potential adverse events, they are mostly
useful, as noted above, in identifying key areas where additional controls or
mitigation plans might be needed or to explore market opportunities.
“There is a prospect of a thrilling time
ahead for you.”

16. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
A goal of developing an effective set of KRIs is to identify
relevant metrics that provide useful insights about potential
risks that have an impact on the achievement of the
organization’s short & long term performance & goals.
the selection and or design of effective KRIs starts with a firm grasp of organizational
objectives and risk-related events - uncertainties that might affect the achievement of those
objectives.
regulatory compliance risks
fraud or corruption risks reputational risks
extended enterprise risks
contract risks competitor actions risks
geopolitical risks
talent related risks
reporting risks
security risks
business interruption risks
market dynamics risks

17. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Linking Objectives to Strategies to KRI’s.
Mapping key risks to core strategic initiatives puts management in a
position to begin identifying the most critical metrics that can serve as
leading key risk indicators to help them oversee the execution of core or
strategic initiatives.
KPI

18. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Opportunities for Proactive Strategic Risk Management.
This strategic use of KRIs increases the likelihood that objectives set by
management are achieved. Proactively monitoring relevant KRIs helps
minimize uncertainty and identify opportunities for strategy or operational
adjustments.

19. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Why are KRIs important for IT?
How to select “right” KRIs for IT?

20. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
IT continues to emerge as a significant source of strategic risk.
the selection and or design of effective KRIs starts with a firm grasp of organizational
objectives and risk-related events - uncertainties that might affect the achievement of those
objectives.
source: Corporate Executive Board

21. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
are them linked?
Traditional IT Risk Areas
*Illustrative

22. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
On top of the traditional IT risk areas, embedded within the enterprise
risk “heat map” lie an array of business risks that, upon further
consideration, reveal a significant IT component.
Emerging IT-related Risk Areas
*Illustrative

23. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
“By establishing the context, the organization articulates its objectives, defines the
external and internal parameters to be taken into account when managing risk, and sets
the scope and risk criteria for the remaining process.” (ISO 31000, p. 15)

24. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
KRIs should be associated with corresponding KPIs measured as
preceding events with causal relationship affecting desired outcomes.
Revenue
KPI
Reputation
KRI
Data Privacy events

25. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
KRIs should be associated with corresponding KPIs measured as
preceding events with causal relationship affecting desired outcomes.
IT Strategic Initiatives & Risks aligned with Company’s core Pillars, Initiatives & Goals
Customer
Satisfaction
Data
KPI Privacy
Operational
Excellence
Systems
KPI Availability
*Illustrative

26. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Start with Credible & Discrete KRIs directly impacting business KPIs
IT Strategic Initiatives aligned with Company’s core Pillars & Initiatives
KPI
KRI
*Illustrative. Source: Gartner

27. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Real-world KRIs and KPIs mappings
KRIs KPIs
*Illustrative. Source Gartner

28. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
How to leverage KRIs and
improve Business performance?

29. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Business case example for a shipping company…
A cross-country shipping company with a fleet of 100 trucks.
KPI and KRI Risk management
KPI: On-time delivery has reputation,
sales and customer service
implications.
Changing oil every 3k mi raises costs
KRI: Lorry breakdown rates have a
but does not significantly lower
causal relationship with on-time
breakdown rates.
delivery.
Changing oil every 10k mi lower costs
KPI: Failure to change oil has a causal
but significantly raises breakdown rates.
relationship and a negative impact with
breakdowns.
Control: Maintenance SLA with oil
change every 5k mi.
Business outcomes:• Alignment of risk-related activities to execution.
• Risk visibility drives better business decisions with a KRI.
*Illustrative

30. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Risk adjusted KPIs improve decisions and increase business value.
on-time delivery oil change
KPI KRI
on-time delivery =
orders delivered on-time / oil-change KRI = lorries w/o
total orders received oil change within last 5,000mi /
total fleet
on-time delivery KPI = oil-change KRI =
912/1,000 = 91% 75/100 = 75%
KPI target = 90%
Risk adjusted on-time delivery KPI = KPI – (4 * KRI)
= 91% - 3% = 88%
*Illustrative

31. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
The Risk Adjusted Value Model and the KRI Catalog
Business Outcomes Key Risk Indicators
aspect
*Illustrative. Source Gartner

32. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
The Risk Adjusted Value Model and the KRI Catalog
KRI Audit Exception Index
Category Compliance
Business Finance and Regulatory
aspect
Outcomes Support Services
Impacted KPI Time to Market
KRI Description Audit findings are a measure of Compliance failures. The Audit
Exception Index is a KRI that a company is accepting more risk than it
is addressing.
KRI Metric The Audit Exception Index measures the % of audit exceptions granted
over the total number of audit findings.
Audit Exception Index = Granted Exceptions / Total Audit Findings
KRI Example The ABC Co. granted 10 critical audit exceptions in the past 12mo.
During the same period, the total number of findings was 40.
Audit Exception Index = (10/40) = 25%
Risk Adjusted ABC Co. is in the heavily regulated pharma industry. Poor compliance
KPI example increases regulatory scrutiny, which increases new drug development
costs while delaying product launch.
RA New Product Index = New Product Index – (4 x Audit Exception
Alternative Index)
Compliance Program Maturity.
Measures Average days out of date for Critical Mandates.

33. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
How to go about developing a Strategy-KRI-KPI mapping exercise?
The “Vertical-Horizontal” analysis
Security I&O CIO COO CEO
dependency links
perspective analysis
Core Competence Execution
function critical
perspective analysis

34. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Three Takeaways
• Management Process need to consider Risk explicitly.
• Risk Adjusted KPIs improve business decisions and increases
business value.
• A Risk Adjusted/Aware Value Model represents the activities
and events that affect the expected or planned outcomes of
your Co.

35. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Communicating & Engaging through KRIs
Organizing, monitoring, reviewing and communicating KRI progress and their
impact on KPIs can be greatly facilitated by having a centralized, automated
system for the company’s Risk Adjusted KPI program, with flexible, audience
oriented, reporting & dashboarding functionality.

36. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Governance
Risk Management
and
Compliance
are
nuisances
without
an holistic strategy
and
proper tooling

37. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
IT GRC needs are often more complicated than those of their
enterprise colleagues.
With PCI, HIPAA, ISO certification, and privacy laws, IT Pros are typically looking for more
sophisticated control mapping, asset management, vulnerability and event data and product
integration functionality.
As we mentioned, KRIs can/need to be linked to multiple KPIs and
controls, across various enterprise key processes.
On top of the KRI-KPI linkage and its management complexity,
creating risk intelligence require embracing all risk related
information as policies, procedures, losses, incidents, source legal
and regulatory content, compliance control actions taken, auditing
, etc.
All this requires proper systems support to help risk owners and
senior management develop a common language and a clearer
vision of the future.
As of today, IT risk and compliance issues don’t usually get the executive visibility they deserve.
Although many firms may list one or two IT risks among their corporate top 10, most IT & Risk
heads struggle to get visibility with their corporate executives and boards.
(until there’s a breach, that is)

38. THE ENDLESS POSSIBILITIES OF RISK IN BUSINESS. KRIs & IT
Even as concerns grow over mounting regulations, cyberwarfare, privacy,
reputation and fraud, it will be a proper KRI to KPI mapping and the existing large
and successful list of deployments and success stories, as much as anything
else, that will pave the way for your ITGRC program.
So buckle up, leverage from both of them and turn your IT into the domain expert
you Co. needs.
“The wise man expects to prepare for the
unexpected.”

39. THE ENDLESS POSSIBILITIES
OF REPUTATION, RISK &
DESIGN IN BUSINESS.
KRIs, KPIs & IT
Maximo Neira Schliemann
maxneira@beyondeconomics.es
@neiraschliemann
July 31st, 2012