4. Visible Ops: Playbook of High Performers
The IT Process Institute has
been studying high-performing
organizations since 1999
What is common to all the high
performers?
What is different between them
and average and low
performers?
How did they become great?
Answers have been codified in
the Visible Ops Methodology
www.ITPI.org
@RealGeneKim, genek@realgenekim.me
5. Agenda
Introductions
Results of the “marriage counseling” questioning
(10m)
Share with you my “top things I wish someone
showed me ten years ago”
ITPI: IT Controls Benchmark Results: controls vs.
performance (5m)
Gartner: Paul Proctor/Michael Smith Risk Adjusted Value
Model: KPIs, KRIs and information security linkage (5m)
Ebay: Dave Cullinane: Infosec risk management (5m)
Open up for what works for you
5
@RealGeneKim, genek@realgenekim.me
6. The Marriage Counseling Questions
What about the business view of IT causes you
to feel uncomfortable?
In your interactions with the business, what
situations don’t feel right to you?
@RealGeneKim, genek@realgenekim.me
7. Gene’s Study of High
Performing IT Organizations
7
@RealGeneKim, genek@realgenekim.me
8. Since 1999, We’ve Benchmarked 1500+
IT Organizations
Source: EMA (2009)
Source: IT Process Institute (2008)
@RealGeneKim, genek@realgenekim.me
9. High Performing IT Organizations
High performers maintain a posture of compliance
Fewest number of repeat audit findings
One-third amount of audit preparation effort
High performers find and fix security breaches faster
5 times more likely to detect breaches by automated control
5 times less likely to have breaches result in a loss event
When high performers implement changes…
14 times more changes
One-half the change failure rate
One-quarter the first fix failure rate
10x faster MTTR for Sev 1 outages
When high performers manage IT resources…
One-third the amount of unplanned work
8 times more projects and IT services
6 times more applications
Source: IT Process Institute, 2008
@RealGeneKim, genek@realgenekim.me
10. 2007: Three Controls Predict 60% Of
Performance
To what extent does an organization define,
monitor and enforce the following?
Standardized configuration strategy
Process discipline
Controlled access to production systems
@RealGeneKim, genek@realgenekim.me
Source: IT Process Institute, 2008
12. The Marriage Counseling Questions
What about the business view of IT causes you
to feel uncomfortable?
In your interactions with the business, what
situations don’t feel right to you?
Source: Gene Kim 2012
@RealGeneKim, genek@realgenekim.me
13. CEO Pains
If IT fails I don't know why, if IT succeeds I don't know why.
By managing inputs and outputs, I can hold any area of the business accountable –
except for IT
I have difficulties holding IT accountable -- IT is often “slippery” (blaming everyone,
especially vendors and suppliers)
I do not have a detailed understanding around the ROI of the IT investments I make.
I need more assurance than my trust in the IT managers.
Failures in IT are often catastrophic and are followed by expensive new projects.
When catastrophic failures in IT happen, I hear “I told you so”
I have no insight into IT productivity or human resource utilization
(aside: Waiting projects imply that service delivery is too slow).
Large investments in IT projects that eventual fail; without warning.
I need data to make informed decisions about IT.
I do not think IT knows how to manage risk well.
Source: Gene Kim 2012
13
@RealGeneKim, genek@realgenekim.me
14. CIO Pains
No visibility into what is actually going on in IT, have to rely on rumors (word on the street).
No sense of security; events in IT seem random that could cause me to lose my job.
The complexity of IT defies detailed understanding; as a result decisions are often made
based on trust or "the best story"
Can communicate expense of IT but cannot calculate value.
Product managers and business people control/drive IT projects with inadequate technical
knowledge.
Cannot isolate who is responsible for IT failures; is it the business, IT, or the tools.
I often have to rely on the CEO trust to decide to "pitch" a project.
I have to rely on my credibility to get projects funded.
Uncoordinated dependencies
CIOs has reverse leverage :everyone can make a mistake so big that can is small to them,
but huge to you – one DBA can light fuses that take years to detonate and destroy the
business (accidentally have reliance on a report that turns into a journal entry)
Source: Gene Kim 2012
14
@RealGeneKim, genek@realgenekim.me
15. CISO Pains
Growing compliance requirements consumes more cycles every day.
Management seems to make poor decisions despite the risks I articulate
Insufficient resources/Cannot respond quickly enough
Need more data to communicate up succinctly
I am perceived to slow down business agility
I have to get projects approved with persuasion rather than data/facts
Last minute projects are able to bypass controls
(implies that doing it with controls takes too long)
Cannot isolate the real risk areas
We find more than can be fixed
Management falsely believes that compliance equals security
Seems like revenue trumps controls
When we apply risk management processes, the probability of bad things happening are
so low that management always chooses to "accept the risk" -- and therefore we can't get
budget.
I have to get projects approved with persuasion rather than data/facts
Source: Gene Kim 2012
15
@RealGeneKim, genek@realgenekim.me
16. Paul Proctor, Michael Smith
Gartner
Risk-Adjusted Value Model
16
@RealGeneKim, genek@realgenekim.me
21. Want more information on RVM?
Contact Paul Proctor, Chief of Research, Risk
and Security, Gartner, Inc.
(mailto:paul.proctor@gartner.com)
or your Gartner rep
21
@RealGeneKim, genek@realgenekim.me
23. Risk Grid Calculation
High Significant DR Event
> $100M Criminal Activity
Data Breach
Regulatory Action
Medium
$50-$100M Operations Security
Impact
SW / Site Security
Low
<$50M
Audit Failure
Low <33% Medium 33-66% High >66%
Source: David Cullinane
Probability
@RealGeneKim, genek@realgenekim.me
24. Information Security Risk
Risk Security Risk Curve
Source: David Cullinane
Investment
@RealGeneKim, genek@realgenekim.me
26. Information Security Risk Tolerance
Risk Security Risk Curve
initial Risk Profile
$300M
Adjusted Risk
Profile with new
funding levels
$140M
Source: David Cullinane
$10M $20M
25HC 50HC Investment
@RealGeneKim, genek@realgenekim.me
27. Information Security Risk Tolerance
Risk Security Risk China
Curve eCrime Threat
Surface/Attacks
Russia (RBN)
E. Europe
$300M
Brazil
$140M
Source: David Cullinane
$10M $20M
25HC 50HC Investment
@RealGeneKim, genek@realgenekim.me
28. Information Security Risk Tolerance
Risk Security Risk China
Curve eCrime Threat
Surface/Attacks
Russia (RBN)
E. Europe
$300M
Brazil
$140M
Added Savings
from Process
improvement
Source: David Cullinane
$10M $20M
25HC 50HC Investment
@RealGeneKim, genek@realgenekim.me
29. Information Security Risk Tolerance
Risk Security Risk China
Curve eCrime Threat
Surface/Attacks
Russia (RBN)
E. Europe
$300M
Brazil
$140M
$60M Added Savings
from Process
2009 Target improvement
Source: David Cullinane
Risk Profile
$10M $20M
25HC 50HC Investment
@RealGeneKim, genek@realgenekim.me
30. Risk of multiple businesses
Need to Focus Here
Financial Impact
A
B
C D E
$100M
F
Legend:
Size – Importance to
company
Color – Effectiveness of
Security controls
Source: David Cullinane
Data at Risk
@RealGeneKim, genek@realgenekim.me
31. Next Generation IRM
Source: David Cullinane
31
@RealGeneKim, genek@realgenekim.me
32. Left Top: Current Controls
Environment as noted using
Cobit Assessment criteria.
Scores reflect support levels
based on existing budgets.
Left Bottom: Controls
Environment as noted using
Cobit Assessment criteria after
budget cuts. Scores reflect
decreased support levels due to
less resources.
Effective Controls
Source: David Cullinane
No Controls
@RealGeneKim, genek@realgenekim.me
33. • Circles sized according to importance to company
• Ability to measure control effectiveness and see impact Risk:
• Ability to determine best expenditure of limited funds to maximize ROSI High
Source: David CullinaneMedium
Low
@RealGeneKim, genek@realgenekim.me
34. When IT Fails: The Novel and The DevOps
Cookbook
Coming in July 2012
“In the tradition of the best MBA case studies, this
book should be mandatory reading for business
and IT graduates alike.”
Paul Muller, VP Software Marketing, Hewlett-
Packard
Gene Kim, Tripwire founder,
“The greatest IT management book of our
Visible Ops co-author generation.”
Branden Williams, CTO Marketing, RSA
@RealGeneKim, genek@realgenekim.me
35. When IT Fails: The Novel and The DevOps
Cookbook
Our mission is to positively affect the
lives of 1 million IT workers by 2017
If you would like the “Top 10 Things
Infosec Needs To Know About DevOps,”
sample chapters and updates on the
book:
Gene Kim, Tripwire founder,
Visible Ops co-author
Sign up at http://itrevolution.com
Email genek@realgenekim.me
Hand me a business card
@RealGeneKim, genek@realgenekim.me
Editor's Notes
4 square picture of where eBay Marketplaces; Corporate IT; and Adjacencies exist utilizing two biggest security & availability risk factors: Financial Impact (associated with availability) and Data at Risk (associated with confidentiality and the potential to disclose or make whole to customers and/or employees)The color represents control effectiveness as determined by: Assessments conducted by GIS; Internal Audit; PwC; external consultants related to security controls and our ability to mitigate against threat environment.