When IT Fails…The Business Fails…Gene KimAuthor, Visible Ops HandbookProKarma SeminarAugust 20, 2012Session ID:           ...
@RealGeneKim, genek@realgenekim.me
Now, More Than Ever… Even in “low-tech industries,” 95% of all capital  projects have an IT component… 50% of all capita...
Comparison Of Turnover For CEOs and    CFOs…         When firms with IT-related material weaknesses are          compared...
There’s a hidden gas, that we can’t see, taste,touch, smell, and it’s killing CEOs everywhere.                 It’s called...
it.fail() == business.fail()                  @RealGeneKim, genek@realgenekim.me
Where Did The High Performers Come From?                        @RealGeneKim, genek@realgenekim.me
Over Ten Years, We Benchmarked 1500+ ITOrgs                        @RealGeneKim, genek@realgenekim.me
High Performing IT Organizations High performers maintain a posture of compliance    Fewest number of repeat audit findi...
Tough Love From Ari Balogh                       @RealGeneKim, genek@realgenekim.me
The Downward Spiral Operations Sees…                           Dev Sees…    Too many fragile and insecure             Mo...
My Mission Chronicle the Hero’s Journey For IT ("When IT  Fails: A Business Novel”) so that everyone can  gain a shared u...
14     @RealGeneKim, genek@realgenekim.me
15     @RealGeneKim, genek@realgenekim.me
16     @RealGeneKim, genek@realgenekim.me
17     @RealGeneKim, genek@realgenekim.me
18     @RealGeneKim, genek@realgenekim.me
19     @RealGeneKim, genek@realgenekim.me
The State Of The Business                       @RealGeneKim, genek@realgenekim.me
Project Phoenix                  @RealGeneKim, genek@realgenekim.me
Day 1: Payroll Outage                        @RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
Day 2: PMO Meeting                     @RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
Day 3: The SOX-404 Audit Meeting                       @RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
My Mission: Figure Out How Break The IT Core     Chronic Conflict      Every IT organization is pressured to       simult...
2007: Three Controls Predict 60% OfPerformance To what extent does an organization define,  monitor and enforce the follo...
Visible Ops: Playbook of High Performers The IT Process Institute has  been studying high-performing  organizations since...
Visible Ops Security: Linking Security and IT      Operations Objectives In 4 Practical Steps                             ...
Source: John Allspaw                       @RealGeneKim, genek@realgenekim.me
Source: John Allspaw                       @RealGeneKim, genek@realgenekim.me
The First Way:Systems Thinking                   @RealGeneKim, genek@realgenekim.me
The First Way:Systems Thinking(Business)                             (Customer)                   @RealGeneKim, genek@real...
The Second Way:Amplify Feedback Loops                         @RealGeneKim, genek@realgenekim.me
The Third Way:Culture Of Continual Experimentation AndLearning                         @RealGeneKim, genek@realgenekim.me
Good News: It Can Be DoneBad News: You Can’t Do It Alone                         @RealGeneKim, genek@realgenekim.me
Ops      @RealGeneKim, genek@realgenekim.me
QA And Test Source: Flickr: vandyll                           @RealGeneKim, genek@realgenekim.me
Development              @RealGeneKim, genek@realgenekim.me
Process And Controls                       @RealGeneKim, genek@realgenekim.me
Product Management And Design Source: Flickr: birdsandanchors                                   @RealGeneKim, genek@realge...
What Does Transformation Feel Like?                  47                       @RealGeneKim, genek@realgenekim.me
Find What’s Most Important First                        @RealGeneKim, genek@realgenekim.me
Quickly Find What Is Different…                        @RealGeneKim, genek@realgenekim.me
Before Something Bad Happens…                     @RealGeneKim, genek@realgenekim.me
Find Risk Early…                   @RealGeneKim, genek@realgenekim.me
Communicate It Effectively To Peers…                       @RealGeneKim, genek@realgenekim.me
Hold People Accountable…                      @RealGeneKim, genek@realgenekim.me
Based On Objective Evidence…                      @RealGeneKim, genek@realgenekim.me
Answer Important Questions…                      @RealGeneKim, genek@realgenekim.me
Recognize Compounding Technical Debt…                      @RealGeneKim, genek@realgenekim.me
That Gets Worse…                   @RealGeneKim, genek@realgenekim.me
And Fixing It… Source: Pingdom                   @RealGeneKim, genek@realgenekim.me
Have What We Need, When We Need It…                     @RealGeneKim, genek@realgenekim.me
Big Things Get Done Quickly…                         @RealGeneKim, genek@realgenekim.me
Ever Increasing Situational Mastery…                        @RealGeneKim, genek@realgenekim.me
Help The Business Win…                         @RealGeneKim, genek@realgenekim.me
With Support From Your Peers…                      @RealGeneKim, genek@realgenekim.me
And Do More With Less Effort…                       @RealGeneKim, genek@realgenekim.me
This Is An Important Problem Operations Sees…                            Dev Sees…  Fragile applications are prone to    ...
66     @RealGeneKim, genek@realgenekim.me
@RealGeneKim, genek@realgenekim.me
When IT Fails: A Business Novel and  The DevOps Cookbook                               Coming in Winter 2012/2013        ...
When IT Fails: The Novel and The DevOps  Cookbook                               Our mission is to positively affect the  ...
If you’d like the slides from today’spresentation…                     Text your first name, email                      a...
Upcoming SlideShare
Loading in...5
×

When IT Fails The Business Fails...

848

Published on

ProKarma Seminar 8/21/12

Published in: Business, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
848
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • How each side Actively impedes the achievement of each other’s goals.
  • “Project Phoenix is essential to closing the gap with the competition,so we can finally do what the competition has been doing for years. Customers need to beable to buy from us from wherever they want, whether it’s on the Internet or in our retailstores. Otherwise, we’ll soon have no customers, at all.”The outage
  • Who are they auditing? IT operations.I love IT operatoins. Why? Because when the developers screw up, the only people who can save the day are the IT operations people. Memory leak? No problem, we’ll do hourly reboots until you figure that out.Who here is from IT operations?Bad day:Not as prepared for the audit as they thoughtSpending 30% of their time scrambling, generating presentation for auditorsOr an outage, and the developer is adamant that they didn’t make the change – they’re saying, “it must be the security guys – they’re always causing outages”Or, there’s 50 systems behind the load balancer, and six systems are acting funny – what different, and who made them differentOr every server is like a snowflake, each having their own personalityWe as Tripwire practitioners can help them make sure changes are made visible, authorized, deployed completely and accurately, find differencesCreate and enforce a culture of change management and causality
  • Who’s introducing variance? Well, it’s often these guys. Show me a developer who isn’t causing an outage, I’ll show you one who is on vacation.Primary measurement is deploy features quickly – get to market.I’ve worked with two of the five largest Internet companies (Google, Microsoft, Yahoo, AOL, Amazon), and I now believe that the biggest differentiator to great time to market is great operations:Bad day: We do 6 weeks of testing, but deployment still fails. Why? QA environment doesn’t match productionOr there’s a failure in testing, and no one can agree whether it’s a code failure or an environment failureOr changes are made in QA, but no one wrote them down, so they didn’t get replicated downstream in productionBelieve it or not, we as Tripwire practitioners can even help them – make sure environments are available when we need them, that they’re properly configured correctly the first time, document all the changes, replicate them downstream
  • So who are all these constituencies that we can help, and increase our relevance as Tripwire practitioners and champions?How many people here are in infosec?Goal: protect critical systems and dataSafeguard organizational commitmentsPrevent security breaches, help quickly detect and recover from themBad day: no security standardsNo one is complyingYes, we’re 3 years behind. “Whaddyagonna do about it?”Vs. we (Tripwire owner) can become more relevant and add value by help infosec by leveraging all the configuration guidance out thereMeasure variance between produciton and those known good statesTrust and verify that when management says, we’ve trued up the configurations, they’ve actually done itWhy? Now, more than ever, there are an ever increasing amount of regulatory and contractual requirements to protect systems and data
  • [ picture of messy data center ] Ten minutes into Bill’s first day on the job, he has to deal with a payroll run failure. Tomorrow is payday, and finance just found out that while all the salaried employees are going to get paid, none of the hourly factory employees will. All their records from the factory timekeeping systems were zeroed out.Was it a SAN failure? A database failure? An application failure? Interface failure? Cabling error?
  • http://www.flickr.com/photos/heritagefutures/3110685470/
  • How each side Actively impedes the achievement of each other’s goals.
  • Transcript of "When IT Fails The Business Fails..."

    1. 1. When IT Fails…The Business Fails…Gene KimAuthor, Visible Ops HandbookProKarma SeminarAugust 20, 2012Session ID: @RealGeneKim, genek@realgenekim.me
    2. 2. @RealGeneKim, genek@realgenekim.me
    3. 3. Now, More Than Ever… Even in “low-tech industries,” 95% of all capital projects have an IT component… 50% of all capital spending is technology-related Where we need to be… IT is always in the way (again…) We are here… @RealGeneKim, genek@realgenekim.me
    4. 4. Comparison Of Turnover For CEOs and CFOs…  When firms with IT-related material weaknesses are compared with the other two groups, there are some startling differences in executive turnover… N=184 Material weakness Material weakness “Clean” vs. (no IT related issues) (with IT related issues) CEO 2.0x higher 8.0x higher* CFO 1.7x higher 3.6x higher CIO 2.2x higher 2.2x higher * These firms also 2.6 less likely to be profitable than “clean” firmsSource: Forthcoming Paper: Richardson, Masli, Watson, Zmud, Sarbanes-Oxley Information TechnologyMaterial Weaknesses And The Disciplining Of The CEO, CFO And CIO @RealGeneKim, genek@realgenekim.me
    5. 5. There’s a hidden gas, that we can’t see, taste,touch, smell, and it’s killing CEOs everywhere. It’s called IT. Or more precisely, unplanned work in IT. 6 @RealGeneKim, genek@realgenekim.me
    6. 6. it.fail() == business.fail() @RealGeneKim, genek@realgenekim.me
    7. 7. Where Did The High Performers Come From? @RealGeneKim, genek@realgenekim.me
    8. 8. Over Ten Years, We Benchmarked 1500+ ITOrgs @RealGeneKim, genek@realgenekim.me
    9. 9. High Performing IT Organizations High performers maintain a posture of compliance  Fewest number of repeat audit findings  One-third amount of audit preparation effort High performers find and fix security breaches faster  5 times more likely to detect breaches by automated control  5 times less likely to have breaches result in a loss event When high performers implement changes…  14 times more changes  One-half the change failure rate  One-quarter the first fix failure rate  10x faster MTTR for Sev 1 outages When high performers manage IT resources…  One-third the amount of unplanned work  8 times more projects and IT services  6 times more applications Source: IT Process Institute, 2008 @RealGeneKim, genek@realgenekim.me
    10. 10. Tough Love From Ari Balogh @RealGeneKim, genek@realgenekim.me
    11. 11. The Downward Spiral Operations Sees… Dev Sees…  Too many fragile and insecure  More urgent, date-driven projects applications in production put into the queue  Too much time required to restore  Even more fragile code (less service secure) put into production  Too much firefighting and unplanned  More releases have increasingly work “turbulent installs”  Planned project work cannot complete  Release cycles lengthen to amortize “cost of deployments”  Frustrated customers leave  Bigger deployment failures  Market share goes down  More time spent on firefighting  Business misses Wall Street commitments  Ever increasing backlog of work that cold help the business win  Business makes even larger promises to Wall Street  Ever increasing amount of tension between IT Ops, Development, Design… These aren’t ITSM or IT Operations problems… These are business problems! @RealGeneKim, genek@realgenekim.me
    12. 12. My Mission Chronicle the Hero’s Journey For IT ("When IT Fails: A Business Novel”) so that everyone can gain a shared understanding of how and why IT fails, so they can fix it 13 @RealGeneKim, genek@realgenekim.me
    13. 13. 14 @RealGeneKim, genek@realgenekim.me
    14. 14. 15 @RealGeneKim, genek@realgenekim.me
    15. 15. 16 @RealGeneKim, genek@realgenekim.me
    16. 16. 17 @RealGeneKim, genek@realgenekim.me
    17. 17. 18 @RealGeneKim, genek@realgenekim.me
    18. 18. 19 @RealGeneKim, genek@realgenekim.me
    19. 19. The State Of The Business @RealGeneKim, genek@realgenekim.me
    20. 20. Project Phoenix @RealGeneKim, genek@realgenekim.me
    21. 21. Day 1: Payroll Outage @RealGeneKim, genek@realgenekim.me
    22. 22. @RealGeneKim, genek@realgenekim.me
    23. 23. Day 2: PMO Meeting @RealGeneKim, genek@realgenekim.me
    24. 24. @RealGeneKim, genek@realgenekim.me
    25. 25. Day 3: The SOX-404 Audit Meeting @RealGeneKim, genek@realgenekim.me
    26. 26. @RealGeneKim, genek@realgenekim.me
    27. 27. @RealGeneKim, genek@realgenekim.me
    28. 28. @RealGeneKim, genek@realgenekim.me
    29. 29. @RealGeneKim, genek@realgenekim.me
    30. 30. My Mission: Figure Out How Break The IT Core Chronic Conflict  Every IT organization is pressured to simultaneously:  Respond more quickly to urgent business needs  Provide stable, secure and predictable IT service Words often used to describe process improvement: “hysterical, irrelevant, bureaucratic, bottleneck, difficult to understand, not aligned with the business, immature, shrill, perpetually focused on irrelevant technical minutiae…” Source: The authors acknowledge Dr. Eliyahu Goldratt, creator of the Theory of Constraints and author of The Goal, has written extensively on the theory and practice of identifying and resolving core, chronic conflicts.31 @RealGeneKim, genek@realgenekim.me
    31. 31. 2007: Three Controls Predict 60% OfPerformance To what extent does an organization define, monitor and enforce the following?  Standardized configuration strategy  Process discipline  Controlled access to production systems @RealGeneKim, genek@realgenekim.me Source: IT Process Institute, 2008
    32. 32. Visible Ops: Playbook of High Performers The IT Process Institute has been studying high-performing organizations since 1999  What is common to all the high performers?  What is different between them and average and low performers?  How did they become great? Answers have been codified in the Visible Ops Methodology www.ITPI.org @RealGeneKim, genek@realgenekim.me
    33. 33. Visible Ops Security: Linking Security and IT Operations Objectives In 4 Practical Steps Service Design & Management Security Management Service Level Management Capacity Management Availability & Contingency Service Reporting Financial Management Management Control Processes Phase 2 Asset & Configuration Management Phase 3 Release Processes Change Management Supplier Processes Catch and Establish Release Management Resolution Processes Customer Relationship release, find Incident Management Management repeatable Problem Management Supplier Management fragile artifacts build library Automation Phase 1 Electrify fence, Phase 4 modify first Continually improve responseSources: ITPI Visible Ops & IT Infrastructure Library (ITIL) / BS 15000 @RealGeneKim, genek@realgenekim.me
    34. 34. Source: John Allspaw @RealGeneKim, genek@realgenekim.me
    35. 35. Source: John Allspaw @RealGeneKim, genek@realgenekim.me
    36. 36. The First Way:Systems Thinking @RealGeneKim, genek@realgenekim.me
    37. 37. The First Way:Systems Thinking(Business) (Customer) @RealGeneKim, genek@realgenekim.me
    38. 38. The Second Way:Amplify Feedback Loops @RealGeneKim, genek@realgenekim.me
    39. 39. The Third Way:Culture Of Continual Experimentation AndLearning @RealGeneKim, genek@realgenekim.me
    40. 40. Good News: It Can Be DoneBad News: You Can’t Do It Alone @RealGeneKim, genek@realgenekim.me
    41. 41. Ops @RealGeneKim, genek@realgenekim.me
    42. 42. QA And Test Source: Flickr: vandyll @RealGeneKim, genek@realgenekim.me
    43. 43. Development @RealGeneKim, genek@realgenekim.me
    44. 44. Process And Controls @RealGeneKim, genek@realgenekim.me
    45. 45. Product Management And Design Source: Flickr: birdsandanchors @RealGeneKim, genek@realgenekim.me
    46. 46. What Does Transformation Feel Like? 47 @RealGeneKim, genek@realgenekim.me
    47. 47. Find What’s Most Important First @RealGeneKim, genek@realgenekim.me
    48. 48. Quickly Find What Is Different… @RealGeneKim, genek@realgenekim.me
    49. 49. Before Something Bad Happens… @RealGeneKim, genek@realgenekim.me
    50. 50. Find Risk Early… @RealGeneKim, genek@realgenekim.me
    51. 51. Communicate It Effectively To Peers… @RealGeneKim, genek@realgenekim.me
    52. 52. Hold People Accountable… @RealGeneKim, genek@realgenekim.me
    53. 53. Based On Objective Evidence… @RealGeneKim, genek@realgenekim.me
    54. 54. Answer Important Questions… @RealGeneKim, genek@realgenekim.me
    55. 55. Recognize Compounding Technical Debt… @RealGeneKim, genek@realgenekim.me
    56. 56. That Gets Worse… @RealGeneKim, genek@realgenekim.me
    57. 57. And Fixing It… Source: Pingdom @RealGeneKim, genek@realgenekim.me
    58. 58. Have What We Need, When We Need It… @RealGeneKim, genek@realgenekim.me
    59. 59. Big Things Get Done Quickly… @RealGeneKim, genek@realgenekim.me
    60. 60. Ever Increasing Situational Mastery… @RealGeneKim, genek@realgenekim.me
    61. 61. Help The Business Win… @RealGeneKim, genek@realgenekim.me
    62. 62. With Support From Your Peers… @RealGeneKim, genek@realgenekim.me
    63. 63. And Do More With Less Effort… @RealGeneKim, genek@realgenekim.me
    64. 64. This Is An Important Problem Operations Sees… Dev Sees…  Fragile applications are prone to  More urgent, date-driven projects failure put into the queue  Long time required to figure out “which  Even more fragile code (less bit got flipped” secure) put into production  Detective control is a salesperson  More releases have increasingly “turbulent installs”  Too much time required to restore service  Release cycles lengthen to amortize “cost of deployments”  Too much firefighting and unplanned work  Failing bigger deployments more difficult to diagnose  Urgent security rework and remediation  Most senior and constrained IT ops resources have less time to  Planned project work cannot complete fix underlying process problems  Frustrated customers leave  Ever increasing backlog of work  Market share goes down that cold help the business win  Business misses Wall Street  Ever increasing amount of commitments tension between IT Ops, Development, Design…  Business makes even larger promises to Wall Street @RealGeneKim, genek@realgenekim.me
    65. 65. 66 @RealGeneKim, genek@realgenekim.me
    66. 66. @RealGeneKim, genek@realgenekim.me
    67. 67. When IT Fails: A Business Novel and The DevOps Cookbook  Coming in Winter 2012/2013  “In the tradition of the best MBA case studies, this book should be mandatory reading for business and IT graduates alike.” Paul Muller, VP Software Marketing, Hewlett- PackardGene Kim, Tripwire founder,  “The greatest IT management book of ourVisible Ops co-author generation.” Branden Williams, CTO Marketing, RSA @RealGeneKim, genek@realgenekim.me
    68. 68. When IT Fails: The Novel and The DevOps Cookbook  Our mission is to positively affect the lives of 1 million IT workers by 2017  If you would like the novel excerpts, “Top 10 Things You Needs To Know About DevOps,” and updates on the book:  Sign up at http://itrevolution.comGene Kim, Tripwire founder,Visible Ops co-author  Email genek@realgenekim.me  Hand me a business card @RealGeneKim, genek@realgenekim.me
    69. 69. If you’d like the slides from today’spresentation…  Text your first name, email address and “68383” to: +1 (858) 598-3980  Or visit: http://www.instantcustomer.c om/go/68383  Or scan this QR Code: 70 @RealGeneKim, genek@realgenekim.me
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×