The Postmortem – It doesn’t have to be messy


Published on

Your company has had a really, really bad day. Bad Days happen. It is when, not if. In a typical scenario, the top executives come in yelling and screaming and fire whoever they think may have had some part in the Bad Day’s events. After most of the yelling has stopped, then the executives want to know what happened. There is no doubt that autopsies can be messy and emit unpleasant odors, but they are a necessary evil and effective learning tool. But there is a right way and a wrong way to conduct a postmortem analysis of your incident. The obvious objective is to learn what happened so that it doesn’t happen again. The focus should be on the root cause and conducting an efficient analysis of the problem, but there is so much more that can be learned. Using the Incident Command System (ICS), this session provides an overview of the same postmortem process used by Fire Departments across the country.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The Postmortem – It doesn’t have to be messy

  1. 1. © 2013 Blackrock 3 Partners LLC 1A Good Post Mortem . . . .Includes a Pulse Check!22013 Events…so far2You are part of Critical Infrastructure!3Postmortem TypicalWorst Case (the UsualCase) Yelling & screaming Fighting & biting Firing Everyone hides Blame finding Many resumesupdated Stock option datereviewed Nothing is fixedBest Case (Utopia) Process oriented rootcause(s) analysis Team effort Focus on positive Focus on prevention Mistakes kept inperspective Completed by non-involved personnel
  2. 2. © 2013 Blackrock 3 Partners LLC 24EVENTA Typical Post Mortem Goes Like This Failure Operations Process Software Hardware Response Responsibilities5Why the Pulse Check?Cause: Factors that produce an effectReason: An explanation or justificationY6Looking for “Some Guy” Failure Operations Process Software Hardware Response Responsibilities
  3. 3. © 2013 Blackrock 3 Partners LLC 37AAR Marketing/Sales want products faster thanEngineering/Operations can design/build/implement them Investors want faster growth thanmanagement can deliver Early technology architecture/equipmentdecisions never get replaced R&D teams love the newest technology Operations teams hate the newesttechnology8It’s a Matter of Perspective A wake-up call A near miss for a larger, moredevastating incident An opportunity to stop and look at theorganization – because you never havetime to stop and look because thebusiness is running so fast You dont know what you dont know Don’t let a good crisis go to waste!9When done rightThe process allowsyou to look at riskdifferentlyPositive Failure!
  4. 4. © 2013 Blackrock 3 Partners LLC 410TECH Possible vs. LikelyOperationsCopyright 2013 BlackRock 3 Partners LLCOperationsNormalTechnologicalEnvironmentalHumanNot NormalCriminal11TALENTTrainingAccountabilityLeadershipEmpowermentNotificationTrust/TemptationExperts or Saviors12IMSIncidentCommanderUnified CommandApplicationsApp 1 App 2DatabaseDBA - 1 DBA - 2ContinuedOperationsCustomer ServiceExecutive LiaisonPlansDisaster Recovery
  5. 5. © 2013 Blackrock 3 Partners LLC 513Change – Failure – AAR Maybe we shouldn’t talk about failureand death together!FailureAARChangeAfter Action Review (AAR)Post Incident Evaluation (PIE)Post Incident Analysis (PIA)Incident Review (IR)Reason- Cause – Analysis (RCA)14EVENTOperations ProcessSoftware HardwareResponseResponsibilitiesSalesMarketingGrowthTechnologyEnvironmentCommunicationHumanAfter Action Review15AARTradeoff between different groups: Marketing Sales Engineering Operations Finance Management Team Board Investors
  6. 6. © 2013 Blackrock 3 Partners LLC 616Establishing a Culture Things WILL go wrong There should be a process to address them Incident Management System (IMS)– Handling of an occurrence– Handling of an issue– Having a really bad day RCA with a new perspective Retrospective and prospective The key is to understand the reason17www.blackrock3.comChris Hawley –Chris@blackrock3.comRob Schnepp –Rob@blackrock3.comRon Vidal –Ron@blackrock3.comSan Francisco & BaltimoreBlackrock 3 Partners LLC