CS5032 Lecture 9: Learning from failure 1Presentation Transcript
LEARNINGFROMFAILURE 1DR JOHN ROOKSBY
IN THIS LECTUREToday - two lectures looking at how organisations can investigateand learn from failure• Incident reporting• Accident investigationsIncident reporting:• It is important to learn from incidents irrespectively of whether they caused harm• Aviation, Healthcare and the Nuclear industry routinely collect incident data, but with varying success• Collecting incident data can be problematic – mainly because people don‟t want to be blamed for highlighting incidents
Accidents are the “tip of an iceberg”• Incidents or “adverse events” are far more common, and underlie accidents.Incident reports highlight faults orreport errors and near misseswhere there was potential for anaccident.If incidents are not reported, theymay never be noticed by anyoneother than those involvedWhat constitutes an incident andwhether it warrants reporting canbe a judgement http://en.wikipedia.org/wiki/File:Iceberg.jpg
INCIDENT REPORTINGSCHEMESIncident Reporting Schemes are mechanisms for learning fromerrors and failuresUsed in many safety critical domains:• Healthcare, Aviation, Marine, Railways, Nuclear Power, Oil and Chemical ProductionIncreasingly used in complex, business critical environments:• For example Data Centres
EXAMPLE REPORT - AVIATIONOn pre-flight check I loaded the Flight Management Computer (FMC),with longitude WEST instead of EAST. Somehow the FMC accepted it(it should have refused it three times). During taxi I noticed thatsomething was wrong, as I could not see the initial route and runwayon the navigation map display, but I got distracted by ATC. After wewere airborne, the senior cabin attendant came to the flight deck to tellus the cabin monitor (which shows the route on a screen topassengers) showed us in the Canaries instead of the WesternMediterranean! We continued the flight on raw data only to find out thatthe Heading was wrong by about 30-40 degrees. With a ceiling of1,000 ft at our destination I could not wait to be on terra firma. Now Ialways check the Latitude/Longitude three times on initialization!”
EXAMPLE REPORT – PHARMACYDate of report: March 6th Reporter: BettyJonesDate and time of incident: March 6th, 4.30pmDescription of incident: Warning message ignored by doctor. A PEP(Post-exposure prophylaxis) medication kit was ordered by a doctor in theemergency department. Standard PEPs contain lopinavir, ritonavir,zidovudine and lamivudine. The patient was already using severalmedications including venlafaxine, amitriptyline, bupropion, and fentanyl.If this standard PEP had been administered to the patient there may havebeen a harmful interaction between the ritonavir and the fentanyl.When the doctor ordered the PEP, the IT system flagged up a warningmessage saying there was a potential drug interaction problem betweenritonavir and fentanyl. This warning was ignored by the doctor, who laterexplained to me that she didn‟t read it because she was in a rush.This interaction is potentially fatal, and I rejected the prescription when Isaw it. An alternative PEP kit was dispensed.
INCIDENT REPORTINGSCHEMESReporting incidents, not just accidents, enables organisationsto:1. Identify why errors and failures occurred.2. Identify why accidents DON‟T occur. • What are the barriers that stop errors or failures escalating to accidents?3. Produce reminders of known hazards and workarounds and generally keep people thinking about safety and improvement4. Share success stories5. Allow information to be shared between sites, and (sometimes) between organisations6. Produce adequate quantities of data for understanding general issues (human factors, regulatory weakness etc) or rare issues.
INCIDENT REPORTINGSCHEMESLessons can be drawn from incident reports on an individualand collective basis:• Individually: • Reports are treated as a “war story”. This way the individual report is recounted in a meeting and discussed or can be posted in a newsletter. • Discussion and learning takes place among practitioners• Collectively: • Reports are collected together and can be analysed to identify themes and patterns • Lessons have to be drawn out from a manager or specialist investigator.
REPORTING RATESThere is an inverse relationship between the number of incidentsreported and the number of accidentsThe number of incident reports is not a measure of incidents• If an organisation has no incident reports, this does not mean it has had no incidents.• If an organisation has many incident reports this does not mean that it has had many incidents.Question: Over a twelve month period, organisation A has 0incident reports; organisation B has 100; organisation C has 1000.Which is the safest organisation?
REPORTING RATESIncident reports in 1997 2000 Commercial Aviation: 22,908 26,623 General Aviation: 8,384 8,501Accident rates per 100,000 flight hours Commercial Aviation: 1.6 3.24 General Aviation: 7.19 6.3Accidents Commercial Aviation: 147 (21) 148 (26) General Aviation: 1845 (350) 1837 (344)
FACTORS INFLUENCINGREPORTING RATESProfessionalism:• Reflection is an essential aspect of professional practice• Professional bodies encourage the accumulation of knowledge and create circumstances in which this can be achieved• In many cases incident reporting schemes arose from professional groups rather than within individual organisations
FACTORS INFLUENCINGREPORTING RATESTrust and Blame• If someone feels they might be blamed for a report they are less likely to write one • Blame may be from other practitioners, managers, the media/public• It can also be socially problematic to write reports that concern actions by other people• Organisations with a “blame culture” therefore have problems in learning from failure• Enabling anonymous reporting can avoid issues of blame to a certain degree, but successful schemes thrive when there is a “culture of trust”.
FACTORS INFLUENCINGREPORTING RATESDesign:• The design of a scheme also influences its success• How easy is it to report? • Incidents are best reported when they are fresh in your mind, and reports are more likely to be completed if it is relatively quick and easy to do so. • Paper vs electronic forms. Complex vs simple forms.• Feedback loops are also very important. People are more likely to report if they see value in writing reports.
FACTORS INFLUENCINGREPORTING RATESWhy do commercial pilots/crew report more?1. There is a „no blame‟ environment. Although reports are screened for serious offences, the orientation is to looking for the root causes of error rather than blaming individuals.2. There is more education about the value of reporting incidents3. There is a pro-reporting culture, pilots are reprimanded for having too few reports4. Commercial pilots have more to lose if they fail to report an incident. They are more likely to report, especially if they think someone else may have seen the incident5. There is a workable separation between accident reporting and incident reporting. The media focus more on accident reports.
FACTORS INFLUENCINGREPORTING RATESIn the NHS (England) incident reporting in anaesthesiology fellmassively when hospital wide reporting schemes were launchedin mid 2000s to replace departmental schemes • No atmosphere of trust across hospitals • Complex and cumbersome forms, or direct computer entry • No visible feedback loops • Led to parallel reporting schemes (in some cases reporting was done in secret)The situation improved over time
ANONYMITY ANDREPORTINGThe Paradox of anonymity: People often happier to reportanonymously but the reports are of less value to an investigationThere are different ways to handle anonymity in incident reporting• Open Schemes: Full disclosure of identity of reporter and those involved• Confidential: Disclosure of identities to trusted third parties• Anonymous: No disclosure of identityDesigning an anonymous system is very difficult. Anonymity isnot a good substitute for trust• Even if it is not clear who reported, it may be obvious who featured in the events described• People might also be seen reporting, even if what they report is not clear.
SIMPLE, SMALLSCALE SCHEME A report is submitted to report coordinator as soon as possible after an error Report coordinator asks secretary to type up report Reports are collated and sent to practitioners monthly Reports are discussed in monthly meeting and corrective actions decided
LARGE SCALE SCHEME Contributor submits report Specialists(human factors, Third party validates and systems, etc) supplements report. contacted if Submits to management and necessary regulator Bank of previous Regional and reports Management and regulator examined for national decide on corrective actions investigators similar incidents contacted if if necessary necessary Incident summary and corrective action published in bulletin
SMALL SCALE SCHEMESMany incident reporting schemes begin as local, small scaleschemes. For example a scheme might be initiated in onedepartment in an organisation, or among a local professional group. • Often higher levels of trust • The analysis can more readily draw from contextual knowledge • Focus on quick fixes (“make do and mend” culture) • Can be quick to react
LARGE SCALESCHEMESLarge, Organisation-Wide and National Schemes • More reports and greater coverage, but with inconsistencies between reports • More opportunities to look for root causes • Greater overheads in analysing reports because of lack of contextual knowledge • Trust much harder to maintainInternational Schemes • Some attempts at this, particularly within European Union. Tend to focus on bulletins and announcements
OPERATION ORCADIANAround ten years ago a boy died during an operation. Hisanaesthetic breathing circuit was blocked by a small plastic object.• An initial assumption of the police was that this was a deliberate act of sabotageIncident data was drawn together from across the NHS and anumber of incidents were found where small plastic objects wereblocking the breathing circuitAn investigation discovered the likely cause was that plastic capsfrom medications were sometimes finding their way into breathingtubes during storageChanges were made to the ways in which breathing equipment isstoredTraining was updatedAnd guidance was issued on how to spot when the breathing circuitis obstructed
MULTIPLE REPORTINGSCHEMESA problem other than a lack of reporting, is that some industries havemultiple reporting schemes • Different purposes and audiences • Different jurisdictions and authorities • Different geographic areas • Different approaches to confidentiality
MULTIPLE REPORTINGSCHEMESUK Nuclear Power Industry• NUPER (Nuclear Plant Event Reporting): Internal, private database of incidents in UK power industry• UK HSE (Health and Safety Executive): Publishes full incident reports, and summary versions• MHIDAS (Major hazards incidents database service): A bibliographic resource, maintained by Safety and Reliability Directorate• INIS (International Nuclear Information System): Coordinated by international atomic agency in Austria• PDR (Public Document Room): USA based resource, links to some incident reports and bulletins
MULTIPLE REPORTINGSCHEMESAviation - Three schemes is the UK • CAA Mandatory Reporting System, and Voluntary Reporting System • CHIRPS (Confidential Human Factors Incident Reporting Programme Scheme) • AAIB Air Accident Investigations Board (UK) produces monthly bulletinsInternational Schemes: Civil Aviation Organisation operate theADREP Accident/Incident Reporting System.Europe: European Commission is trying to overcome reportscheme compatibility issues with ECC-AIRS The European Co-ordination centre for Aircraft Accident Reporting
KEY POINTSIncident reporting schemes are important to safety. Reports canfocus on errors and near misses as well as failuresTrust is important for a successful schemes, people should not beblamed for reporting an incidentReporting schemes need to be well designed if they are to beeffectiveReporting rates do not correlate with incident rates. No reportsdoes not mean there were no incidents.Small scale schemes often focus on quick fixes rather than rootcauses. Large scale schemes are slower and more bureaucraticbut can be more thorough.
FURTHER READINGC.W. Johnson, Failure in Safety-Critical Systems: A Handbook of Accidentand Incident Reporting, University of Glasgow Press, Glasgow, Scotland,October 2003.Full text online: http://www.dcs.gla.ac.uk/~johnson/book/
EXERCISE1. What is the value of the following reports?2. Should the people involved be blamed for these incidents?3. Can you categorise events in these incidents using the GEMS slips-lapse-mistake model?4. What lessons are drawn from these incidents?5. Do the lessons learned from these incidents address the root causes of the problems? If not, why?
AOn pre-flight check I loaded the Flight Management Computer (FMC),with longitude WEST instead of EAST. Somehow the FMC accepted it(it should have refused it three times). During taxi I noticed thatsomething was wrong, as I could not see the initial route and runwayon the navigation map display, but I got distracted by ATC. After wewere airborne, the senior cabin attendant came to the flight deck to tellus the cabin monitor (which shows the route on a screen topassengers) showed us in the Canaries instead of the WesternMediterranean! We continued the flight on raw data only to find out thatthe Heading was wrong by about 30-40 degrees. With a ceiling of1,000 ft at our destination I could not wait to be on terra firma. Now Ialways check the Latitude/Longitude three times on initialization!”
BDate of report: March 6th Reporter: BettyJonesDate and time of incident: March 6th, 4.30pmDescription of incident: Warning message ignored by doctor. A PEP(Post-exposure prophylaxis) medication kit was ordered by a doctor in theemergency department to reduce the risk of HIV infection to a patient whohad been assaulted. Standard PEPs contain lopinavir, ritonavir,zidovudine and lamivudine. The patient was already using severalmedications including venlafaxine, amitriptyline, bupropion, and fentanyl.If this standard PEP had been administered to the patient there may havebeen a harmful interaction between the ritonavir and the fentanyl.When the doctor ordered the PEP, the IT system flagged up a warningmessage saying there was a potential drug interaction problem betweenritonavir and fentanyl. This warning was ignored by the doctor, who laterexplained to me that she didn‟t read it because she was in a rush.This interaction is potentially fatal, and I rejected the prescription when Isaw it. An alternative PEP kit was dispensed.
BOutcome: The pharmacists agree they must be vigilant when druginteractions have been overridden by doctors. The pharmacistsrecognise that doctors in the emergency department are very busyand that the system does not always meaningfully describe theseriousness of particular drug interactions.
Reporter: Anon Patient Sex: Male CASA: 2: Relevant systemic diseaseUrgency: 1: Routine; on distributed listFactors: anaesthetist, organisationalThe incident caused: 3: Transient abnormality with full recoveryHow preventable do you think the incident would be by furtherresource? 1: Probably within current resourceWhat happened? The patient was for direct pharyngoscopy, a short butstimulating procedure so the plan was to use boluses of alfentanil andmivacurium. Both these drugs were in correctly labelled 10 ml syringes.Inadvertently I gave the mivavurium prior to induction instead ofalfentanil. I did not realise my error for a few minutes. The patient initiallyappeared drowsy but agitated, breathing became shallow and saturationdropped to 85%. He developed multple VEs. On realising my error somepropofol was given, the trachea intubated and over a short period of timehis saturation and ECG returned to normal. We continued with theprocedure. On recovery he had recall of what had happened and wasquite distressed by it.
CLessons learned:1. Correctly labelling syringes isn‟t enough, especially when the colour ofthe labels is very similar. In this case both the labels that come with thedrug are white. We use other visual aids first, syringe size probably beingthe most important.2. Avoid drawing up muscle relaxants and induction agents in similar sizesyringes at the same time as other drugs, ie sux and fentanyl,thiopentone and augmentin.3. In this case the part the cause for the error was that I was using anumber of drugs that I dont usually use - thats when you should be extravigilant.
Reporter: Anon Patient Sex: DASA: 1: FitUrgency: 1: Routine; on distributed listFactors: EquipmentThe incident caused: 2: Transient abnormality unnoticed by patientHow preventable do you think the incident would be by furtherresource? 5: Not obviously by any change of practiceWhat happened? Patient was having a rigid bronchoscopy followed bysubmandibular gland excision. The patient was ventilated using a Sandersinjector for the bronchoscopy, connected to the high pressure oxygen outleton the anaesthetic machine (Blease Frontoline). Following thebronchoscopy, the patient was reintubated and conventionally for the nextprocedure. However, when the Sanders injector was disconnected from theoxygen outlet, the outlet valve jammed open, causing a massive leak ofoxygen, enough to cause a complete failure of the anaesthetic gas supply tothe patient. Fortunately, my initial response of fiddling with the leaking valveled to it closing and restoring normal function. If it had not closed, or anotheranaesthetist had reacted differently, the patient would have remainedunventilated until an alternative system of ventilation could be obtained.Cylinder and piped medical air on the machine were of no value as all thegas supply was leaking out.
DLessons Learned: However good and reliable modern anaestheticmachines are, catastrophic oxygen failure can always occur – evenbypassing the normal backup of cylinder supply, or medical air supply asin this case. The new Association of Anaesthetists machine checklistrecommends checking that an alternative means of ventilating a patient isavailable and checked – this incident is a good reminder of how importantthat can be. I intend to use this incident as a teaching scenario from nowon.