Losa Report


Published on

Air Georgian Limited Line Operations Safety Audit

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Losa Report

  1. 1. 2010 Air Georgian Limited  Daniel Bockner Vice President, Flight Operations AIR GEORGIAN LIMITED – 2010 LOSA REPORT Line Operations Safety Audit report 
  2. 2. Air Georgian Limited – 2010 LOSA Report                1
  3. 3. Air Georgian Limited – 2010 LOSA Report Contents Executive Summary ........................................................................................................................ 3 Acknowledgements ......................................................................................................................... 4 ABBREVIATIONS ........................................................................................................................ 5 GLOSSARY ................................................................................................................................... 6 Air Georgian Limited Air Canada Alliance Destinations ............................................................... 7 1. INTRODUCTION ...................................................................................................................... 8  1.1  The Line Operations Safety Audit ................................................................................... 8  1.2  Air Georgian Limited ....................................................................................................... 9  1.3  The Air Georgian LOSA ................................................................................................ 11 2.  LOSA Methodology .............................................................................................................. 12  2.1  The Threat and Error Management Model ..................................................................... 12  2.2  Involving the Union ....................................................................................................... 13  2.3  The LOSA Observation Flights...................................................................................... 15  2.4  The LOSA Observation Forms and Database ................................................................ 16  2.5  The LOSA Safety Survey ............................................................................................... 17 3.  Findings ................................................................................................................................. 18  3.1  Threats ............................................................................................................................ 18  3.2 Errors.............................................................................................................................. 19  3.3  Undesired Aircraft States ............................................................................................... 24  3.4   Crew Performance Marker Worksheet .......................................................................... 25  3.5 The LOSA Safety Survey .............................................................................................. 26 4. Recommendations ..................................................................................................................... 28 Conclusion .................................................................................................................................... 31 APPENDIX A ............................................................................................................................... 32 APPENDIX B ............................................................................................................................... 40 APPENDIX C ............................................................................................................................... 42 APPENDIX D ............................................................................................................................... 43 APPENDIX E ............................................................................................................................... 45  2
  4. 4. Air Georgian Limited – 2010 LOSA Report Executive Summary This report provides a detailed description of the Line Operations Safety Audit (LOSA) that wasconducted by Air Georgian Limited during the summer of 2010. The LOSA is an audit of liveline flights and follows the methodology set out in ICAO Document 9803. Air Georgian Limitedis a Canadian air operator based at Toronto’s Lester B Pearson International Airport. TheCompany operates a scheduled regional airline service as a Tier 3 partner to Air Canada usingBE02 commuter turboprop aircraft. Air Georgian operates flights from both Toronto, Ontarioand Halifax, Nova Scotia on segments as short as twenty-five minutes to up to just over twohours. The end result has been an increased awareness and understanding of the many riskfactors that can play a role in commercial aircraft operations thus providing a commensurateimprovement in safety.Due to the nature of commercial aircraft operations it is often quite difficult for companymanagers and individual pilots to gain a comprehensive, quantifiable picture of the manychallenges involved in day to day operations. The LOSA is a tool that is designed to address thisproblem as it allows for the accumulation of a substantial amount of data from regular line flightsthat can then be studied in detail. This is done by using trained observers who monitor a series offlights and take detailed notes of their observations; the information is then entered into adatabase. Managers are able to use the database to pinpoint recurring concerns so thatcountermeasures can be developed.For the Air Georgian LOSA 111 flights were observed by six trained observers. The observersprovided a narrative of the main points noticed during each particular flight. Also, using theThreat and Error Model, they identified each external threat (for example, adverse weather) tothe flight, each error by the crew, and any resulting undesired aircraft state. Finally, using ascoring system, the observers applied a numerical value to their assessment of the flight crew’sperformance to give an overall rating to each flight.The data collected during the LOSA has provided Air Georgian with a wealth of valuableinformation. Using the LOSA data the Company will be able to make improvements to the flightcrew training program, improve internal Company policies and procedures, and provide pilotswith a better understanding of the many threats that they may have to deal with during day to dayoperations. The 2010 LOSA will be followed up within the next two years with another full auditto assess the success, or lack thereof, that the Company will have in addressing the issues raisedso far. 3
  5. 5. Air Georgian Limited – 2010 LOSA Report Acknowledgements  The LOSA is a large and involved project and requires the dedicated efforts of a number ofpeople. Most importantly, I want to thank the in-flight observers for their hard work:Captain Tim Crits, Captain James Graham, Captain George Kyreakakos, Captain Rob Maahre,First Officer Patrick Pendergast and Captain Salman Syed.I would like to thank the other members of the LOSA Steering Committee:Captain Troy Stephens (Chief Pilot), Captain Dave Ongena (Chief Instructor), Captain RobBooth (Corporate Safety Officer) and Captain Vito Nobrega (Standards Captain). 4
  6. 6. Air Georgian Limited – 2010 LOSA Report ABBREVIATIONS ACP – Approved Check PilotAGL – Above Ground LevelAPG – Aircraft Performance Group (supplier of runway analysis charts)ATC – Air Traffic ControlBE02 – Hawker Beechcraft Corporation Model 1900D AirlinerCARs – Canadian Aviation RegulationsEFIS – Electronic Flight Instrument SystemFPM – Feet Per MinuteGPS – Global Positioning SystemGPWS – Ground Proximity Warning SystemICAO – International Civil Aviation OrganizationIMC – Instrument Meteorological ConditionsLOSA – Line Operations Safety AuditMEL – Minimum Equipment ListMDA – Minimum Descent AltitudeNOTAM – Notice to AirmenOREA – Ontario Regional Employee AssociationPF – Pilot FlyingPM – Pilot MonitoringSOP – Standard Operating ProceduresTCAS – Traffic Collision Avoidance SystemTEM – Threat and Error ManagementTORA – Take-Off Run AvailableUAS – Undesired Aircraft State 5
  7. 7. Air Georgian Limited – 2010 LOSA Report VMC – Visual Meteorological ConditionsVref – Reference Landing Approach Speed GLOSSARY Air Operator – The holder of an Air Operator’s Certificate (a commercial air service).Captain – The pilot assigned by the Company as the Pilot in Command for a particular flight. Tobe assigned as the Pilot in Command the pilot must hold the rank of Captain within the Companyas this includes substantial additional training and experience.Error – A non-compliance with regulations, standard operating procedures and policies, orunexpected deviation from crew, Company or ATC expectations on the part of the flight crew.First Officer – The pilot assigned by the Company as Second in Command for a particular flight.A pilot holding the rank of Captain may be assigned by the Company to act as the First Officerfor a particular flight but generally pilots holding the rank of First Officer are assigned to therole.Minimum Equipment List – A document approved by the Minister that authorizes an operator tooperate an aircraft with aircraft equipment that is inoperative under the conditions specifiedtherein.Snag – An inoperative item on an aircraft. Once discovered the snag is entered into the aircraftJourney Logbook. Depending on the snag an aircraft may or may not be able to remain in service(as per the aircraft Minimum Equipment List).Standard Operating Procedures – In the context of aircraft operations by an air operator, SOPsare a comprehensive company-issued publication that details the precise procedures that theflight crew shall follow from pre-flight procedures to the post-flight duties.Threat – An external factor that may increase risk during a flight.Undesired Aircraft State – An occurrence where the flight crew places the aircraft into a situationthat unnecessarily increases risk. 6
  8. 8. Air Georgian Limited – 2010 LOSA Report Air Georgian Limited Air Canada Alliance Destinations  At the time that the LOSA observation flights were carried out, Air Georgian Limited operatedflights to the following destinations under the Air Canada Alliance service:From Toronto (YYZ)Montreal, P.Q via BDL (YUL)Kingston, On (YGK)Sarnia, On (YZR)Rochester, N.Y (ROC)Syracuse, N.Y (SYR)Albany, N.Y (ALB)Hartford, Connecticut (BDL)Richmond, Virginia (RIC)Portland, Maine (PWM)Allentown, PA (ABE)Harrisburg, PA (MDT)Manchester, N.H (MHT)Grand Rapids, Michigan (GRR)Dayton, Ohio (DAY)Providence, R.I (PVD)From Halifax (YHZ)Moncton, N.B (YQM)Charlottetown, P.E.I (YYG)St. John, N.B (YSJ)Fredericton, N.B (YFC) 7
  9. 9. Air Georgian Limited – 2010 LOSA Report  1. INTRODUCTION 1.1 The Line Operations Safety Audit  Through the summer of 2010 Air Georgian Limited carried out a Line Operations Safety Audit (LOSA). This report will explain the methodology used and the results returned. The LOSA is a tool that was originally developed in the United States by the University of Texas Human Factors Research Project and Continental Airlines. Air Georgian carried out its LOSA independently as a wholly ‘in-house’ exercise (though external expertise was brought in during the preparations) using the methodology prescribed in ICAO Document 9803. ICAO Document 9803 describes the LOSA as ‘a program for the management of human error in aviation operations’ and proposes it ‘as a critical organizational strategy aimed at developing countermeasures to operations errors’1. In essence the LOSA is used to identify threats and errors in the cockpit environment by proactively collecting safety data. Flight crews are closely observed in order to identify the way that they deal with the many threats inherent in the operation of commercial aircraft during day to day operations as well as the errors that all humans make. Data is collected and carefully reviewed so that adjustments and improvements to training and company procedures can be made as applicable. In addition, flight crews can be made aware of common threats and errors so that they may be better able to avoid or deal with them. The LOSA process involves having specially trained observers monitoring regular line flights from a passenger seat. The observers do not participate in any way in the operation of the aircraft and are not members of the crew. During the flight the observer makes careful notes using a specially designed form. A substantial number of flights on all or most company routes must be observed to provide a meaningful data sample. The observation flights should be conducted in a relatively compressed time-frame or changes in weather may create operating conditions that vary too widely. For example, in southern Canada, a LOSA with some observation flights carried out in August and others done in November would return some uneven data due to the different operating conditions between the hot summer months and our rather cold and icy fall weather. Each observer must be fully trained in the LOSA methodology in order to return meaningful data and each should be selected for their intelligence, work ethic, reliability, and general enthusiasm for a complex project of this nature. It is also essential to have the full support of the union or employee association before the LOSA begins; management and the labour leadership must work together to eliminate any concerns among the pilot group regarding the purpose of the LOSA. All company pilots must understand that the LOSA is not about finding individual fault or getting people in trouble for not following procedures and/or policies. Pilots will only participate in a LOSA if they are clear that it is not a game of ‘gotcha’, but rather a useful 1 International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO, 2002), vii. 8
  10. 10. Air Georgian Limited – 2010 LOSA Report  contribution to overall safety; therefore the LOSA is a ‘no-jeopardy’ program for the flight crew. It must also be clearly understood by both management and labour that a LOSA is a voluntary process; flight crews cannot be forced to participate or else the atmosphere in the cockpit will be strained and the data returned may be skewed as a result. In addition, all of the data that is gathered in a LOSA must be completely de-identified. Flight Crew members must be comfortable in the knowledge that there is no way that any of the information gathered during the observation flights could be used against them in any way. With the Air Georgian LOSA management had absolutely no knowledge of the names of the flight crew members operating any of the observed flights and had no way of gaining this information. Managers must completely resist the temptation to identify flight crew members, even when an observed flight was clearly not operated in accordance with company policies and/or procedures. Any failure to do so would poison the process and render any future LOSA within the company impossible. Rather, it is far more effective for management to learn from the deviations that were observed during the LOSA and develop effective systemic counter-measures that will be useful for years to come rather than to try to discipline individuals. The purpose of this report is to provide Air Georgian management with a comprehensive breakdown of the results returned by the LOSA in order for the Company leadership team to put in place corrective measures as required. The list of recommendations at the end of the report is meant to give managers a set of objectives that, if met, will address the shortcomings within our operation that were noted during the LOSA. However the list is not necessarily exhaustive and, using this report and the raw data contained in the Air Georgian LOSA database, managers may well come up with additional goals. It is also hoped that individual Company pilots will take the time to digest the information contained in this report as well. Finally, the Air Georgian LOSA report contains information that is likely relevant to other operators in the regional airline world.1.2 Air Georgian Limited  Air Georgian Limited is a Canadian air operator with a fleet of fifteen Beechcraft 1900D (BE02) turboprop commuter airliners as well as four business jets. Air Georgian Limited operates fourteen BE02s on behalf of Air Canada under the Air Canada Alliance name. At the time of the LOSA twelve BE02s were based in Toronto, Ontario and two were based in Halifax, Nova Scotia; crews based in both cities participated in the audit. Air Georgian Limited has extensive experience operating the BE02 and has been doing so since 1996. The company has been flying on behalf of Air Canada since early 2000 and carries out over 25000 flights per year (not including the charter aircraft). Only crews operating the BE02 on Air Canada Alliance flights were involved in the LOSA. Air Georgian, as a regional turboprop operator, has a relatively young pilot group consisting of 120 flight crew. Pilot employees stay with the company for an average of four years before being hired by major Canadian airlines such as Air Canada. Air Georgian hires First Officers with a 9
  11. 11. Air Georgian Limited – 2010 LOSA Report minimum of 1500 hours of total flying time and the minimum for a Captain upgrade is 2700hours with 500 on the BE02. Earlier in its history Air Georgian was able to hire most of its pilotsfrom other commuter or air taxi operations. Pilots from these backgrounds often have experienceflying in the fairly demanding conditions common in the many sparsely settled areas of Canada.As the pool of available pilots has diminished in recent years Air Georgian has been forced tohire more of its flight crew directly from the ranks of flight instructors. This is unfortunate aspilots whose only professional employment is limited to instructing have a far narrowerexperience base to draw on. Having said that, Air Georgian provides comprehensive training andall of our pilots, regardless of background, bring themselves up to and maintain an acceptablestandard. The company also has a tightly controlled mentorship program where a small numberof pilots (a maximum of six) are hired directly from the aviation programs of either SenecaCollege or the University of New Brunswick with a minimum of 250 hours of flight time. Theprograms run by both of these schools are highly regarded within the industry and produceexcellent entry-level candidates. Air Georgian pilots fly an average of approximately seventyhours per month and are provided eleven guaranteed days off per month. The typical duty day isapproximately nine hours though days can be scheduled up to fourteen hours.The corporate culture at Air Georgian revolves around safety. From the owners of the Companydown through the CEO and Accountable Executive safety is always paramount. Air Georgian isin business to be profitable but never by accepting unsafe conditions or even unnecessarilyincreased risk. This message is passed on to the employee group from their first day of work withthe Company and is reinforced at every opportunity. The regulations and industry best practicesare followed to the letter and, in many aspects of our operation, Air Georgian goes well beyondthe minimum standard that is required. For example, Air Georgian pilots are provided withsubstantially more training time and Line Indoctrination than is required by the CanadianAviation Regulations (CARS); we also have a line check program for our pilots despite the factthat it is not a requirement for a commuter operator.Air Georgian/Air Canada Alliance flights operate sectors that range in length from twenty fiveminutes to over two hours (120 to 460 miles). Flights based in Toronto serve medium sizedmarkets, mostly in the northeastern United States, such as Albany New York, Portland Maine,Manchester New Hampshire etc… Halifax based aircraft fly to domestic destinations within theMaritime Provinces and most of these flights are relatively short. Almost all Air Georgian/AirCanada Alliance flights operate to and from their respective hubs (Toronto and Halifax) ratherthan carrying on to different stations in the course of the day. Only three of Air Georgian’s BE02aircraft are equipped with autopilots and all aircraft come equipped with GPWS, TCAS, GPSand dual EFIS screens for each pilot. 10
  12. 12. Air Georgian Limited – 2010 LOSA Report 1.3 The Air Georgian LOSA  The LOSA at Air Georgian was conducted using six in-flight observers and was overseen collectively by the LOSA Steering Committee. The LOSA Steering Committee consisted of the author (the Vice President, Flight Operations and also the Operations Manager), the Chief Pilot, the Chief Instructor, the Corporate Safety Officer, the Standards Captain and the six in-flight observers. The six in-flight observers were all non-management BE02 pilots; one was an ACP, two were Line Indoctrination Captains (including the ACP), three were line Captains and one was a First Officers. Five of the observers were based in Toronto and one in Halifax. Prior to conducting the in-flight observations the in-flight observers were given four days of training in the LOSA methodology. Detailed instruction was provided to them on the Threat and Error model, the forms used for the observations and specific details about the observation process among other topics. Two days of the training was provided by Dr. Robert Baron of ‘The Aviation Consulting Group’ (based in Myrtle Beach, South Carolina). The LOSA Steering Committee met on a number of occasions to work out the various details involved in carrying out the audit. One of our main challenges was the size of the aircraft; being a small, eighteen passenger seat commuter aircraft, the BE02 is not equipped with a spacious cockpit. The aircraft does not have a jumpseat in the cockpit and the observers had to sit in passenger seat 1B. Extenders for headsets were provided to the observers so that they could plug in and listen to the cockpit intercom and the VHF radios during each flight. Unfortunately, from seat 1B, the observers were faced with a difficult line of sight into the cockpit. It was possible to see almost all of what was going on but this required contortions that made for a fairly uncomfortable series of flights for the observers. Despite this problem the observers were able to effectively monitor the vast majority of the goings on in the cockpit and the reports returned by them contained an enormous amount of valuable information. The targeted number of usable LOSA observation flights was 120 (twenty for each observer) though, due to some issues that came up over the course of the observation flight schedule, we were only able to accomplish 111. Still, 111 observed flights represents an acceptable sample and consists of approximately 5% of the Air Canada Alliance flights that Air Georgian operates in a given month. Of course one must bear in mind that each crew was observed twice (once for the outbound leg and once inbound); therefore a total of fifty-six crews were observed during the audit. 11
  13. 13. Air Georgian Limited – 2010 LOSA Report  An Air Georgian Limited Beechcraft 1900D (BE02)2. LOSA Methodology  2.1 The Threat and Error Management Model   The Threat and Error Management model (TEM) was developed at the University of Texas and is best described as the aviation equivalent to ‘defensive driving for a motorist’2. In essence “the model posits that threats and errors are integral parts of daily flight operations and must be managed”3. With the TEM model the many threats, errors and undesired aircraft states that may be encountered on any given flight are defined and flight crews are trained to deal with them. It is also made clear to flight crews that most threats, errors and undesired aircraft states are regular occurrences that must be dealt with on a daily basis rather than abnormal situations. In the TEM model a threat is something that may increase risk during a flight but it is external and thus beyond the control of the crew (for example, an aircraft maintenance issue, significant weather, company schedule pressures etc…). An error is defined as a “non-compliance with regulations, Standard Operating Procedures (SOPs) and policies, or unexpected deviation from crew, company or ATC expectations”4. An undesired aircraft state is said to occur whenever the flight crew puts the aircraft into a situation that unnecessarily increases risk; this can be done either 2 Ashleigh Merritt, Ph.D. & James Klinect, Ph.D., “Defensive Flying for Pilots: An Introduction to Threat and Error Management”, Flight Safety Foundation website, http://flightsafety.org/archives-and-resources/threat-and-error management-tem. 3 International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO, 2002), 2-1. 4 Ibid 2-3 12
  14. 14. Air Georgian Limited – 2010 LOSA Report  through the actions of the flight crew or the lack thereof. The TEM model provides a simple to understand but comprehensive method of categorizing and quantifying the many threats, errors and undesired aircraft states that can be encountered on any given flight. At the time that the LOSA was conducted formal TEM training was not a part of the training curriculum provided to Air Georgian pilots. Many aspects of the TEM were present in the Company pilot training curriculum but the overall concept was not specifically trained. The LOSA made it clear that the TEM concept is a valuable tool and one that should be thoroughly taught to all Company pilots.2.2 Involving the Union  It is essential to the success of the LOSA that pilots feel comfortable participating. There must be a high level of trust between the pilot group and management or flight crews will not be interested in having an observer sitting behind them making notes while they do their jobs. Pilots must feel assured that the LOSA is not simply a game of ‘gotcha’ in which management is trying to shift blame for operational problems into the cockpit. One of the most effective ways of doing this is by involving the pilot union or employee association in the LOSA from the earliest planning stages. At Air Georgian management and labour generally have a good working relationship. There will always be disagreements between management and labour as no marriage is ever perfect. However, the Company has never had a work disruption due to labour issues and both Company management and the labour leadership are dedicated to creating a fair and equitable workplace while still focusing on safety and profitability (in that order). At Air Georgian the Ontario Regional Employee Association (OREA) represents all Company employees though by far the largest group of employees are pilots and most of the Association leadership are pilots as well. The President of OREA was consulted at the start of the LOSA process and he immediately saw the value in it and provided his full support. The labour leadership appreciated the fact that their input and support was sought out and they were invaluable in allaying any concerns that some members of the pilot group brought to their attention. Two weeks before the first series of observation flights the following memorandum was issued to the pilots providing a full explanation of the LOSA and making it clear that the audit had the full support of the employee association: 13
  15. 15. Air Georgian Limited – 2010 LOSA Report Line Operation Safety AuditPilots,Beginning in mid-July and continuing for about four weeks, Air Georgian will conduct a LineOperations Safety Audit (LOSA). For this audit we will use Air Georgian pilots to conduct in-flight observations. The in-flight observers will sit in seat 1B and will be plugged into theintercom for a full turn. Each in-flight observer will conduct a number of turns during the auditperiod. During the audit period approximately 60 turns (120 flights) will be observed.LOSA has been in use since the mid-1990s and is recognized world-wide as an effective safetytool. Numerous airlines, including Air Canada, have conducted LOSAs with very positive results.To our knowledge Air Georgian will be only the second airline operating turboprop commuteraircraft in the world to carry out a LOSA. LOSA observations are no-jeopardy events, and alldata are confidential and de-identified. LOSA data will be deposited into a data-base by the in-flight observers and none of the information regarding operating crew particulars will beprovided to management. The in-flight observers will themselves select the flights that they willobserve. All of the in-flight observers are non-management line pilots. The in-flight observerswill not be there to critique your performance – their mission is to be an unobtrusive observerand to fill out data collection forms after the flight is completed. Once all of the LOSAobservation flights have been completed and the information has been entered into the databaseby the in-flight observers the LOSA Steering Committee (consisting of the in-flight observers,myself, Troy Stephens, Dave Ongena and Rob Booth) will review the data and put together adetailed report along with recommendations that will be made available to the pilot group. Thedata, once it gets to the Steering Committee, will have been de-identified and the committeemembers will have no idea who the crews involved were or the dates of the observation flights.The recommendations will possibly lead to changes or additions to Company policies,procedures and training.The purpose of the audit is to identify problem areas that affect our operation so that awarenessand, where possible, countermeasures can be developed. There are three main areas that will belooked at, though the scope of the LOSA will not be limited to these. First, external threatscaused by ATC, our ground handlers, airport authorities etc… Secondly we will look at theperformance of our own crews to identify potential problems with training, comprehension ofprocedures and adherence to SOPs and policies. Third, we will be able to determine theeffectiveness of SOPs and company policies and see where changes for the better can be made.The LOSA is being carried out with the full knowledge and approval of OREA. We would like tostress again that the LOSA observation flights are no-jeopardy and will be completely de- 14
  16. 16. Air Georgian Limited – 2010 LOSA Report  identified; there is no need for the flight crew to be concerned about failing anything or being reported to management for making errors. We strongly urge all Air Georgian pilots to participate if an in-flight observer requests that he be allowed to observe your flight. However, it is the right of the crew to deny access to the in-flight observer should they so desire. Again, the LOSA will help to improve safety for all Air Georgian pilots and for our passengers so we hope to have a high level of participation from the pilot group. Please don’t hesitate to contact me if you have any questions about the LOSA. Thanks, Dan Bockner2.3 The LOSA Observation Flights  The six LOSA observers were each given seven straight days to carry out their observation flights. The observers were taken off of the normal flight schedule two at a time. On their first day each pair of observers monitored two Air Canada Alliance turns (from the hub to an outstations and back). On their second day the in-flight observers both brought their first day’s audit forms into the office for review. At this point any misunderstanding or questions that the observers had were addressed. The observers then continued with their observation flights over the subsequent five days doing two turns each day. The observers chose the flights that they wished to observe and used the following guidelines: Over the course of their LOSA week they were to observe flights to as many different destinations as possible; The observers had to coordinate with Crew Scheduling to check flight loads and to be listed on the flights that they chose. Obviously only flights with open seats could be observed (as mentioned above the BE02 does not have a jumpseat in the cockpit); The observers were listed on two flights at a time in case they were turned down by the crew of their first selected flight; Management had no involvement in the selection of the observation flights and had no knowledge of which flights were being observed. Once at the gate the observer would approach the flight crew and inform them that their flight had been selected for the LOSA. A brief explanation of what was to take place was provided to the Captain and the First Officer and both were informed that the LOSA was voluntary and they could feel free to deny boarding to the observer (over the course of the LOSA only one crew 15
  17. 17. Air Georgian Limited – 2010 LOSA Report  refused to participate and denied boarding to the observer). Once the flight crew agreed to take part in the LOSA the observer would board the flight with the rest of the passengers. As mentioned above, the observers sat in seat 1B on the BE02. During their observation flights they remained plugged in to the cockpit intercom system and could thus hear all communication between crew members and with ATC. Due to the nature of our operation the observer was not able to observe the flight crew as they prepared for the flight. This was most unfortunate as the pre-flight preparation process is very important to the successful completion of a safe flight. However, over the course of the LOSA flight the observers were generally able to determine how well the flight crew had prepared for each particular flight. The observers were under strict instructions not to interact with the flight crew in any way once they had completed their introductory briefing, though they were told to review the flight crew’s paperwork at some point during or immediately after the flight. During the flight the observers made very detailed notes and these were later used to fill out the LOSA observation forms. Some of the observers brought laptops though it was quickly determined that this was not practical as these had to be stowed during some of the most noteworthy segments of the flights. Once their first turn of the day was completed the observers took a meal break and then set off for their second turn. They generally completed the LOSA forms (transcribing their notes) each evening once they had completed their flights for the day.2.4 The LOSA Observation Forms and Database  For each LOSA flight specially designed LOSA forms were filled out. The forms used in the Air Georgian LOSA were very similar to the sample forms provided in ICAO Document 9803 and in FAA Advisory Circular 120-90 (a copy of the LOSA form is attached in Appendix A). Nothing that is written on the form provides any indication of the identity of the flight crew. One set of forms is filled out for each leg and this was either done in paper format or electronically at the discretion of the observer. The LOSA form allows for a detailed description of each flight. The observer notes some basic information about the flight such as the city pair, the pilot flying (either the Captain or the First Officer, without naming names of course), the general weather conditions, among other things. The next section on the form provides space for a detailed narrative divided into phases of flight. The observers were instructed to provide a general description of the overall flight situation (i.e. any delays or other stressors) as well as an idea of the working atmosphere in the cockpit (i.e. open communications, stressed, etc…). For each phase of the flight the narrative was to include any noteworthy events that could be defined as threats, errors or undesired aircraft states; as much detail as possible was required. Below the narrative section are boxes for threats, errors and undesired aircraft states. The final section of the form is the Crew Performance Marker Worksheet; here the observer is able to provide an assessment of the overall flight by providing marks on a number of different topics in all phases of flight. 16
  18. 18. Air Georgian Limited – 2010 LOSA Report  The completed forms were then entered into a database by the observers, who were provided office time to perform this task after all of the observation flights had been completed. The information was inputted into the database on a digital form that was identical to the LOSA forms used by the observers. The database is fully searchable and queries on single items or combinations can be made. For example, a query searching for all LOSA forms with the Captain as pilot flying, between Toronto and Albany with adverse weather present could be made and the returned forms could then be reviewed and compared. It is very important that the database be flexible and simple to use as it will be a valuable tool for the Training Department and the Safety Officer far into the future.2.5 The LOSA Safety Survey  The final component of the LOSA was the safety survey. A survey was created that was distributed to the pilot group by the LOSA observers. This was not done at the time of the observation flights because, due to the nature of our operation, the flight crew would likely have been over-tasked had they been pressed to do a survey just prior to or after operating a revenue flight. The safety survey consisted of ten questions that covered the respondent’s experience level and his/her assessment of the level of safety within the Company’s operation. The survey was deliberately kept short in order to increase the number of pilots who would bother to participate. Each copy of the survey was handed out to pilots by a LOSA observer who then, when possible, waited while the individual completed it. As with the observation flights, the survey was completely de-identified and there was no way for management to track who had written what. Some of the feedback from the survey will be included below and the survey questions are included in the appendix. The survey did not include any multiple choice answers as it was determined that written responses were more desirable in order to receive the information in the words of the pilots themselves. The possible downside to this approach is that some pilots could use the survey to air non-safety related complaints; this did turn out to be the case with some of the responses. However, filtering out the non-safety related issues is a fairly simple matter. The completed surveys will be kept for long-term reference so that the information and opinions gathered today can be compared with the concerns that Air Georgian pilots may raise in the future. It will be interesting to see if/how the pilot perception of safety within the Company may change over the next couple of years (until the next LOSA). 17
  19. 19. Air Georgian Limited – 2010 LOSA Report 3. Findings  3.1 Threats  Many of the threats encountered during the observed flights involved everyday airline issues and these were generally well-handled by the pilots. In the summer months extensive thunderstorm activity is quite common in the area of operations that Toronto based Air Georgian crews fly in. For Halifax based Air Georgian crews fog is quite common in July and August. Delays, especially in Toronto, are also a regular occurrence for a variety of reasons and Air Georgian crews take them in stride for the most part. However, there were delayed flights where the observers noted that the crew seemed quite rushed. Rushing is highly undesirable and leads to an unnecessary increase in the level of risk. These crews may have felt self-imposed pressure to make up some of the time lost due to the delay. Taxi instructions in Toronto were misunderstood by the flight crew on a number of occasions. Toronto is a large, busy international airport and taxi routes can involve a number of turns onto different taxiways and across runways. The observers noted instances where ATC used non- standard or otherwise unclear phraseology that caused some confusion among the crew. There were also missed calls during the taxi due to frequency congestion. In some cases the Air Georgian crew simply misunderstood or did not hear a clearly conveyed instruction or clearance. ATC was noted as a threat on a considerable number of the observed flights. Generally the ATC threat involved congested frequencies though the use of non-standard phraseology (as mentioned in the previous paragraph) and unclear transmissions were also noted as ATC threats. Air Georgian aircraft are maintained to a very high standard by our professional maintenance staff. However, due to the realities of airline work, aircraft are often dispatched with deferred items. Air Georgian management insists that flight crews follow the MEL to the letter thus mitigating most of the risk involved. However, there were instances during the observed flights where risk, while still being acceptable, was heightened due to the deferral of one or more aircraft system; the most substantial item being the pressurization system. One surprising threat that came to light from the observations was the scheduling of two Captains together as a crew. When two Captains flew together the observers noted that cockpit discipline was generally substantially lower than when a Captain and First Officer were paired together. For the most part the Captain listed as the Pilot in Command was too deferential to the when another Captain was acting as Second in Command. Also, in some cases, the Captains acting as Second in Command had not flown from the right seat for some time and were not as familiar with the right seat duties as they should have been. However, the real threat here was the heightened level of complacency involved; two Captains often seem to be far too relaxed when operating together, thus heightening the level of risk. 18
  20. 20. Air Georgian Limited – 2010 LOSA Report  The repetitive nature of regional airline flying is in itself a threat to a certain degree. On the one hand risk is lowered by the fact that crews quickly become very familiar with the routes and destinations. Unfortunately the routine and the generally simple operating conditions involved with flying in southern Canada and the northeastern quarter of the United States breeds complacency. Almost all of the crew errors noted below involved a complacent attitude among the flight crew. For example, a charter crew taxiing at an unfamiliar airport would almost certainly not taxi at an excessive speed; this is something crews only do when they know their way around an airport. The lack of compliance with the sterile cockpit SOP also indicates that a complacent attitude is fairly widespread though it varies in degree among individuals. THREATS 35 30 25 20 15 THREATS 10 5 0 Adverse Wx ATC Airport  Gnd and Air  Airline Ops  A/C Mtce +  Condition Traf Pressure MEL The above chart shows the main threat types and the percentage of flights where these were encountered (some flights encountered multiple threats).3.2 Errors  Over the course of 111 flights (the number of observed flights during the LOSA) there are bound to be some noteworthy events. No operation or group of pilots is perfect; hence the reason for the LOSA in the first place. It is natural that some of the observed events will be more significant than others and this section will start with a list of the most unusual occurrences noted by the observers. These are events that could be considered to have increased risk though there were no serious consequences in any of these cases. On a YYZ-ROC flight thunderstorms were active along the route of flight. The crew determined once airborne that the weather radar was malfunctioning. Upon arrival in Rochester the crew did 19
  21. 21. Air Georgian Limited – 2010 LOSA Report  not enter the snag into the journey logbook or take any other action (the aircraft should have been grounded in Rochester). They called for boarding as quickly as possible and departed on the return flight back to Toronto. In this situation the crew violated the Canadian Aviation Regulations (CAR 704.64) by knowingly operating an aircraft with the weather radar inoperative with forecast thunderstorms along the route of flight. During a YYZ-DAY flight the crew failed to properly check the NOTAMs. The Dayton International Airport was closed at the flight’s scheduled arrival time due to an air show. The crew only discovered this on the descent into Dayton when ATC pointed it out. The flight had to hold for more than a half an hour. The crew clearly failed to do the proper flight planning prior to departure. It should also be pointed out that the Flight Following office also missed this NOTAM. At the time of the LOSA Air Georgian used a Type C pilot self-dispatch system thus making the Captain fully responsible for all flight planning. Having said that the Company should have been aware of the temporary airport closure. On a BDL-YYZ flight the crew activated the wrong flight plan in the GPS (BDL-YUL instead of BDL-YYZ). They did not properly check the loaded flight plan as per the ‘Before Take-off’ checklist and only discovered the error when ATC intervened once en route. Prior to starting the approach during a YHZ-YFC flight the crew briefed a Pilot Monitored Approach (PMA). As per the SOPs the First Officer is to fly the aircraft during the approach and, if the runway is seen prior to minimums, the Captain takes over control to land the aircraft. On the flight in question the First Officer flew the aircraft all the way to landing. The failure to properly follow the PMA procedure increased the risk of confusion over which pilot had control of the aircraft at a critical phase of flight at approach minimums. On a YHZ-YFC flight the aircraft assigned to the flight had a deferred pressurization system that was properly logged. The crew was aware of the deferral yet they still opted to fly at a cruising altitude of 12000 feet. This is a relatively short flight (forty-five minutes) and the increased fuel consumption at an altitude below 10000 feet was not a concern (nor were there any other factors that made 12000 feet more desirable than an altitude below 10000 feet). In addition, the crew did not start a timer once they climbed above 10000 feet to comply with the CAR 605.32 allowing no more than thirty minutes unpressurized at an altitude above 10000 feet (to a maximum of 13000 feet). No briefing was made regarding the possible adverse consequences of flying above 10000 feet in an unpressurized aircraft. The decision to operate the flight above 10000 feet needlessly increased the crew’s susceptibility to hypoxia and may have violated the CARs. On a MHT-YYZ flight the aircraft pressurization system malfunctioned and the aircraft did not pressurize during the climb. The crew caught the fault well before they climbed above 10000 feet (and before the ‘Cabin Alt High’ annunciator illuminated) and they briefed a lower altitude for the flight back to Toronto once they had determined that the aircraft had sufficient fuel on-board for the higher burn rate. Unfortunately the crew did not enter the snag into the journey logbook and, upon arrival in Toronto, the aircraft was left on the gate with an undocumented open snag (this is a violation of 3.15 of the Company Operations Manual as well as a number of CARs). 20
  22. 22. Air Georgian Limited – 2010 LOSA Report  A flight departed from YZR for YYZ and the crew opted to take off with a tailwind (Sarnia is an uncontrolled airport). The crew did not refer to the runway analysis chart for the tailwind departure as per SOPs. This despite the fact that the runway is relatively short with a TORA of 5100 feet. There is a significant weight penalty per knot of tailwind for this runway. During the cruise climb portion of a YYZ-MHT flight the observer noted that both crew members were reading magazines. The aircraft was equipped with an autopilot and, during the climb, the vertical speed decreased to 300 FPM without either crewmember noticing. Although a 300 FPM vertical speed is not unsafe it is operationally undesirable and should have been observed and corrected by the crew. Also, had it been the speed that had been decreasing (if, for example, the crew had selected the incorrect climb mode) without being noticed by the crew, the situation would have quickly become dangerous. In addition, it should go without saying (and it is a violation of Company policy) that it is unacceptable for both crewmembers to be engaged in non-operational activities while operating a flight. A number of serious issues were noted that needlessly increased risk and which took place repeatedly during the series of LOSA observation flight such as: Excessive taxi speeds. On a surprising number of flights the observers noted high taxi speeds (above the twenty knot maximum allowed by the BE02 SOPs). Air Georgian has very clear taxi SOPs and the hazards of taxiing too fast are pointed out in Company literature. All Air Georgian flight crews are made aware that rushing (in any form) is never acceptable and yet high taxi speeds seem to be relatively common. From the data returned on the LOSA forms a large number of crews ignored the sterile cockpit SOP; casual conversation during critical phases of flight seems to be endemic. Since the findings from the February 2009 Colgan Air crash in Buffalo began to emerge Air Georgian has regularly publicized the hazards of ignoring the sterile cockpit SOP to flight crews. Apparently these efforts have yet to yield concrete results. Flight crews often discussed non-operational topics or engaged in non-operational activities at inappropriate times and the result during the observed flights included repeatedly missing radio calls from ATC, forgetting checklists and minor losses of situational awareness. As the Colgan Air crash demonstrated, ignoring the sterile cockpit SOP can lead to far worse consequences as well. There were a number of cases of crewmembers doing their checklists from memory. At Air Georgian we make it very clear that checklists must be properly read by the Pilot Monitoring; doing a checklist from memory is unacceptable as it regularly leads to errors. On a small number of flights the attitude of the Captain was deplorably lax. There are clearly a small minority of pilots who enjoy flaunting rules and procedures and who make a show of doing so. On flights operated by these Captains the level of risk is raised unnecessarily from start-up to shut-down. 21
  23. 23. Air Georgian Limited – 2010 LOSA Report  The LOSA data has uncovered less significant, but still notable, crew related errors. All of these findings were noted on numerous flights and point to items that flight crews must pay more attention to and that the Company must make more of an effort to stress during training. Flight crews often did not check their runway analysis charts when required to (Air Georgian uses APG charts). During training and line indoctrination Air Georgian pilots are instructed to refer to the APG charts for each take-off and landing. While it is certainly possible that crews had indeed checked the APG charts before the LOSA in-flight observer joined them it is extremely unlikely that this occurred in all, or even the majority, of cases where the observer noted that the charts had not been checked. In a significant minority of the observed flights the briefings provided between crew members were lacking. Important information was often left out of the briefings and, in a few cases, required briefings were not made at all. The KLN-90B GPS is the primary navigation system on all Air Georgian BE02s. The BE02 SOPs require that the GPS be backed up by traditional ground based navigation aids though this requirement was ignored on the vast majority of observed flights. The BE02 SOPs require that crews brief one emergency or abnormal situation as a refresher prior to, or during, the first flight of each pairing. This SOP was regularly ignored during the observed flights. The TCAS mode is often not set properly during flight (note that this also involves a check list flaw). Crews often started and shut down aircraft with the EFIS power and/or the Avionics Master on despite repeated direction from management not to do so as this procedure may cause damage to the avionics. The PF is required to make a 1000 foot call prior to level off in climbs and descents while the PM is required to make a 100 foot call prior to level off in climbs and descents; these calls are often missed. A number of floated landings were observed due to the PF landing with power on instead of idle power (as per the SOPs). The PM was often observed performing non-essential duties (such as filling out the journey logbook, making Company radio calls, or other non-essential paperwork) at inappropriate times when his/her attention should have been focused on the operation of the aircraft (i.e. during the taxi and while in busy terminal airspace). While these are more examples of violations of the sterile cockpit SOP, these tasks are at least operational in nature. There were many cases where the PF executed duties that are assigned to the PM by the SOPs. This can cause confusion between crew members and distracts the PF from actually flying or taxiing the aircraft. There is almost never any need for the PF to execute PM functions during a flight. This took place most often when the Captain was the PF. Crews often failed to play the automated turbulence briefing prior to entering turbulent air. Passengers were thus not given any warning upon encountering turbulence. 22
  24. 24. Air Georgian Limited – 2010 LOSA Report  The one passenger incident that occurred during the observed flights took place in Rochester. The flight had arrived at the gate and a ‘senior Air Canada employee’ insisted on being allowed off of the aircraft immediately. The BE02 SOPs require that the air stair door not be opened until the ground crew is prepared. However, the crew seems to have been intimidated into opening the door and allowing the passenger off of the aircraft at an inappropriate time. The proper procedure would have been to instruct the passenger to remain seated until the door was opened at the appropriate time. In addition, the crew should have contacted management as soon as practicable to provide information about the totally unacceptable behavior exhibited by the non-revenue passenger. ERRORS 60 50 40 30 20 10 ERRORS 0 The above chart shows the main error types and the percentage of flights where these were encountered (some flights had multiple errors). It should be stressed that the vast majority of errors noted were very minor. Perfection is impossible regardless of who is in the cockpit; the point of the above information is to make pilots aware of the types of errors that occur so that individuals and the Company as a whole can work to reduce the frequency of mistakes. 23
  25. 25. Air Georgian Limited – 2010 LOSA Report 3.3 Undesired Aircraft States  Almost all of the undesired aircraft states that were noted by the in-flight observers during the LOSA were related to crew errors, as one would expect. The undesired aircraft states that were encountered were all relatively minor and most were small deviations that were corrected almost immediately. However, the data returned is quite valuable as it will help the Company to tailor training so that times and locations where increased vigilance is required can be pointed out to pilots. The UAS that gives the most concern involves unstable approaches. The Air Georgian BE02 SOPs define a stabilized approach as follows: “The aircraft shall be stabilized on the approach by 1000 feet AGL in IMC and by 500 feet AGL in VMC. The criteria for a stabilized approach are as follows: (i) In the landing configuration with the Before Landing Checklist complete (ii) Indicated airspeed within plus 10 knots to minus 5 knots of target airspeed. Approach speed should be maintained until the missed approach point, then reduced to Vref following the decision to land. (iii) When conducting a single engine non-precision approach, flap 35 should be selected following the decision to leave MDA for landing. If the aircraft is not stabilized on the approach by 1000 feet AGL in IMC, or by 500 feet AGL in VMC, or if it becomes destabilized below these altitudes, the crew shall immediately execute a go-around.”5 There were three instances of unstable approaches observed and, though relatively minor occurrences, the Company must do a better job at emphasizing the danger of not having the aircraft properly stabilized for landing below the altitudes set out in the SOPs. Industry-wide the majority of landing accidents have followed unstable approaches and an occurrence rate of almost 3% is not acceptable. 5 Air Georgian Limited, Beech 1900D Standard Operating Procedures. (Toronto: Air Georgian Limited, 2009), 2-68. 24
  26. 26. Air Georgian Limited – 2010 LOSA Report  UNDESIRED AIRCRAFT STATES 7 6 5 4 3 2 1 UNDESIRED AIRCRAFT STATES 0 The above chart shows the Undesired Aircraft States that were encountered along with the percentage of flights that these were observed on.3.4 Crew Performance Marker Worksheet  In the Crew Performance Marker Worksheet the observer was able to sum up the flight as a whole. The flight is divided into phases and planning, execution, review and overall performance markers are assessed by assigning a score of one to four for each item (1 = Poor, 2 = Marginal, 3 = Good, 4 = Outstanding). The reasoning for each assigned score can be found in the narrative section of the form. Far more 4s were assigned by the observers than 1s and the narratives seem to justify this for the most part though there are instances where, based on the narrative, a mark of 1 would likely have been more appropriate. Still, the low number of crews assigned a ‘poor’ rating is positive and is indicative of the quality of work performed by most Air Georgian pilots. Still, the Company must work to reduce the number of ‘poor’ ratings, especially since the category with the highest number is ‘Leadership’ (in the Overall Performance Markers section) with a total of five (4.5% of observed flights). A ‘poor’ leadership rating was generally assigned where Captains utterly failed to establish a professional atmosphere in the cockpit. 25
  27. 27. Air Georgian Limited – 2010 LOSA Report  6 5 4 3 Pre‐Dep/Taxi 2 Takeoff/Climb 1 Descent/App/Land Overall 0 This chart shows the number of 1s (Poor) assigned on the Crew Performance Marker Worksheet (note that unlike the previous charts these are total numbers rather than percentages).3.5 The LOSA Safety Survey  The LOSA Safety Survey was intended to compliment the audit flights by allowing Air Georgian pilots to verbalize their impressions of the Company safety culture. The ten questions included in the survey asked pilots about their flying backgrounds, their views on the major safety hazards faced in our operation, as well as their impressions of how the Company approaches safety and the priority placed on it. The survey was not meant to be scientific; rather, the intention was to gain a feel for the general impression that the pilot group has regarding the approach that the Company takes toward safety. Had individuals expressed serious safety concerns or had there been a general frustration among pilots at the way the Company prioritizes safety there would certainly have been grounds for worry. Fortunately, in general, the survey respondents were satisfied with the Company’s approach toward safety and a number were quite complimentary. Of the twenty-five surveys returned sixteen pilots had no significant safety concerns to report, seven had concerns that had more to do with union issues than safety. Only two of the respondents raised what could be considered to be significant safety concerns, one involving a maintenance issue and one involving an internal communication failure that resulted in an aircraft going flying with an expired deferred maintenance item. Also, only two of the twenty- five respondents felt that there would be negative consequences from management if they ever refused to operate a flight because of a safety concern. 26
  28. 28. Air Georgian Limited – 2010 LOSA Report The main issue raised by the respondents had to do with fatigue; many feel that Air Georgian’soperations are fatiguing and this is obviously a safety concern. While it is possible that a pilotmay become fatigued while at work the Company has addressed this with a comprehensivefatigue policy. This policy allows pilots to call in fatigued (for any reason) if necessary or, ifalready at work, to remove themselves from duty if required with no disciplinary consequences.The Company also has a standing offer of a paid hotel room for any pilot (or other employee)who feels too tired to drive home after a duty period. In addition, Air Georgian pilots areguaranteed eleven days off per month and are not allowed to be scheduled with less than twelvehours off between pairings (that is, from the time they leave the airport until they have to arriveat work the next day) and eleven hours off while on a layovers (this is a bare minimum; mostlayovers are considerably longer). Given the above, and considering the available fatigueresearch, Company pilots are clearly provided with enough time free from duty to avoidbecoming fatigued due to the schedule. It should also be mentioned that, because of the unionscheduling rules, the distribution of work is heavily slanted in favour of more senior pilots. If itis an issue for pilots that some of them work much harder than others at Air Georgian the unionscheduling rules that give some pilots twenty-five hours of flying per month while others do 100hours would have to be changed. 27
  29. 29. Air Georgian Limited – 2010 LOSA Report  4. Recommendations   Now that some of the threats, errors and undesired aircraft states that Air Georgian flight crews encounter on a regular basis have been identified and quantified it is up to the Company as a whole to use this information to make improvements that will decrease risk. The following recommendations should, if followed, allow the Company to mitigate threats and substantially reduce errors and undesired aircraft states. 1. Air Georgian must devise a better method of informing flight crews of deferred maintenance items on their assigned aircraft. By informing flight crews in advance of DMIs there will be much less chance of errors (involving the maintenance status of the aircraft) occurring, the most important one being the operation of an aircraft in conditions that are prohibited by the deferral procedure. 2. As much as is operationally possible the Company should not pair two Captains together to operate flights. As this is not always possible the Company should make Captains aware of the higher number of errors that occur when they fly with each other rather than with a First Officer as well as the reasons for this. 3. The Company should make more of an effort in making pilots aware of the dangers involved in complacency. This can be done via the monthly newsletter, in ground school, during line checks and by managers when they fly with line pilots. 4. The leadership of the Flight Operations branch of the Company must make a major, sustained effort to encourage and increase adherence to SOPs. While SOP compliance is not bad within the Company the LOSA has made it clear that it can and should be much better. The following measures should be adopted: i. The Training Department should develop and use more scenario based simulator training that incorporates ‘normal’ SOP use rather than simply throwing one emergency after another at the candidates; ii. Management pilots should fly as often as possible and lead by example by strictly following the SOPs;iii. The very real dangers of non-compliance with SOPs must regularly be advertised to Company pilots via the monthly newsletter, during all training (ground school and simulator), and in person by Managers and Line Indoctrination Captains;iv. The small number of pilots who do not feel that SOPs apply to them should be identified when possible. These pilots should be counseled and, in extreme cases where adequate improvement is not shown, should have their employment terminated if they are unable or unwilling to comply with SOPs; v. It should be made clear to all Company pilots that anything less than full compliance with the SOPs is not acceptable and is unprofessional; it is not something that should be tolerated within the Company. 28
  30. 30. Air Georgian Limited – 2010 LOSA Report 5. Unstable approaches (as defined by the SOPs) must be eliminated completely from Company operations. It must be emphasized to all Company pilots during training, line indoctrination and line checks that unstable approaches cannot be continued; the only acceptable action is to discontinue an unstable approach.6. Too many Company pilots operate flights without referencing the runway performance data. Crews become complacent because almost all of the runways that Air Georgian operates from are more than adequate for the BE02 at maximum weight under all but the most extreme temperature conditions. However, not all of the runways are usable at maximum takeoff weight without a special departure procedure in the event of an engine failure; nor are all runways adequate under tailwind or icing conditions. It is crucial that the runway performance data charts are referenced prior to flight. However, the Company can facilitate this process by highlighting the performance limiting runways (and conditions) used by the scheduled service for pilots. This would be a simple matter of creating an information sheet that could be placed into the route manual.7. The Company must place more emphasis on the proper use of aircraft automation during training, line indoctrination and line checks (automation includes the passenger briefing system).8. It is clear that the Company has to enhance the training that we provide regarding the proper use of the aircraft Minimum Equipment List and the entering of snags in the aircraft Journey Log Book.9. High taxi speeds (outside of SOP limits) were a recurring issue during the observed flights (11% of the flights). Risk is substantially increased by taxiing too fast and the dangers of doing so must be advertised to flight crew more emphatically. Company management must also be on the lookout for fast taxiing and the Captains responsible should be counseled.10. The entire pilot training program for both initial and recurrent candidates requires an extensive review. The training program was first developed over ten years ago and components have been added in a haphazard fashion over time. The training program does not take advantage of many of the latest technologies available and some of these should be incorporated into a revamped program. The type of threats, errors and undesired aircraft states that have been noted in this report should be emphasized in the training program, during line indoctrination and on line checks. The Chief Instructor, working closely with the Chief Pilot, should conduct this review.11. Data from check rides, line indoctrination, line checks and training should be recorded and scrutinized so that deficiencies can be identified and corrected. These data can be used to set Company benchmarks so that progress can be measured over extended periods.12. While there is always more that can be done by any organization to enhance safety much of it will be pointless if the line employees who are actually doing the work do not ‘buy in’. For example, the Company can go on about SOP compliance on a daily basis but if pilots are not willing to make the effort to do the job properly very little will change. It has to be made clear to pilots that professionalism is essential at all times in this line of work. Safety has to involve all of the stakeholders making an effort to continually improve. Pilots must be clear that their own 29
  31. 31. Air Georgian Limited – 2010 LOSA Report  safety depends on always maintaining a high level of professionalism; this includes SOP compliance, regular review of emergency procedures and good aircraft systems knowledge.13. The Company should introduce formal Threat and Error Management training into the pilot initial and recurrent curricula.14. Management must review the BE02 checklists and correct any deficiencies (such as the lack of a reminder to change the TCAS mode at the top of the climb and the top of descent).15. The Flight Following office should become more proactive in the way that flights are handled. While aircraft dispatched under a Type C system do not require licensed dispatchers there is no reason why Company Flight Followers cannot play a more involved role in the way that flights are followed. This includes maintaining a high level of awareness throughout each shift of the maintenance status of aircraft, the weather conditions system-wide, NOTAMs that may affect operations, problematic airport conditions and any other essential operational information. Attaining this level of awareness among Flight Followers will require close supervision from management (at least initially) and a review of the training provided. 30
  32. 32. Air Georgian Limited – 2010 LOSA Report Conclusion  The LOSA was a large undertaking for a regional operator the size of Air Georgian. Asubstantial amount of resources were devoted to the project. However, the process was wellworth the monetary and time commitment involved and a follow-up LOSA will be conducted inthe next year and a half to two years.To a large degree the results of the LOSA are quite reassuring; the data returned shows that ouroperating environment and operation are safe with low levels of risk. Our flight crews aregenerally dedicated professionals who do not take undue risks and who place safety ahead of allother considerations. However, as noted above, the data also shows that there is work to be donein many areas. The Company must make more of an effort to enhance our pilot training byputting to better use the knowledge gained over the fifteen years that Air Georgian has operatedthe BE02. We should also make more use of some of the cutting edge training technologies thatare available today. Company communication to the pilot group should also be improved. Pilotsmust also recognize that maintaining a high level of professionalism is every bit as much theirresponsibility as it is the Company’s and they must strive every day to operate at the highestlevel possible. In short, continuous improvement has to be the goal for both management andpilots so that risk levels can be lowered to the bare minimum. Some of the data andrecommendations that have come out of the LOSA will hardly be surprising to experiencedaviation professionals; the value is derived from the fact that we now have verified, quantifiedinformation rather than assumptions. The LOSA has provided a very clear picture of theoperation at the time that the data was gathered.The findings provided by the LOSA will be used as benchmarks that the results of subsequentLOSAs will be measured against. When the next LOSA is conducted we will be able todetermine the effectiveness of the recommendations included in this report (and/or theeffectiveness of our execution). None of the issues raised in this report are insurmountable; onthe contrary, given the necessary effort and some time, every concern that has been brought tolight by the LOSA is correctable to a large degree. No operation is or can be perfect but a solidsafety culture and a commitment to continuous improvement will result in a sustained reductionin the level of risk inherent in flight operations. 31
  33. 33. Air Georgian Limited – 2010 LOSA Report APPENDIX A Below is a copy of the LOSA form; one form was filled out for each observed flight.Observer Information Observer ID (Employee number) Observation NumberCrew Observation Number Of(e.g., “1 of 2” indicates segment one for a crew that you observed across two segmentsFlight DemographicsCity Pairs (e.g., YYZ-ALB)A/C Type BE02Pilot Flying (Check one) CA FO Time from Pushback to Gate Arrival Local Arrival Time (Hours:Minutes) (Use 24 hour time) Late Departure? (Yes or No) Predeparture/Taxi Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did Narrative the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings. 32
  34. 34. Air Georgian Limited – 2010 LOSA Report  Takeoff/Climb Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How didNarrative the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings. 33
  35. 35. Air Georgian Limited – 2010 LOSA Report  CruiseNarrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly? Descent/Approach/Land/TaxiNarrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly? 34
  36. 36. Air Georgian Limited – 2010 LOSA Report  Overall Flight Narrative This narrative should include your overall impressions of the crew. Threat Management Worksheet Threat Description Threat Management Phase ofThreat ID Flight Linked to Threat flight crew How did the crew manage or Describe the threat 1 Predepart/Taxi error? Type 2 Takeoff/Climb mismanage the threat? 3 Cruise 4 Des/App/Land (Yes/No) 5 Taxi-in T1 T2 35
  37. 37. Air Georgian Limited – 2010 LOSA Report  T3 T4 T5 T6 Threat Codes Environmental Threats Airline Threats 200 Airline Operational Pressure 204 Ground/Ramp 100 Adverse Weather 103 Airport Conditions 201 Cabin 205 Dispatch/Paperwork 101 ATC 104 Heavy Traffic (air or ground) 202 A/C Malfunctions/MEL Items 206 Manuals/Charts 102 Terrain 199 Other Environmental Threats 203 Ground Maintenance 299 Other Airline Threats Error Management Worksheet Error Error Description Error Response/Outcome Management Phase of LinkedError ID Flight Error to Crew Error How did the crew Outcome Describe the crew Threat? Error Response manage or 1 Predepart/Taxi error 2 Takeoff/Climb Type 1 Detected 1 Inconsequential mismanage the (If Yes, 3 Cruise enter the 2 No response 2 Undesired state error? 4 Des/App/Land 3 Additional error 5 Taxi-in Threat ID)E1 36
  38. 38. Air Georgian Limited – 2010 LOSA Report E2E3E4E5E6 Error Type Codes Aircraft Handling Procedural Communication300 Manual Flying 400 SOP Cross-Verification 500 Crew to External Communication301 Flight Control 401 Checklist 501 Crew to Crew Communication302 Automation 402 Callout 599 Other Communication303 Ground Handling 403 Briefing304 Systems/Instruments/Radios 404 Documentation399 Other Aircraft Handling 499 Other Procedural Undesired Aircraft State (UAS) Management Worksheet UAS Description UAS Response/Outcome UAS ManagementUAS ID Linking Crew UAS How did the crew manage Error? Undesired aircraft UAS UAS Outcome Response or mismanage the state description Code (Enter the 1 Inconsequential undesired aircraft state? Error ID) 1 Detected 2 Additional Error 2 No responseUAS 1 37
  39. 39. Air Georgian Limited – 2010 LOSA Report UAS 2UAS 3 Undesired Aircraft State Type Codes Aircraft Handling States- Configuration States Ground States Approach/Landing States All Phases1 Incorrect A/C configuration--- 20 Proceeding toward wrong 40 Vertical deviation 80 Crew induced deviation flight controls, brakes, thrust runway above G/S or FMS path reversers, landing gear 41 Lateral deviation 21 Runway incursion 81 Crew induced deviation below 42 Unnecessary WX penetration G/S or FMS path2 Incorrect A/C configuration--- systems (fuel, electrical, 22 Proceeding toward wrong taxiway/ramp 43 Unauthorized airspace 82 Unstable approach hydraulics, pneumatics, air- penetration conditioning, pressurization, 23 Taxiway/ramp incursion 83 Continued landing---unstable instrumentation 44 Speed too high approach3 Incorrect A/C configuration--- 24 Wrong gate 45 Speed too low 84 Firm landing automation 25 Wrong hold spot 46 Abrupt aircraft control--- 85 Floated landing4 Incorrect A/C configuration--- 26 Abrupt aircraft control—taxi (attitude) engines 86 Landing off C/L 47 Excessive banking 87 Long landing outside TDZ 48 Operation outside aircraft limitations 88 Landing short of TDZ 99 Other Undesired States Crew Performance Marker Worksheet 1 2 3 4 Poor Marginal Good Outstanding Observed performance had Observed performance was Observed performance was Observed performance was truly safety implications adequate but needs improvement effective noteworthy 38
  40. 40. Air Georgian Limited – 2010 LOSA Report  Phase of Flight Ratings Predeparture/ Takeoff/ Descent/Approach/ Planning Performance Markers Taxi Climb Land/Taxi The required briefing wasSOP BRIEFING interactive and operationally thorough. Operational plans and decisions werePLANS STATED communicated and acknowledged. Crew members developedCONTINGENCY effective strategies toMANAGEMENT manage threats to safety. Execution Performance Markers Crew members actively monitored and cross-MONITOR/CROSS-CHECK checked systems and other crew members. Operational tasks were prioritized and properlyWORKLOAD MANAGEMENT managed to handle primary flight duties. Crew members remainedVIGILANCE alert to the environment and position of the aircraft. Automation was properly managed to balanceAUTOMATION MANAGEMENT situational and/or workload requirements. Crew members usedTAXIWAY/RUNWAY caution and kept watchMANAGEMENT outside when navigating taxiways and runways. Review/Modify Performance Markers Existing plans wereEVALUATION OF PLANS reviewed and modified when necessary. Crew members not afraid to ask questions to investigateINQUIRY and/or clarify current plans of action. Overall Performance Markers Ratings Environment for openCOMMUNICATION communication wasENVIRONMENT established and maintained. Captain showed leadershipLEADERSHIP and coordinated flight deck activities.   39
  41. 41. Air Georgian Limited – 2010 LOSA Report APPENDIX B This memo was issued to managers only in order to provide a brief description of the LOSA. Line Operations Safety AuditWe will be carrying out a Line Operations Safety Audit (LOSA) at Air Georgian over thesummer of 2010. LOSA is recognized worldwide as an effective diagnostic tool that enables airoperators to credibly measure their safety performance. By conducting a LOSA those responsiblefor operational safety will be provided with enough data to identify and thus be able to correctweaknesses that would otherwise be difficult to pinpoint.As this will be the first LOSA carried out at Air Georgian the scope will be quite wide; allaspects of the flight operation will be audited. Subsequent LOSAs can be narrowed in scope toconcentrate on issues identified by the 2010 audit. The time frame for the next LOSA would betwo to three years after the report for the 2010 audit is issued.The LOSA will be implemented by the LOSA steering committee which will consist of theOperations Manager, the System Chief Pilot, the Chief Instructor, the Corporate Safety Officerand a representative from the employee association. In general terms the audit will be executedin six stages as follows: 1. Planning – After the training course is completed on June 8th the steering committee will finalize the audit process and procedures in one or more meetings. 2. Audit flights – The actual audit flights will be carried out by six auditors. It is envisaged that each auditor will audit two turns per day over a five day period. The audit flights will be spread out through the month of July so that only two auditors are active at any one time. After their first day of audit flights the auditors will meet with at least two members of the steering committee to go over the process and clarify any points as necessary. The auditors will be given specific audit days on their schedules but will pick the flights to be audited themselves so that management is not in any way involved in selecting the crews of the audit flights. 3. Compile data – Once the entire series of audit flights have been completed the data must be compiled into a database. This will be done by the auditors themselves and each will be given an office day in early August or late July for this task. The data will be totally de-personalized so that there is no way to identify which crew members operated any of the flights. The format of the database is to be announced. 4. Review data – In August the steering committee and the auditors will meet to review the data captured during the audit and come up with a list of findings. 40