Losa Report

2010
Air Georgian Limited

Daniel Bockner
Vice President, Flight
Operations

AIR GEORGIAN LIMITED – 2010
LOSA REPORT
Line Operations Safety Audit report

Air Georgian Limited – 2010 LOSA Report

1


Contents
Executive Summary ........................................................................................................................ 3
Acknowledgements ......................................................................................................................... 4
ABBREVIATIONS ........................................................................................................................ 5
GLOSSARY ................................................................................................................................... 6
Air Georgian Limited Air Canada Alliance Destinations ............................................................... 7
1. INTRODUCTION ...................................................................................................................... 8
1.1 The Line Operations Safety Audit ................................................................................... 8
1.2 Air Georgian Limited ....................................................................................................... 9
1.3 The Air Georgian LOSA ................................................................................................ 11
2. LOSA Methodology .............................................................................................................. 12
2.1 The Threat and Error Management Model ..................................................................... 12
2.2 Involving the Union ....................................................................................................... 13
2.3 The LOSA Observation Flights...................................................................................... 15
2.4 The LOSA Observation Forms and Database ................................................................ 16
2.5 The LOSA Safety Survey ............................................................................................... 17
3. Findings ................................................................................................................................. 18
3.1 Threats ............................................................................................................................ 18
3.2 Errors.............................................................................................................................. 19
3.3 Undesired Aircraft States ............................................................................................... 24
3.4 Crew Performance Marker Worksheet .......................................................................... 25
3.5 The LOSA Safety Survey .............................................................................................. 26
4. Recommendations ..................................................................................................................... 28
Conclusion .................................................................................................................................... 31
APPENDIX A ............................................................................................................................... 32
APPENDIX B ............................................................................................................................... 40
APPENDIX C ............................................................................................................................... 42
APPENDIX D ............................................................................................................................... 43
APPENDIX E ............................................................................................................................... 45

2


Executive Summary

This report provides a detailed description of the Line Operations Safety Audit (LOSA) that was
conducted by Air Georgian Limited during the summer of 2010. The LOSA is an audit of live
line flights and follows the methodology set out in ICAO Document 9803. Air Georgian Limited
is a Canadian air operator based at Toronto’s Lester B Pearson International Airport. The
Company operates a scheduled regional airline service as a Tier 3 partner to Air Canada using
BE02 commuter turboprop aircraft. Air Georgian operates flights from both Toronto, Ontario
and Halifax, Nova Scotia on segments as short as twenty-five minutes to up to just over two
hours. The end result has been an increased awareness and understanding of the many risk
factors that can play a role in commercial aircraft operations thus providing a commensurate
improvement in safety.

Due to the nature of commercial aircraft operations it is often quite difficult for company
managers and individual pilots to gain a comprehensive, quantifiable picture of the many
challenges involved in day to day operations. The LOSA is a tool that is designed to address this
problem as it allows for the accumulation of a substantial amount of data from regular line flights
that can then be studied in detail. This is done by using trained observers who monitor a series of
flights and take detailed notes of their observations; the information is then entered into a
database. Managers are able to use the database to pinpoint recurring concerns so that
countermeasures can be developed.

For the Air Georgian LOSA 111 flights were observed by six trained observers. The observers
provided a narrative of the main points noticed during each particular flight. Also, using the
Threat and Error Model, they identified each external threat (for example, adverse weather) to
the flight, each error by the crew, and any resulting undesired aircraft state. Finally, using a
scoring system, the observers applied a numerical value to their assessment of the flight crew’s
performance to give an overall rating to each flight.

The data collected during the LOSA has provided Air Georgian with a wealth of valuable
information. Using the LOSA data the Company will be able to make improvements to the flight
crew training program, improve internal Company policies and procedures, and provide pilots
with a better understanding of the many threats that they may have to deal with during day to day
operations. The 2010 LOSA will be followed up within the next two years with another full audit
to assess the success, or lack thereof, that the Company will have in addressing the issues raised
so far.

3


Acknowledgements
The LOSA is a large and involved project and requires the dedicated efforts of a number of
people. Most importantly, I want to thank the in-flight observers for their hard work:

Captain Tim Crits, Captain James Graham, Captain George Kyreakakos, Captain Rob Maahre,
First Officer Patrick Pendergast and Captain Salman Syed.

I would like to thank the other members of the LOSA Steering Committee:

Captain Troy Stephens (Chief Pilot), Captain Dave Ongena (Chief Instructor), Captain Rob
Booth (Corporate Safety Officer) and Captain Vito Nobrega (Standards Captain).

4


ABBREVIATIONS
ACP – Approved Check Pilot

AGL – Above Ground Level

APG – Aircraft Performance Group (supplier of runway analysis charts)

ATC – Air Traffic Control

BE02 – Hawker Beechcraft Corporation Model 1900D Airliner

CARs – Canadian Aviation Regulations

EFIS – Electronic Flight Instrument System

FPM – Feet Per Minute

GPS – Global Positioning System

GPWS – Ground Proximity Warning System

ICAO – International Civil Aviation Organization

IMC – Instrument Meteorological Conditions

LOSA – Line Operations Safety Audit

MEL – Minimum Equipment List

MDA – Minimum Descent Altitude

NOTAM – Notice to Airmen

OREA – Ontario Regional Employee Association

PF – Pilot Flying

PM – Pilot Monitoring

SOP – Standard Operating Procedures

TCAS – Traffic Collision Avoidance System

TEM – Threat and Error Management

TORA – Take-Off Run Available

UAS – Undesired Aircraft State

5

VMC – Visual Meteorological Conditions

Vref – Reference Landing Approach Speed

GLOSSARY
Air Operator – The holder of an Air Operator’s Certificate (a commercial air service).

Captain – The pilot assigned by the Company as the Pilot in Command for a particular flight. To
be assigned as the Pilot in Command the pilot must hold the rank of Captain within the Company
as this includes substantial additional training and experience.

Error – A non-compliance with regulations, standard operating procedures and policies, or
unexpected deviation from crew, Company or ATC expectations on the part of the flight crew.

First Officer – The pilot assigned by the Company as Second in Command for a particular flight.
A pilot holding the rank of Captain may be assigned by the Company to act as the First Officer
for a particular flight but generally pilots holding the rank of First Officer are assigned to the
role.

Minimum Equipment List – A document approved by the Minister that authorizes an operator to
operate an aircraft with aircraft equipment that is inoperative under the conditions specified
therein.

Snag – An inoperative item on an aircraft. Once discovered the snag is entered into the aircraft
Journey Logbook. Depending on the snag an aircraft may or may not be able to remain in service
(as per the aircraft Minimum Equipment List).

Standard Operating Procedures – In the context of aircraft operations by an air operator, SOPs
are a comprehensive company-issued publication that details the precise procedures that the
flight crew shall follow from pre-flight procedures to the post-flight duties.

Threat – An external factor that may increase risk during a flight.

Undesired Aircraft State – An occurrence where the flight crew places the aircraft into a situation
that unnecessarily increases risk.

6


Air Georgian Limited Air Canada Alliance Destinations
At the time that the LOSA observation flights were carried out, Air Georgian Limited operated
flights to the following destinations under the Air Canada Alliance service:

From Toronto (YYZ)

Montreal, P.Q via BDL (YUL)

Kingston, On (YGK)

Sarnia, On (YZR)

Rochester, N.Y (ROC)

Syracuse, N.Y (SYR)

Albany, N.Y (ALB)

Hartford, Connecticut (BDL)

Richmond, Virginia (RIC)

Portland, Maine (PWM)

Allentown, PA (ABE)

Harrisburg, PA (MDT)

Manchester, N.H (MHT)

Grand Rapids, Michigan (GRR)

Dayton, Ohio (DAY)

Providence, R.I (PVD)

From Halifax (YHZ)

Moncton, N.B (YQM)

Charlottetown, P.E.I (YYG)

St. John, N.B (YSJ)

Fredericton, N.B (YFC)

7


1. INTRODUCTION
1.1 The Line Operations Safety Audit
Through the summer of 2010 Air Georgian Limited carried out a Line Operations Safety Audit
(LOSA). This report will explain the methodology used and the results returned. The LOSA is a
tool that was originally developed in the United States by the University of Texas Human
Factors Research Project and Continental Airlines. Air Georgian carried out its LOSA
independently as a wholly ‘in-house’ exercise (though external expertise was brought in during
the preparations) using the methodology prescribed in ICAO Document 9803.

ICAO Document 9803 describes the LOSA as ‘a program for the management of human error in
aviation operations’ and proposes it ‘as a critical organizational strategy aimed at developing
countermeasures to operations errors’1. In essence the LOSA is used to identify threats and errors
in the cockpit environment by proactively collecting safety data. Flight crews are closely
observed in order to identify the way that they deal with the many threats inherent in the
operation of commercial aircraft during day to day operations as well as the errors that all
humans make. Data is collected and carefully reviewed so that adjustments and improvements to
training and company procedures can be made as applicable. In addition, flight crews can be
made aware of common threats and errors so that they may be better able to avoid or deal with
them.

The LOSA process involves having specially trained observers monitoring regular line flights
from a passenger seat. The observers do not participate in any way in the operation of the aircraft
and are not members of the crew. During the flight the observer makes careful notes using a
specially designed form. A substantial number of flights on all or most company routes must be
observed to provide a meaningful data sample. The observation flights should be conducted in a
relatively compressed time-frame or changes in weather may create operating conditions that
vary too widely. For example, in southern Canada, a LOSA with some observation flights carried
out in August and others done in November would return some uneven data due to the different
operating conditions between the hot summer months and our rather cold and icy fall weather.
Each observer must be fully trained in the LOSA methodology in order to return meaningful data
and each should be selected for their intelligence, work ethic, reliability, and general enthusiasm
for a complex project of this nature. It is also essential to have the full support of the union or
employee association before the LOSA begins; management and the labour leadership must
work together to eliminate any concerns among the pilot group regarding the purpose of the
LOSA. All company pilots must understand that the LOSA is not about finding individual fault
or getting people in trouble for not following procedures and/or policies. Pilots will only
participate in a LOSA if they are clear that it is not a game of ‘gotcha’, but rather a useful

1
International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO,
2002), vii.

8

contribution to overall safety; therefore the LOSA is a ‘no-jeopardy’ program for the flight crew.
It must also be clearly understood by both management and labour that a LOSA is a voluntary
process; flight crews cannot be forced to participate or else the atmosphere in the cockpit will be
strained and the data returned may be skewed as a result. In addition, all of the data that is
gathered in a LOSA must be completely de-identified. Flight Crew members must be
comfortable in the knowledge that there is no way that any of the information gathered during
the observation flights could be used against them in any way. With the Air Georgian LOSA
management had absolutely no knowledge of the names of the flight crew members operating
any of the observed flights and had no way of gaining this information. Managers must
completely resist the temptation to identify flight crew members, even when an observed flight
was clearly not operated in accordance with company policies and/or procedures. Any failure to
do so would poison the process and render any future LOSA within the company impossible.
Rather, it is far more effective for management to learn from the deviations that were observed
during the LOSA and develop effective systemic counter-measures that will be useful for years
to come rather than to try to discipline individuals.

The purpose of this report is to provide Air Georgian management with a comprehensive
breakdown of the results returned by the LOSA in order for the Company leadership team to put
in place corrective measures as required. The list of recommendations at the end of the report is
meant to give managers a set of objectives that, if met, will address the shortcomings within our
operation that were noted during the LOSA. However the list is not necessarily exhaustive and,
using this report and the raw data contained in the Air Georgian LOSA database, managers may
well come up with additional goals. It is also hoped that individual Company pilots will take the
time to digest the information contained in this report as well. Finally, the Air Georgian LOSA
report contains information that is likely relevant to other operators in the regional airline world.

1.2 Air Georgian Limited
Air Georgian Limited is a Canadian air operator with a fleet of fifteen Beechcraft 1900D (BE02)
turboprop commuter airliners as well as four business jets. Air Georgian Limited operates
fourteen BE02s on behalf of Air Canada under the Air Canada Alliance name. At the time of the
LOSA twelve BE02s were based in Toronto, Ontario and two were based in Halifax, Nova
Scotia; crews based in both cities participated in the audit. Air Georgian Limited has extensive
experience operating the BE02 and has been doing so since 1996. The company has been flying
on behalf of Air Canada since early 2000 and carries out over 25000 flights per year (not
including the charter aircraft). Only crews operating the BE02 on Air Canada Alliance flights
were involved in the LOSA.

Air Georgian, as a regional turboprop operator, has a relatively young pilot group consisting of
120 flight crew. Pilot employees stay with the company for an average of four years before being
hired by major Canadian airlines such as Air Canada. Air Georgian hires First Officers with a

9

minimum of 1500 hours of total flying time and the minimum for a Captain upgrade is 2700
hours with 500 on the BE02. Earlier in its history Air Georgian was able to hire most of its pilots
from other commuter or air taxi operations. Pilots from these backgrounds often have experience
flying in the fairly demanding conditions common in the many sparsely settled areas of Canada.
As the pool of available pilots has diminished in recent years Air Georgian has been forced to
hire more of its flight crew directly from the ranks of flight instructors. This is unfortunate as
pilots whose only professional employment is limited to instructing have a far narrower
experience base to draw on. Having said that, Air Georgian provides comprehensive training and
all of our pilots, regardless of background, bring themselves up to and maintain an acceptable
standard. The company also has a tightly controlled mentorship program where a small number
of pilots (a maximum of six) are hired directly from the aviation programs of either Seneca
College or the University of New Brunswick with a minimum of 250 hours of flight time. The
programs run by both of these schools are highly regarded within the industry and produce
excellent entry-level candidates. Air Georgian pilots fly an average of approximately seventy
hours per month and are provided eleven guaranteed days off per month. The typical duty day is
approximately nine hours though days can be scheduled up to fourteen hours.

The corporate culture at Air Georgian revolves around safety. From the owners of the Company
down through the CEO and Accountable Executive safety is always paramount. Air Georgian is
in business to be profitable but never by accepting unsafe conditions or even unnecessarily
increased risk. This message is passed on to the employee group from their first day of work with
the Company and is reinforced at every opportunity. The regulations and industry best practices
are followed to the letter and, in many aspects of our operation, Air Georgian goes well beyond
the minimum standard that is required. For example, Air Georgian pilots are provided with
substantially more training time and Line Indoctrination than is required by the Canadian
Aviation Regulations (CARS); we also have a line check program for our pilots despite the fact
that it is not a requirement for a commuter operator.

Air Georgian/Air Canada Alliance flights operate sectors that range in length from twenty five
minutes to over two hours (120 to 460 miles). Flights based in Toronto serve medium sized
markets, mostly in the northeastern United States, such as Albany New York, Portland Maine,
Manchester New Hampshire etc… Halifax based aircraft fly to domestic destinations within the
Maritime Provinces and most of these flights are relatively short. Almost all Air Georgian/Air
Canada Alliance flights operate to and from their respective hubs (Toronto and Halifax) rather
than carrying on to different stations in the course of the day. Only three of Air Georgian’s BE02
aircraft are equipped with autopilots and all aircraft come equipped with GPWS, TCAS, GPS
and dual EFIS screens for each pilot.

10


1.3 The Air Georgian LOSA
The LOSA at Air Georgian was conducted using six in-flight observers and was overseen
collectively by the LOSA Steering Committee. The LOSA Steering Committee consisted of the
author (the Vice President, Flight Operations and also the Operations Manager), the Chief Pilot,
the Chief Instructor, the Corporate Safety Officer, the Standards Captain and the six in-flight
observers. The six in-flight observers were all non-management BE02 pilots; one was an ACP,
two were Line Indoctrination Captains (including the ACP), three were line Captains and one
was a First Officers. Five of the observers were based in Toronto and one in Halifax. Prior to
conducting the in-flight observations the in-flight observers were given four days of training in
the LOSA methodology. Detailed instruction was provided to them on the Threat and Error
model, the forms used for the observations and specific details about the observation process
among other topics. Two days of the training was provided by Dr. Robert Baron of ‘The
Aviation Consulting Group’ (based in Myrtle Beach, South Carolina).

The LOSA Steering Committee met on a number of occasions to work out the various details
involved in carrying out the audit. One of our main challenges was the size of the aircraft; being
a small, eighteen passenger seat commuter aircraft, the BE02 is not equipped with a spacious
cockpit. The aircraft does not have a jumpseat in the cockpit and the observers had to sit in
passenger seat 1B. Extenders for headsets were provided to the observers so that they could plug
in and listen to the cockpit intercom and the VHF radios during each flight. Unfortunately, from
seat 1B, the observers were faced with a difficult line of sight into the cockpit. It was possible to
see almost all of what was going on but this required contortions that made for a fairly
uncomfortable series of flights for the observers. Despite this problem the observers were able to
effectively monitor the vast majority of the goings on in the cockpit and the reports returned by
them contained an enormous amount of valuable information. The targeted number of usable
LOSA observation flights was 120 (twenty for each observer) though, due to some issues that
came up over the course of the observation flight schedule, we were only able to accomplish 111.
Still, 111 observed flights represents an acceptable sample and consists of approximately 5% of
the Air Canada Alliance flights that Air Georgian operates in a given month. Of course one must
bear in mind that each crew was observed twice (once for the outbound leg and once inbound);
therefore a total of fifty-six crews were observed during the audit.

11


An Air Georgian Limited Beechcraft 1900D (BE02)

2. LOSA Methodology

2.1 The Threat and Error Management Model
The Threat and Error Management model (TEM) was developed at the University of Texas and
is best described as the aviation equivalent to ‘defensive driving for a motorist’2. In essence “the
model posits that threats and errors are integral parts of daily flight operations and must be
managed”3. With the TEM model the many threats, errors and undesired aircraft states that may
be encountered on any given flight are defined and flight crews are trained to deal with them. It
is also made clear to flight crews that most threats, errors and undesired aircraft states are regular
occurrences that must be dealt with on a daily basis rather than abnormal situations. In the TEM
model a threat is something that may increase risk during a flight but it is external and thus
beyond the control of the crew (for example, an aircraft maintenance issue, significant weather,
company schedule pressures etc…). An error is defined as a “non-compliance with regulations,
Standard Operating Procedures (SOPs) and policies, or unexpected deviation from crew,
company or ATC expectations”4. An undesired aircraft state is said to occur whenever the flight
crew puts the aircraft into a situation that unnecessarily increases risk; this can be done either

2
Ashleigh Merritt, Ph.D. & James Klinect, Ph.D., “Defensive Flying for Pilots: An Introduction to Threat and Error
Management”, Flight Safety Foundation website, http://flightsafety.org/archives-and-resources/threat-and-error
management-tem.
3
International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO,
2002), 2-1.
4
Ibid 2-3

12

through the actions of the flight crew or the lack thereof. The TEM model provides a simple to
understand but comprehensive method of categorizing and quantifying the many threats, errors
and undesired aircraft states that can be encountered on any given flight.

At the time that the LOSA was conducted formal TEM training was not a part of the training
curriculum provided to Air Georgian pilots. Many aspects of the TEM were present in the
Company pilot training curriculum but the overall concept was not specifically trained. The
LOSA made it clear that the TEM concept is a valuable tool and one that should be thoroughly
taught to all Company pilots.

2.2 Involving the Union
It is essential to the success of the LOSA that pilots feel comfortable participating. There must be
a high level of trust between the pilot group and management or flight crews will not be
interested in having an observer sitting behind them making notes while they do their jobs. Pilots
must feel assured that the LOSA is not simply a game of ‘gotcha’ in which management is trying
to shift blame for operational problems into the cockpit. One of the most effective ways of doing
this is by involving the pilot union or employee association in the LOSA from the earliest
planning stages.

At Air Georgian management and labour generally have a good working relationship. There will
always be disagreements between management and labour as no marriage is ever perfect.
However, the Company has never had a work disruption due to labour issues and both Company
management and the labour leadership are dedicated to creating a fair and equitable workplace
while still focusing on safety and profitability (in that order). At Air Georgian the Ontario
Regional Employee Association (OREA) represents all Company employees though by far the
largest group of employees are pilots and most of the Association leadership are pilots as well.
The President of OREA was consulted at the start of the LOSA process and he immediately saw
the value in it and provided his full support. The labour leadership appreciated the fact that their
input and support was sought out and they were invaluable in allaying any concerns that some
members of the pilot group brought to their attention.

Two weeks before the first series of observation flights the following memorandum was issued
to the pilots providing a full explanation of the LOSA and making it clear that the audit had the
full support of the employee association:

13

Line Operation Safety Audit

Pilots,

Beginning in mid-July and continuing for about four weeks, Air Georgian will conduct a Line
Operations Safety Audit (LOSA). For this audit we will use Air Georgian pilots to conduct in-
flight observations. The in-flight observers will sit in seat 1B and will be plugged into the
intercom for a full turn. Each in-flight observer will conduct a number of turns during the audit
period. During the audit period approximately 60 turns (120 flights) will be observed.

LOSA has been in use since the mid-1990s and is recognized world-wide as an effective safety
tool. Numerous airlines, including Air Canada, have conducted LOSAs with very positive results.
To our knowledge Air Georgian will be only the second airline operating turboprop commuter
aircraft in the world to carry out a LOSA. LOSA observations are no-jeopardy events, and all
data are confidential and de-identified. LOSA data will be deposited into a data-base by the in-
flight observers and none of the information regarding operating crew particulars will be
provided to management. The in-flight observers will themselves select the flights that they will
observe. All of the in-flight observers are non-management line pilots. The in-flight observers
will not be there to critique your performance – their mission is to be an unobtrusive observer
and to fill out data collection forms after the flight is completed. Once all of the LOSA
observation flights have been completed and the information has been entered into the database
by the in-flight observers the LOSA Steering Committee (consisting of the in-flight observers,
myself, Troy Stephens, Dave Ongena and Rob Booth) will review the data and put together a
detailed report along with recommendations that will be made available to the pilot group. The
data, once it gets to the Steering Committee, will have been de-identified and the committee
members will have no idea who the crews involved were or the dates of the observation flights.
The recommendations will possibly lead to changes or additions to Company policies,
procedures and training.

The purpose of the audit is to identify problem areas that affect our operation so that awareness
and, where possible, countermeasures can be developed. There are three main areas that will be
looked at, though the scope of the LOSA will not be limited to these. First, external threats
caused by ATC, our ground handlers, airport authorities etc… Secondly we will look at the
performance of our own crews to identify potential problems with training, comprehension of
procedures and adherence to SOPs and policies. Third, we will be able to determine the
effectiveness of SOPs and company policies and see where changes for the better can be made.

The LOSA is being carried out with the full knowledge and approval of OREA. We would like to
stress again that the LOSA observation flights are no-jeopardy and will be completely de-

14

identified; there is no need for the flight crew to be concerned about failing anything or being
reported to management for making errors. We strongly urge all Air Georgian pilots to
participate if an in-flight observer requests that he be allowed to observe your flight. However, it
is the right of the crew to deny access to the in-flight observer should they so desire. Again, the
LOSA will help to improve safety for all Air Georgian pilots and for our passengers so we hope
to have a high level of participation from the pilot group.

Please don’t hesitate to contact me if you have any questions about the LOSA.

Thanks,

Dan Bockner

2.3 The LOSA Observation Flights
The six LOSA observers were each given seven straight days to carry out their observation
flights. The observers were taken off of the normal flight schedule two at a time. On their first
day each pair of observers monitored two Air Canada Alliance turns (from the hub to an
outstations and back). On their second day the in-flight observers both brought their first day’s
audit forms into the office for review. At this point any misunderstanding or questions that the
observers had were addressed. The observers then continued with their observation flights over
the subsequent five days doing two turns each day. The observers chose the flights that they
wished to observe and used the following guidelines:

 Over the course of their LOSA week they were to observe flights to as many different
destinations as possible;
 The observers had to coordinate with Crew Scheduling to check flight loads and to be listed on
the flights that they chose. Obviously only flights with open seats could be observed (as
mentioned above the BE02 does not have a jumpseat in the cockpit);
 The observers were listed on two flights at a time in case they were turned down by the crew of
their first selected flight;
 Management had no involvement in the selection of the observation flights and had no
knowledge of which flights were being observed.

Once at the gate the observer would approach the flight crew and inform them that their flight
had been selected for the LOSA. A brief explanation of what was to take place was provided to
the Captain and the First Officer and both were informed that the LOSA was voluntary and they
could feel free to deny boarding to the observer (over the course of the LOSA only one crew

15

refused to participate and denied boarding to the observer). Once the flight crew agreed to take
part in the LOSA the observer would board the flight with the rest of the passengers. As
mentioned above, the observers sat in seat 1B on the BE02. During their observation flights they
remained plugged in to the cockpit intercom system and could thus hear all communication
between crew members and with ATC.

Due to the nature of our operation the observer was not able to observe the flight crew as they
prepared for the flight. This was most unfortunate as the pre-flight preparation process is very
important to the successful completion of a safe flight. However, over the course of the LOSA
flight the observers were generally able to determine how well the flight crew had prepared for
each particular flight. The observers were under strict instructions not to interact with the flight
crew in any way once they had completed their introductory briefing, though they were told to
review the flight crew’s paperwork at some point during or immediately after the flight. During
the flight the observers made very detailed notes and these were later used to fill out the LOSA
observation forms. Some of the observers brought laptops though it was quickly determined that
this was not practical as these had to be stowed during some of the most noteworthy segments of
the flights. Once their first turn of the day was completed the observers took a meal break and
then set off for their second turn. They generally completed the LOSA forms (transcribing their
notes) each evening once they had completed their flights for the day.

2.4 The LOSA Observation Forms and Database
For each LOSA flight specially designed LOSA forms were filled out. The forms used in the Air
Georgian LOSA were very similar to the sample forms provided in ICAO Document 9803 and in
FAA Advisory Circular 120-90 (a copy of the LOSA form is attached in Appendix A). Nothing
that is written on the form provides any indication of the identity of the flight crew. One set of
forms is filled out for each leg and this was either done in paper format or electronically at the
discretion of the observer. The LOSA form allows for a detailed description of each flight. The
observer notes some basic information about the flight such as the city pair, the pilot flying
(either the Captain or the First Officer, without naming names of course), the general weather
conditions, among other things. The next section on the form provides space for a detailed
narrative divided into phases of flight. The observers were instructed to provide a general
description of the overall flight situation (i.e. any delays or other stressors) as well as an idea of
the working atmosphere in the cockpit (i.e. open communications, stressed, etc…). For each
phase of the flight the narrative was to include any noteworthy events that could be defined as
threats, errors or undesired aircraft states; as much detail as possible was required. Below the
narrative section are boxes for threats, errors and undesired aircraft states. The final section of
the form is the Crew Performance Marker Worksheet; here the observer is able to provide an
assessment of the overall flight by providing marks on a number of different topics in all phases
of flight.

16

The completed forms were then entered into a database by the observers, who were provided
office time to perform this task after all of the observation flights had been completed. The
information was inputted into the database on a digital form that was identical to the LOSA
forms used by the observers. The database is fully searchable and queries on single items or
combinations can be made. For example, a query searching for all LOSA forms with the Captain
as pilot flying, between Toronto and Albany with adverse weather present could be made and the
returned forms could then be reviewed and compared. It is very important that the database be
flexible and simple to use as it will be a valuable tool for the Training Department and the Safety
Officer far into the future.

2.5 The LOSA Safety Survey
The final component of the LOSA was the safety survey. A survey was created that was
distributed to the pilot group by the LOSA observers. This was not done at the time of the
observation flights because, due to the nature of our operation, the flight crew would likely have
been over-tasked had they been pressed to do a survey just prior to or after operating a revenue
flight. The safety survey consisted of ten questions that covered the respondent’s experience
level and his/her assessment of the level of safety within the Company’s operation. The survey
was deliberately kept short in order to increase the number of pilots who would bother to
participate. Each copy of the survey was handed out to pilots by a LOSA observer who then,
when possible, waited while the individual completed it. As with the observation flights, the
survey was completely de-identified and there was no way for management to track who had
written what. Some of the feedback from the survey will be included below and the survey
questions are included in the appendix. The survey did not include any multiple choice answers
as it was determined that written responses were more desirable in order to receive the
information in the words of the pilots themselves. The possible downside to this approach is that
some pilots could use the survey to air non-safety related complaints; this did turn out to be the
case with some of the responses. However, filtering out the non-safety related issues is a fairly
simple matter. The completed surveys will be kept for long-term reference so that the
information and opinions gathered today can be compared with the concerns that Air Georgian
pilots may raise in the future. It will be interesting to see if/how the pilot perception of safety
within the Company may change over the next couple of years (until the next LOSA).

17


3. Findings

3.1 Threats
Many of the threats encountered during the observed flights involved everyday airline issues and
these were generally well-handled by the pilots. In the summer months extensive thunderstorm
activity is quite common in the area of operations that Toronto based Air Georgian crews fly in.
For Halifax based Air Georgian crews fog is quite common in July and August. Delays,
especially in Toronto, are also a regular occurrence for a variety of reasons and Air Georgian
crews take them in stride for the most part. However, there were delayed flights where the
observers noted that the crew seemed quite rushed. Rushing is highly undesirable and leads to an
unnecessary increase in the level of risk. These crews may have felt self-imposed pressure to
make up some of the time lost due to the delay.

Taxi instructions in Toronto were misunderstood by the flight crew on a number of occasions.
Toronto is a large, busy international airport and taxi routes can involve a number of turns onto
different taxiways and across runways. The observers noted instances where ATC used non-
standard or otherwise unclear phraseology that caused some confusion among the crew. There
were also missed calls during the taxi due to frequency congestion. In some cases the Air
Georgian crew simply misunderstood or did not hear a clearly conveyed instruction or clearance.

ATC was noted as a threat on a considerable number of the observed flights. Generally the ATC
threat involved congested frequencies though the use of non-standard phraseology (as mentioned
in the previous paragraph) and unclear transmissions were also noted as ATC threats.

Air Georgian aircraft are maintained to a very high standard by our professional maintenance
staff. However, due to the realities of airline work, aircraft are often dispatched with deferred
items. Air Georgian management insists that flight crews follow the MEL to the letter thus
mitigating most of the risk involved. However, there were instances during the observed flights
where risk, while still being acceptable, was heightened due to the deferral of one or more
aircraft system; the most substantial item being the pressurization system.

One surprising threat that came to light from the observations was the scheduling of two
Captains together as a crew. When two Captains flew together the observers noted that cockpit
discipline was generally substantially lower than when a Captain and First Officer were paired
together. For the most part the Captain listed as the Pilot in Command was too deferential to the
when another Captain was acting as Second in Command. Also, in some cases, the Captains
acting as Second in Command had not flown from the right seat for some time and were not as
familiar with the right seat duties as they should have been. However, the real threat here was the
heightened level of complacency involved; two Captains often seem to be far too relaxed when
operating together, thus heightening the level of risk.

18

The repetitive nature of regional airline flying is in itself a threat to a certain degree. On the one
hand risk is lowered by the fact that crews quickly become very familiar with the routes and
destinations. Unfortunately the routine and the generally simple operating conditions involved
with flying in southern Canada and the northeastern quarter of the United States breeds
complacency. Almost all of the crew errors noted below involved a complacent attitude among
the flight crew. For example, a charter crew taxiing at an unfamiliar airport would almost
certainly not taxi at an excessive speed; this is something crews only do when they know their
way around an airport. The lack of compliance with the sterile cockpit SOP also indicates that a
complacent attitude is fairly widespread though it varies in degree among individuals.

THREATS
35

30

25

20

15 THREATS
10

5

0
Adverse Wx ATC Airport Gnd and Air Airline Ops A/C Mtce +
Condition Traf Pressure MEL

The above chart shows the main threat types and the percentage of flights where these were
encountered (some flights encountered multiple threats).

3.2 Errors
Over the course of 111 flights (the number of observed flights during the LOSA) there are bound
to be some noteworthy events. No operation or group of pilots is perfect; hence the reason for the
LOSA in the first place. It is natural that some of the observed events will be more significant
than others and this section will start with a list of the most unusual occurrences noted by the
observers. These are events that could be considered to have increased risk though there were no
serious consequences in any of these cases.

 On a YYZ-ROC flight thunderstorms were active along the route of flight. The crew determined
once airborne that the weather radar was malfunctioning. Upon arrival in Rochester the crew did

19

not enter the snag into the journey logbook or take any other action (the aircraft should have
been grounded in Rochester). They called for boarding as quickly as possible and departed on the
return flight back to Toronto. In this situation the crew violated the Canadian Aviation
Regulations (CAR 704.64) by knowingly operating an aircraft with the weather radar inoperative
with forecast thunderstorms along the route of flight.
 During a YYZ-DAY flight the crew failed to properly check the NOTAMs. The Dayton
International Airport was closed at the flight’s scheduled arrival time due to an air show. The
crew only discovered this on the descent into Dayton when ATC pointed it out. The flight had to
hold for more than a half an hour. The crew clearly failed to do the proper flight planning prior to
departure. It should also be pointed out that the Flight Following office also missed this
NOTAM. At the time of the LOSA Air Georgian used a Type C pilot self-dispatch system thus
making the Captain fully responsible for all flight planning. Having said that the Company
should have been aware of the temporary airport closure.
 On a BDL-YYZ flight the crew activated the wrong flight plan in the GPS (BDL-YUL instead of
BDL-YYZ). They did not properly check the loaded flight plan as per the ‘Before Take-off’
checklist and only discovered the error when ATC intervened once en route.
 Prior to starting the approach during a YHZ-YFC flight the crew briefed a Pilot Monitored
Approach (PMA). As per the SOPs the First Officer is to fly the aircraft during the approach and,
if the runway is seen prior to minimums, the Captain takes over control to land the aircraft. On
the flight in question the First Officer flew the aircraft all the way to landing. The failure to
properly follow the PMA procedure increased the risk of confusion over which pilot had control
of the aircraft at a critical phase of flight at approach minimums.
 On a YHZ-YFC flight the aircraft assigned to the flight had a deferred pressurization system that
was properly logged. The crew was aware of the deferral yet they still opted to fly at a cruising
altitude of 12000 feet. This is a relatively short flight (forty-five minutes) and the increased fuel
consumption at an altitude below 10000 feet was not a concern (nor were there any other factors
that made 12000 feet more desirable than an altitude below 10000 feet). In addition, the crew did
not start a timer once they climbed above 10000 feet to comply with the CAR 605.32 allowing
no more than thirty minutes unpressurized at an altitude above 10000 feet (to a maximum of
13000 feet). No briefing was made regarding the possible adverse consequences of flying above
10000 feet in an unpressurized aircraft. The decision to operate the flight above 10000 feet
needlessly increased the crew’s susceptibility to hypoxia and may have violated the CARs.
 On a MHT-YYZ flight the aircraft pressurization system malfunctioned and the aircraft did not
pressurize during the climb. The crew caught the fault well before they climbed above 10000 feet
(and before the ‘Cabin Alt High’ annunciator illuminated) and they briefed a lower altitude for
the flight back to Toronto once they had determined that the aircraft had sufficient fuel on-board
for the higher burn rate. Unfortunately the crew did not enter the snag into the journey logbook
and, upon arrival in Toronto, the aircraft was left on the gate with an undocumented open snag
(this is a violation of 3.15 of the Company Operations Manual as well as a number of CARs).

20


 A flight departed from YZR for YYZ and the crew opted to take off with a tailwind (Sarnia is an
uncontrolled airport). The crew did not refer to the runway analysis chart for the tailwind
departure as per SOPs. This despite the fact that the runway is relatively short with a TORA of
5100 feet. There is a significant weight penalty per knot of tailwind for this runway.
 During the cruise climb portion of a YYZ-MHT flight the observer noted that both crew
members were reading magazines. The aircraft was equipped with an autopilot and, during the
climb, the vertical speed decreased to 300 FPM without either crewmember noticing. Although a
300 FPM vertical speed is not unsafe it is operationally undesirable and should have been
observed and corrected by the crew. Also, had it been the speed that had been decreasing (if, for
example, the crew had selected the incorrect climb mode) without being noticed by the crew, the
situation would have quickly become dangerous. In addition, it should go without saying (and it
is a violation of Company policy) that it is unacceptable for both crewmembers to be engaged in
non-operational activities while operating a flight.

A number of serious issues were noted that needlessly increased risk and which took place
repeatedly during the series of LOSA observation flight such as:

 Excessive taxi speeds. On a surprising number of flights the observers noted high taxi speeds
(above the twenty knot maximum allowed by the BE02 SOPs). Air Georgian has very clear taxi
SOPs and the hazards of taxiing too fast are pointed out in Company literature. All Air Georgian
flight crews are made aware that rushing (in any form) is never acceptable and yet high taxi
speeds seem to be relatively common.
 From the data returned on the LOSA forms a large number of crews ignored the sterile cockpit
SOP; casual conversation during critical phases of flight seems to be endemic. Since the findings
from the February 2009 Colgan Air crash in Buffalo began to emerge Air Georgian has regularly
publicized the hazards of ignoring the sterile cockpit SOP to flight crews. Apparently these
efforts have yet to yield concrete results. Flight crews often discussed non-operational topics or
engaged in non-operational activities at inappropriate times and the result during the observed
flights included repeatedly missing radio calls from ATC, forgetting checklists and minor losses
of situational awareness. As the Colgan Air crash demonstrated, ignoring the sterile cockpit SOP
can lead to far worse consequences as well.
 There were a number of cases of crewmembers doing their checklists from memory. At Air
Georgian we make it very clear that checklists must be properly read by the Pilot Monitoring;
doing a checklist from memory is unacceptable as it regularly leads to errors.
 On a small number of flights the attitude of the Captain was deplorably lax. There are clearly a
small minority of pilots who enjoy flaunting rules and procedures and who make a show of doing
so. On flights operated by these Captains the level of risk is raised unnecessarily from start-up to
shut-down.

21

The LOSA data has uncovered less significant, but still notable, crew related errors. All of these
findings were noted on numerous flights and point to items that flight crews must pay more
attention to and that the Company must make more of an effort to stress during training.
 Flight crews often did not check their runway analysis charts when required to (Air Georgian
uses APG charts). During training and line indoctrination Air Georgian pilots are instructed to
refer to the APG charts for each take-off and landing. While it is certainly possible that crews
had indeed checked the APG charts before the LOSA in-flight observer joined them it is
extremely unlikely that this occurred in all, or even the majority, of cases where the observer
noted that the charts had not been checked.
 In a significant minority of the observed flights the briefings provided between crew members
were lacking. Important information was often left out of the briefings and, in a few cases,
required briefings were not made at all.
 The KLN-90B GPS is the primary navigation system on all Air Georgian BE02s. The BE02
SOPs require that the GPS be backed up by traditional ground based navigation aids though this
requirement was ignored on the vast majority of observed flights.
 The BE02 SOPs require that crews brief one emergency or abnormal situation as a refresher
prior to, or during, the first flight of each pairing. This SOP was regularly ignored during the
observed flights.
 The TCAS mode is often not set properly during flight (note that this also involves a check list
flaw).
 Crews often started and shut down aircraft with the EFIS power and/or the Avionics Master on
despite repeated direction from management not to do so as this procedure may cause damage to
the avionics.
 The PF is required to make a 1000 foot call prior to level off in climbs and descents while the
PM is required to make a 100 foot call prior to level off in climbs and descents; these calls are
often missed.
 A number of floated landings were observed due to the PF landing with power on instead of idle
power (as per the SOPs).
 The PM was often observed performing non-essential duties (such as filling out the journey
logbook, making Company radio calls, or other non-essential paperwork) at inappropriate times
when his/her attention should have been focused on the operation of the aircraft (i.e. during the
taxi and while in busy terminal airspace). While these are more examples of violations of the
sterile cockpit SOP, these tasks are at least operational in nature.
 There were many cases where the PF executed duties that are assigned to the PM by the SOPs.
This can cause confusion between crew members and distracts the PF from actually flying or
taxiing the aircraft. There is almost never any need for the PF to execute PM functions during a
flight. This took place most often when the Captain was the PF.
 Crews often failed to play the automated turbulence briefing prior to entering turbulent air.
Passengers were thus not given any warning upon encountering turbulence.

22


 The one passenger incident that occurred during the observed flights took place in Rochester.
The flight had arrived at the gate and a ‘senior Air Canada employee’ insisted on being allowed
off of the aircraft immediately. The BE02 SOPs require that the air stair door not be opened until
the ground crew is prepared. However, the crew seems to have been intimidated into opening the
door and allowing the passenger off of the aircraft at an inappropriate time. The proper procedure
would have been to instruct the passenger to remain seated until the door was opened at the
appropriate time. In addition, the crew should have contacted management as soon as practicable
to provide information about the totally unacceptable behavior exhibited by the non-revenue
passenger.

ERRORS
60
50
40
30
20
10
ERRORS
0

The above chart shows the main error types and the percentage of flights where these were
encountered (some flights had multiple errors).

It should be stressed that the vast majority of errors noted were very minor. Perfection is
impossible regardless of who is in the cockpit; the point of the above information is to make
pilots aware of the types of errors that occur so that individuals and the Company as a whole can
work to reduce the frequency of mistakes.

23


3.3 Undesired Aircraft States
Almost all of the undesired aircraft states that were noted by the in-flight observers during the
LOSA were related to crew errors, as one would expect. The undesired aircraft states that were
encountered were all relatively minor and most were small deviations that were corrected almost
immediately. However, the data returned is quite valuable as it will help the Company to tailor
training so that times and locations where increased vigilance is required can be pointed out to
pilots.

The UAS that gives the most concern involves unstable approaches. The Air Georgian BE02
SOPs define a stabilized approach as follows:

“The aircraft shall be stabilized on the approach by 1000 feet AGL in IMC and by 500 feet AGL
in VMC.
The criteria for a stabilized approach are as follows:
(i) In the landing configuration with the Before Landing Checklist complete
(ii) Indicated airspeed within plus 10 knots to minus 5 knots of target airspeed. Approach
speed should be maintained until the missed approach point, then reduced to Vref following the
decision to land.
(iii) When conducting a single engine non-precision approach, flap 35 should be selected
following the decision to leave MDA for landing.
If the aircraft is not stabilized on the approach by 1000 feet AGL in IMC, or by 500 feet AGL in
VMC, or if it becomes destabilized below these altitudes, the crew shall immediately execute a
go-around.”5
There were three instances of unstable approaches observed and, though relatively minor
occurrences, the Company must do a better job at emphasizing the danger of not having the
aircraft properly stabilized for landing below the altitudes set out in the SOPs. Industry-wide the
majority of landing accidents have followed unstable approaches and an occurrence rate of
almost 3% is not acceptable.

5
Air Georgian Limited, Beech 1900D Standard Operating Procedures. (Toronto: Air Georgian Limited, 2009), 2-68.

24


UNDESIRED AIRCRAFT STATES
7
6
5
4
3
2
1 UNDESIRED AIRCRAFT STATES
0

The above chart shows the Undesired Aircraft States that were encountered along with the
percentage of flights that these were observed on.

3.4 Crew Performance Marker Worksheet
In the Crew Performance Marker Worksheet the observer was able to sum up the flight as a
whole. The flight is divided into phases and planning, execution, review and overall performance
markers are assessed by assigning a score of one to four for each item (1 = Poor, 2 = Marginal, 3
= Good, 4 = Outstanding). The reasoning for each assigned score can be found in the narrative
section of the form. Far more 4s were assigned by the observers than 1s and the narratives seem
to justify this for the most part though there are instances where, based on the narrative, a mark
of 1 would likely have been more appropriate. Still, the low number of crews assigned a ‘poor’
rating is positive and is indicative of the quality of work performed by most Air Georgian pilots.
Still, the Company must work to reduce the number of ‘poor’ ratings, especially since the
category with the highest number is ‘Leadership’ (in the Overall Performance Markers section)
with a total of five (4.5% of observed flights). A ‘poor’ leadership rating was generally assigned
where Captains utterly failed to establish a professional atmosphere in the cockpit.

25


6

5

4

3
Pre‐Dep/Taxi
2
Takeoff/Climb
1 Descent/App/Land
Overall
0

This chart shows the number of 1s (Poor) assigned on the Crew Performance Marker Worksheet
(note that unlike the previous charts these are total numbers rather than percentages).

3.5 The LOSA Safety Survey
The LOSA Safety Survey was intended to compliment the audit flights by allowing Air Georgian
pilots to verbalize their impressions of the Company safety culture. The ten questions included in
the survey asked pilots about their flying backgrounds, their views on the major safety hazards
faced in our operation, as well as their impressions of how the Company approaches safety and
the priority placed on it. The survey was not meant to be scientific; rather, the intention was to
gain a feel for the general impression that the pilot group has regarding the approach that the
Company takes toward safety. Had individuals expressed serious safety concerns or had there
been a general frustration among pilots at the way the Company prioritizes safety there would
certainly have been grounds for worry. Fortunately, in general, the survey respondents were
satisfied with the Company’s approach toward safety and a number were quite complimentary.
Of the twenty-five surveys returned sixteen pilots had no significant safety concerns to report,
seven had concerns that had more to do with union issues than safety. Only two of the
respondents raised what could be considered to be significant safety concerns, one involving a
maintenance issue and one involving an internal communication failure that resulted in an
aircraft going flying with an expired deferred maintenance item. Also, only two of the twenty-
five respondents felt that there would be negative consequences from management if they ever
refused to operate a flight because of a safety concern.

26

The main issue raised by the respondents had to do with fatigue; many feel that Air Georgian’s
operations are fatiguing and this is obviously a safety concern. While it is possible that a pilot
may become fatigued while at work the Company has addressed this with a comprehensive
fatigue policy. This policy allows pilots to call in fatigued (for any reason) if necessary or, if
already at work, to remove themselves from duty if required with no disciplinary consequences.
The Company also has a standing offer of a paid hotel room for any pilot (or other employee)
who feels too tired to drive home after a duty period. In addition, Air Georgian pilots are
guaranteed eleven days off per month and are not allowed to be scheduled with less than twelve
hours off between pairings (that is, from the time they leave the airport until they have to arrive
at work the next day) and eleven hours off while on a layovers (this is a bare minimum; most
layovers are considerably longer). Given the above, and considering the available fatigue
research, Company pilots are clearly provided with enough time free from duty to avoid
becoming fatigued due to the schedule. It should also be mentioned that, because of the union
scheduling rules, the distribution of work is heavily slanted in favour of more senior pilots. If it
is an issue for pilots that some of them work much harder than others at Air Georgian the union
scheduling rules that give some pilots twenty-five hours of flying per month while others do 100
hours would have to be changed.

27


4. Recommendations

Now that some of the threats, errors and undesired aircraft states that Air Georgian flight crews
encounter on a regular basis have been identified and quantified it is up to the Company as a
whole to use this information to make improvements that will decrease risk. The following
recommendations should, if followed, allow the Company to mitigate threats and substantially
reduce errors and undesired aircraft states.

1. Air Georgian must devise a better method of informing flight crews of deferred maintenance
items on their assigned aircraft. By informing flight crews in advance of DMIs there will be
much less chance of errors (involving the maintenance status of the aircraft) occurring, the most
important one being the operation of an aircraft in conditions that are prohibited by the deferral
procedure.
2. As much as is operationally possible the Company should not pair two Captains together to
operate flights. As this is not always possible the Company should make Captains aware of the
higher number of errors that occur when they fly with each other rather than with a First Officer
as well as the reasons for this.
3. The Company should make more of an effort in making pilots aware of the dangers involved in
complacency. This can be done via the monthly newsletter, in ground school, during line checks
and by managers when they fly with line pilots.
4. The leadership of the Flight Operations branch of the Company must make a major, sustained
effort to encourage and increase adherence to SOPs. While SOP compliance is not bad within the
Company the LOSA has made it clear that it can and should be much better. The following
measures should be adopted:
i. The Training Department should develop and use more scenario based simulator training that
incorporates ‘normal’ SOP use rather than simply throwing one emergency after another at the
candidates;
ii. Management pilots should fly as often as possible and lead by example by strictly following the
SOPs;
iii. The very real dangers of non-compliance with SOPs must regularly be advertised to Company
pilots via the monthly newsletter, during all training (ground school and simulator), and in
person by Managers and Line Indoctrination Captains;
iv. The small number of pilots who do not feel that SOPs apply to them should be identified when
possible. These pilots should be counseled and, in extreme cases where adequate improvement is
not shown, should have their employment terminated if they are unable or unwilling to comply
with SOPs;
v. It should be made clear to all Company pilots that anything less than full compliance with the
SOPs is not acceptable and is unprofessional; it is not something that should be tolerated within
the Company.

28

5. Unstable approaches (as defined by the SOPs) must be eliminated completely from Company
operations. It must be emphasized to all Company pilots during training, line indoctrination and
line checks that unstable approaches cannot be continued; the only acceptable action is to
discontinue an unstable approach.
6. Too many Company pilots operate flights without referencing the runway performance data.
Crews become complacent because almost all of the runways that Air Georgian operates from
are more than adequate for the BE02 at maximum weight under all but the most extreme
temperature conditions. However, not all of the runways are usable at maximum takeoff weight
without a special departure procedure in the event of an engine failure; nor are all runways
adequate under tailwind or icing conditions. It is crucial that the runway performance data charts
are referenced prior to flight. However, the Company can facilitate this process by highlighting
the performance limiting runways (and conditions) used by the scheduled service for pilots. This
would be a simple matter of creating an information sheet that could be placed into the route
manual.
7. The Company must place more emphasis on the proper use of aircraft automation during
training, line indoctrination and line checks (automation includes the passenger briefing system).
8. It is clear that the Company has to enhance the training that we provide regarding the proper use
of the aircraft Minimum Equipment List and the entering of snags in the aircraft Journey Log
Book.
9. High taxi speeds (outside of SOP limits) were a recurring issue during the observed flights (11%
of the flights). Risk is substantially increased by taxiing too fast and the dangers of doing so must
be advertised to flight crew more emphatically. Company management must also be on the
lookout for fast taxiing and the Captains responsible should be counseled.
10. The entire pilot training program for both initial and recurrent candidates requires an extensive
review. The training program was first developed over ten years ago and components have been
added in a haphazard fashion over time. The training program does not take advantage of many
of the latest technologies available and some of these should be incorporated into a revamped
program. The type of threats, errors and undesired aircraft states that have been noted in this
report should be emphasized in the training program, during line indoctrination and on line
checks. The Chief Instructor, working closely with the Chief Pilot, should conduct this review.
11. Data from check rides, line indoctrination, line checks and training should be recorded and
scrutinized so that deficiencies can be identified and corrected. These data can be used to set
Company benchmarks so that progress can be measured over extended periods.
12. While there is always more that can be done by any organization to enhance safety much of it
will be pointless if the line employees who are actually doing the work do not ‘buy in’. For
example, the Company can go on about SOP compliance on a daily basis but if pilots are not
willing to make the effort to do the job properly very little will change. It has to be made clear to
pilots that professionalism is essential at all times in this line of work. Safety has to involve all of
the stakeholders making an effort to continually improve. Pilots must be clear that their own

29

safety depends on always maintaining a high level of professionalism; this includes SOP
compliance, regular review of emergency procedures and good aircraft systems knowledge.
13. The Company should introduce formal Threat and Error Management training into the pilot
initial and recurrent curricula.
14. Management must review the BE02 checklists and correct any deficiencies (such as the lack of a
reminder to change the TCAS mode at the top of the climb and the top of descent).
15. The Flight Following office should become more proactive in the way that flights are handled.
While aircraft dispatched under a Type C system do not require licensed dispatchers there is no
reason why Company Flight Followers cannot play a more involved role in the way that flights
are followed. This includes maintaining a high level of awareness throughout each shift of the
maintenance status of aircraft, the weather conditions system-wide, NOTAMs that may affect
operations, problematic airport conditions and any other essential operational information.
Attaining this level of awareness among Flight Followers will require close supervision from
management (at least initially) and a review of the training provided.

30


Conclusion

The LOSA was a large undertaking for a regional operator the size of Air Georgian. A
substantial amount of resources were devoted to the project. However, the process was well
worth the monetary and time commitment involved and a follow-up LOSA will be conducted in
the next year and a half to two years.

To a large degree the results of the LOSA are quite reassuring; the data returned shows that our
operating environment and operation are safe with low levels of risk. Our flight crews are
generally dedicated professionals who do not take undue risks and who place safety ahead of all
other considerations. However, as noted above, the data also shows that there is work to be done
in many areas. The Company must make more of an effort to enhance our pilot training by
putting to better use the knowledge gained over the fifteen years that Air Georgian has operated
the BE02. We should also make more use of some of the cutting edge training technologies that
are available today. Company communication to the pilot group should also be improved. Pilots
must also recognize that maintaining a high level of professionalism is every bit as much their
responsibility as it is the Company’s and they must strive every day to operate at the highest
level possible. In short, continuous improvement has to be the goal for both management and
pilots so that risk levels can be lowered to the bare minimum. Some of the data and
recommendations that have come out of the LOSA will hardly be surprising to experienced
aviation professionals; the value is derived from the fact that we now have verified, quantified
information rather than assumptions. The LOSA has provided a very clear picture of the
operation at the time that the data was gathered.

The findings provided by the LOSA will be used as benchmarks that the results of subsequent
LOSAs will be measured against. When the next LOSA is conducted we will be able to
determine the effectiveness of the recommendations included in this report (and/or the
effectiveness of our execution). None of the issues raised in this report are insurmountable; on
the contrary, given the necessary effort and some time, every concern that has been brought to
light by the LOSA is correctable to a large degree. No operation is or can be perfect but a solid
safety culture and a commitment to continuous improvement will result in a sustained reduction
in the level of risk inherent in flight operations.

31


APPENDIX A

Below is a copy of the LOSA form; one form was filled out for each observed flight.

Observer Information

Observer ID (Employee number) Observation Number

Crew Observation Number Of
(e.g., “1 of 2” indicates segment one for a crew that you observed across two segments

Flight Demographics

City Pairs (e.g., YYZ-ALB)
A/C Type BE02
Pilot Flying (Check one) CA FO

Time from Pushback to Gate Arrival Local Arrival Time
(Hours:Minutes) (Use 24 hour time)
Late Departure?
(Yes or No)

Predeparture/Taxi

Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did
Narrative the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral
ratings.

32


Takeoff/Climb

Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did
Narrative the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral
ratings.

33

Cruise

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly?

Descent/Approach/Land/Taxi

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly?

34


Overall Flight

Narrative This narrative should include your overall impressions of the crew.

Threat Management Worksheet

Threat Description Threat Management
Phase of
Threat ID

Flight Linked to
Threat flight crew How did the crew manage or
Describe the threat 1 Predepart/Taxi error?
Type 2 Takeoff/Climb mismanage the threat?
3 Cruise
4 Des/App/Land (Yes/No)
5 Taxi-in

T1

T2

35


T3

T4

T5

T6

Threat Codes
Environmental Threats Airline Threats
200 Airline Operational Pressure 204 Ground/Ramp
100 Adverse Weather 103 Airport Conditions
201 Cabin 205 Dispatch/Paperwork
101 ATC 104 Heavy Traffic (air or ground)
202 A/C Malfunctions/MEL Items 206 Manuals/Charts
102 Terrain 199 Other Environmental Threats
203 Ground Maintenance 299 Other Airline Threats

Error Management Worksheet

Error
Error Description Error Response/Outcome
Management
Phase of Linked
Error ID

Flight Error
to Crew Error How did the crew
Outcome
Describe the crew Threat? Error Response manage or
1 Predepart/Taxi
error 2 Takeoff/Climb Type 1 Detected 1 Inconsequential mismanage the
(If Yes,
3 Cruise
enter the 2 No response 2 Undesired state error?
4 Des/App/Land 3 Additional error
5 Taxi-in Threat ID)

E1

36


E2

E3

E4

E5

E6

Error Type Codes
Aircraft Handling Procedural Communication
300 Manual Flying 400 SOP Cross-Verification 500 Crew to External Communication
301 Flight Control 401 Checklist 501 Crew to Crew Communication
302 Automation 402 Callout 599 Other Communication
303 Ground Handling 403 Briefing
304 Systems/Instruments/Radios 404 Documentation
399 Other Aircraft Handling 499 Other Procedural

Undesired Aircraft State (UAS) Management Worksheet

UAS Description UAS Response/Outcome UAS Management
UAS ID

Linking
Crew UAS How did the crew manage
Error? Undesired aircraft UAS UAS Outcome
Response or mismanage the
state description Code
(Enter the 1 Inconsequential undesired aircraft state?
Error ID) 1 Detected 2 Additional Error
2 No response

UAS
1

37


UAS
2

UAS
3

Undesired Aircraft State Type Codes
Aircraft Handling States-
Configuration States Ground States Approach/Landing States
All Phases

1 Incorrect A/C configuration--- 20 Proceeding toward wrong 40 Vertical deviation 80 Crew induced deviation
flight controls, brakes, thrust runway above G/S or FMS path
reversers, landing gear 41 Lateral deviation
21 Runway incursion 81 Crew induced deviation below
42 Unnecessary WX penetration G/S or FMS path
2 Incorrect A/C configuration---
systems (fuel, electrical, 22 Proceeding toward wrong
taxiway/ramp 43 Unauthorized airspace 82 Unstable approach
hydraulics, pneumatics, air-
penetration
conditioning, pressurization,
23 Taxiway/ramp incursion 83 Continued landing---unstable
instrumentation
44 Speed too high approach
3 Incorrect A/C configuration--- 24 Wrong gate
45 Speed too low 84 Firm landing
automation
25 Wrong hold spot
46 Abrupt aircraft control--- 85 Floated landing
4 Incorrect A/C configuration---
26 Abrupt aircraft control—taxi (attitude)
engines
86 Landing off C/L
47 Excessive banking
87 Long landing outside TDZ
48 Operation outside aircraft
limitations 88 Landing short of TDZ

99 Other Undesired States

Crew Performance Marker Worksheet

1 2 3 4
Poor Marginal Good Outstanding

Observed performance had Observed performance was Observed performance was Observed performance was truly
safety implications adequate but needs improvement effective noteworthy

38

Phase of Flight Ratings
Predeparture/ Takeoff/ Descent/Approach/
Planning Performance Markers
Taxi Climb Land/Taxi
The required briefing was
SOP BRIEFING interactive and operationally
thorough.
Operational plans and
decisions were
PLANS STATED communicated and
acknowledged.
Crew members developed
CONTINGENCY effective strategies to
MANAGEMENT manage threats to safety.

Execution Performance Markers
Crew members actively
monitored and cross-
MONITOR/CROSS-CHECK checked systems and other
crew members.
Operational tasks were
prioritized and properly
WORKLOAD MANAGEMENT managed to handle primary
flight duties.
Crew members remained
VIGILANCE alert to the environment and
position of the aircraft.
Automation was properly
managed to balance
AUTOMATION MANAGEMENT situational and/or workload
requirements.
Crew members used
TAXIWAY/RUNWAY caution and kept watch
MANAGEMENT outside when navigating
taxiways and runways.

Review/Modify Performance Markers
Existing plans were
EVALUATION OF PLANS reviewed and modified when
necessary.
Crew members not afraid to
ask questions to investigate
INQUIRY and/or clarify current plans
of action.

Overall Performance Markers Ratings
Environment for open
COMMUNICATION communication was
ENVIRONMENT established and maintained.
Captain showed leadership
LEADERSHIP and coordinated flight deck
activities.

39


APPENDIX B

This memo was issued to managers only in order to provide a brief description of the LOSA.

Line Operations Safety Audit

We will be carrying out a Line Operations Safety Audit (LOSA) at Air Georgian over the
summer of 2010. LOSA is recognized worldwide as an effective diagnostic tool that enables air
operators to credibly measure their safety performance. By conducting a LOSA those responsible
for operational safety will be provided with enough data to identify and thus be able to correct
weaknesses that would otherwise be difficult to pinpoint.

As this will be the first LOSA carried out at Air Georgian the scope will be quite wide; all
aspects of the flight operation will be audited. Subsequent LOSAs can be narrowed in scope to
concentrate on issues identified by the 2010 audit. The time frame for the next LOSA would be
two to three years after the report for the 2010 audit is issued.

The LOSA will be implemented by the LOSA steering committee which will consist of the
Operations Manager, the System Chief Pilot, the Chief Instructor, the Corporate Safety Officer
and a representative from the employee association. In general terms the audit will be executed
in six stages as follows:

1. Planning – After the training course is completed on June 8th the steering committee will
finalize the audit process and procedures in one or more meetings.
2. Audit flights – The actual audit flights will be carried out by six auditors. It is envisaged
that each auditor will audit two turns per day over a five day period. The audit flights will
be spread out through the month of July so that only two auditors are active at any one
time. After their first day of audit flights the auditors will meet with at least two members
of the steering committee to go over the process and clarify any points as necessary. The
auditors will be given specific audit days on their schedules but will pick the flights to be
audited themselves so that management is not in any way involved in selecting the crews
of the audit flights.
3. Compile data – Once the entire series of audit flights have been completed the data must
be compiled into a database. This will be done by the auditors themselves and each will
be given an office day in early August or late July for this task. The data will be totally
de-personalized so that there is no way to identify which crew members operated any of
the flights. The format of the database is to be announced.
4. Review data – In August the steering committee and the auditors will meet to review the
data captured during the audit and come up with a list of findings.

40

5. Issue report – Once the findings have been identified the Operations Manager, with input
from all involved, will write up the audit report. This document will be detailed and will
include not only the findings but the responses as well.
6. Implement fixes – All findings must be addressed and corrected or the audit will have
been a waste of time and resources. Correcting any findings will require the full buy-in of
managers at all levels.

Please bear in mind that the time-line will be as flexible as it needs to be. The LOSA is a priority
project but if our crewing situation becomes too tight it may well get pushed back into the fall.

41


APPENDIX C

Below is a copy of the LOSA Safety Survey.

Air Georgian Limited LOSA Safety
Survey
1. What are your total hours?
Under 2000 2000 – 4000 4000 – 6000 6000+
2. Before joining Air Georgian, what type of operation did you work for (i.e. air taxi, 704 commuter
operation, flight training unit etc…)?
3. What do you feel are the top issues that adversely affect safety in our operation (name up to 3
issues)?
4. Have you ever encountered a significant safety deficiency at Air Georgian? If yes, please
elaborate.
5. How seriously do you think that Air Georgian management takes safety on a day to day basis?
6. What could Air Georgian management do better when it comes to enhancing safety?
7. Have you ever felt that an ATC clearance or instruction that you have been issued has
unnecessarily increased risk or has been unsafe? If yes, please elaborate and state how you dealt
with the situation.
8. While working at Air Georgian, have you ever felt that your safety has been put at risk due to an
encounter with adverse weather?
9. How do you think you would be treated by Air Georgian management if you ever refused to carry
out a flight due to a safety concern?
10. Do you have any comments or concerns not addressed by the above questions?

42


APPENDIX D

These instructions should be issued to the in-flight observers in future LOSAs.

LOSA Auditor Checklist
LOSA Auditors should plan to complete one flight in the morning and one flight in the
afternoon. It is the Auditor's responsibility to plan sufficient time between flights for customs,
LOSA Crew briefings, and meals.

Equipment: Box type splitter- This splitter has a 9 volt battery inside the control box. (There is
no external indication of this.) Its function is to power the Auditor's microphone. This selection
should be turned off for our LOSA purposes as the Auditor is not to be talking with the flight
crew. If the battery is dead/nearly dead and the MIC is turned on, gross amounts of feedback
will be heard in the Auditor's headset, preventing effective observation. ALWAYS LEAVE
THE MICROPHONE TURNED OFF.

All splitters are to be plugged into the left seat jacks. The captain will then plug into the splitter
box.

The day before your LOSA flights:

-Call Crew Sched to plan your LOSA flights. It is imperative that you do so early as there may
be more than one LOSA auditor operating when you are. This means you may not have access
to the exact flights you want.

-Copy down the flight numbers for your planned flights and request that the Crew Scheduler list
you for these flights.

On arrival at the North End:

-Get your flight locator numbers from Crew Sched.

-Check in for your first flight on-line if possible.

-If it is your first LOSA flight, get the splitter from the office.

-Print a weather package/NOTAMS for your flights.

On arrival at the Departure Gate:

-Arrive at the gate 30 minutes prior to departure so that you can contact the crew.

-Supply the crew with the LOSA briefing and give the captain the headset splitter and advise
him/her of its operation.

43

-Advise the sales agent of the LOSA observation, and confirm that you are to be seated in seat
1A or 1B.

On arrival at the aircraft:

-Check the logbook for deferred items that might affect the flight/observation.

On departure from the airport after your LOSA observations:

-Confer with Crew Sched and plan out the following days LOSA flights.

-Copy down flight numbers for your planned flights, and request that the Crew Scheduler list you
for these flights.

Preparation of LOSA paperwork:

-Complete full LOSA write-ups prior to your data entry day. Make sure it is checked and
complete so that the only task required at the office is transferring it onto the company database.

44


APPENDIX E

LOSA Lessons Learned

This was our first attempt at conducting a LOSA at Air Georgian; for the most part we are very
pleased with how it went though we did learn a number of lessons for next time.

1. The Crew Performance Marker Worksheet section of the LOSA form should include an
area to mark overall SOP compliance during the flight;
2. Before the series of observation flights begin the LOSA Steering Committee should
ensure that the headset splitter boxes work with all of the Company aircraft involved in
the audit. Unfortunately we did have some complications as one of the headset splitters
was not compatible with one or two of the aircraft (the plug fit but it simply did not
work);
3. In the future the observers should not be scheduled to do seven days of in-flight
observation in a row. All of the observers complained that this schedule was too
demanding and that at least one day off should be inserted. This is understandable as the
observers had long days at the airport and then they had to type their notes in the
evenings;
4. Observer training needs to more heavily emphasize correct and standardized form
marking. The observers all made errors in the way that they categorized the various
threats, errors and undesired aircraft states. All of these were subsequently corrected but
this required quite a bit of additional work and could have been avoided with more
thorough training;
5. Crew Schedulers and Flight Followers should have been briefed about the LOSA in
person by management. Instead we simply issued them a memo and this was not
sufficient. There were quite a few misunderstandings regarding the job being done by the
observers. Better communication from management would have reduced or eliminated
the day of operations confusion that occurred on a number of occasions;
6. For the next LOSA a much higher number of first turns of the day should be observed.
These would include both early morning departures and afternoon departures depending
on the flight crew’s start time. The observer will thus be able to observe the crew as they
flight plan so that they can note how thoroughly it is done.

45

Losa Report

More Related Content

What's hot

Viewers also liked

Losa Report