A 360-DEGREE PEER EVALUATION OF LEADERSHIP COMPETENCIES
Anthony Rogers, MBA
Presented to the Faculty of the Department
of Center for Leadership Studies
Our Lady of the Lake University,
in Partial Fulfillment of the Requirements
For the degree of
Doctor of Philosophy
in Leadership Studies
Our Lady of the Lake University
San Antonio, Texas
July 31, 2001
This study examined a model of leadership in the fire service and evaluated the selection
process of two of the seven leadership roles in the San Antonio Fire Department—fire
apparatus operator (FAO) and lieutenant. The selection process is based primarily on the
results of a written examination that ranks firefighters on their raw test scores and number
of years of experience, which is referred to as seniority points. Although there are
numerous studies on leadership theories and behavior patterns, the amount of literature
available on the leadership selection process—especially in the fire service—is severely
limited. The purposes of this study were to: (1) determine whether the leadership
selection process in the San Antonio Fire Department is identifying firefighters with
leadership competencies as identified by the Booth Research Group (1998) and the
Rogers’ Firefighters Survey (2001) conducted on firefighters to determine leadership
competencies relevant to the fire service, and (2) suggest relevant criteria for selecting
firefighters for leadership roles in the fire service. A 360-degree peer evaluation that
contains 57 questions that are used as the criterion variable was used to rate the
leadership performance of FAOs and lieutenants. Data collected were correlated to
determine whether the leadership selection process is related to selecting firefighters with
leadership competencies. To determine the degree of relationship, four variables—a
criterion and three predictors that include scores on the written promotional examination,
years of service, years of formal education, and scores on the 360-degree peer
I wish first to thank God for the strength, courage, and wisdom with which He
equipped me to complete my dissertation and the lengthy journey toward a doctor’s
degree. I pray that the Lord will enable me to use the knowledge and wisdom I have
attained to better serve and honor Him. I also wish to thank my children, Catherine and
Amanda, who sacrificed their quality time with me so that I could complete my studies. I
want to thank my fellow firefighters who place their lives on the line 24 hours a day.
They have supported me and cheered me on during the journey to complete my Ph.D.
Furthermore, I extend my gratitude to my chair, Dr. Malcolm J. Ree, and the committee
members for their support and encouragement. I want to thank Sally for her continuous
support. Above all, I want to thank my devoted wife, Patsy Mae, without whose constant
encouragement and support I never could have completed the program. I am eternally
TABLE OF CONTENTS
ABSTRACT ……………..……………………………………………………….. ii
TABLE OF CONTENTS………………………………………………………….. iv
Research Questions..………………………………………………….…… 6
Problem Statement….……………………………………………………… 7
Purpose of This Study..……………………………………………………. 7
Theoretical Literature on Leadership ……………………………………... 12
Definitions of Leadership …………………………………………………. 12
Models and Theories of Leadership ………………………………………. 13
Leadership Selection Process in the Fire Service ………………………… 18
Collective Bargaining Contract Agreement ………………………………. 22
Promotional Examination Process ……………………………………….... 23
Leadership Prediction Studies………………………………………………29
Methods of Verification and Measurement of
Leadership In the Fire Service………………………………………….. 33
Research Design …………………………………………………………... 41
Criterion and Predictor Variables …………………………………………. 41
Null Hypotheses ……………………………………………………………42
The Participants ………………………………………………………….. 42
The Instrument …………………………………………………………... 43
Data Collection …………………………………………………………… 45
Data Analysis ……………………………………………………………... 46
Ethical Decisions ………………………………………………………….. 46
Descriptive Statistics of Participants ……………………………………… 47
Hypotheses ………………………………………………………………... 52
Conclusions ……………………………………………………………….. 61
Discussion ………………………………………………………………… 62
Recommendations ………………………………………………………… 67
REFERENCES ……………………………………………………………………. 69
Appendix A—FAO Practical Skills ……………………………………… 76
Appendix B—Lieutenant Performance Dimensions ……………………… 77
Appendix C—Rogers Firefighters Survey …………………………………78
Appendix D—Five Competencies Identified
by the Booth Research Group ………………………………………….. 79
Appendix E-Six Firefighters Leadership Competencies Identified
by the Rogers Survey ………………………………………….……….. 80
Appendix F—360-Degree Peer Evaluation ……………………………….. 81
Appendix G-Hunter and Schmidt Predictive
Validity Table ………………………………………………………….. 85
Appendix H-Letter Granting Permission
To Conduct Study ……………………………………………………… 86
Appendix I-Consent Form ………………………………………………... 87
In today’s dangerous world, 9-1-1 fire and medical emergencies occur every five
seconds. Fire kills thousands of people each year, injures hundreds of thousands of
others, destroys billions of dollars in property, and costs tens of billions of dollars
(USFA, 1999). Firefighters help protect the public against these dangers by quickly
responding to a variety of emergencies. Frequently they are the first emergency personnel
at the scene of an accident or medical emergency and may be called upon to put out a
fire, treat injuries, rescue trapped victims, or perform other vital functions (Bureau of
Labor Statistics, 2000).
The fire service is facing a growing spectrum of job responsibilities and the need
for effective leadership is even greater than in the past. If the need for effective leadership
in the fire service is growing, what might be done to select firefighters with effective
leadership competencies? What are the leadership competencies relevant to the fire
service? What selection process would accurately measure or validate the need for
leadership competencies? Part of the answer is to look at the selection and promotion
process and compare it with the leadership competencies needed in the fire service, as
identified by the Booth Research Group (1998) and the Rogers’ Firefighters Survey
Applicants for municipal fire-fighting jobs generally must pass a written exam;
tests of strength, physical stamina, coordination, and agility; and a medical examination.
Entry-level examinations generally are open to individuals who are at least 19 years of
age and have a high school education or the equivalent. Those who receive the highest
scores in all phases of testing have the best chance of receiving an appointment to the fire
department (Miller, 1997).
As a rule, entry-level cadets in large fire departments are trained for several weeks
at the department’s training center or academy. The recruits study fire-fighting
techniques, fire prevention, hazardous-materials control, local building codes, and
emergency medical procedures that include first aid and cardiopulmonary resuscitation
(CPR). They also learn how to use axes, chain saws, fire extinguishers, ladders, and other
fire-fighting equipment and rescue equipment. After successfully completing this course
of study, they are assigned to a fire company where they undergo a one-year probationary
period of on-the-job training (Bureau of Labor Statistics, 2000).
During this time the probationary firefighter is assigned to a fire station where he
or she will work directly under the supervision of the company officer and the other
senior firefighters assigned to the company. Throughout this period, the probationary
firefighter is exposed to all the elements of the fire service, thereby gaining valuable
experience. Upon completion of the probationary period, the probationary firefighter is
upgraded to the classification of firefighter (IAFF Local 624, 1999).
According to the International City Management Association (1979), leadership
in the fire service is a critical role because life-saving decisions often are dependent on
the leader’s ability to act decisively and make sound decisions on the fire ground as well
as in administrative situations. Traditionally, firefighters were selected for leadership
roles based on job seniority as well as their ability to fight fire by frequently withstanding
a smoky inferno without the protection of a breathing apparatus; hence, the term smoke
eater was coined. Currently, opportunities for promotion to leadership in the fire service
are based on seniority and the results of a multiple-choice, written examination. The steps
to leadership begin with promotion to fire apparatus operator (FAO), and move up the
hierarchy to lieutenant, captain, battalion chief, assistant chief, deputy chief, and finally,
Fire and police municipal civil service laws regulate the classification and
appointment of firefighters to leadership roles. Chapter 143 of the Texas Local
Government Codes provides specific guidelines regarding the entrance examination as
well as the written promotional examination process for municipal fire service (Miller,
1997). In addition, collective bargaining agreements between firefighters unions and
municipal governments also regulate the leadership promotion process in the fire service.
Collective bargaining agreements often describe the eligibility criteria, seniority points,
study material, written examination, assessment-center process, assessment-center
promotion dispute resolution procedures, probationary period, appointments to assistant
chief and deputy chief, and the mechanics of posting promotional examination results
(IAFF Local 624, 1999).
According to the Texas Local Government Codes (Miller, 1987), the leadership
selection process in the fire service in Texas begins with a vacancy. When a vacancy
occurs, a written multiple-choice promotion examination is administered. On November
1 of each year, the San Antonio Fire Department’s fire chief establishes a committee(s) to
select study material for examinees to use in preparing for the written promotion
examination. The fire chief makes the final selection of the study material, subject to
approval by the Civil Service Commission (IAFF Local 624, 1999). Exam questions are
formulated from the study material, which is based on a job analysis designed to assess
the knowledge, skills, abilities, and other characteristics (KSAOs) needed for effective
performance in the San Antonio Fire Department (Booth Research Group, 1998). The
names of examinees whose scores meet the minimum score required for promotion
consideration are placed on an eligibility list.
The Booth Research Group (2000) is a consulting firm that develops written
promotional examinations for hundreds of positions in various organizations, including
the fire service. Their written examinations are customized to reflect the organizational
needs of their clients. This firm also develops the protocols and procedures for the
The Booth Research Group was hired by the City of San Antonio to develop a
valid, multiple-choice written promotional examination for the positions of FAO and
lieutenant. Additionally, a questionnaire designed to assess the knowledge, skills,
abilities, and other characteristics needed for performance as an FAO and a lieutenant in
the San Antonio Fire Department was developed to analyze the positions. From analysis
of these positions, practical skills and competencies were identified (Booth Research
For the position of FAO, 13 practical skills were identified (see Appendix A).
These skills include; (1) fire inspection and compliance, (2) water pump operation, (3)
apparatus driving, (4) medical responsiveness, (5) fire-fighting, (6) training, (7)
hazardous materials identification, (8) communication and interaction, (9) public
education and community service, (10) search and rescue abilities, (11) performance of
station duties, (12) ability to perform inspections, and (13) self-development. For the
lieutenant position, five competencies and 18 different performance dimensions were
identified (see Appendix B). The five competencies include (1) supervision, (2) general
administration, (3) personal development, (4) emergency scene management, and (5)
community interaction. The 18 performance dimensions include; (1) conflict resolution,
(2) creativity, (3) customer service, (4) decision- making, (5) effectiveness under stress,
(6) independent thinking, (7) interpersonal skills, (8) job knowledge, (9) leadership, (10)
oral communication, (11) organization and planning, (12) personnel management, (13)
persuasiveness, (14) problem- solving, (15) sensitivity toward diverse populations, (16)
supervisory skills, (17) teamwork, and (18) written skills (Booth, 1998).
A pilot study of the Rogers’ Firefighters Survey (2001) was conducted by the
researcher and writer of this dissertation (see Appendix C). The purpose was to gather
data regarding effective leadership competencies relevant to the fire service, as identified
by 35 firefighters who work in the firefighting division. The survey identified six
competencies that are considered relevant to the fire service (see Appendix D). The six
competencies include; (1) technical skill, (2) self-confidence, (3) communication, (4)
ethical behavior, (5) interpersonal skills, (6) and emotional balance.
The performance dimensions from which the competencies were identified are
based on the role and responsibilities of the position. To gain an understanding of the
requirements of the position, information was collected on several levels, including
conducting site visits and interviews; examining documentation on the position; and
distributing, collecting, and analyzing a questionnaire (Booth Research Group, 1998).
A built-in hazard of service as a firefighter is regular exposure to the dangers of
physical injury or death. Risk-taking is a common competency that firefighters often are
exposed to operationally as well as administratively. An example of administrative risk-
taking would be management’s critical decision on whether to close a fire station,
possibly leaving the area that the station serves farther away from rescue and fire-fighting
services. However, bravery and risk-taking, were not identified by the Booth Research
Group (1998) or the Rogers’ Firefighters Survey (2001) as relevant competencies. In
general, firefighters tend to suppress the fact that their occupation is dangerous; this
probably was the reason that it was not identified as one of the relevant competencies. It
is reasonable to hypothesize that risk-taking behaviors will be similar to other previously
identified predictor variables, and that leadership effectiveness in dangerous
environments may then be conceptually and empirically linked to organizational theory
(Anderson, Fiedler, & Frost, 1983).
In the past, a good fire department was measured by its ability to extinguish a fire
in a reasonable amount of time and provide the community with the most appropriate
level of fire protection, based on community resources. Today the fire service is faced
with growing job responsibilities, and the efficient operation of a municipal fire
department mandates that fire- service leaders be competent in administrative,
operational, and leadership skills (Hoover, 1993).
The following research questions were answered:
1. Does the leadership selection process in the San Antonio Fire Department
identify firefighters with leadership competencies relevant to the fire service?
2. What are the relevant predictors used as criteria for selecting firefighters for
leadership in the fire service?
There are extensive literature studies on measuring leadership skills and
identifying leadership behavior patterns (Bass, 1990). Currently, however, there are no
studies that evaluate the leadership selection process in the fire service.
Rationale For The Hypotheses.
This study is intended to increase our knowledge of leadership competencies from
the perspective of subordinates, peers, and supervisors in the fire service. In particular,
the researcher and writer of this dissertation combined the leadership competencies
identified by the Booth Research Group (1998) with those identified by the Rogers’
Firefighters Survey (2001) to create a 360-degree peer evaluation (see Appendix E). The
results of the 360-degree peer evaluation will be used as a criterion variable to determine
whether written promotion examination scores, years of service, and years of formal
education correlate with effective leadership in the fire service.
In the past, fire departments have been spared the scrutiny of accountability. City
government, as well as the citizens, are now holding the fire service accountable for
effective and efficient operations of the department (Hoover, 1993). They expect a fire
service leader to be proficient in fire-ground tactics, computer technology, and various
leadership skills that include transformational and transactional leadership. The citizens
expect fire service leaders to be innovative and aggressive in handling the fire service
problems that plague their community (Hoover, 1993).
Purpose Of This Study
The major intents of this study were to:
1. Determine whether the present leadership selection process was identifying
firefighters who have leadership competencies as identified by the Booth
Research Group (1998) and the Rogers’ Firefighters Survey (2001), which was
conducted on 35 San Antonio firefighters regarding leadership competencies
relevant to the fire service.
2. Suggest relevant predictors for selecting firefighters for leadership roles in the
The statistical null hypotheses are stated in chapter 3. The research hypotheses
Hypothesis 1: A score on the written promotional examination is predictive of
effective leadership competencies, as rated by the 360-degree peer evaluation. In
accordance with Chapter 143 of the Texas Local Government Codes and collective
bargaining agreements, the firefighters who earn the highest raw score on the
promotional written examination and have adequate seniority points are promoted to
vacant leadership roles in the fire service. Currently there are no data to support a
conclusion that test scores are predictive of effective leadership.
Hypothesis 2: Years of service is predictive of effective leadership competencies,
as rated by the 360-degree peer evaluation. The two variables used for selection into
leadership roles are the raw test score on the written promotion examination and seniority
points given for each year of service—a maximum of 10 points.
Hypothesis 3: Years of formal education is predictive of effective leadership
competencies, as rated by the 360-degree peer evaluation. According to the data,
education is a predictive factor of an effective leader. Progressive fire departments
mandate that personnel selected for the positions of battalion chief and above be required
to earn a master’s degree within three years of appointment.
The fire service’s hierarchical chain of command has seven leadership roles,
beginning with the FAO and ending with the fire chief. This study evaluates the two
lowest leadership roles only—the FAO and lieutenant positions—which represent a large
cross section of subjects involved in the study. Each position in the fire service is
classified and based on the job description.
Seniority is defined as total years of service in the San Antonio Fire Department,
interrupted or uninterrupted, and not merely the first continuous period of service. The
term firefighter refers to volunteer firefighters; paid, on-call firefighters; and career
firefighters. This study examines the career firefighter category only. Career firefighters
are assigned predominantly to municipal fire departments, while the majority of paid, on-
call firefighters and volunteer firefighters work in the rural areas. Firefighters are public
servants who provide emergency and non-emergency fire services.
A collective bargaining agreement is a binding contractual agreement between the
firefighters union and the municipal government that describes various terms of
employment (IAFF Local 624, 1999).
According to the Wage and Hour Division of the Department of Labor (Mathis &
Maine, 2000), a firefighter is an employee whose functions include those of firefighter,
paramedic, emergency medical technician (EMT), rescue worker, ambulance personnel,
and hazardous- materials technician. A firefighter is (1) trained in fire suppression; has
the legal authority and responsibility to engage in fire suppression; and is employed by a
fire department of a municipality, county, fire district, or state; and is (2) engaged in the
prevention, control, and extinguishment of fires, or in the response to emergency
situations where life, property, or the environment is at risk [29 U.S.C. sec 203 (y)].
Four variables were used in this correlational study. Three predictor variables that
include; (1) scores on a written promotion examination; (2) years of experience which
also referred to as seniority; and (3) years of formal education will be correlated with the
criterion variable (4) scores on a 360-degree peer evaluation of the ranks of FAO and
This study made the following assumptions:
1. The 360-degree evaluators have sufficient knowledge to determine the
existence to which the rated exhibit leadership competencies.
2. Given the guarantee of anonymity, participants will respond to the 360-degree
peer evaluation questionnaire honestly and objectively.
This study has the following limitations:
1. Because the variables were correlated without experimental manipulation,
casual relationships cannot be inferred.
2. This design was limited to the ranks of FAO and lieutenant; therefore, it
cannot necessarily be generalized to the other ranks.
3. Because of their work and personal relationships with their co-workers,
participants may exhibit bias in answering the 360-degree peer evaluation.
4. This study is applicable to the San Antonio Fire Department.
5. Because of the small number of female participants (N= 3) in this study, the
results of this study cannot necessarily be generalized to other females in the
6. Because of the small number of African American participants (N= 16) in this
study, the results of this study cannot necessarily be generalized to other
African Americans in the fire service.
The significance of this study is to contribute to the research on the leadership
selection process in the fire service. First, the study identified leadership competencies
identified by researchers to be relevant to the fire service. Second, it evaluated the results
of the Rogers’ Firefighters Survey (2001) of 35 San Antonio firefighters who identified
leadership competencies that they perceived as relevant to the fire service. Third, it
combined the data to develop a 360-degree peer evaluation that was used to determine
whether the leadership selection process is indeed selecting firefighters who have the
leadership competencies—as perceived by the subordinates, peers, and supervisors—that
are necessary for the fire service. Finally, this study determined how well the current
selection process is working. The results of the study may suggest relevant predictors for
criteria for improving the leadership selection process.
Theoretical Literature On Leadership
A review of the current literature on leadership selection in the fire service
provided little information on the reliability and validity of the process. This chapter
identified several definitions of leadership and evaluated the leadership theories and
models that are applicable to leadership selection. Leadership theories were compared
and contrasted in a search for common elements and coherent perspectives. In addition,
methods of measuring leadership competencies were evaluated. Finally, a summary
discusses the importance of identifying effective competencies for leadership selection
criteria in the fire service.
Definitions Of Leadership
The definition of leadership can sometimes seem esoteric and illusive; one of the
many reasons is the number of definitions, theories, and researchers that exist. Stogdill
(1974), a prominent leadership researcher said, “… there are almost as many different
definitions of leadership as there are people who have attempted to define the concept”
(p. 7). Although the concept of leadership can seem mystifying, its various definitions are
critical to understanding the concepts and principles of leadership.
Hollander (1978) defined leadership as “a process of influence between a leader
and those who are followers” (p. 16), whereas Bennis (1959) defined it as “ a process by
which an agent induces a subordinate to behave in a desired manner” (p. 125). Robert
Kennedy defined leadership as “inspiring people to exercise their best qualities,” while
Dwight D. Eisenhower said leadership is “the art of getting someone else to do what you
want done because he wants to do it” (as cited by Gratza, 1972, p 199). “A definition of
leadership that would be widely accepted by the majority of theorists and researchers
might say that leadership is a process of social influence in which one person is able to
enlist the aid and support of others in the accomplishment of a common task” (Chemers,
1997, pg. 1).
The many definitions of leadership describe its various roles. The definition of
leadership used by the researcher and writer of this dissertation is “a process of social
influence in which one person is able to enlist the aid and support of others in the
accomplishment of a common task” (Chemers, 1997). This definition will be used to
study the leadership competencies of the firefighters selected for leadership roles. In
addition, the leadership theories and models were used to evaluate whether the selection
process was successfully identifying leadership characteristics and traits, as well as
whether these theories and models were used as selection criteria.
Models And Theories Of Leadership
A common view of leadership is that there is something rare in the personality of
an individual who has the unique qualifications to ascend to leadership. Some people
refer to these unique qualifications as leadership traits. Because of the various internal
and external elements involved in leadership, it remains controversial and continues to be
the subject of many studies (Kirkpatrick & Locke, 2000).
Great Man Theory.
Nineteenth-century philosopher Thomas Caryle offered the Great Man Theory,
which asserted that leadership qualities are inherited and that great men are born, not
made. The leader, who is endowed with unique qualities, contributes regardless of the
situation. The Great Man Theory is a method used to select individuals who are perceived
to be great leaders to transform and inspire individuals and organizations (Bass, 1990).
The theory promotes the idea that anyone in a leadership position must deserve to be
there by virtue of his or her characteristics or personal traits (Chemers, 1997). One
weakness of this theory is that there is little evidence to support the claim that inherited
traits are good predictors of leadership effectiveness. Currently, leadership is viewed as a
complex interaction between the leader and the social, organizational, and economic
environment. This interaction includes the leader’s ability to successfully integrate
situational components while transforming and inspiring individuals and organizations
Before the 1950s, the study of leadership was based mainly on the Great Man
Theory; however, it later was challenged by the trait theory (Goldbach, 1989). Trait
refers to a person’s general characteristics, including his or her capacities, motives, or
patterns of behavior. The trait theory is derived from the statistical treatment of large
numbers of observations presented as norms. Kirkpatrick & Locke (1991) identified six
traits that differentiate leaders from followers. These traits include: (1) drive, (2)
integrity, (3) self-confidence, (4) cognitive ability, (5) desire to lead, and (6) knowledge
of the job. Jenkins (1947) and Stogdill (1948) found that selecting leaders by means of
their traits met with little success.
One major weakness of the trait theory is that traits do not explain the complexity
and intertwined behavior of the total person (Allen, 1965). Additionally, this theory does
not take into account the interaction between the leader and his or her group. Another
weakness of the trait theory is that traits often are confused for being skills. A skill is a
technical ability, knowledge, or expertness; a trait is a characteristic (Stogdill, 1948).
Situational Leadership Theory.
The situational leadership theory suggests that leadership styles should be
matched to the maturity of the subordinates (Hersey & Blanchard, 1997). The theory is
primarily a model that classifies the subordinate’s maturity in two dimensions: (a)
psychological maturity and (b) job maturity. Psychological maturity assesses the
subordinate’s commitment, motivation, and willingness to accept responsibility; job
maturity examines the subordinate’s experience, knowledge, and understanding of the
job. As the subordinate’s maturity grows, his or her relationship with the leader should be
more relationship-motivated than task-motivated. Little is known about the validity of
this theory; however, it has much in common with the path-goal theory.
Path-Goal Theory Of Leadership.
The path-goal theory of leadership suggests that the performance, satisfaction, and
motivation of a group of people can be affected by a leader who: (a) offers rewards for
achieving performance goals, (b) clarifies paths toward the goals, and (c) removes
performance obstacles. The path-goal approach identifies the specific variables that
define the motivation, and can adapt the leadership style according to the situation. The
leadership style should be conducive to the situation. It can be directive, supportive,
participative, or achievement-oriented (Evans, 1970).
Fiedler’s Contingency Model.
The Fiedler’s contingency model assumes that group performance depends on: (a)
leadership styles described in terms of task and relationship motivation, and (b)
situational favorableness, which is determined by leadership-member relations, the task
structure, and the position of power. Fiedler argued that leadership involves social
influence and the ease with which a leader is able to influence his or her followers.
Fiedler further argued that the quality of interpersonal relations between the leader and
his or her follower is such that if the leader is well liked and respected by the followers,
his or her ability to influence them is easier and more likely to be successful (Fiedler,
Cognitive Resource Theory.
The Cognitive Resource Theory (CRT) suggests that leaders and group members
can make effective use of their intellectual abilities, technical knowledge, and experience
to contribute to group performance (Fiedler & Garcia, 1987). However, research on the
relationship between intelligence, leadership styles, and performance demonstrated that
intelligence contributed highly to performance only if the leader was directed and
working in a stress-free relationship with his or her supportive group (Fiedler, 1987).
Vroom-Yetton Normative Leadership Decision Theory.
The Vroom-Yetton Normative Leadership Decision Theory affects a whole group
or team that includes individual subordinates (Vroom, 1964). The theory maintains that
one of a leader’s important prerogatives is controlling the process by which decisions are
made. It further suggests that the decision strategies range across the dimension of degree
of subordinate participation and fall into three categories—autocratic, consultative, and
The decision model for groups falls under five strategies: (a) autocratic–1, where
the leader makes the decision using information already available, (b) autocratic–2,
where additional information is obtained from the subordinates but the leader makes the
decision alone, (c) consultative–1, where the leader discusses the problem with the
subordinates individually but preserves the authority to make the decision alone, (d)
consultative–2, where the problem is discussed with subordinates as a group before the
leader decides, and (e) Group–2, where the leader shares the problem with the
subordinates as a group and invites them to participate fully and equally in the decision,
with the leader acting as the group chair (Chemers, 1997).
Managerial Grid Theory.
The Blake-Mouton Managerial Grid Theory is two-dimensional and focuses on
the manager’s concern for: (a) production and (b) concern for people. The managerial
grid is set up on a horizontal and vertical axis that ranges from zero to nine. Concern for
people is on the vertical axis while concern for production is on the horizontal axis. Five
managerial styles are formulated from the grid: (a) supportive, (b) delegating, (c)
organizing, (d) coaching, and (e) authoritative. The grid represents the various leadership
styles recommended, based on the need for concern or a combination of concerns.
According to Blake and Mouton (1964), coaching is the most desirable managerial style
because it is based on effective integration of the task and the people (Chemers, 1997).
A brief overview of the leadership theories and models, which were useful in
identifying leadership traits, suggests the elements that make an effective leader. This
study draws upon the leadership theories and models to evaluate whether leadership traits
and styles are being identified and used toward the selection of leaders in the fire service.
The theories that are used in this study include: (a) the trait theory, (b) the
situational theory, and (c) the Vroom-Yetton Normative Leadership Decision Theory.
The trait theory is important in identifying individuals who have personal traits that could
lead to effective leadership. The situational theory was used to evaluate whether the
psychological maturity and job maturity levels are measured in selecting leaders.
Maturity is an important characteristic of an effective leader, especially in the dangerous
occupation of fire fighting. Finally, the Vroom-Yetton Normative Leadership Decision
Theory will be used to evaluate decision-making competencies. To be an effective leader
in the fire service, a firefighter must be decisive, especially when human lives are hinging
on the leader’s ability to make sound decisions (Dewey, 1910).
Leadership Selection Process In The Fire Service
National Survey Of Leader Development And Promotional Testing In The Fire Service.
A common practice in almost every fire department is the system used to test
firefighters for leadership roles in the fire service (Maxfield, 1996). Dr. Walter S. Booth
(1999) of the Booth Research Group conducted a nationwide survey of fire departments
in the United States to find out their: (a) most commonly used promotional test, (b)
minimum educational requirements, and (c) average time-in rate for testing eligibility
requirements. The survey was mailed to 408 paid fire departments in the U.S., each of
which has more than 100 firefighters.
The survey revealed that the written exam was frequently used to promote
firefighters to the ranks of FAO through battalion chief, as shown in Table 1. The
examination requirement for the most part was reduced for promotion to assistant chief;
however, as the firefighter ranks moved up the hierarchy, the use of oral boards,
assessment centers, and appointments increased. The use of practical testing methods
appears significant for promotion to FAO; however, their use is insignificant for
promotion to the higher ranks (Booth, 1999).
Most Commonly Used Testing Method by the Fire Service
Testing Methods FAO Lieutenant Captain Battalion Chief Assistant Chief
Written Exam 85% 85% 89% 66% 38%
Practical 59% 18% 19% 17% 10%
Assessment Center 16% 47% 50% 55% 42%
Oral Board 7% 14% 16% 20% 19%
Appointment 1% 1% 2% 4% 19%
Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43
(8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with
permission of the author.
The survey also asked for the minimum educational requirements. A majority of
fire departments required that most of their firefighters have a high school diploma, as
reflected in Table 2. As firefighters moved up the rank structure, the education level
required increased to a bachelor’s degree. With added responsibilities, the educational
requirements should increase. The minimum educational requirement of the battalion fire
chief and assistant fire chief were almost non-existent at the post-graduate level reflecting
0% for the battalion fire chief and 4% for the assistant fire chief. The percentage is higher
in the bachelor degree level reflecting 18% for the battalion fire chief and 36% for the
assistant fire chief. Table 2 indicates that associate degree level was the mode for the
minimum educational requirement for the battalion fire chief and assistant fire chief.
Minimum Educational Requirement
Education FAO Lieutenant Captain Battalion Chief Assistant Chief
High School 72% 48% 35% 21% 6%
Some College 14% 32% 34% 24% 23%
Associates 4% 9% 21% 37% 32%
Bachelors 0% 0% 2% 18% 36%
Post Graduate 0% 0% 0% 0% 4%
No Requirements 11% 11% 8% 0% 0%
Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43
(8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with
permission of the author.
Table 3 shows the average time in grade required for test eligibility, indicating
that a firefighter remains in grade for an average of 2.3 years before becoming eligible for
promotion to the next rank. The average 2.3 years, however, is not indicative of the
amount of time a firefighter remains in a position before being promoted to the next rank.
As a firefighter moves up the leadership hierarchy, especially to the battalion chief and
assistant chief ranks, the potential for promotion to the next rank decreases because of the
fewer number of available positions (Booth, 1999).
Average Time In Grade For Test Eligibility
Rank Modal time in grade Average time in Modal time in Average time in
grade department department
FAO 3 2.7 0 0.0 years
Lieutenant 2 2.5 5 5.6 years
Captain 2 2.1 7 6.3 years
Bttn. Chief 2 2.2 10 9.1 years
Asst. Chief 2 2.1 10 9.6 years
Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43
(8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with
permission of the author.
Municipal Civil Service Laws And State Statutes.
According to Miller (1997), the purpose of the Fire and Police Civil Service Law
is to secure efficient fire and police departments comprised of capable personnel who are
free from political influence and have permanent employment tenure as public servants.
The authority is granted under provisions of the Texas statute Chapter 143 Municipal
Civil Service, also known as Chapter 143 of the Local Government Codes, or Vernon’s
Texas Code Annotated. In Chapter 143 Subchapter B, entitled “Classification and
Appointments,” the promotional procedures are identified: (a) eligibility for promotion,
(b) examination notice, (c) exam procedures, and (d) exam grades.
According to Miller (1997), a firefighter is not eligible for promotion unless he or
she has served in the current rank for at least two years before the examination. The Fire
and Police Civil Service Commission posts a notice that lists the study material from
which the examination questions will be taken. Applicants must earn a minimum raw
score of 70 points to pass the examination. Applicants who pass the exam receive one
point added to their raw score for each year of service, for a maximum of 10 additional
points. A tie in the promotion competition is broken by the highest score on the written
exam. If there still is a tie, the promotion is awarded to the candidate who earned the
highest score on the entrance examination (Miller, 1997).
Collective Bargaining Contract Agreement
In 1935, the Wagner Act established the National Labor Relations Act Board,
which regulates negotiations and the collective bargaining process between employers
and labor organizations (Fossum, 1992). According to the Society of Human Resource
Management (1998), collective bargaining is an exclusive representation by the union of
the group of people in a bargaining unit. Bargaining is the process by which management
and union representatives negotiate employment conditions. Subjects for collective
bargaining negotiations fall into three categories: (1) mandatory subjects, (2) voluntary
subjects, and (3) illegal subjects.
In 1973, the citizens of San Antonio passed a citywide referendum granting
collective bargaining rights to fire fighters (personal communication from H. Cardenas,
January 9, 2001). During the collective bargaining negotiations, the promotional process
from which firefighters are selected to leadership roles falls under mandatory subjects.
The collective bargaining contract between the City of San Antonio and Local 624
International Association of Fire Fighters, Article XXXI title Promotions, Section 1-8,
clearly identifies the promotional process used to select a firefighter for leadership in the
fire service (IAFF Local 624, 1999).
Section 1 of the contract identifies eligibility for promotion in addition to
granting seniority points, for a maximum of 10 points. Section 2—Study
Material—addresses the appointment of a committee to determine the material that will
compose the examination, pending the approval of the Fire and Police Civil Service
Commission. Section 3—Promotion to Fire Apparatus Operator, Lieutenant, and
Captain—states that position vacancies will be filled by means of a competitive written
examination in accordance with Chapter 143 of the Local Government Codes; however, a
score of 70 will be considered the minimum for eligibility for a promotion. For
promotion to district chief, candidates must pass a written examination, with seniority
points added, in addition to passing an assessment-center process, as stipulated in Section
4 (IAFF Local 624, 1999).
The collective bargaining agreement between the City of San Antonio and the
firefighters is a city ordinance that supersedes municipal and state statutes, including
Chapter 143 of the Local Government Codes. During collective bargaining talks, the city
and the firefighters have domain over the promotional process and can alter it to fit their
needs better, as long as both parties are in agreement (IAFF Local 624, 1999).
Promotional Examination Process
The San Antonio Fire Department’s tool of choice as a predictor is a written
promotional examination, the results of which reveal the candidates for each
classification who are eligible for promotion to leadership (Miller, 1997). The written
exam is the first part of the selection process. Candidates have two hours to answer 100
questions formulated from the assigned study material, which covers critical job
knowledge and skills. The Booth Research Group developed the test questions from job
analyses of the positions for which candidate are being tested (Booth, 1998).
The written exam is based primarily on technical and administrative skills related
to job knowledge. For the rank of district chief, a captain who is eligible for the position
must also pass an assessment center in order to be placed on the eligibility list. By
definition, an assessment center is a program in which candidates proceed through a
number of job-related assessment exercises that typically include: (1) in-basket exercises,
(2) leaderless group discussions, (3) confrontation exercises, (4) an interview, and (5)
analysis exercises. The purpose of the assessment center is to elicit behavior relevant to
dimensions that are crucial to job success (Prudy & Filer, 1980). When a vacancy occurs,
the top candidate is selected for promotion to this leadership role (Booth, 1999).
The study material for the written promotional examination can be a source of
concern if it is not job-related. Often there are contradictions—among the different
sources and even within a single source—regarding the relevancy of the material to the
fire service. The study material is carefully reviewed to avoid such contradictions. The
questions are from areas in a text that are about the position. In addition, the
examinations are administered easily with the use of an administration guide and other
instructions available to the proctor. The written promotional tests are custom-designed to
reflect the needs, concerns, and mission of the fire service (Booth, 1999).
Qualifications For The Promotional Examination.
Promotional examinations are open to firefighters who for at least two
consecutive years have been in a position that is below the rank of that for which the
exam is being administered, according to the Collective Bargaining Agreement between
the City of San Antonio and Local 624 of the International Association of Firefighters
(1999, pg. 53). For example, a firefighter who has been in the rank of FAO for at least
two consecutive years is eligible for the written exam for lieutenant, which is the next
rank in the hierarchy.
Promotional Examination Procedures.
According to the Texas Firemen’s and Policemen’s Civil Service Law (Miller,
1997), the Civil Service Commission is authorized to adopt rules governing promotions
and to administer promotional exams to provide eligibility lists for each classification in
the fire service. Candidates are given an identical written examination, which comprises
100 multiple-choice questions. The questions, which must originate from study material
that is approved and posted by the Civil Service Commission, also must be relevant to the
duties of the position for which the exam is being administered. The study material must
be a reasonably current publication that is available to all candidates eligible for the
Firefighters from the ranks of firefighter to captain are screened for eligibility to
take the written promotional exam. Part of the eligibility requirement is that they have a
minimum of two years in rank.. The assigned study material begins with the practical
fire-ground operations of the FAO and lower ranks, then covers leadership and
supervision for mid-level ranks such as lieutenant and captain, and ends with the
administrative-type operations of the higher ranks such as district chief. While some of
the training and study material is common to all ranks, other study areas such as strategy
and tactics, leadership, management, and supervision are required for the lieutenant
through district chief positions (Booth, 1999).
The director of the Civil Service Commission is responsible for preparation and
security for each promotional exam (Miller, 1997).
Analysis Of The Job Knowledge, Skills, And Abilities (KSAs).
One of the many definitions of job analysis is “a purposeful, systematic process of
collecting information on the important, work-related aspects of a job” (Gatewood &
Field, 1990, p. 251). The components of a job analysis include: (1) work activities, (2)
tools and equipment, (3) the work environment, and (4) personnel requirements such as
knowledge, skills and abilities, and other characteristics (KSAOs) such as physical traits,
job interest, and personality (Gatewood & Field, 1990).
Job analysis data frequently are used to develop two types of measures: (1)
predictors of job applicants who are likely to be successful in the position being
measured, and (2) standards of performance that an applicant must meet to be considered
successful on the job. By examining tasks performed on the job as well as the KSAOs,
examiners can establish the predictors and criteria for measurement in order to select the
applicant with the greatest potential for success (Gatewood & Field, 1990).
Most psychologists believe that the best measure of job performance is a work-or
job-sample measure. However, this is usually infeasible for public safety organizations.
For example, for firefighters, there are 440 different tasks in firefighting. Second, task
such as accurate work in presence of physical danger or under stresses such as
emergencies may be difficult or dangerous to stage (Hunter & Schmidt, 1996).
The City of San Antonio hired the Booth Research Group (1998) to develop a
valid, multiple-choice written promotional examination for the positions of FAO and
lieutenant. Conducting a job analysis is an essential part of developing a valid testing
instrument. The Booth Research Group conducted job analyses on the FAO and
The Booth Research Group (1998) designed a questionnaire to assess the
knowledge, skills, abilities, and other characteristics (KSAOs) needed for effective
performance in each of these ranks. From the job analysis data, scales were designed to
assess the functional skills required for each job. Seven job analysis scales, which range
from low to high, provide a method for measuring basic/generic occupational skills. The
scales, which are widely used to objectively define firefighting tasks, can be applied to all
One hundred twenty-nine (N=129) questionnaires were distributed to FAOs and
lieutenants, of whom 108 (84%) responded. The FAOs and lieutenants reviewed the rate
of importance and complexity of all job tasks. Promotional examination questions were
developed based on the job dimensions relevant to the classifications being tested (Booth,
Reliability And Validity Of The Selection Process.
According to the American Psychological Association’s (APA) 1980 publication,
Principles for Validation and Use of Personnel Selection Procedures, “when any selection
procedure is used, the essential procedure is that evidence be accumulated to show a
relationship between the decisions, based on assessments made by that procedure and
criteria such as job performance, training performance, advancement, or other pertinent
behavior” (p. 1). Validity, as defined by the APA (1980), is the degree to which
inferences from scores on tests or assessments are supported by evidence. The concept of
reliability is used in terms of dependability, consistency, and precision of measurement.
According to Field & Gatewood (1990), the types of validity strategies involve:
1. Criterion validity, which includes concurrent or predictive validity. In a
concurrent validation, the predictor and the criterion are collected
simultaneously. In a predictive validation, the predictor is administered
before beginning the job, and the criterion data are collected later.
2. Content validity, which is based on descriptive samples of job behaviors
that are necessary for effective performance. These content items are then
placed in the test.
3. Construct validity, which is based on the concepts, attributes, or
This study examines the criterion-related concurrent validity of cognitive-ability
test scores, job tenure, and scores from the 360-degree feedback instrument. The process
is accomplished by (a) conducting a job analysis; (b) determining relevant knowledge,
skills, abilities, and other characteristics required to successfully perform the job; (c)
choosing one or more predictors of the KSAOs; (d) weighing the predictors against the
criteria; and (e) administering them to the employees.
The criterion-related test consists of the critical elements of the job, which can be
performance or technical elements, and is defined in terms of skills. The predictor is a
sample of performance that is used to predict future performance. A predictor can be a
written test, an interview, or a personal reference check. Weighting predictors can
provide the leadership selection process with the best possible forecast of the candidate’s
success on the job (Field & Gatewood, 1990).
Leadership Prediction Studies
Remarkable strides have been made in developing reliable, valid tests for
selecting non-managerial employees in various occupations. Developing a process for
selecting effective leaders, however, has proven less successful (Bass, 1990). Supervisors
often are chosen from the rank and file based on their technical skills rather than their
leadership skills. For example, a police sergeant is more likely to be promoted to
supervisor because he is good at the technical aspects of being a police officer rather than
because he has the qualities to be an effective leader (Hogan, Curphy, & Hogan, 1994).
The most commonly used approach to predicting the readiness of the job
incumbent for leadership is the interview process (Bass, 1990). According to Hogan,
Curphy, and Hogan (1994), the best way to predict leadership capability is to use a
combination of cognitive ability, personality, simulation, role-playing, and multirater
assessment instruments and techniques.
Organizations such as AT&T, Sears Roebuck, IBM, the United States Military,
police departments, and many others use the judgment from simulation techniques to
predict leadership capability. The simulations are based on observation of leadership
performance using work samples and situational tests (Bass, 1990). The AT&T
Corporation developed a managerial selection tool known as an assessment center, which
identifies performance dimensions relevant to the job. The company uses the results to
select job incumbents for leadership roles within its organization (Bray, 1982).
IBM also uses an assessment center process to select it leaders. The company
conducted an in-basket test on its supervisors that revealed a correlation of r= .32 when
the subjects were rated again after facing the challenge of increases in management
responsibilities within three years after testing (Wollowick & McNamara, 1969).
Organizations commonly use performance ratings to promote and to forecast
future success in candidates for leadership (Bass, 1990). In the Yammarino & Bass study
(1989), the military superiors of promotion candidates in the U.S. Navy used the
candidates’ performance grades as students in the U.S. Naval Academy as predictors of
whether they would become charismatic Naval officers. The cumulative military
performance grades that were awarded did not prove to predict subsequent performance
fitness reports of 186 U.S. Navy officers serving in the fleet one decade later. The results
are consistent with other studies that fail to find a significant correlation between
undergraduate college grades and success in business and industry (Schick & Kunnecke,
1982) or higher ratings on job performance (Pallet & Hoyt, 1968). Most of these studies,
however, have one or more of the nine flaws noted by Ree (1995) as examples of
conducting ability research incorrectly. One of these examples is the disregard for group
differences such as race and sex. Group differences should be present; however, the less-
known differences should be ignored. Another example is use of the wrong model of
reliability. Although test-retest measures would be appropriate, it is suggested that high
reliability values could result frequently from this model misspecification (Ree, 1995).
Although performance measures have not been predictive of readiness of job
incumbents for leadership positions, peer ratings have been found to be an effective
predictor in some instances. For example, a peer rating by cadets at West Point Military
Academy was found to be the best single predictor of leadership success as an officer in
the U.S. Army. For example, a correlation of r= .51 was calculated between the peer
ratings at West Point and the success rating of the same subjects as infantry officers 18
months later (Haggerty, Johnson, & King, 1954).
Studies show that many public and private organizations operate on the belief that
it is appropriate to hire intelligent and experienced job incumbents for leadership
positions. Studies indicate that the correlation between leadership performance and
various experience measures is less than r= .10 (Fiedler, 1995; Fiedler & Garcia, 1987).
Although some studies indicate a correlation of less than r= .10, there are other studies
that report mixed correlations where the magnitude or degree of relationship is about the
same. Studies have shown that under stressful conditions, experience and performance
were correlated in the range of r= .40 to r= .60. For example, the experience of fire
department officers and their performance was correlated to be r= -.66 (N=22, p< .01) in
low-stress administrative jobs, but in stressful fire combat, it was correlated to be r= .68
(N=22, p< .01) (Frost, 1981).
A study conducted by Hunter and Schmidt (1998) summarizes the practical
implications of 85 years of research in personnel selection. On the basis of meta-analytic
findings, the validity of 19 selection procedures for predicting job performance and
training performance and the validity of general mental ability (GMA) and the 18 other
selection procedures was presented (see appendix F). The selection procedure with the
highest validity and utility for job performance were GMA plus a work sample test (mean
validity of .63), GMA plus an integrity test (mean validity of .65) and GMA plus a
structured interview (mean validity of .63).
Many labor leaders assume that the predictive validity of experience is higher
than the predictive validity of ability, especially in highly experienced workers. Studies
predicting performance ratings for predictive validity of both experience and ability
indicates that the average predictive validity for experience to be .18, where as the
predictive validity of ability is .51. Therefore, predictive validity is three times larger than
the predictive validity of experience (Hunter and Schmidt, 1996).
In 1945, Shartle (1950) organized the Ohio State Leadership Studies. He and his
colleagues integrated the most comprehensive research program on identifying leadership
behavior patterns. The group generated a list of about 1,800 descriptive statements about
various aspects of leadership behavior. From this list, a leadership-behavior-description
questionnaire was developed. On the questionnaire, respondents would rate a leader by
picking one of five choices to indicate the frequency of behavior that described the
leader. The data were correlated and factor-analyzed, resulting in two elements that
emerged to describe leadership behavior—consideration and initiation of structure (Bass,
By contrast, Frank (1955) believed that the compilation of data based on sheer
numbers of subjects could not adduce a completely unified theory of behavior because of
the variability of the individual. According to Fiedler and House (1944), there is no
evidence of a specific leadership trait, behavior, or personality. Leadership effectiveness
is the ability to get a group to accomplish its mission; it does not depend solely on the
leader’s abilities and attributes but also on how well the leader’s personality, abilities,
and behaviors match the situation. Two major behaviors that identify leaders have been
identified; one was how well a leader treats his or her subordinates; the other was the
degree to which a leader structures the roles and working relationships of his or her
Methods Of Verification And Measurement Of Leadership In The Fire Service
Psychologists know that theories of leadership depend on different methods of
verification, while models of leadership depend on different methods of measurement
(Bass & Stogdill, 1990). According to Shores (1984), “a selection procedure for
firefighters is considered to be related to the criterion when the relationship between
performance on the procedure and performance on the criteria measure is statistically
significant at .05 level of significance” (p. 8).
In this study, the degree of relationship between the criterion and the predictive
variables was evaluated and computed using acceptable statistical procedures to
determine whether a relationship exists. The variables examined included: (1) the Booth
Research Group Questionnaire, (2) the Rogers’ Firefighters Survey (2001), (3) the
written criterion test, (4) the years of service, (5) the years of formal education, and (5)
the criterion variable—a 360-degree peer evaluation.
Booth Research Group Questionnaire.
A questionnaire is a formal measure of characteristics, attitudes, motivations, and
opinions as well as past, current, and potential behaviors. The information elicited from a
questionnaire can be used to describe or compare facts or make predictions. Depending
on the objectives, the survey design must be flexible (Salant & Dillman, 1994). The
questionnaire is not a test; it is simply a method of collecting information (Booth, 1998).
It is important to remember that getting a subject to complete a long questionnaire may
prove difficult. The questions should be written clearly and reflect the purpose of the
questionnaire (Astous, 2001).
The City of San Antonio hired the Booth Research Group (1998) to develop a job
analysis questionnaire to assess the knowledge, skills, abilities, and other characteristics
(KSAOs) required for effective performance in the positions of FAO and lieutenant. The
purpose of the questionnaire was to gather information in order to paint a big picture of
the job characteristics that accurately reflect the position being evaluated.
The job analysis identified several competencies relevant to the job. When a
competency was identified, it received a number of ratings based on the importance and
frequency of the performance dimensions. A competency is a specific aspect of a
position. For the FAO position, 13 different groups and tasks were identified. For the
lieutenant position, five competencies were identified from which 18 dimensions were
rated based on their importance and frequency (Booth, 1998).
The Booth Research Group (1998) demonstrated criterion-related validity by
using a construct-base approach that included (1) a series of job analysis activities; (2) a
series of workshops to identify the abilities and characteristics required to perform the
job; and (3) a questionnaire to established a linkage between the abilities, constructs, and
specific task dimensions.
Rogers’ Firefighters Survey.
A survey is a powerful scientific tool for gathering accurate, useful facts and
sound data that can be translated into valuable information for its intended users (Salant
& Dillman, 1994). In a study conducted by the Selection Consultant Center for the
California Personnel Board (Hubbard, Hunt, & Krase, 1975), a study of 284 respondents
in 62 fire departments in California and Nevada rated six-job criteria that are relevant to
the fire service. These criteria included: (a) mechanical and trade knowledge, (b) physical
agility, (c) personal attributes, (d) communication skills, (e) problem-solving, and (f)
The National Fire Academy in Emmitsburg, Maryland, conducted a national
survey of more than 5,000 Emergency Medical Service (EMS) firefighters and leaders.
The survey identified management and leadership traits that the respondents believed
were relevant to the fire service (Dyar, 2000). The leadership traits most often identified
in the survey were: (a) honesty, (b) consistency, (c) vision, and (d) competency.
In a pilot study, the Rogers’ Firefighters Survey (2001) was randomly
administered to 35 San Antonio firefighters in order to identify leadership competencies
relevant to the fire service. One of the purposes of the survey was to gather data from
firefighters assigned to the fire-fighting division, as opposed to those in the EMS
division. Included in the questionnaire was a list of 42 leadership traits and competencies
from which the firefighters were to select those relevant to the fire service. They also
were to provide good and bad examples of each leadership behavior they selected, based
on what they had encountered on the job. Their examples provided a variety of
descriptions of relevant leadership behavior in the fire service. From the survey, six
leadership competencies were identified as the most relevant: (1) communication, (2)
self-confidence, (3) ethical behavior, (4) professional and technical skills, (5)
interpersonal skills, and (6) emotional balance (Rogers, 2001).
Written, Criteria-Based Test.
The written promotional examination is the first part of the leadership selection
process in the fire service. Research on the value of written tests has revealed that despite
their limitations, written tests are a source of valuable information about the probability
of the individual performing the job successfully (National Academy of Science, 1982).
In this study, the writer and researcher of this dissertation examines the written
promotional examination conducted by the Booth Research Group (1998). The Booth
Research Group developed the written promotional test based on the criteria-related
concurrent process. It involves:
1. A series of job analysis activities that identify the tasks and dimensions that
compose the target job.
2. A series of workshops and interviews that identify the abilities and
characteristics required to perform the job effectively.
3. A survey to assess the skills, knowledge, abilities, and other characteristics
required for effective performance of the target jobs, which in this study are
the positions of FAO and lieutenant.
The validated written promotional examination is used to identify firefighters who
have the greatest potential for success on the job.
Years Of Service.
Years of service, also known as seniority, represent all the years of service in the
San Antonio Fire Department. It could be viewed as a predictor of job performance in the
fire service. However, current studies indicate that over time, the validity of job
experience for predicting performance declines (Hunter & Schmidt, 1996).
Years Of Formal Education.
The years of formal education represents the attainment of credit hours from a
college or university. It could be viewed as a predictor of job performance in the fire
service. In the general population, the correlation between education and ability is about
.55 (Hunter & Schmidt, 1998).
A 360-Degree Peer Evaluation.
Companies and organizations use a multi-source or 360-degree feedback to rate
the leadership and managerial performance of (1) supervisors, (2) peers, (3) subordinates,
and (4) internal and external customers, according to Goudy-Keith-Lyne (1999). This
360-degree feedback is used as a rating tool of managerial performance because it allows
peers and leaders to identify and compare leadership competencies of the individual who
is being rated. The extent to which the comparison of feedback provides meaningful
information is dependent on whether the rater-sample interprets the performance scales
The choice of questionnaire is a critical step in the 360-degree feedback process,
and the degree of feedback is determined by the quality of the questionnaire (Goudy-
Keith-Lyne, 1999). The dimensions of the questionnaire should be based on job-related
criteria that are relevant to what is being measured. The 360-degree peer evaluation has
face validity based on observed behaviors and skills.
In 1992 Riggio and Cole conducted a peer-evaluation study on 261 members of
the fire service who represented three ranks in the organizational hierarchy: chiefs
(N=12), captains (N=76), and firefighters (N=173). The mean age of the subjects was
35.6 years (SD=7.9 years) and the years-of-service mean was 12.9. An investigation was
conducted on the concurrence between the performance ratings of the superiors and those
of the subordinates as well as the correlation between these performance ratings and
standardized measures of job satisfaction. The results revealed significant agreement
between superiors (chiefs) and subordinates (firefighters) in rating the performance of the
captains. These performance ratings also correlated positively at r = .58 with the job
satisfaction of the subordinates (firefighters) and supervisors (captains) (Cole & Riggio,
One issue of concern with the 360-degree peer evaluation is a lack of convergence
across multi-source ratings because of the different raters of the job responsibilities that
may give them different conceptualizations of leadership (Tornow & London, 1998).
Research on peer appraisals reveals that they could be prone to bias. Under certain
circumstances, workers may react negatively toward peer evaluations (Kane & Lawler,
1978). Recent research has raised concerns about the rating of leadership performance by
peers and superiors (Harris & Schaubroeck, 1988). Bias is a major source of error in peer
evaluations, according to Bass (1990), Podsakoff and Organ (1986), and Harris and
Schaubroeck (1988). It is hoped that the anonymity afforded the participants will lessen
any potential for bias in this study.
Effective leadership in the fire service is critical to the safety and welfare of
firefighters and the community. Ability testing, which is available to the fire service,
gives management the capability of predicting job performance. Although the written test
can be viewed as a predictor of effective performance, it tends to demonstrate ethnic and
racial bias resulting in adverse impact (Booth, 1998). Several lawsuits were filed
regarding the test procedures that evaluate candidates for promotion in the fire service. In
1974, the percentage of African-American employees in the St. Louis Fire Department
was 11% of the 1,000 uniformed personnel. Of the 180 fire captains, only four (2%) were
African-American. Although this city had a population that was 40% African-American,
no African-American firefighter had ever held a position above the rank of fire captain.
The fire department’s written promotional test appeared to adversely impact the African-
American firefighters (Gebhart, Duffe, & McCurley, 1998).
Adverse impact is a form of discrimination in which organizational selection
standards are applied uniformly to applicants of both genders and all racial and ethnic
groups; the net result, however, is denial of these groups into leadership, according to
Gatewood and Field (1990). For example, the height requirement once used by police
departments to recruit applicants discriminated against Hispanic, Japanese, and women
Title VII of the Civil Rights Act of 1964 is the principle body of federal
legislation in the area of employee selection. Section 703 (b) of Title VII addresses the
use of tests in selection procedures, stating: “… nor shall it be unlawful employment
practice for an employer to give and to act upon any professional developed ability test,
provided that such test is not designed, intended, or used to discriminate because of race,
color, sex, religion, or national origin.” (Shores, 1984, p. 3 & Federal Uniform Guidelines
on Employee Selection Procedures, 1978).
Selecting firefighters with effective leadership competencies is critical to the fire
service. Administering an ability test that identifies competencies and is within the
parameters of the law is every bit as critical. One legal defense that can be applied to the
selection process is use of the Uniform Guidelines on Employee Selection Procedures
(1978). These guidelines serve as a primary reference in the selection process and
represent the viewpoint of the federal agencies charged with enforcement of the Equal
Employment Opportunity (EEO) laws. The guidelines also provide employers with one
set of government requirements to meet in order to avoid discrimination in testing and
other selection processes (Field & Gatewood, 1990).
This chapter presents the research design that identifies the four variables used,
states the null hypotheses, and describes the participants and instrument used to collect
data. It also describes the data-collection and data-analysis procedures and concludes
with a statement of the ethical decisions involved in this study.
The research design for this study is a correlation/regression cross-sectional
design. It employed a 360-degree peer evaluation to gather data for the criterion variable.
The study focused on the relationship between the criterion and three predictor variables.
A correlation/regression design was used because the predictor variables do not lend
themselves to experimental manipulation.
Criterion And Predictor Variables
The degree of relationship between any two variables is the extent to which they
vary in a systemic manner. A correlation/regression analysis was conducted on the
variables to determine the degree of relationship between the criterion variables and the
predictor variables. The predictor variables were (1) scores on a written promotional
examination, (2) years of experience or seniority in the San Antonio Fire Department,
and (3) years of formal education. The criterion variables were the scores of the 360-
degree peer evaluation, which measures leadership competencies relevant to the fire
One hundred eighty-one firefighters participated in the 360-degree peer
evaluation, which had 57 variables that measured leadership competencies for the FAO
and lieutenant, and were ranked on a scale of 1 to 5. Once the scores were collected, a
factor analysis was conducted on the variables to reveal patterns of interrelationship
among variables and detect clusters of variables that inter-correlated and redundant. The
initial Eigenvalue of the first factor were rated at 55%, with the others reporting nothing
greater than 5%. Consequentially, all the scores of the variables were totaled to reflect the
criterion variable, which is described as the 360-degree peer evaluation.
Hypothesis 1 Ho: ρ = 0 there will be no significant relationship between the
scores on the written promotional examination and the scores of the 360-degree peer
evaluation of effective leadership competencies.
Hypothesis 2 Ho: ρ = 0 there will be no significant relationship between years in
the fire service and the scores of the 360-degree peer evaluation.
Hypothesis 3 Ho: ρ = 0 there will be no significant relationship between the years
of formal education and the scores of the 360-degree peer evaluation.
Hypothesis 4 Ho: ρ = 0 there will be no significant relationship between the
criterion and the set of all predictors. This null hypothesis could also be stated as: The
values of the regression weights are equal and all equal to zero.
The participants were San Antonio Fire Department firefighters, FAOs,
lieutenants, and captains. Verification of their ranks is documented in official department
records. The participants rated the FAO and lieutenant ranks, which are low- to midlevel
leadership positions. These ranks were identified among the seven leadership positions in
the hierarchy in order to ensure an adequate sample size.
Participants came from the Fire Suppression Division only; the other divisions
were excluded from the study because their hierarchical structures are not conducive to a
360-degree peer evaluation. About 60% (N= 840) of the firefighters in the department
(N=1400) are assigned to the Fire Suppression Division. Permission to conduct the
research on this sample group was granted by the president of the International
Association of Firefighters Local 624 (see Appendix A).
The age of the participants ranged from 31 to 59 years; the mean was 45 years.
Their sex was predominantly male; a small percentage was female. Years of service
ranged from four to 35 years, with a mean of 21 years of service. The years of formal
education ranged from 12 (General Equivalency Degree) to 18 years (master’s degree). A
high school diploma with one and a half years of college (N=13.8 years) was the average
amount of formal education. Although participants were exposed to similar leadership
selection procedures, the study material for the written promotional exam changes
slightly each year.
The criterion variable was the score on the 360-degree peer evaluation that rated
FAOs and lieutenants. The surveys were distributed to 225 potential participants, with an
expected response rate of about 80% (N=181).
In this study, the 360-degree peer evaluation was used to provide feedback
regarding leadership competencies relevant to the fire service. The survey called for
ratings of leadership performance from the full circle of key personnel that included (1)
supervisors, (2) peers, and (3) subordinates.
The 360-degree peer evaluation was created from two sources. The first source
was the Booth Research Group (1998), which conducted a job analysis that identified five
leadership competencies relevant to the positions of FAO and lieutenant. The second
source of data was the Rogers’ Firefighters Survey (2001), which surveyed 35 San
Antonio firefighters who identified six leadership competencies relevant to the fire
service. A total of eleven leadership competencies were identified from the two sources;
however, two of the competencies identified by the Rogers’ Firefighters Survey (2001)
overlapped with those identified by the Booth Research Group (1998), yielding a net
result of nine competencies to measure the leadership performance of FAOs and
lieutenants in the San Antonio Fire Department.
The 360-degree peer evaluation was based on nine leadership competencies from
which 57 rating questions were created. The competencies are (a) communication, (b)
teamwork, (c) ethical behavior, (d) general administration, (e) emergency scene
management, (f) personal development, (g) community interaction, (h) technical skill,
and (i) self-confidence. Communication was one of the competencies from which seven
rating questions were developed. Examples of how communication skills were rated
include observing how the leadership candidate: (1) conveys his or her thoughts through
speech and in writing, (2) exhibits knowledge of communication tactics and techniques,
and (3) listens and responds to verbal orders at the fire station and the fire scene.
A rating scale was used on the 360-degree feedback to measure the leadership
performance of FAOs and lieutenants. The scale ranges from 1 to 5, representing the
lowest to the highest ratings, respectively. For example, a rating of 5 reflects exceptional
skills, meaning that the individual consistently exceeds expectations of behavior and
skills in this area. A rating of 1 reflects low skills; the individual consistently fails to
reach expectations of behavior and skills in this area. The results of the 360-degree peer
evaluation were used in correlational and regression analysis to determine the
relationship between the criterion and predictor variables.
The performance rating scales of the 360-degree peer evaluation were based on
questions relevant to the positions of FAO and lieutenant in the fire service. There are no
data on the validity and reliability of this instrument; however, the 360-degree peer
evaluation was based on observation of behaviors and skills offering face and content
The instrument was delivered to 225 participants at 12 randomly selected fire
stations. Written and verbal instructions were administered to the participants. The
estimated time expected for completion of the survey was 20 minutes or less. Upon their
completion, the surveys were collected and secured, ensuring the participants’
The criterion variable scores were collected from the results of the survey. The
predictor variables included (a) the scores earned on the written promotional test, (b)
years of service, and (c) years of formal education. The exam scores were obtained from
the City of San Antonio Fire and Police Civil Service Commission. The years of service
and years of formal education were gathered from the San Antonio Fire Department
The Pearson r statistical index was used to describe the degree of strength and
direction of relationship between the criterion and predictive variables. Multiple
regressions also were conducted to determine the ability of the predictor variables,
individually and together, to predict the criterion variable. The multiple regressions were
evaluated in the usual manner using the F-test of significance (Agresti & Findlay, 1997).
The data collected from the surveys were entered into the SPSS for analysis,
which consisted of correlation and regression analyses of the criterion and predictive
One of the ethical aspects of this study was the anonymity afforded the
participants, since the 360-degree peer evaluation involved performance ratings of
superiors by their subordinates. To ensure honest and accurate answers, anonymity was
enforced by the use of numbers to identify the individual being rated and his or her job
performance rating. A separate confidential identification sheet identified the names with
their assigned number, but the sheet is kept separate from the 360-degree peer evaluation
and was available only to the researcher.
The purpose of this study was to determine whether the present leadership
selection process in the San Antonio Fire Department is identifying firefighters with
leadership competencies as recognized by the Booth Research Group (1998) and the
Rogers’ Firefighters Survey (2001). In addition, the study results suggest relevant
predictors used as criteria for selecting firefighters for leadership roles. The results of the
study, which are described in the following sections, provide clear and definitive
information for the fire service to use in selecting firefighters for leadership.
Descriptive Statistics Of Participants
The study was conducted on firefighters assigned to the San Antonio Fire
Department, which has a total of 48 fire stations. The participants held either the rank of
firefighter, FAO, lieutenant, or captain and were assigned to the Fire Suppression
Division, to which about 60% (N= 840) of the firefighters in the department (N=1,400)
are assigned. Two hundred twenty-five surveys were distributed to 12 randomly selected
fire stations where firefighters (N=181) agreed to participate.
The surveys, which were completed between March 14 and May 15, 2001, were
randomly distributed to 12 fire stations, then collected and secured by the researcher.
Although the amount of time needed to complete the surveys varied, on average,
participants completed the survey within 45 minutes.
The ethnic groups measured in this study were Hispanic, White, and African-
American. Of the 181 participants, 52.5% (N=95) were Hispanic, 38.7% (N=70) were
White, and 8.8% (N=16) were African-American. The ethnic composition of San
Antonio’s 1.1 million people is 58.7% Hispanic, 32.5% White, and 6.8% African-
American (U.S. Census Bureau, 2000).
Descriptive Statistics of Participant Ethnicity
Rank White Hispanic African American Total
FAO 48 49 8 105
Lieutenant 22 46 8 76
Percentage 38.7% 52.5% 8.8% 100%
Note. Percentage from U.S. Census Bureau, 2000
Participants in the study ranged from 31 to 59 years of age. The mean was 45.92,
with a mode of 53 and a median age of 48 years. There was an increase in the number of
firefighters in the 36– to 38–year age bracket and the 49– to 53–year age bracket.
Descriptive Statistics of Participant Age
Age FAO Lieutenant Total
31–35 years 7 3 10
36–40 years 17 29 46
41–45 years 13 9 22
46–50 years 19 27 46
51–55 years 42 4 46
55–59 years 5 6 11
Mean 46.95 44.5 45.92
The San Antonio Fire Department’s sex makeup is 96 % male and 4 % female.
The vast majority of the females are assigned to the EMS Division, with .2% assigned to
the fire- fighting division. In this study, the sex of the participants was 98.3% (N=178)
males and 1.7% (N=3) females.
Descriptive Statistics of Participant Sex
Rank Male Female Total
FAO 102 3 105
Lieutenant 76 0 76
Written Promotional Examination Scores.
The written promotional examination scores ranged from 74 to 96 points. The
mean was 84.56 with a mode of 86 and a median of 84 points. The standard deviation for
the examination scores for the FAO was 6.00 and 4.33 for the lieutenant.
Descriptive Statistics of Participant Examination Scores
Exam Scores FAO Lieutenant Total
71–75 18 0 18
76–80 28 0 28
81–85 21 33 54
86–90 25 23 48
1–95 13 16 29
95–100 0 4 4
Mean 82.50 87.48 84.56
Years Of Service.
The years of service ranged from 4 years to 35 years. The mean was 21.46 years
with a mode of 25 years of service. The years of service of 15, 25, and 28 years contained
the largest number of participants. The number of participants between 4 and 14 years
Descriptive Statistics of Participant Years of Service
Years of Service FAO Lieutenant Total
01–10 years 14 1 15
11–20 years 21 40 61
21–30 years 63 25 88
31–40 years 7 10 17
Mean 22.07 20.62 21.46
Years Of Formal Education.
The years of formal education ranged from 12 (GED; General Equivalency
Degree) to 18 years (master’s degree). The mean was 13.88 years with a mode of 14
years. The majority of the participants (N= 100) had a fire science associate’s degree; the
second-largest group of participants (N= 28) were those who had a high school diploma.
The number of participants with an undergraduate degree was 15.5% (N=28); only 1.1 %
(N= 3) had a graduate degree. The percentage of participants with a graduate degree was
Descriptive Statistics of Participant Years of Formal Education
Years of Education FAO Lieutenant Total
12 years 30 6 36
13 years 10 4 14
14 years 50 50 100
15 years 0 1 1
16 years 13 15 28
17 years 0 0 0
18 years 2 0 2
Mean 13.66 14.20 13.88
The first hypothesis of this study was to determine whether there was a significant
correlation between the scores on the written promotional examination and the scores of
the 360-degree peer evaluation. The written promotional examinations for the ranks of
FAO and lieutenant differ in that the study material provided to the FAOs focused on
skilled, base questions while the lieutenant’s study material focused on performance
dimensions and leadership. The written examination questions reflected the study
material. Therefore, the data for the ranks of FAO and lieutenant were split to reflect the
difference in examination scores of the two ranks. Although the Pearson r correlation was
used, the variables were not significantly correlated at the .05 level.
In Table 10, the resultant correlation coefficient for the rank of FAO was .079, p>
.05. For the rank of lieutenant, the correlation coefficient was .082, p> .05. These
correlation coefficients indicate that there was no significant relationship at the .05 level
between the scores of the written examination and the scores of the 360-degree peer
Research Hypothesis-1 Correlation Promotional Examination Scores Versus 360-Degree
Rank r Significance (one-tailed)
FAO .079 p> .05
Lieutenant .082 p> .05
In Figures 1-A and 1-B, the scattergram for Research Hypothesis 1 correlations
indicates that there was no clear pattern of relationship between the scores of the written
promotional examination and the scores of the 360-degree peer evaluation. Although
somewhat clustered in pairs at the lower right corner of the scattergram, Figure 1-A
shows that the plots were scattered throughout the range of scores, indicating no
significant correlation between the variables at the .05 level. Figure 1-B shows that the
plots were scattered in a positive direction, however, there is no clear patterned of
relations between the variables.
Exams score 90
100 200 300
Scattergram for Research Hypothesis 1—Correlation Promotional
Examination Scores versus 360-Degree Peer Evaluation for the Rank of FAO
100 200 300
Scattergram for Research Hypothesis 1—Correlations Promotional Examination
Scores versus 360-Degree Peer Evaluation for the Rank of Lieutenant
The second hypothesis of this study was to determine whether there was a
significant relationship between the scores of the years of service and the scores of the
360-degree peer evaluation. The data for the ranks of FAO and lieutenant were split to
reflect the difference in the years of service of the two ranks. Although the Pearson r
correlation was used, the variables were not significantly correlated at the .05 level.
In Table 11, the resultant correlation coefficient for the rank of FAO, was .128, p>
.05. For the rank of lieutenant, the correlation coefficient was .035, p> .05. These
correlation coefficients indicate that there was not a significant relationship between the
scores of the years of service and the scores of the 360-degree peer evaluation.
Research Hypothesis 2—Correlation Years of Service Versus 360-Degree Peer
Rank r Significance (1-tailed)
FAO .128 p> .05
Lieutenant .035 p> .05
In Figures 2-A and 2-B, the scattergram for Research Hypothesis-2 indicates that
there was no clear pattern of relationship between the scores of the years of service for
the FAO and lieutenant ranks and the scores of the 360-degree peer evaluation.
Years of Service
100 200 300
Scattergram for Research Hypothesis 2—Correlations
Years of Service versus 360-Degree Peer Evaluation for the Rank of FAO
Years of Service
100 200 300
Scattergram for Research Hypothesis 2—Correlations
Years of Service versus 360-Degree Peer Evaluation for the Rank of Lieutenant
The third hypothesis of this study was to determine whether there was a
significant relationship between the scores of the years of formal education and the scores
of the 360-degree peer evaluation. The data for the ranks of FAO and lieutenant were
split to reflect the difference in years of formal education of the two ranks. Using the
Pearson r correlation, the variables were not significantly correlated at the .05 level.
In Table 12, the resultant correlation coefficient for the rank of FAO was .079, p>
.05. For the rank of lieutenant, the correlation coefficient was -.159, p> .05. These
correlation coefficients indicate that there was no significant relationship between the
scores on the years of formal education and the scores of the 360-degree peer evaluation.
Research Hypothesis 3—Correlation Years of Formal Education Versus 360-Degree Peer
Rank r Significance (1-tailed)
FAO .079 p> .05
Lieutenant -.159 p> .05
In Figures 3-A and 3-B, the scattergram for Research Hypothesis 3 correlations
indicated that there was no clear pattern of relationship between the scores of the years of
formal education with the scores of the 360-degree peer evaluation.
Years of Education
100 200 300
Scattergram for Research Hypothesis 3—Correlations
Years of Formal Education versus 360-Degree Peer Evaluation for the Rank of FAO
Years of Education
100 200 300
Scattergram for Research Hypothesis 3—Correlations
Years of Formal Education versus 360-Degree Peer Evaluation for the Rank of
The fourth hypothesis of this study was to determine whether there was a
significant relationship between the scores of the 360-degree peer evaluation and the
combination of the scores of the three predictors that included the written promotional
examination, years of service, and years of formal education of the FAO and lieutenant
ranks. In Table 13, the results of the regression analyses indicated a multiple R of .187,
p> .05. Therefore, the hypothesis could not be rejected.
Regression Analyses of the Predictors and Criterion Variables
R F Significance
.187 2.128 p> .05
Although not listed as a hypothesis, it is worth noting that the written promotion
examination of the combined written promotional scores of the FAO and lieutenant, was
significant as a predictor of leadership in the fire service with a correlation coefficient of
r= .138, p< .05. As indicated in Table 14, the written promotion examination was the
only significant predictor of the three predictors of leadership, as rated by the scores of
the 360-degree peer evaluation. Although the written examination was not found to be a
significant predictor of leadership in the correlation analysis, it was found to be a
significant predictor of leadership in the regression analyses.
Regression Analyses of the Predictors and Criterion
Predictors r Significance
Written Promotion Examination .138 p< .05
Years of Service .072 p> .05
Years of Education .020 p> .05
Note: Both FAO and Lieutenant included in the analysis
The first three hypotheses consisted of a correlational analysis using the Pearson r
and a one-tailed t-test of significance at the .05 level for each of the FAO and lieutenant
ranks. The fourth hypothesis consisted of a regression analysis and an F-test to determine
significance with both groups combined. The null hypotheses could not be rejected.
However, the results of a regression analysis revealed that the written exam is a
significant predictor of leadership when both FAO’s and lieutenants were included in the
analysis, as rated by the scores of the 360-degree peer evaluation.
This study was undertaken in part to determine whether the leadership selection
process in the San Antonio Fire Department identified firefighters with leadership
competencies that are relevant to the fire service. Four hypotheses were examined to shed
more light on the factors that relate to leadership competencies in the fire service. The
first three hypotheses consisted of a correlational analysis using the Pearson r and a one-
tailed t-test of significance at the .05 level. The fourth hypothesis consisted of a
regression analyses and an F-test to determine significance. The null hypotheses could
not be rejected. The regression analyses indicated that the written promotion examination
is a significant predictor of leadership, as rated by the scores of the 360-degree peer
The findings of this study have led to several conclusions about the unique
population of its subjects. These conclusions will be summarized in the following section.
The first research question that was posed and answered was—Does the
leadership selection process in the San Antonio Fire Department identify firefighters with
leadership competencies that are relevant to the fire service? The answer is yes. The
current leadership selection process relies heavily on the written promotional
examination. This study has shown that the written promotional exam is a significant
predictor of effective leadership in the fire service, with a correlation coefficient of
r= .138, p< .05.
The second research question was—What are the relevant predictor criteria used
for selecting firefighters for leadership roles in the fire service? Each promotional
vacancy requires that firefighters meet certain minimum qualifications in order to be
eligible to take the written promotional exam. These qualifications are time in rank and
testing for the next rank. For example, an individual who has served in the rank of
lieutenant for at least two consecutive years is eligible to take the examination for the
next rank, which is captain. Furthermore, a firefighter must attain a score on the written
promotional exam high enough to rank he or she for selection to a leadership role.
The primary criterion for selecting firefighters for leadership roles relies heavily
on the written examination. A regression analysis was conducted on the scores of the
360-degree peer evaluation that identified leadership competencies in the fire service and
three predictor variables that included the written promotional exam scores, years of
service, and years of formal education.
As shown in Table 13, the regression analysis indicated that the written
promotional examination was the only predictor among the three that was significant,
with a correlation coefficient of r= .138, p< .05. The conclusion is that the written
promotional examination continues to be one of the best predictors of effective leadership
in the fire service.
The Written Promotional Examination As A Predictor.
The definition of ability differs from one study to another. The ability most
studied is general cognitive ability also known as general mental ability (GMA), which
measures the ability to learn (Hunter and Schmidt, 1996). A sizable amount of research
has been conducted and data collected on the value of the written examination as a
predictor and selection tool. If used correctly, the written promotional exam can be a
source of valuable information for predicting success on the job. Data on the research of
protective occupations—mainly the police and firefighters—has revealed that written
promotional examinations of supervisor candidates can predict their job-performance
evaluations when they are supervisors, with a positive correlation that ranged from .16 to
.22 (Ghiselli, 1966).
Hunter and Schmidt (1998) conducted a meta-analysis of research in personnel
selection. On the basis of the meta-analytic findings, their study presented the validity of
19 selection procedures for predicting job performance and training performance. Job-
knowledge tests were identified as one of the best predictors of job performance; their
predictive validity was .48, with a multiple R of .58, and a gain in validity of .07.
Integrity tests and employment interviews also were rated high among predictors. The
validity rating of the integrity tests was .41, with a multiple R of .65; structured
employment interviews were .51, with a multiple R of .63. Integrity tests and the
employment interviews had high percentage increases in validity (see Appendix G).
The general mental ability test (GMA)—equivalent to the fire service’s entrance
examination—as well as job knowledge (equivalent to a written promotional
examination) and a job sample, all contribute to the overall validity of the selection
process. The written promotional examination is regarded as a criterion-related measure
of critical job elements. Although the type of criterion-related performance test varies
considerably with the content of the job, it is a criterion predictor nevertheless (Field &
A common view of leadership is that of an individual who possesses the rare
characteristics, traits, and skills that render him or her uniquely qualified to ascend to
leadership, according to Kirkpatrick & Locke (2000). In this study, a 360-degree peer
evaluation measured leadership competencies such as traits. The trait theory was useful in
identifying these leadership traits, characteristics, and styles. The leadership traits that
were identified as relevant to the fire service were communication, teamwork, ethics,
technical skills, ability to lead, and self-confidence.
This study shows that the written promotional examination, which was developed
to measure leadership traits, is the best predictor of effective leadership in the fire service.
Promotion often is associated with additional leadership responsibilities; therefore,
passing the written exam with a high score is an indication of a firefighter’s desire to
become a leader. This achievement also could be an indication of the drive and cognitive
ability of the firefighter to learn the various skills and concepts of the job, as well as his
or her confidence and ability to lead and make decisions.
Years Of Service As A Predictor.
Situational leadership theory suggests that leadership styles be matched to the
maturity of the subordinates (Hersey & Blanchard, 1997). This theory primarily is a
model that classifies the subordinate’s maturity at two levels: (a) psychological maturity
and (b) job maturity. The years-of-service requirement can be viewed as a measurement
of psychological maturity that assesses the subordinate’s commitment, motivation, and
willingness to accept responsibility. Job maturity examines the subordinate’s experience,
knowledge, and understanding of the job.
The results of this study, however, indicate that the years of service is not a
significant factor in predicting leadership in the fire service. According to Hunter and
Schmidt (1998), the predictive validity for overall job performance of GMA scores
combined with a second predictor using multiple regressions indicated that job
experience (equivalent to years of service) has a validity of .18. In addition, the validity
of job experience for predicting performance declines over time (Hunter & Schmidt,
Furthermore, studies predicting performance ratings for the predictive validity of
both experience and ability indicate that the average predictive validity for experience to
be .18, whereas the predictive validity of ability is .51. Predictive validity of ability is
nearly 3 times larger than the predictive validity of experience. Not only are seniority
(years of service) advocates wrong, but the difference in validity is very strongly in the
opposite direction (Hunter & Schmidt, 1996).
Years Of Formal Education As A Predictor.
The Vroom-Yetton Normative Leadership Decision Theory dictates that the
leader’s important prerogatives is controlling the process by which decisions are made,
and that the decision affects the whole group as well as the individual team subordinates.
Formal education is viewed as enhancing the legitimacy and expertise of the leader (Daft,
1998). Additionally, educational accomplishments may be perceived as leadership
candidate’s ability to learn.
The third predictor identified was educational level, which is years of formal
education attained from an accredited educational institution. The results of this study
indicated that years of formal education were not a significant factor in predicting
leadership in the fire service. However, according to Hunter and Schmidt (1998),
predictive validity of GMA scores combined with the predictor of years of education for
overall job performance was r= .10. In addition, path analyses indicate that the major
reason that ability is able to predict performance with such accuracy is that individuals
with higher abilities learn job knowledge more quickly. (Hunter & Schmidt, 1996).
A number of limitations affected the ability for general conclusions to be drawn
from this study. The participants who were evaluated by means of the 360-degree peer
evaluation were FAOs and lieutenants—the two lowest leadership ranks in the fire
service; therefore, the results cannot necessarily be generalized to draw inferences about
the other ranks.
This study cannot be used to determine how the predictors will correspond to
leadership competencies in the upper ranks, as a firefighter moves up to the mid-
management and executive levels.
Of the 225 firefighters who were asked to participate in the study, about 80 %
agreed. No attempt was made to determine the characteristics of those who did not
participate in the study; however, it was later determined that many of them could not
participate because they were unavailable.
The three ethnic groups—Hispanic, Whites, and African American—were broad
and generalized. The ethnic composition of the participants reflects that of San Antonio.
The group comprised of males and females. A disproportionately larger number of males
than females participated.
Because the basic foundation of leadership is evident in all organizations, the 360-
degree peer evaluation survey could prove a valuable tool, if adapted appropriately for
studies of organizations outside the fire service.
The number of participants in this study also could have been increased. Although
the years-of-service requirement was not found to be a significant predictor of
leadership—with a correlation of r= .072, p >.05—it is suspected that an increase in the
number of participants could alter the results of the analysis. The results could then
indicate that the years-of-service requirement is a significant factor in predicting
leadership in the fire service.
The validation of selection measures frequently is accomplished by using
correlation and regression analyses of the criterion and predictor variables (Ree, Carreta,
& Earles, 1998). This study was conducted by using a correlation and regression analyses
of the criterion and predictor variables. The criterion variable has face and content
validity but has not been tested for other methods of validity and reliability.
Recommendations For Future Research
Several areas of potential research arose out of the need for continued studies of
leadership in the fire service. The first area would expand the study to participants in the
mid-management and executive ranks of the fire service, including captain, district fire
chief, assistant fire chief, deputy fire chief, and the fire chief. This would provide a
broader spectrum for evaluating leadership competencies of the fire service.
A second area in need of study would study what effects would leadership
training have on the operational and administrative functions of the fire service? The
study could analyze the short- and long-term effects of leadership training on the
leadership selection process. Other fire departments also could be studied in each of the
aforementioned categories in order to compare the results with this study. The data could
be useful for future studies of the fire service.
A third area in need of study would be outside the realm of the fire
service—public-safety organizations. The need for leadership training in these
organizations is on the rise, and a comprehensive study in this area is warranted.
A fourth area in need of study would be the benefits that could be derived from
use of the 360-degree peer evaluation for organizations that use job performance
examinations to predict job performance.
A fifth area in need of study would be the effect of female leadership in the fire
service. As females move up the hierarchical structure in the department, what effects
would it have on the leadership of the fire service?
Finally, study the use of the 360-degree peer evaluation for other areas of the fire
service such as Emergency Medical Service Division or Fire Prevention Division.
Agresti, A., & Finlay, B. (1986). Multiple regression and correlation. In J. Grant,
(Ed.), Statistical methods for the social science. (pp. 383-419). Englewood Cliffs, N J:
Allen, G. W. (1965). The problem of personality theory. In Harrower. Variables
in personality theory and personality testing (24-28) Springfield, Il: Charles C. Thomas.
American Psychological Association. (1980). Principles of validation and use of
personnel selection procedures. Washington, DC: American Psychological Association.
Anderson, J. W., Fiedler, F. E., & Frost, D. E. (1983). The role of personal risk-
taking in effective leadership. Human Relations, 36 (2), 185-202.
Astous, F. D. (2001). Guide to questionnaires and surveys [On-line]. Available:
Bass, B. M. (1981). Bass & Stogdill’s Handbook of leadership (2nd ed.). New
York: Free Press.
Bass, B. M. (1990). Bass & Stogdill’s Handbook of leadership (3rd ed.). New
York: Free Press.
Bass, B. M., & Yammarino, F. J. (1989). Long-term forecasting of
transformational leadership and its effects among naval officers: Some preliminary
findings (ONR Tech. Rep. No.2). Binghamton: State University of New York, Center for
Bennis, W. G. (1959). Leadership theory and administrative behavior: The
problem of authority. Administrative Science Quarterly, 8, 125-165.
Blake, R. R., & Mouton, J. S. (1964). The managerial grid: The key to leadership.
Houston: Gulf Publishing.
Blanchard, K. H., & Hersey, P. (1977). Management of organizational behavior.
Englewood Cliffs, NJ: Prentice-Hall.
Booth, W. S. (1998). Fire Lieutenant and FAO, Job Analysis Questionnaire, San
Antonio Fire Department. Parker, Colorado: Booth Research Group, Inc.
Booth, W. S. (1999, August 1). Raising the bar. Fire Chief Magazine, 43 (8), 78-
Booth, W. S. (2000). Promotional testing [On-line]. Available:
Bray, D. W. (1982, Feb). The assessment center and the study of lives. US:
American Psychological Assn., 37 (2), 180-189.
Bureau of Labor Statistics. (2000). Occupational outlook handbook: Fire-fighting
occupation (BLS Publication No. 61002A-63008B). Washington, DC: U.S. Government
Chemers, M. M. (1997). An integrated theory of leadership. Mahwah, N J:
Lawrence Eribaum Associates.
Cole, E. J., & Riggio, R. E. (1992). Agreement between subordinates and superior
ratings of supervisory performance and effects on self and subordinate job satisfaction.
Journal of Occupational & Organizational Psychology, 65 (2), 151-158.
Dewey, J. (1910). How we think. New York: D.C. Heath.
Dillman, D. A., & Salant, P. (1994). How to conduct your own survey. New
York: John Wiley & Son.
Duffe, W. C., Gebhart, G. M., & McCurley, R. A. (1998). Fire-service testing in a
litigious environment: A case history. Public Personnel Management. 27 (4), 447-458.
Dyar, J. (2000, March). Becoming a supervisor: The road to advancement. EMS
Magazine, 29 (3), 45-46.
Evans, M. G. (1970). The effects of supervisory behavior on the path-goal
relationship. Organizational Behavior and Human Performance, 5, 277-298.
Federal Register (1978). Federal Uniform Guidelines on Employee Selection
Procedures. (FR Publication No. 38290-315). Washington, DC: U.S. Government
Fiedler, F. E. (1967). A theory of leadership effectiveness. New York: McGraw
Fiedler, F. E. (1987). When to lead, when to stand back. U.S.: American
Psychological Association, 21 (9), 26-27.
Fiedler, F. E. (1996). Research on leadership selection and training: One view of
the future. Administrative Science Quarterly, 41 (2), 241-250.
Fiedler, F. E., Frost, D. E., & Anderson, J. W. (1983). The roles of personal risk-
taking in effective leadership. Human Relations, 36 (2), 185-202.
Fiedler, F. E., & Garcia, J. E. (1987). New approach to effective leadership:
Cognitive resources and organization of performance. New York: Wiley.
Fiedler, F. E., & House, R. J. (1994). Leadership theory and research: A report of
progress. In Cary L. Copper and Ivan T. Robertson (Eds.), Managerial Psychology (pp.
96-116). Chichester, U.K.: Wiley.
Field, H. S., & Gatewood, R. D. (1990). Human Resource Selection. Orlando,
FL: Dryden Publisher.
Filer, R. J., & Purdy, D.C. (1980, January). Assessment centers: A new
managerial technique. Fire Chief Magazine, 24 (1), 55-56.
Fossum, J. A. (1991). The evolution of American labor. In K. Strand & J. Hess
(Eds.), Labor Relations (pp. 22-41). Boston, MA: Irwin Publisher.
Frost, D. E. (1981). The effects of stress on leadership effectiveness. Unpublished
doctoral dissertation, University of Washington.
Ghieselli, E. E. (1966). The validity of occupational aptitude tests. New York:
Goldbach, G. (1989, March). Leadership: A review of the literature. Fire
Engineering, 142 (3), 101-110.
Goudy, K. L. (1999). The measurement equivalence of 360-degree feedback
ratings across rater populations. Dissertation Abstracts International: Section B. 59 (7-B),
3745. (University Microfilm No. 3745) The Sciences and Engineering.
Gratz, D. B. (1972). Fire department management: Scope and method (p. 199).
Beverly Hills, CA: Glencoe Press.
Haggerty, H. R., Johnson, C. C., & King, S. H. (1954). Evaluation of ratings on
combat performance of officers, obtained by mail [Abstract]. American Psychologist, 9,
Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-
peer, and peer-supervisor ratings. Personnel Psychology, 41, 43-62.
Hogan, J., Curphy, G. J., & Hogan, R. (1994). What we know about leadership:
Effectiveness and personality. American Psychologist, 49 (6), 493-504.
Hollander, E. P. (1978). Leadership dynamics: A practical guide to effective
relationships. New York: Free Press.
Hollander, E. P. (1978). Leadership: What is it? Leadership Dynamics. New
Hoover, D. (1993). Preparing fire officers for administrative responsibilities.
(Doctoral dissertation, University of Akron, 1993). UMI Dissertation Abstract Services:
54 (05-A), 1944.
Hoyt, D. P., & Pallett, J. E. (1968). College curriculum and the success in the
general business. Journal of College Personnel, 9 (4), 238-245.
Hubbard, H. F., Hunt, T., Krause, R. D. (1975). Job-related strength and agility
tests. Public Personnel Management, 4 (5), 305-310.
Hunter, J. E., & Schmidt, F. L. (1998). The validity and utility of selection
methods in personnel psychology: Practical and theoretical implications of 85 years of
research findings. Psychological Bulletin, 124 (2), 262-274.
International Association of Fire Fighters (IAFF) Local 624 (September
1999–September 2001) and the City of San Antonio, (1999, August). Collective
bargaining agreement between the City of San Antonio and Local 624 International
Association of Fire Fighters. City of San Antonio Ordinance Number 62206.
International City Management Association (1979). Managing fire service.
College Park, MD: International City Management Association.
International City Management Association (1984). Effective supervisory
practices. Washington, D.C., International City Management Association.
International Fire Service Training Association (1999). The company as a group.
In M. Sneed (Ed.), Fire department company officer (3rd ed., pp. 29-30 ). Stillwater,
Oklahoma, Fire Protection Publication.
Jenkins, W. O. (1947). A review of leadership studies with particular reference to
military problems. Psychological Bulletin, 44, 54-79.
Kane, J. S., & Lawler, E. E. (1978). Method of peer assessment. Psychological
Bulletin, 85, 555-586.
Kirkpatrick, S. A., & Locke, E. A. (1991). Leadership: Do traits matter?
Academics of Management Executive, 5 (2), 48.
Kunnecke, B. F., & Schick, G. J. (1982). Do high grades, top schools, or an
advanced degree lead to job security and extraordinary salary progression? Interfaces, 12,
London, M., & Tornow, W. W. (1998). Maximizing the value of 360-degree
feedback. San Francisco: Jossey-Bass Publisher.
Maine, D. K., & Mathis, B. J. (2000). When are firefighters truly firefighters?
Defense Counsel Journal, 67 (2), 260-262.
Maxfield, J. (1996, Jan). Ethics and example. American Fire Journal, 48 (1), 28-
McNamara, W. J, & Wollowick, H. B. (1969). Relationship of the components of
an assessment center to management success. Journal of Applied Psychology, 53, 348-
Miller, R. (1997). Texas fire and police civil service law: Chapter 143. Boerne,
TX: Barrister Publishing.
Organ, D. W., & Podsakoff, P. M. (1986). Self-serving biases in the attribution of
causality: Fact of friction? Journal of Management, 12, 531-544.
Ree, M. J. (1995). Nine rules for doing ability research wrong. The Industrial
Organizational Psychologist, 32, 64-68.
Ree, M. J., Carreta, T. R., & Earles, J. A. (1999). In validation, sometimes two
sexes are one too many: A tutorial. Human Performance, 72 (1), 79-88.
Shartle, C. L. (1950). Studies of leadership by interdisciplinary methods. In A. G.
Grace (Ed.), Leadership in American education. Chicago: University of Chicago.
Shores, J. A. (1984). Validation of a civil service selection procedure for
firefighters. Doctoral dissertation, California State University, Long Beach.
Society of Human Resource Management. (1998). Collective bargaining. [On-
line]. Available: www.shrm.org/whitepapers/default.asp?page=wplist.html
Stogdill, R. M. (1948). Personal factors associated with leadership: A survey of
the literature. Journal of Psychology, 25, 35-71.
Stogdill, R. M. (1974). Cited in Commonwealth of Australia. National Fire
Academy, pp. 7.
United States Census Bureau (2000). United States Census 2000 [On-line].
United States Fire Administration. (2000, July). NFDC statistics report (USFA
Data Center). Washington, DC: U.S. Government Printing Office.
Vroom, V. (1964). Work and motivation. New York: John Wiley and Sons.
13 Practical Skills For the FAO
Communication and Interaction
Public Education & Community Service
Search and Rescue
Fire Inspection and Compliance
Source: Booth Research Group (1998)
18 Performance Dimensions for the Lieutenant
Effectiveness Under Stress
Personnel Resource Management
Source: Booth Research Group (1998)
SURVEY OF EFFECTIVE LEADERSHIP BEHAVIOR PATTERNS
OF FIRE SERVICE LEADER
Name: _________________ Date: __________________
Below is a list of leadership traits. Please check the behavior patterns that you feel are
important to be an effective leader in the fire service.
----Technical fire fighting skills ----Social nearness, friendliness
----Task motivation and application ----Supportive of group task
----Social and interpersonal skills ----Emotional balance and control
----Leadership effectiveness and achievement ----Administrative skills
----General impression ----Intellectual skills
----Dominance and decisiveness ----Willingness to assume
----Ethical conduct ----Personal Integrity
----Maintaining a cohesive work group ----Ability to communicate;
----Physical energy ----Maintaining standards of
----Creative, independent ----Conforming
----Courageous, daring ----Experience and activity
----Nurturing behavior ----Maintaining informal control of
----Mature, cultured ----Aloof, distant
----Ability to take risk ----Independent
----Self-confidence ----Union affiliation
----Promotes tradition ----Promotes diversity
----Physically Fit ----Aggressive fire fighting
----Promote teamwork ----Disciplinarian
----Tolerance of stress ----Education
----Visionary ----One of the troops
Examples of Good Behavior
Examples of Bad Behavior
Five Lieutenant Competencies Identified by the Booth Research Group
Overall Competencies Ratings
Emergency Scene Management
Source: Booth Research Group (1998)
Six Firefighters Leadership Competencies Identified by the Rogers’ Firefighters Survey
Firefighters Leadership Competencies
Professional & Technical Skills
Source: Rogers’ Firefighters Survey (2001)
FIRE SERVICE LEADERSHIP
360 Degree Peer Evaluation Survey
Individual Rated Identification No._____________
Evaluator Identification No._______________
What is your relationship to the individual you are rating (please check one from the
1 Captain “I am evaluating the Lieutenant.”
2 Captain “I am evaluating the FAO.”
3 Lieutenant “I am evaluating a colleague (Lieutenant) within my department.”
4 Lieutenant “I am evaluating the FAO.”
5 FAO “I am evaluating the Lieutenant.”
6 FAO “I am evaluating a colleague (FAO) within my department.”
7 Firefighter “I am evaluating the Lieutenant.”
8 Firefighter “I am evaluating the FAO.”
You have been identified as one of a number of individuals who can provide valuable
information regarding the leadership performance of the rank of FAO and Lieutenant.
Your individual responses will remain anonymous.
Rating Scale (1-5)
How well does this person perform this competency? Please use the following scale for
5 An Exceptional Skill: This individual consistently exceeds behavior and skills expectations
in this area.
4 A Strength: The individual meets most and exceeds some of the behavior and skills
expectations in this area.
3 Appropriate Skill Level: The individual meets a majority of the behavior and skills
expectations in this area for this job. There is generally a positive perspective towards
2 Not a Strength: The individual meets some behavior and skills expectations in this area but
sometimes falls short.
1 Least Skilled: The individual consistently fails to reach behavior and skills expectations in
Please use the above scale to rate the FAO or Lieutenant:
1. Ability to convey one’s thoughts both written and verbally. 1 2 3 4 5
2. Effectively communicates with officer/incident commander 1 2 3 4 5
3. Persuasive and articulate when communicating. 1 2 3 4 5
4. Knowledge of communication tactics and techniques. 1 2 3 4 5
5. Listens and respond to verbal orders at the fire station & 1 2 3 4 5
6. Verbally communicates suggestions on tactics at the 1 2 3 4 5
incident of scene.
7. Communicates during critique meetings. 1 2 3 4 5
8. Supports team goals by facilitating a cooperative spirit 1 2 3 4 5
9. Puts interest of team ahead of self. 1 2 3 4 5
10. Builds consensus and shares relevant information. 1 2 3 4 5
11. Recognizes and respects the contributions and needs of 1 2 3 4 5
12. Actively seeks involvement/uses input from people with 1 2 3 4 5
13. Builds and maintains productive working relationships. 1 2 3 4 5
14. Facilitates trust, honesty, openness, and integrity. 1 2 3 4 5
15. Practices ethical behaviors, for example with dealing with 1 2 3 4 5
personnel and the public.
16. Promotes departmental ethical values. 1 2 3 4 5
17. Supports equality. 1 2 3 4 5
18. Supports and complies with departmental rules and 1 2 3 4 5
19. Enforces departmental rules and policies. 1 2 3 4 5
20. Reports unethical behavior to chain-of-command. 1 2 3 4 5
21. Ethical in decision making. 1 2 3 4 5
22. Compiles staffing reports. 1 2 3 4 5
23. Transmits orders and information to others. 1 2 3 4 5
24. Maintains various inventories, records, and logbooks. 1 2 3 4 5
25. Carries out special administrative duties as assigned by 1 2 3 4 5
26. Inspects personal protected gear and various other 1 2 3 4 5
27. Assigns and directs the cleaning of quarters and equipment. 1 2 3 4 5
28. Conducts roll call. 1 2 3 4 5
EMERGENCY SCENE MANAGEMENT
29. Assesses the condition at a scene (size-up the situation). 1 2 3 4 5
30. Determines tactical priorities and strategies based on size-up. 1 2 3 4 5
31. Maintains scene accountability and safety of crew. 1 2 3 4 5
32. Establishes a command post at an emergency scene. 1 2 3 4 5
33 Ensures that personnel operate under the Incident Command 1 2 3 4 5
34. Determines the necessary equipment and apparatus for the 1 2 3 4 5
level of the emergency situation.
35. Establish effective channels of communication at the scene. 1 2 3 4 5
36. Acquires and maintains all required licenses and 1 2 3 4 5
certifications for assignments.
37. Maintains knowledge of the latest fire fighting equipment and 1 2 3 4 5
38. Maintains knowledge of building structures related to fire 1 2 3 4 5
39. Maintains knowledge of chemicals or other hazardous 1 2 3 4 5
40. Maintains knowledge of the geographical area. 1 2 3 4 5
41. Maintains physical fitness. 1 2 3 4 5
42. Continues with formal education. 1 2 3 4 5
43. Controls and assists the public at emergency scenes. 1 2 3 4 5
44. Deals with stressful individuals at the scene. 1 2 3 4 5
45. Meets with property owners to discuss fire prevention and 1 2 3 4 5
46. Conduct’s tours and provides information to the public. 1 2 3 4 5
47. Provides information to civilians on service runs. 1 2 3 4 5
48. Provides formal public education presentations. 1 2 3 4 5
49. Provides information to the public regarding fire codes and 1 2 3 4 5
50. Ability to make tactical judgments under emergency 1 2 3 4 5
51. Firm knowledge of equipment and its use. 1 2 3 4 5
52. Ability to request for assistance when needed. 1 2 3 4 5
53. Confident in making decisions under emergency conditions. 1 2 3 4 5
54. Confident in managing personnel at the fire station and 1 2 3 4 5
55. Confident and emotionally mature in handling 1 2 3 4 5
responsibilities as a leader.
56. Confident in their ability to take risks. 1 2 3 4 5
57. Confident in assuming additional responsibilities. 1 2 3 4 5
Predictive Validity for Overall Job Performance of General Mental Ability (GMA)
Scores Combined With a Second Predictor Using (Standardized) Multiple Regression
Personnel Validity Multiple adding in
measures r R supplement validity GMA Supplement
GMA Tests .51
Work Sample .54 .63 .12 24% .36 .41
Integrity Tests .41 .65 .14 27% .51 .41
Conscientiousness .31 .60 .09 18% .51 .31
Interviews .51 .63 .12 24% .39 .39
Interviews .38 .55 .04 8% .43 .22
Job Knowledge .48 .58 .07 14% .36 .31
Job Tryout .44 .58 .07 14% .40 .20
Peer Ratings .49 .58 .07 14% .35 .31
T & E Behavioral .45 .58 .07 14% .39 .31
Reference checks .26 .57 .06 12% .51 .26
Job experience .18 .54 .03 6% .51 .18
Biographical data .35 .52 .01 2% .45 .13
Assessment .37 .53 .02 4% .43 .15
T & E point .11 .52 .01 2% .39 .29
Years of .10 .52 .01 2% .51 .10
Interest .10 .52 .01 2% .51 .10
Graphology .02 .51 .00 0% .51 .02
Age -.01 .51 .00 0% .51 -.01
Source: Schmidt and Hunter (1998)
July 23, 2000
Brother Anthony Rogers
Member of Local 624
San Antonio, TX 78250
You requested permission from the local to conduct a survey of our union members
concerning leadership in the fire service. It is my understanding that your survey
methods will consist of a mail questionnaire and structured interviews. The participant’s
identity will remain confidential. A copy of the results of the survey will be provided to
the local. You also informed me that participation in the survey is voluntary and at no
expense to the local.
It is the position of Local 624 to encourage our members to pursue their educational goals
through higher learning. Your efforts as a Ph.D. candidate, with the Center for
Leadership Studies at Our Lady of the Lake University, are to be applauded.
Your request is approved. Good luck in your educational goals. If we can be of further
assistance, please do not hesitate to contact us.
President Local 624
Subject Consent To Take Part In A Quantitative Study
Of The Leadership Selection Process
In The Fire Service
Our Lady of the Lake University at Which This Study Is Conducted
Thank you for agreeing to participate in this study. Your response is very important to us. It will help us
better understand the leadership selection process in the fire service. The questionnaire you will complete
is a 360-degree peer evaluation where you will be asked to rate either a fire apparatus operator or lieutenant
regarding their leadership performance. It should take no more that 40-50 minutes to complete.
You will be asked to provide some background information about yourself, such as your name and
relationship to the individual you are rating. Your identification will remain completely anonymous. Your
name will not be tied in any way to the questionnaire.
The only risk that you will be subjected to in this study will be disclosure of your background information.
You will not incur any financial costs because of participating in this study. You will not receive any
reimbursement because of electing to participate.
Your participation in this study is voluntary. You are free to withdraw from participating at any time. You
will suffer no negative consequences of any kind if you choose to withdraw.
You may contact the Institutional Review Board, at Our Lady of the Lake University (210) 434-6711 if you
have any questions about this study.
You will receive a signed copy of this consent form for your records.
If you would like to see the results of this study after its completion, please check the box at the bottom of
the page and provide a mailing address.
YOUR SIGNATURE ON THIS CONSENT FORM INDICATES THAT YOU HAVE DECIDED TO
TAKE PART IN THIS RESEARCH STUDY AND THAT YOU HAVE READ AND UNDERSTAND
THE INFORMATION GIVEN ABOVE AND EXPLAINED TO YOU.
Signature of Participant Signature of Witness
Date/Time Signature of OLLU Faculty Member
Signature of Student Researcher
Check box if you wish to have the results of this study mailed to
you. Include a mailing address:
Copyright Anthony Rogers 2001
All Rights Reserved