A 360-DEGREE PEER EVALUATION OF LEADERSHIP COMPETENCIES
By
Anthony Rogers, MBA
DISSERTATION
Presented to the Faculty of th...
2
Abstract
This study examined a model of leadership in the fire service and evaluated the selection
process of two of the...
3
ACKNOWLEDGEMENTS
I wish first to thank God for the strength, courage, and wisdom with which He
equipped me to complete m...
4
TABLE OF CONTENTS
ABSTRACT ……………..……………………………………………………….. ii
ACKNOWLEDGEMENTS …….………………………………………………….iii
TABLE OF CONTEN...
5
METHODOLGY
Research Design …………………………………………………………... 41
Criterion and Predictor Variables …………………………………………. 41
Null Hypo...
6
Appendix E-Six Firefighters Leadership Competencies Identified
by the Rogers Survey ………………………………………….……….. 80
Appendix F...
7
CHAPTER ONE
INTRODUCTION
In today’s dangerous world, 9-1-1 fire and medical emergencies occur every five
seconds. Fire k...
8
scores in all phases of testing have the best chance of receiving an appointment to the fire
department (Miller, 1997).
...
9
are based on seniority and the results of a multiple-choice, written examination. The steps
to leadership begin with pro...
10
the knowledge, skills, abilities, and other characteristics (KSAOs) needed for effective
performance in the San Antonio...
11
identified (see Appendix B). The five competencies include (1) supervision, (2) general
administration, (3) personal de...
12
taking would be management’s critical decision on whether to close a fire station,
possibly leaving the area that the s...
13
Problem Statement
There are extensive literature studies on measuring leadership skills and
identifying leadership beha...
14
1. Determine whether the present leadership selection process was identifying
firefighters who have leadership competen...
15
education is a predictive factor of an effective leader. Progressive fire departments
mandate that personnel selected f...
16
the legal authority and responsibility to engage in fire suppression; and is employed by a
fire department of a municip...
17
3. Because of their work and personal relationships with their co-workers,
participants may exhibit bias in answering t...
18
CHAPTER TWO
LITERATURE REVIEW
Theoretical Literature On Leadership
A review of the current literature on leadership sel...
19
want done because he wants to do it” (as cited by Gratza, 1972, p 199). “A definition of
leadership that would be widel...
20
situation. The Great Man Theory is a method used to select individuals who are perceived
to be great leaders to transfo...
21
weakness of the trait theory is that traits often are confused for being skills. A skill is a
technical ability, knowle...
22
situational favorableness, which is determined by leadership-member relations, the task
structure, and the position of ...
23
where additional information is obtained from the subordinates but the leader makes the
decision alone, (c) consultativ...
24
The trait theory is important in identifying individuals who have personal traits that could
lead to effective leadersh...
25
appears significant for promotion to FAO; however, their use is insignificant for
promotion to the higher ranks (Booth,...
26
Table 2
Minimum Educational Requirement
Education FAO Lieutenant Captain Battalion Chief Assistant Chief
High School 72...
27
Table 3
Average Time In Grade For Test Eligibility
Rank Modal time in grade Average time in
grade
Modal time in
departm...
28
which the examination questions will be taken. Applicants must earn a minimum raw
score of 70 points to pass the examin...
29
Section 1 of the contract identifies eligibility for promotion in addition to
granting seniority points, for a maximum ...
30
knowledge and skills. The Booth Research Group developed the test questions from job
analyses of the positions for whic...
31
the City of San Antonio and Local 624 of the International Association of Firefighters
(1999, pg. 53). For example, a f...
32
The director of the Civil Service Commission is responsible for preparation and
security for each promotional exam (Mil...
33
instrument. The Booth Research Group conducted job analyses on the FAO and
lieutenant positions.
The Booth Research Gro...
34
According to Field & Gatewood (1990), the types of validity strategies involve:
1. Criterion validity, which includes c...
35
Leadership Prediction Studies
Remarkable strides have been made in developing reliable, valid tests for
selecting non-m...
36
the subjects were rated again after facing the challenge of increases in management
responsibilities within three years...
37
ratings at West Point and the success rating of the same subjects as infantry officers 18
months later (Haggerty, Johns...
38
predicting performance ratings for predictive validity of both experience and ability
indicates that the average predic...
39
Methods Of Verification And Measurement Of Leadership In The Fire Service
Psychologists know that theories of leadershi...
40
The City of San Antonio hired the Booth Research Group (1998) to develop a job
analysis questionnaire to assess the kno...
41
agility, (c) personal attributes, (d) communication skills, (e) problem-solving, and (f)
spatial abilities.
The Nationa...
42
their limitations, written tests are a source of valuable information about the probability
of the individual performin...
43
service. In the general population, the correlation between education and ability is about
.55 (Hunter & Schmidt, 1998)...
44
between superiors (chiefs) and subordinates (firefighters) in rating the performance of the
captains. These performance...
45
African-American. Although this city had a population that was 40% African-American,
no African-American firefighter ha...
46
Employment Opportunity (EEO) laws. The guidelines also provide employers with one
set of government requirements to mee...
47
CHAPTER THREE
METHODOLGY
This chapter presents the research design that identifies the four variables used,
states the ...
48
and lieutenant, and were ranked on a scale of 1 to 5. Once the scores were collected, a
factor analysis was conducted o...
49
Participants came from the Fire Suppression Division only; the other divisions
were excluded from the study because the...
50
The 360-degree peer evaluation was created from two sources. The first source
was the Booth Research Group (1998), whic...
51
skills in this area. A rating of 1 reflects low skills; the individual consistently fails to
reach expectations of beha...
52
Data Analysis
The Pearson r statistical index was used to describe the degree of strength and
direction of relationship...
53
CHAPTER FOUR
RESULTS
The purpose of this study was to determine whether the present leadership
selection process in the...
54
White, and 8.8% (N=16) were African-American. The ethnic composition of San
Antonio’s 1.1 million people is 58.7% Hispa...
55
46–50 years 19 27 46
51–55 years 42 4 46
55–59 years 5 6 11
Mean 46.95 44.5 45.92
Sex.
The San Antonio Fire Department’...
56
Table 7
Descriptive Statistics of Participant Examination Scores
Exam Scores FAO Lieutenant Total
71–75 18 0 18
76–80 2...
57
Table 8
Descriptive Statistics of Participant Years of Service
Years of Service FAO Lieutenant Total
01–10 years 14 1 1...
58
Table 9
Descriptive Statistics of Participant Years of Formal Education
Years of Education FAO Lieutenant Total
12 year...
59
used, the variables were not significantly correlated at the .05 level.
In Table 10, the resultant correlation coeffici...
60
300200100
Examsscore
100
90
80
70
Figure 1-A
Scattergram for Research Hypothesis 1—Correlation Promotional
Examination ...
61
Hypothesis 2.
The second hypothesis of this study was to determine whether there was a
significant relationship between...
62
300200100
YearsofService
40
30
20
10
0
Figure 2-A
Scattergram for Research Hypothesis 2—Correlations
Years of Service v...
63
Hypothesis 3.
The third hypothesis of this study was to determine whether there was a
significant relationship between ...
64
300200100
YearsofEducation
19
18
17
16
15
14
13
12
11
Figure 3-A
Scattergram for Research Hypothesis 3—Correlations
Yea...
65
Scattergram for Research Hypothesis 3—Correlations
Years of Formal Education versus 360-Degree Peer Evaluation for the ...
66
Table 14
Regression Analyses of the Predictors and Criterion
Predictors r Significance
Written Promotion Examination .1...
67
CHAPTER FIVE
DISCUSSION
This study was undertaken in part to determine whether the leadership selection
process in the ...
68
The second research question was—What are the relevant predictor criteria used
for selecting firefighters for leadershi...
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Anthony .d
Upcoming SlideShare
Loading in …5
×

Anthony .d

1,081
-1

Published on

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,081
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Anthony .d

  1. 1. A 360-DEGREE PEER EVALUATION OF LEADERSHIP COMPETENCIES By Anthony Rogers, MBA DISSERTATION Presented to the Faculty of the Department of Center for Leadership Studies Our Lady of the Lake University, in Partial Fulfillment of the Requirements For the degree of Doctor of Philosophy in Leadership Studies Our Lady of the Lake University San Antonio, Texas July 31, 2001
  2. 2. 2 Abstract This study examined a model of leadership in the fire service and evaluated the selection process of two of the seven leadership roles in the San Antonio Fire Department—fire apparatus operator (FAO) and lieutenant. The selection process is based primarily on the results of a written examination that ranks firefighters on their raw test scores and number of years of experience, which is referred to as seniority points. Although there are numerous studies on leadership theories and behavior patterns, the amount of literature available on the leadership selection process—especially in the fire service—is severely limited. The purposes of this study were to: (1) determine whether the leadership selection process in the San Antonio Fire Department is identifying firefighters with leadership competencies as identified by the Booth Research Group (1998) and the Rogers’ Firefighters Survey (2001) conducted on firefighters to determine leadership competencies relevant to the fire service, and (2) suggest relevant criteria for selecting firefighters for leadership roles in the fire service. A 360-degree peer evaluation that contains 57 questions that are used as the criterion variable was used to rate the leadership performance of FAOs and lieutenants. Data collected were correlated to determine whether the leadership selection process is related to selecting firefighters with leadership competencies. To determine the degree of relationship, four variables—a criterion and three predictors that include scores on the written promotional examination, years of service, years of formal education, and scores on the 360-degree peer evaluation—were correlated.
  3. 3. 3 ACKNOWLEDGEMENTS I wish first to thank God for the strength, courage, and wisdom with which He equipped me to complete my dissertation and the lengthy journey toward a doctor’s degree. I pray that the Lord will enable me to use the knowledge and wisdom I have attained to better serve and honor Him. I also wish to thank my children, Catherine and Amanda, who sacrificed their quality time with me so that I could complete my studies. I want to thank my fellow firefighters who place their lives on the line 24 hours a day. They have supported me and cheered me on during the journey to complete my Ph.D. Furthermore, I extend my gratitude to my chair, Dr. Malcolm J. Ree, and the committee members for their support and encouragement. I want to thank Sally for her continuous support. Above all, I want to thank my devoted wife, Patsy Mae, without whose constant encouragement and support I never could have completed the program. I am eternally grateful.
  4. 4. 4 TABLE OF CONTENTS ABSTRACT ……………..……………………………………………………….. ii ACKNOWLEDGEMENTS …….………………………………………………….iii TABLE OF CONTENTS………………………………………………………….. iv INTRODUCTION Research Questions..………………………………………………….…… 6 Problem Statement….……………………………………………………… 7 Purpose of This Study..……………………………………………………. 7 Hypotheses.………………………………………………………………… 8 Definitions………………………………………………………………….. 9 Variables…………………………………………………………………… 10 Assumptions……………………………………………………………….. 10 Limitations………………………………………………………………….10 Significance…………………………………………………………………11 LITERATURE REVIEW Theoretical Literature on Leadership ……………………………………... 12 Definitions of Leadership …………………………………………………. 12 Models and Theories of Leadership ………………………………………. 13 Leadership Selection Process in the Fire Service ………………………… 18 Collective Bargaining Contract Agreement ………………………………. 22 Promotional Examination Process ………………………………………....23 Leadership Prediction Studies………………………………………………29 Methods of Verification and Measurement of Leadership In the Fire Service………………………………………….. 33 Summary…………………………………………………………………… 38
  5. 5. 5 METHODOLGY Research Design …………………………………………………………... 41 Criterion and Predictor Variables …………………………………………. 41 Null Hypotheses ……………………………………………………………42 The Participants ………………………………………………………….. 42 The Instrument …………………………………………………………... 43 Data Collection …………………………………………………………… 45 Data Analysis ……………………………………………………………... 46 Ethical Decisions ………………………………………………………….. 46 RESULTS Descriptive Statistics of Participants ……………………………………… 47 Hypotheses ………………………………………………………………... 52 Summary……………………………………………………………………60 DISCUSSION Conclusions ……………………………………………………………….. 61 Discussion ………………………………………………………………… 62 Recommendations ………………………………………………………… 67 REFERENCES ……………………………………………………………………. 69 APPENDIXES Appendix A—FAO Practical Skills ……………………………………… 76 Appendix B—Lieutenant Performance Dimensions ……………………… 77 Appendix C—Rogers Firefighters Survey …………………………………78 Appendix D—Five Competencies Identified by the Booth Research Group ………………………………………….. 79
  6. 6. 6 Appendix E-Six Firefighters Leadership Competencies Identified by the Rogers Survey ………………………………………….……….. 80 Appendix F—360-Degree Peer Evaluation ……………………………….. 81 Appendix G-Hunter and Schmidt Predictive Validity Table ………………………………………………………….. 85 Appendix H-Letter Granting Permission To Conduct Study ……………………………………………………… 86 Appendix I-Consent Form ………………………………………………... 87
  7. 7. 7 CHAPTER ONE INTRODUCTION In today’s dangerous world, 9-1-1 fire and medical emergencies occur every five seconds. Fire kills thousands of people each year, injures hundreds of thousands of others, destroys billions of dollars in property, and costs tens of billions of dollars (USFA, 1999). Firefighters help protect the public against these dangers by quickly responding to a variety of emergencies. Frequently they are the first emergency personnel at the scene of an accident or medical emergency and may be called upon to put out a fire, treat injuries, rescue trapped victims, or perform other vital functions (Bureau of Labor Statistics, 2000). The fire service is facing a growing spectrum of job responsibilities and the need for effective leadership is even greater than in the past. If the need for effective leadership in the fire service is growing, what might be done to select firefighters with effective leadership competencies? What are the leadership competencies relevant to the fire service? What selection process would accurately measure or validate the need for leadership competencies? Part of the answer is to look at the selection and promotion process and compare it with the leadership competencies needed in the fire service, as identified by the Booth Research Group (1998) and the Rogers’ Firefighters Survey (2001). Applicants for municipal fire-fighting jobs generally must pass a written exam; tests of strength, physical stamina, coordination, and agility; and a medical examination. Entry-level examinations generally are open to individuals who are at least 19 years of age and have a high school education or the equivalent. Those who receive the highest
  8. 8. 8 scores in all phases of testing have the best chance of receiving an appointment to the fire department (Miller, 1997). As a rule, entry-level cadets in large fire departments are trained for several weeks at the department’s training center or academy. The recruits study fire-fighting techniques, fire prevention, hazardous-materials control, local building codes, and emergency medical procedures that include first aid and cardiopulmonary resuscitation (CPR). They also learn how to use axes, chain saws, fire extinguishers, ladders, and other fire-fighting equipment and rescue equipment. After successfully completing this course of study, they are assigned to a fire company where they undergo a one-year probationary period of on-the-job training (Bureau of Labor Statistics, 2000). During this time the probationary firefighter is assigned to a fire station where he or she will work directly under the supervision of the company officer and the other senior firefighters assigned to the company. Throughout this period, the probationary firefighter is exposed to all the elements of the fire service, thereby gaining valuable experience. Upon completion of the probationary period, the probationary firefighter is upgraded to the classification of firefighter (IAFF Local 624, 1999). According to the International City Management Association (1979), leadership in the fire service is a critical role because life-saving decisions often are dependent on the leader’s ability to act decisively and make sound decisions on the fire ground as well as in administrative situations. Traditionally, firefighters were selected for leadership roles based on job seniority as well as their ability to fight fire by frequently withstanding a smoky inferno without the protection of a breathing apparatus; hence, the term smoke eater was coined. Currently, opportunities for promotion to leadership in the fire service
  9. 9. 9 are based on seniority and the results of a multiple-choice, written examination. The steps to leadership begin with promotion to fire apparatus operator (FAO), and move up the hierarchy to lieutenant, captain, battalion chief, assistant chief, deputy chief, and finally, fire chief. Fire and police municipal civil service laws regulate the classification and appointment of firefighters to leadership roles. Chapter 143 of the Texas Local Government Codes provides specific guidelines regarding the entrance examination as well as the written promotional examination process for municipal fire service (Miller, 1997). In addition, collective bargaining agreements between firefighters unions and municipal governments also regulate the leadership promotion process in the fire service. Collective bargaining agreements often describe the eligibility criteria, seniority points, study material, written examination, assessment-center process, assessment-center promotion dispute resolution procedures, probationary period, appointments to assistant chief and deputy chief, and the mechanics of posting promotional examination results (IAFF Local 624, 1999). According to the Texas Local Government Codes (Miller, 1987), the leadership selection process in the fire service in Texas begins with a vacancy. When a vacancy occurs, a written multiple-choice promotion examination is administered. On November 1 of each year, the San Antonio Fire Department’s fire chief establishes a committee(s) to select study material for examinees to use in preparing for the written promotion examination. The fire chief makes the final selection of the study material, subject to approval by the Civil Service Commission (IAFF Local 624, 1999). Exam questions are formulated from the study material, which is based on a job analysis designed to assess
  10. 10. 10 the knowledge, skills, abilities, and other characteristics (KSAOs) needed for effective performance in the San Antonio Fire Department (Booth Research Group, 1998). The names of examinees whose scores meet the minimum score required for promotion consideration are placed on an eligibility list. The Booth Research Group (2000) is a consulting firm that develops written promotional examinations for hundreds of positions in various organizations, including the fire service. Their written examinations are customized to reflect the organizational needs of their clients. This firm also develops the protocols and procedures for the examination process. The Booth Research Group was hired by the City of San Antonio to develop a valid, multiple-choice written promotional examination for the positions of FAO and lieutenant. Additionally, a questionnaire designed to assess the knowledge, skills, abilities, and other characteristics needed for performance as an FAO and a lieutenant in the San Antonio Fire Department was developed to analyze the positions. From analysis of these positions, practical skills and competencies were identified (Booth Research Group, 1998). For the position of FAO, 13 practical skills were identified (see Appendix A). These skills include; (1) fire inspection and compliance, (2) water pump operation, (3) apparatus driving, (4) medical responsiveness, (5) fire-fighting, (6) training, (7) hazardous materials identification, (8) communication and interaction, (9) public education and community service, (10) search and rescue abilities, (11) performance of station duties, (12) ability to perform inspections, and (13) self-development. For the lieutenant position, five competencies and 18 different performance dimensions were
  11. 11. 11 identified (see Appendix B). The five competencies include (1) supervision, (2) general administration, (3) personal development, (4) emergency scene management, and (5) community interaction. The 18 performance dimensions include; (1) conflict resolution, (2) creativity, (3) customer service, (4) decision- making, (5) effectiveness under stress, (6) independent thinking, (7) interpersonal skills, (8) job knowledge, (9) leadership, (10) oral communication, (11) organization and planning, (12) personnel management, (13) persuasiveness, (14) problem- solving, (15) sensitivity toward diverse populations, (16) supervisory skills, (17) teamwork, and (18) written skills (Booth, 1998). A pilot study of the Rogers’ Firefighters Survey (2001) was conducted by the researcher and writer of this dissertation (see Appendix C). The purpose was to gather data regarding effective leadership competencies relevant to the fire service, as identified by 35 firefighters who work in the firefighting division. The survey identified six competencies that are considered relevant to the fire service (see Appendix D). The six competencies include; (1) technical skill, (2) self-confidence, (3) communication, (4) ethical behavior, (5) interpersonal skills, (6) and emotional balance. The performance dimensions from which the competencies were identified are based on the role and responsibilities of the position. To gain an understanding of the requirements of the position, information was collected on several levels, including conducting site visits and interviews; examining documentation on the position; and distributing, collecting, and analyzing a questionnaire (Booth Research Group, 1998). A built-in hazard of service as a firefighter is regular exposure to the dangers of physical injury or death. Risk-taking is a common competency that firefighters often are exposed to operationally as well as administratively. An example of administrative risk-
  12. 12. 12 taking would be management’s critical decision on whether to close a fire station, possibly leaving the area that the station serves farther away from rescue and fire-fighting services. However, bravery and risk-taking, were not identified by the Booth Research Group (1998) or the Rogers’ Firefighters Survey (2001) as relevant competencies. In general, firefighters tend to suppress the fact that their occupation is dangerous; this probably was the reason that it was not identified as one of the relevant competencies. It is reasonable to hypothesize that risk-taking behaviors will be similar to other previously identified predictor variables, and that leadership effectiveness in dangerous environments may then be conceptually and empirically linked to organizational theory (Anderson, Fiedler, & Frost, 1983). In the past, a good fire department was measured by its ability to extinguish a fire in a reasonable amount of time and provide the community with the most appropriate level of fire protection, based on community resources. Today the fire service is faced with growing job responsibilities, and the efficient operation of a municipal fire department mandates that fire- service leaders be competent in administrative, operational, and leadership skills (Hoover, 1993). Research Questions The following research questions were answered: 1. Does the leadership selection process in the San Antonio Fire Department identify firefighters with leadership competencies relevant to the fire service? 2. What are the relevant predictors used as criteria for selecting firefighters for leadership in the fire service?
  13. 13. 13 Problem Statement There are extensive literature studies on measuring leadership skills and identifying leadership behavior patterns (Bass, 1990). Currently, however, there are no studies that evaluate the leadership selection process in the fire service. Rationale For The Hypotheses. This study is intended to increase our knowledge of leadership competencies from the perspective of subordinates, peers, and supervisors in the fire service. In particular, the researcher and writer of this dissertation combined the leadership competencies identified by the Booth Research Group (1998) with those identified by the Rogers’ Firefighters Survey (2001) to create a 360-degree peer evaluation (see Appendix E). The results of the 360-degree peer evaluation will be used as a criterion variable to determine whether written promotion examination scores, years of service, and years of formal education correlate with effective leadership in the fire service. In the past, fire departments have been spared the scrutiny of accountability. City government, as well as the citizens, are now holding the fire service accountable for effective and efficient operations of the department (Hoover, 1993). They expect a fire service leader to be proficient in fire-ground tactics, computer technology, and various leadership skills that include transformational and transactional leadership. The citizens expect fire service leaders to be innovative and aggressive in handling the fire service problems that plague their community (Hoover, 1993). Purpose Of This Study The major intents of this study were to:
  14. 14. 14 1. Determine whether the present leadership selection process was identifying firefighters who have leadership competencies as identified by the Booth Research Group (1998) and the Rogers’ Firefighters Survey (2001), which was conducted on 35 San Antonio firefighters regarding leadership competencies relevant to the fire service. 2. Suggest relevant predictors for selecting firefighters for leadership roles in the fire service. Hypotheses The statistical null hypotheses are stated in chapter 3. The research hypotheses follow. Hypothesis 1: A score on the written promotional examination is predictive of effective leadership competencies, as rated by the 360-degree peer evaluation. In accordance with Chapter 143 of the Texas Local Government Codes and collective bargaining agreements, the firefighters who earn the highest raw score on the promotional written examination and have adequate seniority points are promoted to vacant leadership roles in the fire service. Currently there are no data to support a conclusion that test scores are predictive of effective leadership. Hypothesis 2: Years of service is predictive of effective leadership competencies, as rated by the 360-degree peer evaluation. The two variables used for selection into leadership roles are the raw test score on the written promotion examination and seniority points given for each year of service—a maximum of 10 points. Hypothesis 3: Years of formal education is predictive of effective leadership competencies, as rated by the 360-degree peer evaluation. According to the data,
  15. 15. 15 education is a predictive factor of an effective leader. Progressive fire departments mandate that personnel selected for the positions of battalion chief and above be required to earn a master’s degree within three years of appointment. Definitions The fire service’s hierarchical chain of command has seven leadership roles, beginning with the FAO and ending with the fire chief. This study evaluates the two lowest leadership roles only—the FAO and lieutenant positions—which represent a large cross section of subjects involved in the study. Each position in the fire service is classified and based on the job description. Seniority is defined as total years of service in the San Antonio Fire Department, interrupted or uninterrupted, and not merely the first continuous period of service. The term firefighter refers to volunteer firefighters; paid, on-call firefighters; and career firefighters. This study examines the career firefighter category only. Career firefighters are assigned predominantly to municipal fire departments, while the majority of paid, on- call firefighters and volunteer firefighters work in the rural areas. Firefighters are public servants who provide emergency and non-emergency fire services. A collective bargaining agreement is a binding contractual agreement between the firefighters union and the municipal government that describes various terms of employment (IAFF Local 624, 1999). According to the Wage and Hour Division of the Department of Labor (Mathis & Maine, 2000), a firefighter is an employee whose functions include those of firefighter, paramedic, emergency medical technician (EMT), rescue worker, ambulance personnel, and hazardous- materials technician. A firefighter is (1) trained in fire suppression; has
  16. 16. 16 the legal authority and responsibility to engage in fire suppression; and is employed by a fire department of a municipality, county, fire district, or state; and is (2) engaged in the prevention, control, and extinguishment of fires, or in the response to emergency situations where life, property, or the environment is at risk [29 U.S.C. sec 203 (y)]. Variables Four variables were used in this correlational study. Three predictor variables that include; (1) scores on a written promotion examination; (2) years of experience which also referred to as seniority; and (3) years of formal education will be correlated with the criterion variable (4) scores on a 360-degree peer evaluation of the ranks of FAO and lieutenant. Assumptions This study made the following assumptions: 1. The 360-degree evaluators have sufficient knowledge to determine the existence to which the rated exhibit leadership competencies. 2. Given the guarantee of anonymity, participants will respond to the 360-degree peer evaluation questionnaire honestly and objectively. Limitations This study has the following limitations: 1. Because the variables were correlated without experimental manipulation, casual relationships cannot be inferred. 2. This design was limited to the ranks of FAO and lieutenant; therefore, it cannot necessarily be generalized to the other ranks.
  17. 17. 17 3. Because of their work and personal relationships with their co-workers, participants may exhibit bias in answering the 360-degree peer evaluation. 4. This study is applicable to the San Antonio Fire Department. 5. Because of the small number of female participants (N= 3) in this study, the results of this study cannot necessarily be generalized to other females in the fire service. 6. Because of the small number of African American participants (N= 16) in this study, the results of this study cannot necessarily be generalized to other African Americans in the fire service. Significance The significance of this study is to contribute to the research on the leadership selection process in the fire service. First, the study identified leadership competencies identified by researchers to be relevant to the fire service. Second, it evaluated the results of the Rogers’ Firefighters Survey (2001) of 35 San Antonio firefighters who identified leadership competencies that they perceived as relevant to the fire service. Third, it combined the data to develop a 360-degree peer evaluation that was used to determine whether the leadership selection process is indeed selecting firefighters who have the leadership competencies—as perceived by the subordinates, peers, and supervisors—that are necessary for the fire service. Finally, this study determined how well the current selection process is working. The results of the study may suggest relevant predictors for criteria for improving the leadership selection process.
  18. 18. 18 CHAPTER TWO LITERATURE REVIEW Theoretical Literature On Leadership A review of the current literature on leadership selection in the fire service provided little information on the reliability and validity of the process. This chapter identified several definitions of leadership and evaluated the leadership theories and models that are applicable to leadership selection. Leadership theories were compared and contrasted in a search for common elements and coherent perspectives. In addition, methods of measuring leadership competencies were evaluated. Finally, a summary discusses the importance of identifying effective competencies for leadership selection criteria in the fire service. Definitions Of Leadership The definition of leadership can sometimes seem esoteric and illusive; one of the many reasons is the number of definitions, theories, and researchers that exist. Stogdill (1974), a prominent leadership researcher said, “… there are almost as many different definitions of leadership as there are people who have attempted to define the concept” (p. 7). Although the concept of leadership can seem mystifying, its various definitions are critical to understanding the concepts and principles of leadership. Hollander (1978) defined leadership as “a process of influence between a leader and those who are followers” (p. 16), whereas Bennis (1959) defined it as “ a process by which an agent induces a subordinate to behave in a desired manner” (p. 125). Robert Kennedy defined leadership as “inspiring people to exercise their best qualities,” while Dwight D. Eisenhower said leadership is “the art of getting someone else to do what you
  19. 19. 19 want done because he wants to do it” (as cited by Gratza, 1972, p 199). “A definition of leadership that would be widely accepted by the majority of theorists and researchers might say that leadership is a process of social influence in which one person is able to enlist the aid and support of others in the accomplishment of a common task” (Chemers, 1997, pg. 1). The many definitions of leadership describe its various roles. The definition of leadership used by the researcher and writer of this dissertation is “a process of social influence in which one person is able to enlist the aid and support of others in the accomplishment of a common task” (Chemers, 1997). This definition will be used to study the leadership competencies of the firefighters selected for leadership roles. In addition, the leadership theories and models were used to evaluate whether the selection process was successfully identifying leadership characteristics and traits, as well as whether these theories and models were used as selection criteria. Models And Theories Of Leadership A common view of leadership is that there is something rare in the personality of an individual who has the unique qualifications to ascend to leadership. Some people refer to these unique qualifications as leadership traits. Because of the various internal and external elements involved in leadership, it remains controversial and continues to be the subject of many studies (Kirkpatrick & Locke, 2000). Great Man Theory. Nineteenth-century philosopher Thomas Caryle offered the Great Man Theory, which asserted that leadership qualities are inherited and that great men are born, not made. The leader, who is endowed with unique qualities, contributes regardless of the
  20. 20. 20 situation. The Great Man Theory is a method used to select individuals who are perceived to be great leaders to transform and inspire individuals and organizations (Bass, 1990). The theory promotes the idea that anyone in a leadership position must deserve to be there by virtue of his or her characteristics or personal traits (Chemers, 1997). One weakness of this theory is that there is little evidence to support the claim that inherited traits are good predictors of leadership effectiveness. Currently, leadership is viewed as a complex interaction between the leader and the social, organizational, and economic environment. This interaction includes the leader’s ability to successfully integrate situational components while transforming and inspiring individuals and organizations (Fiedler, 1996). Trait Theory. Before the 1950s, the study of leadership was based mainly on the Great Man Theory; however, it later was challenged by the trait theory (Goldbach, 1989). Trait refers to a person’s general characteristics, including his or her capacities, motives, or patterns of behavior. The trait theory is derived from the statistical treatment of large numbers of observations presented as norms. Kirkpatrick & Locke (1991) identified six traits that differentiate leaders from followers. These traits include: (1) drive, (2) integrity, (3) self-confidence, (4) cognitive ability, (5) desire to lead, and (6) knowledge of the job. Jenkins (1947) and Stogdill (1948) found that selecting leaders by means of their traits met with little success. One major weakness of the trait theory is that traits do not explain the complexity and intertwined behavior of the total person (Allen, 1965). Additionally, this theory does not take into account the interaction between the leader and his or her group. Another
  21. 21. 21 weakness of the trait theory is that traits often are confused for being skills. A skill is a technical ability, knowledge, or expertness; a trait is a characteristic (Stogdill, 1948). Situational Leadership Theory. The situational leadership theory suggests that leadership styles should be matched to the maturity of the subordinates (Hersey & Blanchard, 1997). The theory is primarily a model that classifies the subordinate’s maturity in two dimensions: (a) psychological maturity and (b) job maturity. Psychological maturity assesses the subordinate’s commitment, motivation, and willingness to accept responsibility; job maturity examines the subordinate’s experience, knowledge, and understanding of the job. As the subordinate’s maturity grows, his or her relationship with the leader should be more relationship-motivated than task-motivated. Little is known about the validity of this theory; however, it has much in common with the path-goal theory. Path-Goal Theory Of Leadership. The path-goal theory of leadership suggests that the performance, satisfaction, and motivation of a group of people can be affected by a leader who: (a) offers rewards for achieving performance goals, (b) clarifies paths toward the goals, and (c) removes performance obstacles. The path-goal approach identifies the specific variables that define the motivation, and can adapt the leadership style according to the situation. The leadership style should be conducive to the situation. It can be directive, supportive, participative, or achievement-oriented (Evans, 1970). Fiedler’s Contingency Model. The Fiedler’s contingency model assumes that group performance depends on: (a) leadership styles described in terms of task and relationship motivation, and (b)
  22. 22. 22 situational favorableness, which is determined by leadership-member relations, the task structure, and the position of power. Fiedler argued that leadership involves social influence and the ease with which a leader is able to influence his or her followers. Fiedler further argued that the quality of interpersonal relations between the leader and his or her follower is such that if the leader is well liked and respected by the followers, his or her ability to influence them is easier and more likely to be successful (Fiedler, 1967). Cognitive Resource Theory. The Cognitive Resource Theory (CRT) suggests that leaders and group members can make effective use of their intellectual abilities, technical knowledge, and experience to contribute to group performance (Fiedler & Garcia, 1987). However, research on the relationship between intelligence, leadership styles, and performance demonstrated that intelligence contributed highly to performance only if the leader was directed and working in a stress-free relationship with his or her supportive group (Fiedler, 1987). Vroom-Yetton Normative Leadership Decision Theory. The Vroom-Yetton Normative Leadership Decision Theory affects a whole group or team that includes individual subordinates (Vroom, 1964). The theory maintains that one of a leader’s important prerogatives is controlling the process by which decisions are made. It further suggests that the decision strategies range across the dimension of degree of subordinate participation and fall into three categories—autocratic, consultative, and democratic group. The decision model for groups falls under five strategies: (a) autocratic–1, where the leader makes the decision using information already available, (b) autocratic–2,
  23. 23. 23 where additional information is obtained from the subordinates but the leader makes the decision alone, (c) consultative–1, where the leader discusses the problem with the subordinates individually but preserves the authority to make the decision alone, (d) consultative–2, where the problem is discussed with subordinates as a group before the leader decides, and (e) Group–2, where the leader shares the problem with the subordinates as a group and invites them to participate fully and equally in the decision, with the leader acting as the group chair (Chemers, 1997). Managerial Grid Theory. The Blake-Mouton Managerial Grid Theory is two-dimensional and focuses on the manager’s concern for: (a) production and (b) concern for people. The managerial grid is set up on a horizontal and vertical axis that ranges from zero to nine. Concern for people is on the vertical axis while concern for production is on the horizontal axis. Five managerial styles are formulated from the grid: (a) supportive, (b) delegating, (c) organizing, (d) coaching, and (e) authoritative. The grid represents the various leadership styles recommended, based on the need for concern or a combination of concerns. According to Blake and Mouton (1964), coaching is the most desirable managerial style because it is based on effective integration of the task and the people (Chemers, 1997). A brief overview of the leadership theories and models, which were useful in identifying leadership traits, suggests the elements that make an effective leader. This study draws upon the leadership theories and models to evaluate whether leadership traits and styles are being identified and used toward the selection of leaders in the fire service. The theories that are used in this study include: (a) the trait theory, (b) the situational theory, and (c) the Vroom-Yetton Normative Leadership Decision Theory.
  24. 24. 24 The trait theory is important in identifying individuals who have personal traits that could lead to effective leadership. The situational theory was used to evaluate whether the psychological maturity and job maturity levels are measured in selecting leaders. Maturity is an important characteristic of an effective leader, especially in the dangerous occupation of fire fighting. Finally, the Vroom-Yetton Normative Leadership Decision Theory will be used to evaluate decision-making competencies. To be an effective leader in the fire service, a firefighter must be decisive, especially when human lives are hinging on the leader’s ability to make sound decisions (Dewey, 1910). Leadership Selection Process In The Fire Service National Survey Of Leader Development And Promotional Testing In The Fire Service. A common practice in almost every fire department is the system used to test firefighters for leadership roles in the fire service (Maxfield, 1996). Dr. Walter S. Booth (1999) of the Booth Research Group conducted a nationwide survey of fire departments in the United States to find out their: (a) most commonly used promotional test, (b) minimum educational requirements, and (c) average time-in rate for testing eligibility requirements. The survey was mailed to 408 paid fire departments in the U.S., each of which has more than 100 firefighters. The survey revealed that the written exam was frequently used to promote firefighters to the ranks of FAO through battalion chief, as shown in Table 1. The examination requirement for the most part was reduced for promotion to assistant chief; however, as the firefighter ranks moved up the hierarchy, the use of oral boards, assessment centers, and appointments increased. The use of practical testing methods
  25. 25. 25 appears significant for promotion to FAO; however, their use is insignificant for promotion to the higher ranks (Booth, 1999). Table 1 Most Commonly Used Testing Method by the Fire Service Testing Methods FAO Lieutenant Captain Battalion Chief Assistant Chief Written Exam 85% 85% 89% 66% 38% Practical 59% 18% 19% 17% 10% Assessment Center 16% 47% 50% 55% 42% Oral Board 7% 14% 16% 20% 19% Appointment 1% 1% 2% 4% 19% Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43 (8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with permission of the author. The survey also asked for the minimum educational requirements. A majority of fire departments required that most of their firefighters have a high school diploma, as reflected in Table 2. As firefighters moved up the rank structure, the education level required increased to a bachelor’s degree. With added responsibilities, the educational requirements should increase. The minimum educational requirement of the battalion fire chief and assistant fire chief were almost non-existent at the post-graduate level reflecting 0% for the battalion fire chief and 4% for the assistant fire chief. The percentage is higher in the bachelor degree level reflecting 18% for the battalion fire chief and 36% for the assistant fire chief. Table 2 indicates that associate degree level was the mode for the minimum educational requirement for the battalion fire chief and assistant fire chief.
  26. 26. 26 Table 2 Minimum Educational Requirement Education FAO Lieutenant Captain Battalion Chief Assistant Chief High School 72% 48% 35% 21% 6% Some College 14% 32% 34% 24% 23% Associates 4% 9% 21% 37% 32% Bachelors 0% 0% 2% 18% 36% Post Graduate 0% 0% 0% 0% 4% No Requirements 11% 11% 8% 0% 0% Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43 (8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with permission of the author. Table 3 shows the average time in grade required for test eligibility, indicating that a firefighter remains in grade for an average of 2.3 years before becoming eligible for promotion to the next rank. The average 2.3 years, however, is not indicative of the amount of time a firefighter remains in a position before being promoted to the next rank. As a firefighter moves up the leadership hierarchy, especially to the battalion chief and assistant chief ranks, the potential for promotion to the next rank decreases because of the fewer number of available positions (Booth, 1999).
  27. 27. 27 Table 3 Average Time In Grade For Test Eligibility Rank Modal time in grade Average time in grade Modal time in department Average time in department FAO 3 2.7 0 0.0 years Lieutenant 2 2.5 5 5.6 years Captain 2 2.1 7 6.3 years Bttn. Chief 2 2.2 10 9.1 years Asst. Chief 2 2.1 10 9.6 years Note. From “Raising the Bar,” by W. S. Booth, 1999, Fire Chief Magazine, Volume 43 (8), pp. 78-82. Copyright 1999 by the Booth Research Group Inc. Adapted with permission of the author. Municipal Civil Service Laws And State Statutes. According to Miller (1997), the purpose of the Fire and Police Civil Service Law is to secure efficient fire and police departments comprised of capable personnel who are free from political influence and have permanent employment tenure as public servants. The authority is granted under provisions of the Texas statute Chapter 143 Municipal Civil Service, also known as Chapter 143 of the Local Government Codes, or Vernon’s Texas Code Annotated. In Chapter 143 Subchapter B, entitled “Classification and Appointments,” the promotional procedures are identified: (a) eligibility for promotion, (b) examination notice, (c) exam procedures, and (d) exam grades. According to Miller (1997), a firefighter is not eligible for promotion unless he or she has served in the current rank for at least two years before the examination. The Fire and Police Civil Service Commission posts a notice that lists the study material from
  28. 28. 28 which the examination questions will be taken. Applicants must earn a minimum raw score of 70 points to pass the examination. Applicants who pass the exam receive one point added to their raw score for each year of service, for a maximum of 10 additional points. A tie in the promotion competition is broken by the highest score on the written exam. If there still is a tie, the promotion is awarded to the candidate who earned the highest score on the entrance examination (Miller, 1997). Collective Bargaining Contract Agreement In 1935, the Wagner Act established the National Labor Relations Act Board, which regulates negotiations and the collective bargaining process between employers and labor organizations (Fossum, 1992). According to the Society of Human Resource Management (1998), collective bargaining is an exclusive representation by the union of the group of people in a bargaining unit. Bargaining is the process by which management and union representatives negotiate employment conditions. Subjects for collective bargaining negotiations fall into three categories: (1) mandatory subjects, (2) voluntary subjects, and (3) illegal subjects. In 1973, the citizens of San Antonio passed a citywide referendum granting collective bargaining rights to fire fighters (personal communication from H. Cardenas, January 9, 2001). During the collective bargaining negotiations, the promotional process from which firefighters are selected to leadership roles falls under mandatory subjects. The collective bargaining contract between the City of San Antonio and Local 624 International Association of Fire Fighters, Article XXXI title Promotions, Section 1-8, clearly identifies the promotional process used to select a firefighter for leadership in the fire service (IAFF Local 624, 1999).
  29. 29. 29 Section 1 of the contract identifies eligibility for promotion in addition to granting seniority points, for a maximum of 10 points. Section 2—Study Material—addresses the appointment of a committee to determine the material that will compose the examination, pending the approval of the Fire and Police Civil Service Commission. Section 3—Promotion to Fire Apparatus Operator, Lieutenant, and Captain—states that position vacancies will be filled by means of a competitive written examination in accordance with Chapter 143 of the Local Government Codes; however, a score of 70 will be considered the minimum for eligibility for a promotion. For promotion to district chief, candidates must pass a written examination, with seniority points added, in addition to passing an assessment-center process, as stipulated in Section 4 (IAFF Local 624, 1999). The collective bargaining agreement between the City of San Antonio and the firefighters is a city ordinance that supersedes municipal and state statutes, including Chapter 143 of the Local Government Codes. During collective bargaining talks, the city and the firefighters have domain over the promotional process and can alter it to fit their needs better, as long as both parties are in agreement (IAFF Local 624, 1999). Promotional Examination Process The San Antonio Fire Department’s tool of choice as a predictor is a written promotional examination, the results of which reveal the candidates for each classification who are eligible for promotion to leadership (Miller, 1997). The written exam is the first part of the selection process. Candidates have two hours to answer 100 questions formulated from the assigned study material, which covers critical job
  30. 30. 30 knowledge and skills. The Booth Research Group developed the test questions from job analyses of the positions for which candidate are being tested (Booth, 1998). The written exam is based primarily on technical and administrative skills related to job knowledge. For the rank of district chief, a captain who is eligible for the position must also pass an assessment center in order to be placed on the eligibility list. By definition, an assessment center is a program in which candidates proceed through a number of job-related assessment exercises that typically include: (1) in-basket exercises, (2) leaderless group discussions, (3) confrontation exercises, (4) an interview, and (5) analysis exercises. The purpose of the assessment center is to elicit behavior relevant to dimensions that are crucial to job success (Prudy & Filer, 1980). When a vacancy occurs, the top candidate is selected for promotion to this leadership role (Booth, 1999). The study material for the written promotional examination can be a source of concern if it is not job-related. Often there are contradictions—among the different sources and even within a single source—regarding the relevancy of the material to the fire service. The study material is carefully reviewed to avoid such contradictions. The questions are from areas in a text that are about the position. In addition, the examinations are administered easily with the use of an administration guide and other instructions available to the proctor. The written promotional tests are custom-designed to reflect the needs, concerns, and mission of the fire service (Booth, 1999). Qualifications For The Promotional Examination. Promotional examinations are open to firefighters who for at least two consecutive years have been in a position that is below the rank of that for which the exam is being administered, according to the Collective Bargaining Agreement between
  31. 31. 31 the City of San Antonio and Local 624 of the International Association of Firefighters (1999, pg. 53). For example, a firefighter who has been in the rank of FAO for at least two consecutive years is eligible for the written exam for lieutenant, which is the next rank in the hierarchy. Promotional Examination Procedures. According to the Texas Firemen’s and Policemen’s Civil Service Law (Miller, 1997), the Civil Service Commission is authorized to adopt rules governing promotions and to administer promotional exams to provide eligibility lists for each classification in the fire service. Candidates are given an identical written examination, which comprises 100 multiple-choice questions. The questions, which must originate from study material that is approved and posted by the Civil Service Commission, also must be relevant to the duties of the position for which the exam is being administered. The study material must be a reasonably current publication that is available to all candidates eligible for the exam. Firefighters from the ranks of firefighter to captain are screened for eligibility to take the written promotional exam. Part of the eligibility requirement is that they have a minimum of two years in rank.. The assigned study material begins with the practical fire-ground operations of the FAO and lower ranks, then covers leadership and supervision for mid-level ranks such as lieutenant and captain, and ends with the administrative-type operations of the higher ranks such as district chief. While some of the training and study material is common to all ranks, other study areas such as strategy and tactics, leadership, management, and supervision are required for the lieutenant through district chief positions (Booth, 1999).
  32. 32. 32 The director of the Civil Service Commission is responsible for preparation and security for each promotional exam (Miller, 1997). Analysis Of The Job Knowledge, Skills, And Abilities (KSAs). One of the many definitions of job analysis is “a purposeful, systematic process of collecting information on the important, work-related aspects of a job” (Gatewood & Field, 1990, p. 251). The components of a job analysis include: (1) work activities, (2) tools and equipment, (3) the work environment, and (4) personnel requirements such as knowledge, skills and abilities, and other characteristics (KSAOs) such as physical traits, job interest, and personality (Gatewood & Field, 1990). Job analysis data frequently are used to develop two types of measures: (1) predictors of job applicants who are likely to be successful in the position being measured, and (2) standards of performance that an applicant must meet to be considered successful on the job. By examining tasks performed on the job as well as the KSAOs, examiners can establish the predictors and criteria for measurement in order to select the applicant with the greatest potential for success (Gatewood & Field, 1990). Most psychologists believe that the best measure of job performance is a work-or job-sample measure. However, this is usually infeasible for public safety organizations. For example, for firefighters, there are 440 different tasks in firefighting. Second, task such as accurate work in presence of physical danger or under stresses such as emergencies may be difficult or dangerous to stage (Hunter & Schmidt, 1996). The City of San Antonio hired the Booth Research Group (1998) to develop a valid, multiple-choice written promotional examination for the positions of FAO and lieutenant. Conducting a job analysis is an essential part of developing a valid testing
  33. 33. 33 instrument. The Booth Research Group conducted job analyses on the FAO and lieutenant positions. The Booth Research Group (1998) designed a questionnaire to assess the knowledge, skills, abilities, and other characteristics (KSAOs) needed for effective performance in each of these ranks. From the job analysis data, scales were designed to assess the functional skills required for each job. Seven job analysis scales, which range from low to high, provide a method for measuring basic/generic occupational skills. The scales, which are widely used to objectively define firefighting tasks, can be applied to all jobs. One hundred twenty-nine (N=129) questionnaires were distributed to FAOs and lieutenants, of whom 108 (84%) responded. The FAOs and lieutenants reviewed the rate of importance and complexity of all job tasks. Promotional examination questions were developed based on the job dimensions relevant to the classifications being tested (Booth, 1998). Reliability And Validity Of The Selection Process. According to the American Psychological Association’s (APA) 1980 publication, Principles for Validation and Use of Personnel Selection Procedures, “when any selection procedure is used, the essential procedure is that evidence be accumulated to show a relationship between the decisions, based on assessments made by that procedure and criteria such as job performance, training performance, advancement, or other pertinent behavior” (p. 1). Validity, as defined by the APA (1980), is the degree to which inferences from scores on tests or assessments are supported by evidence. The concept of reliability is used in terms of dependability, consistency, and precision of measurement.
  34. 34. 34 According to Field & Gatewood (1990), the types of validity strategies involve: 1. Criterion validity, which includes concurrent or predictive validity. In a concurrent validation, the predictor and the criterion are collected simultaneously. In a predictive validation, the predictor is administered before beginning the job, and the criterion data are collected later. 2. Content validity, which is based on descriptive samples of job behaviors that are necessary for effective performance. These content items are then placed in the test. 3. Construct validity, which is based on the concepts, attributes, or characteristics measured. This study examines the criterion-related concurrent validity of cognitive-ability test scores, job tenure, and scores from the 360-degree feedback instrument. The process is accomplished by (a) conducting a job analysis; (b) determining relevant knowledge, skills, abilities, and other characteristics required to successfully perform the job; (c) choosing one or more predictors of the KSAOs; (d) weighing the predictors against the criteria; and (e) administering them to the employees. The criterion-related test consists of the critical elements of the job, which can be performance or technical elements, and is defined in terms of skills. The predictor is a sample of performance that is used to predict future performance. A predictor can be a written test, an interview, or a personal reference check. Weighting predictors can provide the leadership selection process with the best possible forecast of the candidate’s success on the job (Field & Gatewood, 1990).
  35. 35. 35 Leadership Prediction Studies Remarkable strides have been made in developing reliable, valid tests for selecting non-managerial employees in various occupations. Developing a process for selecting effective leaders, however, has proven less successful (Bass, 1990). Supervisors often are chosen from the rank and file based on their technical skills rather than their leadership skills. For example, a police sergeant is more likely to be promoted to supervisor because he is good at the technical aspects of being a police officer rather than because he has the qualities to be an effective leader (Hogan, Curphy, & Hogan, 1994). The most commonly used approach to predicting the readiness of the job incumbent for leadership is the interview process (Bass, 1990). According to Hogan, Curphy, and Hogan (1994), the best way to predict leadership capability is to use a combination of cognitive ability, personality, simulation, role-playing, and multirater assessment instruments and techniques. Organizations such as AT&T, Sears Roebuck, IBM, the United States Military, police departments, and many others use the judgment from simulation techniques to predict leadership capability. The simulations are based on observation of leadership performance using work samples and situational tests (Bass, 1990). The AT&T Corporation developed a managerial selection tool known as an assessment center, which identifies performance dimensions relevant to the job. The company uses the results to select job incumbents for leadership roles within its organization (Bray, 1982). IBM also uses an assessment center process to select it leaders. The company conducted an in-basket test on its supervisors that revealed a correlation of r= .32 when
  36. 36. 36 the subjects were rated again after facing the challenge of increases in management responsibilities within three years after testing (Wollowick & McNamara, 1969). Organizations commonly use performance ratings to promote and to forecast future success in candidates for leadership (Bass, 1990). In the Yammarino & Bass study (1989), the military superiors of promotion candidates in the U.S. Navy used the candidates’ performance grades as students in the U.S. Naval Academy as predictors of whether they would become charismatic Naval officers. The cumulative military performance grades that were awarded did not prove to predict subsequent performance fitness reports of 186 U.S. Navy officers serving in the fleet one decade later. The results are consistent with other studies that fail to find a significant correlation between undergraduate college grades and success in business and industry (Schick & Kunnecke, 1982) or higher ratings on job performance (Pallet & Hoyt, 1968). Most of these studies, however, have one or more of the nine flaws noted by Ree (1995) as examples of conducting ability research incorrectly. One of these examples is the disregard for group differences such as race and sex. Group differences should be present; however, the less- known differences should be ignored. Another example is use of the wrong model of reliability. Although test-retest measures would be appropriate, it is suggested that high reliability values could result frequently from this model misspecification (Ree, 1995). Although performance measures have not been predictive of readiness of job incumbents for leadership positions, peer ratings have been found to be an effective predictor in some instances. For example, a peer rating by cadets at West Point Military Academy was found to be the best single predictor of leadership success as an officer in the U.S. Army. For example, a correlation of r= .51 was calculated between the peer
  37. 37. 37 ratings at West Point and the success rating of the same subjects as infantry officers 18 months later (Haggerty, Johnson, & King, 1954). Studies show that many public and private organizations operate on the belief that it is appropriate to hire intelligent and experienced job incumbents for leadership positions. Studies indicate that the correlation between leadership performance and various experience measures is less than r= .10 (Fiedler, 1995; Fiedler & Garcia, 1987). Although some studies indicate a correlation of less than r= .10, there are other studies that report mixed correlations where the magnitude or degree of relationship is about the same. Studies have shown that under stressful conditions, experience and performance were correlated in the range of r= .40 to r= .60. For example, the experience of fire department officers and their performance was correlated to be r= -.66 (N=22, p< .01) in low-stress administrative jobs, but in stressful fire combat, it was correlated to be r= .68 (N=22, p< .01) (Frost, 1981). A study conducted by Hunter and Schmidt (1998) summarizes the practical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, the validity of 19 selection procedures for predicting job performance and training performance and the validity of general mental ability (GMA) and the 18 other selection procedures was presented (see appendix F). The selection procedure with the highest validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65) and GMA plus a structured interview (mean validity of .63). Many labor leaders assume that the predictive validity of experience is higher than the predictive validity of ability, especially in highly experienced workers. Studies
  38. 38. 38 predicting performance ratings for predictive validity of both experience and ability indicates that the average predictive validity for experience to be .18, where as the predictive validity of ability is .51. Therefore, predictive validity is three times larger than the predictive validity of experience (Hunter and Schmidt, 1996). In 1945, Shartle (1950) organized the Ohio State Leadership Studies. He and his colleagues integrated the most comprehensive research program on identifying leadership behavior patterns. The group generated a list of about 1,800 descriptive statements about various aspects of leadership behavior. From this list, a leadership-behavior-description questionnaire was developed. On the questionnaire, respondents would rate a leader by picking one of five choices to indicate the frequency of behavior that described the leader. The data were correlated and factor-analyzed, resulting in two elements that emerged to describe leadership behavior—consideration and initiation of structure (Bass, 1990). By contrast, Frank (1955) believed that the compilation of data based on sheer numbers of subjects could not adduce a completely unified theory of behavior because of the variability of the individual. According to Fiedler and House (1944), there is no evidence of a specific leadership trait, behavior, or personality. Leadership effectiveness is the ability to get a group to accomplish its mission; it does not depend solely on the leader’s abilities and attributes but also on how well the leader’s personality, abilities, and behaviors match the situation. Two major behaviors that identify leaders have been identified; one was how well a leader treats his or her subordinates; the other was the degree to which a leader structures the roles and working relationships of his or her subordinates.
  39. 39. 39 Methods Of Verification And Measurement Of Leadership In The Fire Service Psychologists know that theories of leadership depend on different methods of verification, while models of leadership depend on different methods of measurement (Bass & Stogdill, 1990). According to Shores (1984), “a selection procedure for firefighters is considered to be related to the criterion when the relationship between performance on the procedure and performance on the criteria measure is statistically significant at .05 level of significance” (p. 8). In this study, the degree of relationship between the criterion and the predictive variables was evaluated and computed using acceptable statistical procedures to determine whether a relationship exists. The variables examined included: (1) the Booth Research Group Questionnaire, (2) the Rogers’ Firefighters Survey (2001), (3) the written criterion test, (4) the years of service, (5) the years of formal education, and (5) the criterion variable—a 360-degree peer evaluation. Booth Research Group Questionnaire. A questionnaire is a formal measure of characteristics, attitudes, motivations, and opinions as well as past, current, and potential behaviors. The information elicited from a questionnaire can be used to describe or compare facts or make predictions. Depending on the objectives, the survey design must be flexible (Salant & Dillman, 1994). The questionnaire is not a test; it is simply a method of collecting information (Booth, 1998). It is important to remember that getting a subject to complete a long questionnaire may prove difficult. The questions should be written clearly and reflect the purpose of the questionnaire (Astous, 2001).
  40. 40. 40 The City of San Antonio hired the Booth Research Group (1998) to develop a job analysis questionnaire to assess the knowledge, skills, abilities, and other characteristics (KSAOs) required for effective performance in the positions of FAO and lieutenant. The purpose of the questionnaire was to gather information in order to paint a big picture of the job characteristics that accurately reflect the position being evaluated. The job analysis identified several competencies relevant to the job. When a competency was identified, it received a number of ratings based on the importance and frequency of the performance dimensions. A competency is a specific aspect of a position. For the FAO position, 13 different groups and tasks were identified. For the lieutenant position, five competencies were identified from which 18 dimensions were rated based on their importance and frequency (Booth, 1998). The Booth Research Group (1998) demonstrated criterion-related validity by using a construct-base approach that included (1) a series of job analysis activities; (2) a series of workshops to identify the abilities and characteristics required to perform the job; and (3) a questionnaire to established a linkage between the abilities, constructs, and specific task dimensions. Rogers’ Firefighters Survey. A survey is a powerful scientific tool for gathering accurate, useful facts and sound data that can be translated into valuable information for its intended users (Salant & Dillman, 1994). In a study conducted by the Selection Consultant Center for the California Personnel Board (Hubbard, Hunt, & Krase, 1975), a study of 284 respondents in 62 fire departments in California and Nevada rated six-job criteria that are relevant to the fire service. These criteria included: (a) mechanical and trade knowledge, (b) physical
  41. 41. 41 agility, (c) personal attributes, (d) communication skills, (e) problem-solving, and (f) spatial abilities. The National Fire Academy in Emmitsburg, Maryland, conducted a national survey of more than 5,000 Emergency Medical Service (EMS) firefighters and leaders. The survey identified management and leadership traits that the respondents believed were relevant to the fire service (Dyar, 2000). The leadership traits most often identified in the survey were: (a) honesty, (b) consistency, (c) vision, and (d) competency. In a pilot study, the Rogers’ Firefighters Survey (2001) was randomly administered to 35 San Antonio firefighters in order to identify leadership competencies relevant to the fire service. One of the purposes of the survey was to gather data from firefighters assigned to the fire-fighting division, as opposed to those in the EMS division. Included in the questionnaire was a list of 42 leadership traits and competencies from which the firefighters were to select those relevant to the fire service. They also were to provide good and bad examples of each leadership behavior they selected, based on what they had encountered on the job. Their examples provided a variety of descriptions of relevant leadership behavior in the fire service. From the survey, six leadership competencies were identified as the most relevant: (1) communication, (2) self-confidence, (3) ethical behavior, (4) professional and technical skills, (5) interpersonal skills, and (6) emotional balance (Rogers, 2001). Written, Criteria-Based Test. The written promotional examination is the first part of the leadership selection process in the fire service. Research on the value of written tests has revealed that despite
  42. 42. 42 their limitations, written tests are a source of valuable information about the probability of the individual performing the job successfully (National Academy of Science, 1982). In this study, the writer and researcher of this dissertation examines the written promotional examination conducted by the Booth Research Group (1998). The Booth Research Group developed the written promotional test based on the criteria-related concurrent process. It involves: 1. A series of job analysis activities that identify the tasks and dimensions that compose the target job. 2. A series of workshops and interviews that identify the abilities and characteristics required to perform the job effectively. 3. A survey to assess the skills, knowledge, abilities, and other characteristics required for effective performance of the target jobs, which in this study are the positions of FAO and lieutenant. The validated written promotional examination is used to identify firefighters who have the greatest potential for success on the job. Years Of Service. Years of service, also known as seniority, represent all the years of service in the San Antonio Fire Department. It could be viewed as a predictor of job performance in the fire service. However, current studies indicate that over time, the validity of job experience for predicting performance declines (Hunter & Schmidt, 1996). Years Of Formal Education. The years of formal education represents the attainment of credit hours from a college or university. It could be viewed as a predictor of job performance in the fire
  43. 43. 43 service. In the general population, the correlation between education and ability is about .55 (Hunter & Schmidt, 1998). A 360-Degree Peer Evaluation. Companies and organizations use a multi-source or 360-degree feedback to rate the leadership and managerial performance of (1) supervisors, (2) peers, (3) subordinates, and (4) internal and external customers, according to Goudy-Keith-Lyne (1999). This 360-degree feedback is used as a rating tool of managerial performance because it allows peers and leaders to identify and compare leadership competencies of the individual who is being rated. The extent to which the comparison of feedback provides meaningful information is dependent on whether the rater-sample interprets the performance scales similarly. The choice of questionnaire is a critical step in the 360-degree feedback process, and the degree of feedback is determined by the quality of the questionnaire (Goudy- Keith-Lyne, 1999). The dimensions of the questionnaire should be based on job-related criteria that are relevant to what is being measured. The 360-degree peer evaluation has face validity based on observed behaviors and skills. In 1992 Riggio and Cole conducted a peer-evaluation study on 261 members of the fire service who represented three ranks in the organizational hierarchy: chiefs (N=12), captains (N=76), and firefighters (N=173). The mean age of the subjects was 35.6 years (SD=7.9 years) and the years-of-service mean was 12.9. An investigation was conducted on the concurrence between the performance ratings of the superiors and those of the subordinates as well as the correlation between these performance ratings and standardized measures of job satisfaction. The results revealed significant agreement
  44. 44. 44 between superiors (chiefs) and subordinates (firefighters) in rating the performance of the captains. These performance ratings also correlated positively at r = .58 with the job satisfaction of the subordinates (firefighters) and supervisors (captains) (Cole & Riggio, 1992). One issue of concern with the 360-degree peer evaluation is a lack of convergence across multi-source ratings because of the different raters of the job responsibilities that may give them different conceptualizations of leadership (Tornow & London, 1998). Research on peer appraisals reveals that they could be prone to bias. Under certain circumstances, workers may react negatively toward peer evaluations (Kane & Lawler, 1978). Recent research has raised concerns about the rating of leadership performance by peers and superiors (Harris & Schaubroeck, 1988). Bias is a major source of error in peer evaluations, according to Bass (1990), Podsakoff and Organ (1986), and Harris and Schaubroeck (1988). It is hoped that the anonymity afforded the participants will lessen any potential for bias in this study. Summary Effective leadership in the fire service is critical to the safety and welfare of firefighters and the community. Ability testing, which is available to the fire service, gives management the capability of predicting job performance. Although the written test can be viewed as a predictor of effective performance, it tends to demonstrate ethnic and racial bias resulting in adverse impact (Booth, 1998). Several lawsuits were filed regarding the test procedures that evaluate candidates for promotion in the fire service. In 1974, the percentage of African-American employees in the St. Louis Fire Department was 11% of the 1,000 uniformed personnel. Of the 180 fire captains, only four (2%) were
  45. 45. 45 African-American. Although this city had a population that was 40% African-American, no African-American firefighter had ever held a position above the rank of fire captain. The fire department’s written promotional test appeared to adversely impact the African- American firefighters (Gebhart, Duffe, & McCurley, 1998). Adverse impact is a form of discrimination in which organizational selection standards are applied uniformly to applicants of both genders and all racial and ethnic groups; the net result, however, is denial of these groups into leadership, according to Gatewood and Field (1990). For example, the height requirement once used by police departments to recruit applicants discriminated against Hispanic, Japanese, and women applicants. Title VII of the Civil Rights Act of 1964 is the principle body of federal legislation in the area of employee selection. Section 703 (b) of Title VII addresses the use of tests in selection procedures, stating: “… nor shall it be unlawful employment practice for an employer to give and to act upon any professional developed ability test, provided that such test is not designed, intended, or used to discriminate because of race, color, sex, religion, or national origin.” (Shores, 1984, p. 3 & Federal Uniform Guidelines on Employee Selection Procedures, 1978). Selecting firefighters with effective leadership competencies is critical to the fire service. Administering an ability test that identifies competencies and is within the parameters of the law is every bit as critical. One legal defense that can be applied to the selection process is use of the Uniform Guidelines on Employee Selection Procedures (1978). These guidelines serve as a primary reference in the selection process and represent the viewpoint of the federal agencies charged with enforcement of the Equal
  46. 46. 46 Employment Opportunity (EEO) laws. The guidelines also provide employers with one set of government requirements to meet in order to avoid discrimination in testing and other selection processes (Field & Gatewood, 1990).
  47. 47. 47 CHAPTER THREE METHODOLGY This chapter presents the research design that identifies the four variables used, states the null hypotheses, and describes the participants and instrument used to collect data. It also describes the data-collection and data-analysis procedures and concludes with a statement of the ethical decisions involved in this study. Research Design The research design for this study is a correlation/regression cross-sectional design. It employed a 360-degree peer evaluation to gather data for the criterion variable. The study focused on the relationship between the criterion and three predictor variables. A correlation/regression design was used because the predictor variables do not lend themselves to experimental manipulation. Criterion And Predictor Variables The degree of relationship between any two variables is the extent to which they vary in a systemic manner. A correlation/regression analysis was conducted on the variables to determine the degree of relationship between the criterion variables and the predictor variables. The predictor variables were (1) scores on a written promotional examination, (2) years of experience or seniority in the San Antonio Fire Department, and (3) years of formal education. The criterion variables were the scores of the 360- degree peer evaluation, which measures leadership competencies relevant to the fire service. One hundred eighty-one firefighters participated in the 360-degree peer evaluation, which had 57 variables that measured leadership competencies for the FAO
  48. 48. 48 and lieutenant, and were ranked on a scale of 1 to 5. Once the scores were collected, a factor analysis was conducted on the variables to reveal patterns of interrelationship among variables and detect clusters of variables that inter-correlated and redundant. The initial Eigenvalue of the first factor were rated at 55%, with the others reporting nothing greater than 5%. Consequentially, all the scores of the variables were totaled to reflect the criterion variable, which is described as the 360-degree peer evaluation. Null Hypotheses Hypothesis 1 Ho: ρ = 0 there will be no significant relationship between the scores on the written promotional examination and the scores of the 360-degree peer evaluation of effective leadership competencies. Hypothesis 2 Ho: ρ = 0 there will be no significant relationship between years in the fire service and the scores of the 360-degree peer evaluation. Hypothesis 3 Ho: ρ = 0 there will be no significant relationship between the years of formal education and the scores of the 360-degree peer evaluation. Hypothesis 4 Ho: ρ = 0 there will be no significant relationship between the criterion and the set of all predictors. This null hypothesis could also be stated as: The values of the regression weights are equal and all equal to zero. The Participants The participants were San Antonio Fire Department firefighters, FAOs, lieutenants, and captains. Verification of their ranks is documented in official department records. The participants rated the FAO and lieutenant ranks, which are low- to midlevel leadership positions. These ranks were identified among the seven leadership positions in the hierarchy in order to ensure an adequate sample size.
  49. 49. 49 Participants came from the Fire Suppression Division only; the other divisions were excluded from the study because their hierarchical structures are not conducive to a 360-degree peer evaluation. About 60% (N= 840) of the firefighters in the department (N=1400) are assigned to the Fire Suppression Division. Permission to conduct the research on this sample group was granted by the president of the International Association of Firefighters Local 624 (see Appendix A). The age of the participants ranged from 31 to 59 years; the mean was 45 years. Their sex was predominantly male; a small percentage was female. Years of service ranged from four to 35 years, with a mean of 21 years of service. The years of formal education ranged from 12 (General Equivalency Degree) to 18 years (master’s degree). A high school diploma with one and a half years of college (N=13.8 years) was the average amount of formal education. Although participants were exposed to similar leadership selection procedures, the study material for the written promotional exam changes slightly each year. The criterion variable was the score on the 360-degree peer evaluation that rated FAOs and lieutenants. The surveys were distributed to 225 potential participants, with an expected response rate of about 80% (N=181). The Instrument In this study, the 360-degree peer evaluation was used to provide feedback regarding leadership competencies relevant to the fire service. The survey called for ratings of leadership performance from the full circle of key personnel that included (1) supervisors, (2) peers, and (3) subordinates.
  50. 50. 50 The 360-degree peer evaluation was created from two sources. The first source was the Booth Research Group (1998), which conducted a job analysis that identified five leadership competencies relevant to the positions of FAO and lieutenant. The second source of data was the Rogers’ Firefighters Survey (2001), which surveyed 35 San Antonio firefighters who identified six leadership competencies relevant to the fire service. A total of eleven leadership competencies were identified from the two sources; however, two of the competencies identified by the Rogers’ Firefighters Survey (2001) overlapped with those identified by the Booth Research Group (1998), yielding a net result of nine competencies to measure the leadership performance of FAOs and lieutenants in the San Antonio Fire Department. The 360-degree peer evaluation was based on nine leadership competencies from which 57 rating questions were created. The competencies are (a) communication, (b) teamwork, (c) ethical behavior, (d) general administration, (e) emergency scene management, (f) personal development, (g) community interaction, (h) technical skill, and (i) self-confidence. Communication was one of the competencies from which seven rating questions were developed. Examples of how communication skills were rated include observing how the leadership candidate: (1) conveys his or her thoughts through speech and in writing, (2) exhibits knowledge of communication tactics and techniques, and (3) listens and responds to verbal orders at the fire station and the fire scene. A rating scale was used on the 360-degree feedback to measure the leadership performance of FAOs and lieutenants. The scale ranges from 1 to 5, representing the lowest to the highest ratings, respectively. For example, a rating of 5 reflects exceptional skills, meaning that the individual consistently exceeds expectations of behavior and
  51. 51. 51 skills in this area. A rating of 1 reflects low skills; the individual consistently fails to reach expectations of behavior and skills in this area. The results of the 360-degree peer evaluation were used in correlational and regression analysis to determine the relationship between the criterion and predictor variables. The performance rating scales of the 360-degree peer evaluation were based on questions relevant to the positions of FAO and lieutenant in the fire service. There are no data on the validity and reliability of this instrument; however, the 360-degree peer evaluation was based on observation of behaviors and skills offering face and content validity. Data Collection The instrument was delivered to 225 participants at 12 randomly selected fire stations. Written and verbal instructions were administered to the participants. The estimated time expected for completion of the survey was 20 minutes or less. Upon their completion, the surveys were collected and secured, ensuring the participants’ anonymity. The criterion variable scores were collected from the results of the survey. The predictor variables included (a) the scores earned on the written promotional test, (b) years of service, and (c) years of formal education. The exam scores were obtained from the City of San Antonio Fire and Police Civil Service Commission. The years of service and years of formal education were gathered from the San Antonio Fire Department personnel database.
  52. 52. 52 Data Analysis The Pearson r statistical index was used to describe the degree of strength and direction of relationship between the criterion and predictive variables. Multiple regressions also were conducted to determine the ability of the predictor variables, individually and together, to predict the criterion variable. The multiple regressions were evaluated in the usual manner using the F-test of significance (Agresti & Findlay, 1997). The data collected from the surveys were entered into the SPSS for analysis, which consisted of correlation and regression analyses of the criterion and predictive variables. Ethical Decisions One of the ethical aspects of this study was the anonymity afforded the participants, since the 360-degree peer evaluation involved performance ratings of superiors by their subordinates. To ensure honest and accurate answers, anonymity was enforced by the use of numbers to identify the individual being rated and his or her job performance rating. A separate confidential identification sheet identified the names with their assigned number, but the sheet is kept separate from the 360-degree peer evaluation and was available only to the researcher.
  53. 53. 53 CHAPTER FOUR RESULTS The purpose of this study was to determine whether the present leadership selection process in the San Antonio Fire Department is identifying firefighters with leadership competencies as recognized by the Booth Research Group (1998) and the Rogers’ Firefighters Survey (2001). In addition, the study results suggest relevant predictors used as criteria for selecting firefighters for leadership roles. The results of the study, which are described in the following sections, provide clear and definitive information for the fire service to use in selecting firefighters for leadership. Descriptive Statistics Of Participants The study was conducted on firefighters assigned to the San Antonio Fire Department, which has a total of 48 fire stations. The participants held either the rank of firefighter, FAO, lieutenant, or captain and were assigned to the Fire Suppression Division, to which about 60% (N= 840) of the firefighters in the department (N=1,400) are assigned. Two hundred twenty-five surveys were distributed to 12 randomly selected fire stations where firefighters (N=181) agreed to participate. The surveys, which were completed between March 14 and May 15, 2001, were randomly distributed to 12 fire stations, then collected and secured by the researcher. Although the amount of time needed to complete the surveys varied, on average, participants completed the survey within 45 minutes. Ethnicity. The ethnic groups measured in this study were Hispanic, White, and African- American. Of the 181 participants, 52.5% (N=95) were Hispanic, 38.7% (N=70) were
  54. 54. 54 White, and 8.8% (N=16) were African-American. The ethnic composition of San Antonio’s 1.1 million people is 58.7% Hispanic, 32.5% White, and 6.8% African- American (U.S. Census Bureau, 2000). Table 4 Descriptive Statistics of Participant Ethnicity Rank White Hispanic African American Total FAO 48 49 8 105 Lieutenant 22 46 8 76 Percentage 38.7% 52.5% 8.8% 100% Note. Percentage from U.S. Census Bureau, 2000 Age. Participants in the study ranged from 31 to 59 years of age. The mean was 45.92, with a mode of 53 and a median age of 48 years. There was an increase in the number of firefighters in the 36– to 38–year age bracket and the 49– to 53–year age bracket. Table 5 Descriptive Statistics of Participant Age Age FAO Lieutenant Total 31–35 years 7 3 10 36–40 years 17 29 46 41–45 years 13 9 22
  55. 55. 55 46–50 years 19 27 46 51–55 years 42 4 46 55–59 years 5 6 11 Mean 46.95 44.5 45.92 Sex. The San Antonio Fire Department’s sex makeup is 96 % male and 4 % female. The vast majority of the females are assigned to the EMS Division, with .2% assigned to the fire- fighting division. In this study, the sex of the participants was 98.3% (N=178) males and 1.7% (N=3) females. Table 6 Descriptive Statistics of Participant Sex Rank Male Female Total FAO 102 3 105 Lieutenant 76 0 76 Written Promotional Examination Scores. The written promotional examination scores ranged from 74 to 96 points. The mean was 84.56 with a mode of 86 and a median of 84 points. The standard deviation for the examination scores for the FAO was 6.00 and 4.33 for the lieutenant.
  56. 56. 56 Table 7 Descriptive Statistics of Participant Examination Scores Exam Scores FAO Lieutenant Total 71–75 18 0 18 76–80 28 0 28 81–85 21 33 54 86–90 25 23 48 1–95 13 16 29 95–100 0 4 4 Mean 82.50 87.48 84.56 Years Of Service. The years of service ranged from 4 years to 35 years. The mean was 21.46 years with a mode of 25 years of service. The years of service of 15, 25, and 28 years contained the largest number of participants. The number of participants between 4 and 14 years was few.
  57. 57. 57 Table 8 Descriptive Statistics of Participant Years of Service Years of Service FAO Lieutenant Total 01–10 years 14 1 15 11–20 years 21 40 61 21–30 years 63 25 88 31–40 years 7 10 17 Mean 22.07 20.62 21.46 Years Of Formal Education. The years of formal education ranged from 12 (GED; General Equivalency Degree) to 18 years (master’s degree). The mean was 13.88 years with a mode of 14 years. The majority of the participants (N= 100) had a fire science associate’s degree; the second-largest group of participants (N= 28) were those who had a high school diploma. The number of participants with an undergraduate degree was 15.5% (N=28); only 1.1 % (N= 3) had a graduate degree. The percentage of participants with a graduate degree was almost insignificant.
  58. 58. 58 Table 9 Descriptive Statistics of Participant Years of Formal Education Years of Education FAO Lieutenant Total 12 years 30 6 36 13 years 10 4 14 14 years 50 50 100 15 years 0 1 1 16 years 13 15 28 17 years 0 0 0 18 years 2 0 2 Mean 13.66 14.20 13.88 Hypotheses Hypothesis 1. The first hypothesis of this study was to determine whether there was a significant correlation between the scores on the written promotional examination and the scores of the 360-degree peer evaluation. The written promotional examinations for the ranks of FAO and lieutenant differ in that the study material provided to the FAOs focused on skilled, base questions while the lieutenant’s study material focused on performance dimensions and leadership. The written examination questions reflected the study material. Therefore, the data for the ranks of FAO and lieutenant were split to reflect the difference in examination scores of the two ranks. Although the Pearson r correlation was
  59. 59. 59 used, the variables were not significantly correlated at the .05 level. In Table 10, the resultant correlation coefficient for the rank of FAO was .079, p> .05. For the rank of lieutenant, the correlation coefficient was .082, p> .05. These correlation coefficients indicate that there was no significant relationship at the .05 level between the scores of the written examination and the scores of the 360-degree peer evaluation. Table 10 Research Hypothesis-1 Correlation Promotional Examination Scores Versus 360-Degree Peer Evaluation Rank r Significance (one-tailed) FAO .079 p> .05 Lieutenant .082 p> .05 In Figures 1-A and 1-B, the scattergram for Research Hypothesis 1 correlations indicates that there was no clear pattern of relationship between the scores of the written promotional examination and the scores of the 360-degree peer evaluation. Although somewhat clustered in pairs at the lower right corner of the scattergram, Figure 1-A shows that the plots were scattered throughout the range of scores, indicating no significant correlation between the variables at the .05 level. Figure 1-B shows that the plots were scattered in a positive direction, however, there is no clear patterned of relations between the variables.
  60. 60. 60 300200100 Examsscore 100 90 80 70 Figure 1-A Scattergram for Research Hypothesis 1—Correlation Promotional Examination Scores versus 360-Degree Peer Evaluation for the Rank of FAO 300200100 Examsscore 100 90 80 70 Figure 1-B Scattergram for Research Hypothesis 1—Correlations Promotional Examination Scores versus 360-Degree Peer Evaluation for the Rank of Lieutenant
  61. 61. 61 Hypothesis 2. The second hypothesis of this study was to determine whether there was a significant relationship between the scores of the years of service and the scores of the 360-degree peer evaluation. The data for the ranks of FAO and lieutenant were split to reflect the difference in the years of service of the two ranks. Although the Pearson r correlation was used, the variables were not significantly correlated at the .05 level. In Table 11, the resultant correlation coefficient for the rank of FAO, was .128, p> .05. For the rank of lieutenant, the correlation coefficient was .035, p> .05. These correlation coefficients indicate that there was not a significant relationship between the scores of the years of service and the scores of the 360-degree peer evaluation. Table 11 Research Hypothesis 2—Correlation Years of Service Versus 360-Degree Peer Evaluation Rank r Significance (1-tailed) FAO .128 p> .05 Lieutenant .035 p> .05 In Figures 2-A and 2-B, the scattergram for Research Hypothesis-2 indicates that there was no clear pattern of relationship between the scores of the years of service for the FAO and lieutenant ranks and the scores of the 360-degree peer evaluation.
  62. 62. 62 300200100 YearsofService 40 30 20 10 0 Figure 2-A Scattergram for Research Hypothesis 2—Correlations Years of Service versus 360-Degree Peer Evaluation for the Rank of FAO 300200100 YearsofService 40 30 20 10 0 Figure 2-B Scattergram for Research Hypothesis 2—Correlations Years of Service versus 360-Degree Peer Evaluation for the Rank of Lieutenant
  63. 63. 63 Hypothesis 3. The third hypothesis of this study was to determine whether there was a significant relationship between the scores of the years of formal education and the scores of the 360-degree peer evaluation. The data for the ranks of FAO and lieutenant were split to reflect the difference in years of formal education of the two ranks. Using the Pearson r correlation, the variables were not significantly correlated at the .05 level. In Table 12, the resultant correlation coefficient for the rank of FAO was .079, p> .05. For the rank of lieutenant, the correlation coefficient was -.159, p> .05. These correlation coefficients indicate that there was no significant relationship between the scores on the years of formal education and the scores of the 360-degree peer evaluation. Table 12 Research Hypothesis 3—Correlation Years of Formal Education Versus 360-Degree Peer Evaluation Rank r Significance (1-tailed) FAO .079 p> .05 Lieutenant -.159 p> .05 In Figures 3-A and 3-B, the scattergram for Research Hypothesis 3 correlations indicated that there was no clear pattern of relationship between the scores of the years of formal education with the scores of the 360-degree peer evaluation.
  64. 64. 64 300200100 YearsofEducation 19 18 17 16 15 14 13 12 11 Figure 3-A Scattergram for Research Hypothesis 3—Correlations Years of Formal Education versus 360-Degree Peer Evaluation for the Rank of FAO 300200100 YearsofEducation 19 18 17 16 15 14 13 12 11 Figure 3-B
  65. 65. 65 Scattergram for Research Hypothesis 3—Correlations Years of Formal Education versus 360-Degree Peer Evaluation for the Rank of Lieutenant Hypothesis 4. The fourth hypothesis of this study was to determine whether there was a significant relationship between the scores of the 360-degree peer evaluation and the combination of the scores of the three predictors that included the written promotional examination, years of service, and years of formal education of the FAO and lieutenant ranks. In Table 13, the results of the regression analyses indicated a multiple R of .187, p> .05. Therefore, the hypothesis could not be rejected. Table 13 Regression Analyses of the Predictors and Criterion Variables R F Significance .187 2.128 p> .05 Although not listed as a hypothesis, it is worth noting that the written promotion examination of the combined written promotional scores of the FAO and lieutenant, was significant as a predictor of leadership in the fire service with a correlation coefficient of r= .138, p< .05. As indicated in Table 14, the written promotion examination was the only significant predictor of the three predictors of leadership, as rated by the scores of the 360-degree peer evaluation. Although the written examination was not found to be a significant predictor of leadership in the correlation analysis, it was found to be a significant predictor of leadership in the regression analyses.
  66. 66. 66 Table 14 Regression Analyses of the Predictors and Criterion Predictors r Significance Written Promotion Examination .138 p< .05 Years of Service .072 p> .05 Years of Education .020 p> .05 Note: Both FAO and Lieutenant included in the analysis Summary The first three hypotheses consisted of a correlational analysis using the Pearson r and a one-tailed t-test of significance at the .05 level for each of the FAO and lieutenant ranks. The fourth hypothesis consisted of a regression analysis and an F-test to determine significance with both groups combined. The null hypotheses could not be rejected. However, the results of a regression analysis revealed that the written exam is a significant predictor of leadership when both FAO’s and lieutenants were included in the analysis, as rated by the scores of the 360-degree peer evaluation.
  67. 67. 67 CHAPTER FIVE DISCUSSION This study was undertaken in part to determine whether the leadership selection process in the San Antonio Fire Department identified firefighters with leadership competencies that are relevant to the fire service. Four hypotheses were examined to shed more light on the factors that relate to leadership competencies in the fire service. The first three hypotheses consisted of a correlational analysis using the Pearson r and a one- tailed t-test of significance at the .05 level. The fourth hypothesis consisted of a regression analyses and an F-test to determine significance. The null hypotheses could not be rejected. The regression analyses indicated that the written promotion examination is a significant predictor of leadership, as rated by the scores of the 360-degree peer evaluation. The findings of this study have led to several conclusions about the unique population of its subjects. These conclusions will be summarized in the following section. Conclusions The first research question that was posed and answered was—Does the leadership selection process in the San Antonio Fire Department identify firefighters with leadership competencies that are relevant to the fire service? The answer is yes. The current leadership selection process relies heavily on the written promotional examination. This study has shown that the written promotional exam is a significant predictor of effective leadership in the fire service, with a correlation coefficient of r= .138, p< .05.
  68. 68. 68 The second research question was—What are the relevant predictor criteria used for selecting firefighters for leadership roles in the fire service? Each promotional vacancy requires that firefighters meet certain minimum qualifications in order to be eligible to take the written promotional exam. These qualifications are time in rank and testing for the next rank. For example, an individual who has served in the rank of lieutenant for at least two consecutive years is eligible to take the examination for the next rank, which is captain. Furthermore, a firefighter must attain a score on the written promotional exam high enough to rank he or she for selection to a leadership role. The primary criterion for selecting firefighters for leadership roles relies heavily on the written examination. A regression analysis was conducted on the scores of the 360-degree peer evaluation that identified leadership competencies in the fire service and three predictor variables that included the written promotional exam scores, years of service, and years of formal education. As shown in Table 13, the regression analysis indicated that the written promotional examination was the only predictor among the three that was significant, with a correlation coefficient of r= .138, p< .05. The conclusion is that the written promotional examination continues to be one of the best predictors of effective leadership in the fire service. Discussion The Written Promotional Examination As A Predictor. The definition of ability differs from one study to another. The ability most studied is general cognitive ability also known as general mental ability (GMA), which measures the ability to learn (Hunter and Schmidt, 1996). A sizable amount of research

×