Public Officials' Views on Subjective Performance Measures

Public Officials' Attitudes toward Subjective Performance
Measures
Author(s): Xiaohu Wang and Gerasimos A. Gianakis
Source: Public Productivity & Management Review, Vol. 22,
No. 4 (Jun., 1999), pp. 537-553
Published by: M.E. Sharpe, Inc.
Stable URL: http://www.jstor.org/stable/3380935 .
Accessed: 18/06/2014 16:41
Your use of the JSTOR archive indicates your acceptance of the
Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp
.
JSTOR is a not-for-profit service that helps scholars,
researchers, and students discover, use, and build upon a wide
range of
content in a trusted digital archive. We use information
technology and tools to increase productivity and facilitate new
forms
of scholarship. For more information about JSTOR, please
contact [email protected]
.
M.E. Sharpe, Inc. is collaborating with JSTOR to digitize,
preserve and extend access to Public Productivity
&Management Review.
http://www.jstor.org
This content downloaded from 144.125.40.254 on Wed, 18 Jun
2014 16:41:08 PM

All use subject to JSTOR Terms and Conditions
http://www.jstor.org/action/showPublisher?publisherCode=mes
http://www.jstor.org/stable/3380935?origin=JSTOR-pdf
PUBLIC OFFICIALS' ATTITUDES
TOWARD SUBJECTIVE
PERFORMANCE MEASURES
XIAOHU WANG
GERASIMOS A. GIANAKIS
University of Central Florida
GJovernments have employed measures of agency since the
1940s (Mikesell, 1995).
Recently, there has been a renewed emphasis on
institutionalizing this tool (Gore,
1993; Governmental Accounting Standards Board, 1990;
Osborne & Plastrik, 1997).
Unlike any of the previous performance measurement reforms,
however, this new
effort is mainly a response to external pressures from elected
officials, citizens, and
service clients for management results and accountability rather
than an internal man-
agement tactic for the efficient or effective use of resources
(Cope, 1997; Gore, 1993;
U.S. General Accounting Office, 1997). Consequently, the focus
of performance
measurement is shifted from traditional output and outcome
indicators to service quality

and citizen/customer satisfaction measures. An increasing
number of governments have
been using citizen surveys to assess service quality (Poister &
Henry, 1994; Poister &
Streib, 1998; Streib, 1990).
Despite this shift of focus in performance measurement, critical
questions in using
citizen evaluations have not been addressed in the public
management literature. The
literature focuses almost exclusively on anecdotal discussions
regarding the utilities of
citizen/customer evaluations (Gore, 1993; Osborne & Plastrik,
1997), technical
designs of citizen evaluation measures (Hatry et al., 1992;
Miller & Miller, 1991), and
the influence of service delivery activities on citizen
evaluations (Brudney & England,
1982; Stipak, 1979, 1987). Few studies explore the extent to
which citizen evaluations
influence organizational decision making or governmental
officials' attitudes toward
the use of citizen evaluations-important issues affecting the
ultimate role of citizen
evaluations in government. In this research, we examine the
following questions:
What are the attitudes of public officials toward the use of
citizen evaluations for
assessing public service deliveries? What factors determine
their attitudes? Do differ-
ent public officials prefer different types of performance
measures? and How do offi-
cials expect to use citizen evaluation data in decision making?
A good understanding
of decision makers' attitudes toward citizen evaluations can
facilitate the use of citizen

surveys in the outcome-oriented performance measure systems
expected to be estab-
lished by many governments. Responsiveness and accountability
are enhanced when
Public Productivity & Management Review, Vol. 22 No. 4, June
1999 537-553
? 1999 Sage Publications, Inc.
537
2014 16:41:08 PM
538 PPMR / June 1999
public officials achieve consensus on the use of performance
measures in their
agencies.
Attitudes Toward Subjective Measures:
An Analytical Framework
A subjective performance measure is defined as an indicator
used to assess indi-
viduals' aggregated perceptions, attitudes, or assessments
toward a public sector prod-
uct or service. Subjective measures can be collected from
surveys or interviews with
targeted individuals. Subjective measures are often compared
with objective mea-
sures: service activities, workloads and output levels, or service

achievements docu-
mented and maintained by service providers. The distinction
between objective and
subjective measures lies in whether the measure is based on
empirical observation or,
alternatively, on beliefs, perceptions, or attitudes. This
distinction has been widely
used in public management and program evaluation literature
(Stipak, 1987, pp. 45-46).
In a local police department, for instance, number of arrests and
the crime rate index
are empirically observed and collected; they are objective
measures. On the other
hand, percentage of residents who feel safe and percentage of
residents who rate police
services as excellent are derived from individuals' perceptions
and attitudes; these are
subjective measures. In this article, we focus on citizen
evaluations, although the
intraorganizational assessments of service delivery agencies
conducted by central
management agencies can also be subjective in nature.
Objective measures are widely used by the governments in the
United States
(Poister & Streib, 1998; Tigue & Strachota, 1994). In recent
years, however, there has
been a call for decision makers to use citizen or customer
surveys, citizen focus groups,
and written customer comments (Berman & West, 1995; Gore,
1993; Osborne & Plastrik,
1997). This effort is part of the so-called new performance
measurement, which differs
from the traditional performance measurement systems in value
(accountability vs.
efficiency), focus (service impacts and results vs. internal

management procedures
and activities), performance measure criteria (goal achievement
vs. measurement
validity and reliability), key decision makers (managers vs.
various stakeholders), and
uses of performance measures in decision making (budgeting vs.
management). This
section explores these dichotomies, which form the foundation
for public officials'
attitudes toward subjective measures.
First, the new performance measurement changes the criteria of
evaluating perfor-
mance measures. The purpose of the new performance
measurement is accountability-
that is, through standardized measures and indicators, citizens
and their representa-
tives attain a better understanding of where public resources go
and how effectively
these resources are used (Cope, 1997). Internal managerial
efficiency is secondary to the
ultimate goal of meeting public demands for services.
Individual measures are
designed to evaluate the achievement of public-supported
service goals instead of
internal service workloads, activities, and processes.
Accordingly, the criteria to deter-
mine the merit of a performance measurement system change. A
measure's ability to
address organizational goal achievement is emphasized, while
the values of measure-
ment validity and reliability, which are the core in the
traditional performance mea-
surement systems, are reconsidered.

2014 16:41:08 PM
Wang, Gianakis / SUBJECTIVE PERFORMANCE MEASURES
539
Measurement validity and reliability are core criteria in
traditional performance
measurement systems (Ammons, 1995; Bouckaert, 1993;
Grizzle, 1985; Salzer,
Nixon, Schut, Karver, & Bickman, 1997). They require that a
measure properly
address a service effort or its consequence and that a measure
yield accurate and con-
sistent information (Ammons, 1995; Babbie, 1990). The new
performance measure-
ment minimizes the salience of traditional measurement validity
and reliability criteria
by emphasizing a measure's ability to reflect goal achievement.
In this case, a measure
is considered invalid if it does not address at least one aspect of
goal achievement
(Bouckaert, 1993; Hatry et al., 1992). Many objective measures
used in government
are procedure-, activity-, or workload-related, and they do not
directly address goal
achievement; they have limited utility in the new approach,
even though they may be
valid and reliable in measuring service activities. For example,
apprehension rate, a
frequently used objective measure in local police departments,
does not hold much
value in the new system because it does not have much to say

about the goal of local
policing-how much crime is being reduced or prevented. On the
other hand, because
subjective measures are often associated with goal achievement,
they play a signifi-
cant role in the performance measurement system, even though
they often lack meas-
urement validity and reliability due to evaluators' subjectivity
and their sensitivity to
socioeconomic conditions (Brudney & England, 1982; Stipak,
1979, 1987). For exam-
ple, citizen satisfaction rates for public safety, a subjective
measure, may change
because of the changes in the local economy, community
recreation activities, or in
response to a critical incident, rather than as a function of any
effort of the police. A
detailed discussion of the relationship between measurement
validity, reliability, and
subjective measures appears in the results section below and in
the notes.
Second, the dominant role of public managers in the design and
implementation of
performance measurement gives way to a decision-making
model that involves a vari-
ety of governmental stakeholders. Public managers have been
the key decision makers
in traditional performance measurement systems that target
management operations
and policy implementation. Objective measures are often used
because they empiri-
cally record managers' main responsibilities in these activities:
to transform resources
into services or products. The new performance measurement,
on the other hand, asks

for active participation of citizens and their representatives
(Broom, 1995; Melkers &
Willoughby, 1998), which ultimately demands the use of
subjective measures.
Third, performance budgeting is emphasized. Governments use
performance
measures for management and budgeting purposes
(Congressional Budget Office,
1993; U.S. General Accounting Office, 1993). As a management
tool, performance
measures help managers monitor operations and improve
communication among
managers, elected officials, and citizens (Swiss, 1995).
Performance measures also
help managers specify management objectives, identify
operational problems in man-
agement, and present alternative solutions to solve service
delivery problems (Poister,
1995). As a budgeting tool, a measure helps an agency outline
its service objectives,
identify funding alternatives, establish funding priorities, and
service funding levels.
The new performance measurement advocates performance
budgeting in which gov-
ernments are held financially accountable and responsible for
their performances
(Martin, 1997; Melkers & Willoughby, 1998). Performance
information becomes a
part of future resource allocation deliberation.
2014 16:41:08 PM

Fourth, performance measurement systems should become more
responsive to
socioeconomic and organizational changes. The traditional
ment focuses on internal managerial efficiency, which is
determined by levels of
resource consumption (inputs), system outputs, and the state of
available technology.
In the new performance measurement system, a government's
organizational and
socioeconomic situations are also considered, and the design
and implementation of
performance measures should reflect differences in
organizational and socioeconomic
variables (Kravchuk & Shack, 1996; "Management Information
Service Report,"
1995). For instance, in a populous and culturally diversified
area, public officials may
feel that it is difficult to satisfy different groups of residents
simultaneously, and there-
fore, they may prefer objective to subjective measures. Public
officials in wealthy
communities may be more likely to prefer subjective measures
because they may per-
ceive a stronger taxpayer demand for quality service. Objective
measures such as
criminal arrests and investigations may be perceived as critical
in a high-crime area.
An official may find such measures inappropriate for a low-
crime community. An
organization that centralizes its power at the executive level
may emphasize resource

control rather than the effective use of resources to meet
residents' demands (Gore,
1993; Osborne & Gaebler, 1992). Objective measures are more
suitable for resource
control purposes because, unlike subjective measures of
individual perceptions, objec-
tive measures are directly related to the processes of production
and service delivery.
Consequently, public officials in a centralized management
system may prefer objec-
tive measures to subjective citizen evaluations.
In addition, fiscal stress can stimulate an organization to
improve managerial effi-
ciency and productivity, which are often operationalized as a
variety of objective
measures (Morley, 1986). Thus, fiscal stress may encourage
public officials to focus
on objective measures. Moreover, existing performance
measurement systems may
also influence public officials' attitudes. Public officials in
agencies that have adopted
subjective measures may have a better understanding of these
measures and less fear of
the potential pitfalls of implementing them. Thus, they are more
likely to have a posi-
tive attitude toward these measures than do other officials.
This research examines these characteristics of the new
approach to performance
measurement in an effort to assess the viability of this
approach. Clearly, potential con-
flicts with existing measurement systems are apparent. These
conflicts may center on
additional criteria for effective measurements identified by
Bouckaert (1993), namely,

the legitimacy and acceptability of the measures as perceived by
those who use them
and are affected by them.
Methodology
To measure public officials' attitudes toward performance
measures, we asked
respondents to evaluate a list of actual performance measures
that are widely used in
government. This way of measuring attitudes creates a close-to-
reality situation for
respondents and avoids the random errors that are associated
with the ambiguity inher-
ent in abstract concepts. This method necessitates that we focus
on one type of munici-
pal service. Municipal police services were selected for several
reasons. Policing is
one of the most important and expensive services in local
governments (Mikesell,
2014 16:41:08 PM
541
1995). Performance measures are more frequently reported in
local police depart-
ments than in any other local agency or service (Tigue, 1994).
The implementation of
subjective measures in local police departments has been

studied frequently (Brown &
Coulter, 1983; Parks, 1984; Stipak, 1979, 1987). Local officials
often understand and
are concerned with performance information in their police
departments. Finally, the
scope of the local police function is well defined. A typical
local police department is
responsible for patrol, apprehension, investigation, traffic
control, and administration.
Well-defined policing functions facilitate the development of
performance measures
and the generalization of research results.
This study combines a case study and a survey. Because the
literature regarding
public officials' attitudes toward performance measures is very
limited, we first con-
ducted an exploratory case study to generate appropriate
propositions to be explained
in the survey. The case study included interviews and field
observations and was con-
ducted in three municipalities in south Florida. All the cities
reported using perfor-
mance measures. One municipality had surveyed its citizens for
two consecutive fiscal
years. That municipality used the survey "to provide city
government and citizens with
a systematic understanding of the attitudes of residents toward
key issues facing the
city as well as an assessment of the services provided by the
city." The second city had
used performance measures in a strategic management plan for
5 years. In that plan,
each department displays its "strategic management program in
a format that indicates
objectives, workload indicators, and performance effectiveness

measures as they
relate to city-wide goals." The city manager's office defined
performance measures as
efficiency indicators ("measures of cost per unit of output"),
productivity indicators
("measures of man-hours needed to complete a task or unit of
work"), and citizen satis-
faction indicators ("measure the satisfaction of citizens who
receive or use the work
product or service"). The performance measures in the third city
were mostly output
measures. In a budget instruction that indicated management
expectations for achieve-
ments of each department, an assistant city manager emphasized
efficiency measures.
Over a 6-month period, we interviewed elected officials, city
managers, police
chiefs, and finance directors in these cities. These interviews
consisted of a series of
questions concerning interviewees' attitudes and reasoning
toward subjective mea-
sures. In addition, we examined relevant local documents such
as budgets, manage-
ment memoranda, tape recordings of community and council
meetings, and commu-
nity newsletters.
A survey instrument was then developed and sent to public
officials in all general-
purpose municipalities with populations of 2,500 or greater in
Dade, Broward, and
Palm Beach counties, Florida. The selected municipalities
comprise 99.1% of the
municipal population in this tricounty area. This narrow focus
limits the generalizabil-

ity of the findings, but the police agencies in the targeted
municipalities have similar
functions as established by state statutes, and we were able to
monitor recent public
events that may have affected the survey responses. In addition,
the case study data are
more directly related to the survey data. This combined case
study and survey
approach is highly recommended for a poorly defined area of
study, such as perfor-
mance measurement, where survey data are not always valid and
reliable (Congres-
sional Budget Office, 1993). This article centers on the results
of the survey, and the
case study data are used to explore the findings.
2014 16:41:08 PM
Four categories of local personnel were surveyed: (a) elected
officials (including
mayors, vice mayors, and council or commission members), (b)
city managers and
assistant city managers, (c) police chiefs and assistant police
chiefs, and (d) finance
directors and assistant finance directors. Two persons were
randomly selected in each
category in each city. For those cities that did not have assistant
city managers (or assis-
tant finance directors or assistant police chiefs), only one

person was selected. A pre-
test of 10 public officials was conducted to modify the content,
wording, and sequence
of the survey. Of the 446 people in the sample, 178 completed
and returned the survey,
for a response rate of 40%. This level of response is considered
to be adequate for a
mailed survey (Nachmias & Nachmias, 1987). The income and
population distribu-
tions of the respondents' cities match those of the surveyed
cities, which indicates that
the respondents are representative of the sampled public
officials on these dimensions.
Results
WHAT TYPES OF PERFORMANCE MEASURES
DO PUBLIC OFFICIALS PREFER?
In the survey, the respondents were asked to evaluate 12 police
performance mea-
sures including 3 subjective measures and 9 objective measures.
These performance
measures were selected because they have been frequently used
to measure police per-
formance in the criminal justice literature, they are crime
related, and they are all self-
explanatory. To ensure the valid representation of these
measures, we consulted law
enforcement officers, criminal justice scholars, and municipal
decision makers about
the selection.
Among the 78.7% (140 of 178) of the respondents who
indicated that their jurisdic-
tions use performance measures, 23.6% (42 of 178) said that

they used citizen evalua-
tions. Table 1 shows the means and standard deviations of the
12 selected police per-
formance indicators on a scale of 1 (not favorable) to 5 (very
favorable). The results
indicate that public officials rate the three subjective measures
higher than most objec-
tive measures and that the overall mean difference between
objective and subjective
performance measures (0.53) is statistically significant at .01
level.
Table 2 shows that the preference differences between elected
and appointed offi-
cials are limited for performance measures. The mean difference
between elected offi-
cials and public managers is 0.14 on a 5-point scale for
objective measures, and it is
0.02 for subjective measures. Table 3 shows a break down of
public managers and
depicts details regarding how city managers, police chiefs, and
finance chiefs evaluate
subjective and objective measures. One interesting finding is
that among all public
officials finance officials are least favorable toward subjective
measures. They are also
most favorable toward objective measures. The preference
difference between these
two types of performance measures (0.02) is not statistically
significant. Financial
officials' preference for objective measures may be attributed to
their roles in an
organization's decision-making processes. Local finance
officers' primary duties in
controlling, monitoring, and auditing service costs require them
to handle objective

information almost exclusively. They may also perceive
conceptual difficulty in using
subjective measures in their work. One finance officer argues
that "residents' demand
2014 16:41:08 PM
543
Table 1. Public Officials' Attitudes Toward Individual
Performance Measures
Number of Standard
Respondents Mean Deviation
Subjective measures
Percentage of residents rating police
service as excellent or good, by reason
for satisfaction 175 4.14 0.86
Percentage of residents who feel police
are generally fair and courteous in dealing
with them 176 4.16 0.80
Percentage of residents who feel safe 176 4.23 0.80
Overall for subjective measures 175 4.16 0.69
Objective measures
Number of calls for services 176 4.20 0.84
Percentage of crimes solved by police

(clearance rates) 174 4.13 0.91
Number of reported crimes per 1,000
population (crime rates) 175 3.85 1.05
Number of reported plus unreported
crimes (victimization rate) 174 3.41 1.13
Patrol miles per police employee 174 2.87 1.27
Number of police investigations per year 175 3.72 1.08
Response time 175 4.11 1.05
Number of arrests per police employee 175 3.18 1.09
Number of patrol dispatches per
police employee 175 3.29 1.12
Overall for objective measures 171 3.64 0.68
Overall mean difference between subjective
and objective measures 0.53*
Notes. The respondents' attitudes toward performance measures
are coded on a 5-point scale: I = notfavor-
able with the specified type ofperformance measures; 2 =
lessfavorable; 3 = neutral; 4 =favorable; 5 = very
favorable. Means and standard deviations calculated use this
scale.
*p < .01 (for a t test for mean difference).
for a level of police services changes very often.... It is
extremely difficult to satisfy
it." Another doubts his agency's capability to obtain reliable
citizen survey data and
suggested the establishment of an auditing entity in his agency
for the purpose of over-
sight. In addition, some finance officers may perceive that the
operation of local
finance departments could not be enhanced directly by citizen

evaluations because
these departments do not serve citizens directly.
Interestingly, it is police chiefs, not elected officials, who rate
the subjective mea-
sures the highest. They are also least favorable to objective
measures. Many police
chiefs noticed the drawbacks of traditional objective measures
such as the crime rate
index, response time, and clearance rate. Interviews indicate
that they believe that
these measures can be manipulated easily. "Increasing police
patrol effort could even-
tually lead to the increase of reported crimes [therefore the
crime rate index], not the
decrease of the crimes, because police report more crimes."
"The crime rates in my
community could be influenced by our neighbor community's
crackdown on crimes."
Additionally, police chiefs' attitudes appear to be influenced by
the concept of
2014 16:41:08 PM
Table 2. Elected and Appointed Officials' Attitudes Toward
Performance Measures
Objective Measures Subjective Measures

M SD M SD
Elected officialsa 3.75 0.58 4.17 0.53
Public managers 3.61 0.70 4.19 0.69
Mean differencesb 0.14 0.02
a. The means and standard deviations are calculated from
respondents' preferences for individual perfor-
mance measures. The measure scale is 1 = notfavorable with the
specified type ofperformance measures;
2 = less favorable; 3 = neutral; 4 = favorable; 5 = very
favorable.
b. The mean differences between elected officials and public
managers are not significant at the .05 level
from a t test of mean difference.
Table 3. Public Officials' Attitudes Toward Performance
Measures by Position
Objective Measures Subjective Measures
M SD M SD Mean Differencea
Elected officialsb 3.75 0.58 4.17 0.53 0.42*
City managers 3.55 0.71 4.16 0.69 0.61*
Police chiefs 3.50 0.74 4.32 0.58 0.82*
Finance directors 3.90 0.54 3.92 0.81 0.02
a. These are the mean differences between objective and
subjective measures for each group of public offi-
cials.
b. The means and standard deviations are calculated from public
officials' preferences for individual per-
formance measures. The measure scale is I = notfavorable with
the specified type ofperformance measures;
2 = less favorable; 3 = neutral; 4 =favorable; 5 = very

favorable.
*p < .01 (for a t test of mean difference).
community-oriented policing, which encourages proactive
policing activities includ-
ing the active participation of residents in policing in various
ways. Many chiefs indi-
cate that community-oriented policing techniques can influence
citizen evaluations
toward policing and police departments, as one police chief
clearly illustrated:
If everyone comes out to walk at 9 o'clock at night, how could
you have any crime? No
criminal is going to commit a crime when there are people
watching. So the safer they
feel, the more people come out. The more they come out, less
crimes are committed.
In sum, disillusionment with traditional objective measures and
high expectations for
citizen-related policing techniques may determine police chiefs'
overwhelming pref-
erence for subjective measures.
WHY DO PUBLIC OFFICIALS PREFER
SUBJECTIVE PREFERENCE MEASURES?
We constructed an index to measure public officials' attitudes
toward subjective
measures. This index, the dependent variable, consists of the
three subjective mea-
sures. A score of 5 on this index indicates that a respondent
strongly prefers subjective
measures, whereas a score of 1 indicates that a respondent does
not favor subjective

2014 16:41:08 PM
545
Table 4. Determinants of Public Officials' Preferences for
Subjective Measures
Variable Beta t Value Significance Level
GOAL .195 2.464 .015*
VALIDITY -.141 -1.777 .077
RELIABILITY -.067 -.802 .424
MANAGEMENT .313 4.110 .000*
BUDGET .145 1.860 .072
STATUS .010 .155 .877
POPULATION -.128 -1.801 .073
INCOME .109 1.449 .149
STRUCTURE .090 1.362 .175
STRESS -.076 -1.051 .295
USESUB .314 4.647 .000*
Constant 3.900 .000*
2 R =313.
F probability = .000.
Note. GOAL = assessment of goal relevance of performance
measures; VALIDITY = assessment of validity
of performance measures; RELIABILITY = assessment of
reliability of performance measures; MANAGE-

MENT = use of performance measures for management;
BUDGET = use of performance measures for
budgeting; STATUS = a respondent's status (1 = elected
officials; 0 = appointed officials); POPULATION =
population of 1994; INCOME = median household income in
1994; STRUCTURE = proportion of general
government expenditures in general fund expenditures-a
measure of the degree of centralization of opera-
tion; STRESS = local taxes per capita-a measure of fiscal stress;
USESUB = use of subjective measures in
a respondent's city.
*p < .05.
measures at all. A statistical model is built to test the
significance of the new perfor-
mance measurement model outlined above. The F test in Table 4
shows that this model
is significant at the .000 level with 31.3% of variation of the
subjective index being
explained. For comparison, this model was also regressed on an
index that consists of
the nine objective measures. This objective model is also
significant at the .000 level;
however, the model explains only 27.7% of variation of the
objective index. Correla-
tion studies show that there is no multicollinearity for
respondents' attitudinal vari-
ables in the models, and residual analyses show no violation of
heteroskedasticity in
the models. We also find that, under alternative model
specifications, there is little
change in the directions and sizes of estimated coefficients. The
findings examined
below are robust.
First, public officials prefer subjective measures to assess an

agency's goals or mis-
sions rather than daily managerial operations and procedures. In
the survey, respon-
dents were asked to rate a survey statement concerning two
main policing goals in
local governments: crime prevention and community well-being.
On a 5-point scale
from 1 (strongly disagree) to 5 (strongly agree), respondents
were asked if perfor-
mance measures must address the achievement of the goal of
local policing: protecting
people and properties (GOAL). Table 4 shows that GOAL is
significantly associated
with public officials' preferences for subjective measures, which
indicates public offi-
cials' tendency to address organizational goals and
achievements with subjective
measures instead of objective measures. Table 5 shows that
GOAL is not significantly
2014 16:41:08 PM
associated with a preference for objective measures. The
interviews show that objec-
tive performance measures, such as number of arrests and patrol
miles, mean little for
many public officials if residents are not satisfied. They believe
that "residents are the
customers of public services and only residents know if the

police department does a
good job." Citizen evaluation is the bottom line of police
services.
This finding reflects the tenants of the new performance
measurement. It supports
the recent effort of many governments to build mission- and
result-oriented govern-
ments using outcome measures. This finding challenges current
performance mea-
surement systems, which consist mainly of objective output
information for internal
managerial and control purposes (Tigue & Strachota, 1994; U.S.
General Accounting
Office, 1997), to include more outcome- and result-oriented
measures such as citizen
evaluations.
Second, public officials do not consider measurement validity
and reliability when
they select subjective measures. In this study, measurement
validity is defined as a
measure's capacity to measure what it is supposed to measure. A
performance measure
is valid oniy if it measures the activities or results of a
governmental service. This defi-
nition is widely accepted in the public management and program
evaluation literature
(Ammons, 1995; Grizzle, 1985; Salzer et al., 1997).' During the
case study portion of
this research project, some public officials were concerned
about performance mea-
sures' capacities to address "true" government performances.
For example, one police
chief challenged the validity of the crime rate index, arguing
that reducing crimes is an

unrealistic demand for the police "because crimes happen no
matter what the police do
to prevent them." One financial officer concluded that "police
performance could be
better measured by number of investigations because it is the
direct result of police per-
formances.... Crime reduction is often not," and "number of
patrols is a realistic and
attainable measure for the police." In the survey, respondents
were asked to evaluate
these statements on a 5-point scale from strongly disagree to
strongly agree. An index
consisting of all these statements was constructed (VALIDITY),
a larger value of
VALIDITY indicates a higher concern about measurement
validity of performance
measures.
Measurement reliability is defined as the extent to which a
measure is free from ran-
dom measurement error (Judd, Smith, & Kidder, 199 1).2
Random errors occur when
the definition of a measure changes (e.g., extending the
definition for serious crimes)
or when evaluators change their reporting methods or standards.
A few officials,
including a city manager and a finance director, pointed out that
citizens are not reli-
able evaluators because "their demands for a level of police
services changes very
often," and "residents' perception of police performance often
does not reflect the real-
ity of police performance." In the survey, we measured public
officials' assessments of
measurement reliability through their evaluations of several
statements concerning

subjective measures. An index consisting of these statements
was created
(RELIABILITY), with a larger value indicating a higher
concern for measurement
reliability.
Table 4 shows that neither VALIDITY nor RELIABILITY is
associated with public
officials' preferences for subjective measures. Public officials'
concern for measure-
ment validity and reliability does not influence their selections
of subjective measures.
One explanation of this finding is that many public officials do
not fully understand
2014 16:41:08 PM
547
Table 5. Determinants of Public Officials' Preferences for
Objective Measures
Variable Beta t Value Significance Level
GOAL -.095 -1.197 .097
VALIDITY .112 1.412 .160
RELIABILITY -.226 -2.675 .008*
MANAGEMENT .156 1.944 .054
BUDGET .320 4.120 .000*
STATUS .018 .260 .796

POPULATION -.216 -2.927 .004*
INCOME -.028 -.360 .720
STRUCTURE .019 .279 .781
STRESS .003 .035 .972
USEOBJ .116 1.670 .097
Constant 4.797 .000*
2 R =277
F probability = .000.
Note. GOAL = assessment of goal relevance of performance
measures; VALIDITY = assessment of validity
of performance measures; RELIABILITY = assessment of
reliability of performance measures; MAN-
AGEMENT = use of performance measures for management;
BUDGET = use of performance measures for
budgeting; STATUS = arespondent's status (I = elected
officials; 0 = appointed officials); POPULATION =
population of 1994; INCOME = median household income in
1994; STRUCTURE = proportion of general
government expenditures in general fund expenditures-a
measure of the degree of centralization of opera-
tion; STRESS = local taxes per capita-a measure of fiscal stress;
USEOBJ = use of objective measures in a
respondent's city.
*p< .05.
subjective measures or the consequences of using invalid and
unreliable measures. For
example, some interviewees wrongly believed that the targeted
levels of citizen
evaluations could be achieved with ease. One police chief
claimed that his department
had achieved and sustained a 90% citizen approval rating. A
careful analysis of the sur-
vey questions and the responses showed that the citizens'

approval rate for major local
police services had been around 75% for two consecutive years
(1994 and 1995). At
the same time, the city's crime statistics did not show a
significant improvement, and
more than half of the residents believed that crime was a
problem for the city (58% in
1994 and 63% in 1995).
Another explanation of public officials' dismissal of
measurement validity and reli-
ability is that, in general, they do not feel pressure to use valid
and reliable measures.
As one interviewee described, "When performance measures are
used for superficial
purposes, people do not care about the validity of performance
measures. They select
sound-good ... measures." On the other hand, this result
suggests that a perfectly valid
and reliable measure may not have much utility to public
officials if it does not address
what they want-service achievements and results.
Third, the study finds that public officials are more likely to
select subjective mea-
sures for management purposes. As discussed, performance
measures can be used in
management and resource allocation decisions. In the survey,
respondents were asked
to rate several items assessing these uses. These items included
"monitoring and evalu-
ating police service procedures to identify service problems"
(management), "improv-
ing communication in management" (management), "comparing
with benchmarks"

2014 16:41:08 PM
(management), "making budget allocation decisions in setting
up funding priority and
funding levels" (budgeting), and "evaluating performance to
assign financial benefits"
(budgeting). We developed a management index that consists of
all items concerning
the management functions of performance measures
(MANAGEMENT) and a budg-
eting index that integrates all items regarding the budgeting
function of performance
measures (BUDGETING). A significant relationship between
MANAGEMENT and
public officials' preferences for subjective measures in Table 4
indicates that public
officials believe that subjective measures offer appropriate
guidance to improve man-
agement operations. At the same time, public officials prefer
objective measures in
resource allocation processes (see Table 5).
Because performance budgeting entails financial risks, such as
losing funds and
positions, public officials may feel an intensified need to have
some level of control
over performance results. Citizen evaluations are often more
difficult to control than
objective measures are. When it comes to rating these

subjective measures, those pub-
lic officials who favor the budgeting use of performance
measures are more likely to
select objective measures because these measures are more
closely associated with
managerial operations and activities. The results of these
measures are relatively easier
to control. In the case study, two interviewees-a city manager
and a finance director-
who thought that performance measures should be brought into
their budgeting sys-
tems, stated that "citizens' perception is not reality" and
"residents' demand for public
services changes over time, so it is difficult to satisfy it." In the
interviews, we asked
them to select five favorites among a list of performance
indicators. Nine of their 10
selections were objective measures. This objective-subjective
ratio is much higher
than that of the selections of the rest of interviewees who did
not emphasize the impor-
tance of using performance measures in budgetary decision
making.
These findings suggest that the governments must articulate the
purposes of their
performance measurement systems. Although subjective
measures are proper for a
performance management system, governments could face
potential resistance from
inside when subjective measures are used for allocating
budgetary resources. These
results also imply that the implementation of subjective
measures in government
requires a gradual learning process. Consistent performance
management practice

may improve communication among public officials and,
therefore, reduce officials'
fear and misunderstanding of performance measurement, which
eventually helps a
government build a successful performance budgeting system.
Finally, socioeconomic situations and organizational
arrangements do not appear
to have a significant impact on public officials' preferences for
subjective measures.
As discussed in the literature, socioeconomic variation could
influence public offi-
cials' selection of performance measures. In this study, we also
collected demographic
information for population (POPULATION), median house
income (INCOME), and
tax per capita (TAXPER) to measure the tax burden and tax
potential in a community.
These variables were selected because the data are readily
available and are often used
in social science studies as indicators of a community's
socioeconomic condition. We
were trying to minimize the number of such variables in the
regression model because
of the threat of multicollinearity (socioeconomic variables are
often highly associ-
ated). No significant relationship emerged between public
officials' preferences for
2014 16:41:08 PM

549
subjective measures and these variables. This finding does not
support the suggestion
of the new performance measurement that the focus of
performance measurement
should change with the change of socioeconomic conditions in
communities. How-
ever, objective demographic variables may influence public
officials' subjective pref-
erences in a complex and indirect way.
In the model, the proportion of central management
expenditures in the general
fund expenditures (STRUCTURE) was used as a proximate
estimate of governmental
structure. A high ratio of this proportion suggests that
management functions-such
as finance, personnel, purchasing, and fleet-are centralized in
central management
offices, whereas a low ratio indicates a government may
delegate these management
functions to individual departments. The result in Table 4 shows
there is no relation-
ship between public officials' preferences for subjective
measures and STRUCTURE.
Another interesting observation in the case study was that those
officials in the
communities where subjective measures were used seemed more
likely to support
these measures. The survey data supports this observation. In
Table 4, a positive sig-
nificant relation is found between use of subjective measures
(USESUB) and public

officials' preferences for subjective measures. This result
implies that knowledge and
information about subjective measures increase public officials'
confidence to use
these measures.
Conclusion
Governments are facing increasing citizen demands for service
quality and man-
agement accountability. It is clear that some governments have
made an effort to incor-
porate citizen evaluations into their measurement systems. They
are using citizen sur-
veys to identify service needs, to evaluate service trends, and to
communicate with
other stakeholders. This study shows that public officials prefer
subjective measures,
such as citizen assessment of agency performance, because
these measures are directly
related to an agency's public-supported goals rather than simply
reporting the activi-
ties or outputs of an agency's operational processes. This
finding suggests a need to
change the typical performance measurement system that mainly
collects process-
related, objective performance indicators. Governments should
improve their capaci-
ties to develop and implement subjective measures. Public
administration scholars and
public managers should further their understanding of some
critical issues such as
validity and reliability of subjective measures and the
relationships between objective
and subjective measures.

However, public officials' preferences for subjective measures
are conditional.
This study shows that public officials prefer the use of
subjective measures only for
management purposes. They become more cautious in using the
measures for allocat-
ing financial resources. The attitudes of public officials toward
subjective measures
may change with a shift in the focus of performance
measurement toward resource
allocation. Public officials should have a clear purpose for their
ment systems. They should realize that different purposes might
result in different atti-
tudes toward performance measures. This finding also suggests
a need to build a per-
formance measurement system that includes a variety of
measures. This variety,
2014 16:41:08 PM
however, may threaten the perceived legitimacy of the
performance measurement sys-
tem, which requires a consistent focus in order to avoid
contradictory results. Mea-
sures that satisfy agency personnel may not be acceptable to
other stakeholders.
This study has served to highlight the critical issue of how

subjective measures are
perceived by public officials. We acknowledge the limited
generalization of the find-
ings: They are based on the police function in the municipalities
of three south Florida
counties. We combined a case study approach with a controlled
survey so that we
would observe what was occurring in this area. This laboratory
approach to survey
research yielded important and surprising findings regarding
public officials' percep-
tions of subjective assessments. We speculate here that these
favorable attitudes may
reflect the idea that citizen surveys draw each citizen to focus
on his or her own idea on
what the police are and should be doing rather than on
particular elements of the police
mission.
Clearly, legislatures are pushing public agencies to adopt
outcome-oriented per-
formance measure systems (Broom, 1995; Gore, 1993; Melkers
& Willoughby, 1998;
U.S. General Accounting Office, 1997). The next goal for the
agencies will be to apply
outcome measures, including subjective assessments, in judging
their management
and budgetary decisions (Martin, 1997; U.S. General
Accounting Office, 1997). A
critical question for public officials will be, How do a
government's management and
budget decisions affect citizen subjective evaluations? More
specifically, Does an
increase in service effort lead to a higher rating of residents'
evaluation for the service?
For example, Do increased patrol hours and patrol visibility

improve the residents' sat-
isfaction rate for public safety? Research has shown that
detecting such a relationship
is often difficult. Citizen evaluations may be influenced by
many factors in complex
and changing ways. Unraveling this complex relationship poses
a real challenge for
public administration scholars and practitioners. Failing to meet
this challenge, the
current performance management movement may be relegated to
fad status and gradu-
ally fade away without resulting in any substantial progress in
the way governments
operate.
Notes
1. Validity refers to a construct's ability to measure an intended
attribute (Judd, Smith, & Kidder, 1991).
The same reasoning applies to performance measures.
Performance measures are quantifiable indicators
used to measure activities or their consequences in the provision
of a service. A performance measure in-
cludes two core constructs. First, a performance measure must
measure activities or the consequences of ac-
tivities. A performance measure lacks validity when it fails to
measure activities or their consequences. For
instance, expenditures per capita, or average cost, lacks validity
when used to assess the performance of a
service delivery system because it indicates the dollar values of
the service spent rather than detailing the ac-
tivities of supplying the service (Greytak, Phares, & Morley,
1976). Second, the activities or their conse-
quences must come from a service being measured rather than
from other factors (Salzer, Nixon, Schut,
Karver, & Bickman, 1997). A performance measure lacks

validity when it measures factors rather than the
service. Forinstance, aperformance measure in apolice
department is valid only if it measures the portion of
police efforts devoted to public safety. More specifically, if the
objective of a police department is to reduce
crime, a valid measure of this objective is the number of crimes
reduced by police efforts. By comparison,
the index crime rate lacks validity because it also measures
undesired impacts of numerous socioeconomic
factors that are not under police control and does not measure
the amount of unreported crime reduced by po-
lice efforts (Hatry et al., 1992, p. 75; Wycoff & Manning, 1983,
p. 16). In general, compared with objective
2014 16:41:08 PM
551
measures, valid subjective measures are more difficult to
develop because citizen perceptions are also a
function of many psychological and socioeconomic variables
besides service effort (Salzer et al., 1997).
2. Measurement reliability of a measure refers to its ability to
yield the same result when it is applied re-
peatedly to the same construct (Babbie, 1990). The
measurement standards of a reliable measure do not
change. Most output measures satisfy this condition. For
instance, the change in the number of arrests from
10 to 20 or from 100 to 200 represents the same percentage

increase (100%) for a given population. Reliabil-
ity concerns two conditions: (a) The individual evaluator must
be well informed on the object he or she
measures, and (b) the scales an evaluator uses to measure the
object must be identical if he or she is asked to
measure the object again (Babbie, 1990; Pedhazur & Schmelkin,
1991). It is often difficult for a subjective
measure to satisfy these conditions. For instance, when
percentage of residents who rate police performance
as excellent is used to measure police performance, better
police performance (measured by output or objec-
tive outcome measures) may not improve the rating simply
because residents' demands and standards for the
police services have changed. Asking residents to count the
crime events they encounter may not be reliable
because they may not accurately remember the information. In
general, subjective measures are considered
less reliable than objective measures. A public official who is
concerned about the reliability of performance
measures may hesitate to select such measures.
References
Ammons, D. N., (1995). Performance measurement in local
government. In D. Ammons (Ed.), Account-
ability for performance measurement and monitoring in local
government (pp. 15-32). Washington,
DC: International City/County Management Association.
Babbie, E. (1990). Survey research methods. Belmont, CA:
Wadsworth.
Berman, E. M., & West, J. P. (1995). TQM in American cities:
Hypotheses regarding commitment and im-
pact. Journal of Public Administration Research and Theory,
5(2), 213-230.

Bouckaert, G. (1993). Measurement and meaningful
management. Public Productivity & Management Re-
view, 17(1), 31-43.
Broom, C. A. (1995). Performance-based government models:
Building a track record. Public Budgeting
and Finance, 15, 3-17.
Brown, K., & Coulter, P. B., (1983). Subjective and objective
measures of police service delivery. PublicAd-
ministration Review, 43, 50-57.
Brudney, J. L., & England, R. E. (1982). Urban policy making
and subjective service evaluations: Are they
compatible? Public Administration Review, 42, 127-135.
Congressional Budget Office. (1993). Using performance
measures in thefederal budget process. Washing-
ton, DC: Author.
Cope, B. H., (1997). Bureaucratic reform and issues of political
responsiveness. Journal of Public Admin-
istration Research and Theory, 7(3), 461-471.
Gore, A. (1993). Creating a government that works better and
costs less: Report of the National Perfor-
mance Review. Washington, DC: Government Printing Office.
Governmental Accounting Standards Board. (1990). Service
efforts and accomplishments reporting: Its
time has come: An overview. Norwalk, CT: Governmental
Accounting Standards Board of the Financial
Accounting Foundation.
Greytak, D., Phares, D., & Morley, E. (1976). Municipal output

and performance in New York City. Lexing-
ton, MA: D.C. Heath.
Grizzle, G. A. (1985). Performance measures for budget
justifications: Developing a selection strategy.
Public Productivity Review, 9, 328-341.
Hatry, H. P., Blair, L., Fisk, D., Greiner, J., Hall, J. Jr., &
Schaenman, P. (1992). How effective are your com-
munity services? (2nd ed.). Washington, DC: Urban Institute
and International City/County Manage-
ment Association.
Judd, C. M., Smith, E., & Kidder, L. (1991). Research methods
in social relations (6th ed.). Chicago: Holt,
Rinehart & Winston.
Kravchuk, R. S., & Shack, R. W. (1996). Designing effective
performance measurement systems under the
Government Performance and Results Act of 1993. Public
Administration Review, 56, 133-140.
2014 16:41:08 PM
Management Information Service Report. (1995). In
Performance measurement in law enforcement (Vol.
27, No. 3). Washington, DC: International City/County
Management Association.

Martin, L. (1997). Outcome budgeting: A new entrepreneurial
approach to budgeting. Journal of Public
Budgeting, Accounting and Financial Management, 9, 108-126.
Melkers, J., & Willoughby, K., (1998). The state of the states:
Performance-based budgeting requirements in
47 out of 50. Public Administration Review, 58, 66-73.
Mikesell, J. L. (1995). Fiscal administration: Analysis and
application for the public sector. Belmont, CA:
Wadsworth.
Miller, T. I., & Miller M. A. (1991). Citizen survey.
Washington, DC: Intemational City/County Manage-
ment Association.
Morley, E. (1986). A practitioner's guide to public sector
productivity improvement. New York: Van Nos-
trand Reinhold.
Nachmias, D., & Nachmias, C. (1987). Research methods in the
social sciences (3rd ed.). New York: St.
Martin's.
Osborne, D., & Gaebler, T. (1992). Reinventing government.
New York: Addison-Wesley.
Osborne, D., & Plastrik, P. (1997). Banishing bureaucracy. New
York: Addison-Wesley.
Parks, R. B. (1984). Linking objective and subjective measures
of performance. Public Administration Re-
view, 44, 118-127.
Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement,
design, and analysis: An integrated approach.
Hillsdale, NJ: Lawrence Erlbaum Associates.

Poister, T. (1995). Productivity monitoring: Systems, indicators,
and analysis. In D. Ammons (Ed.), Ac-
countabilityforperformance: Measurement and monitoring in
local government. Washington, DC: In-
ternational City/County Management Association.
Poister, T., & Henry, G. (1994). Citizen ratings of public and
private service quality: A comparative perspec-
tive. Public Administration Review, 54, 155-160.
Poister, T. H., & Streib, G. (1998, May). Performance
measurement in municipal governments: Progress,
problems, and prospects. Paper presented at the annual
conference of American Society of Public Ad-
ministration, Seattle, Washington.
Salzer, M., Nixon, C., Schut, J., Karver, M., & Bickman, L.
(1997). Validating quality indicators: Quality as
relationship between structure, process, and outcome.
Evaluation Review, 21, 292-309.
Stipak, B. (1979). Citizen satisfaction with urban services:
Potential misuse as a performance indicator.
Public Administration Review, 39,46-52.
Stipak, B. (1987). Using subjective measures in program
evaluation. In T. Busson & P. Coulter (Eds.), Policy
evaluation for local government (pp. 45-61). New York:
Greenwood.
Streib, G. (1990). Dusting off a forgotten management tool: The
citizen survey. Public Management, 72,
17-19.
Swiss, J. E. (1995). Performance monitoring systems. In D.

Ammons (Ed.), Accountability for perfor-
mance: Measurement and monitoring in local government (pp.
67-97). Washington, DC: International
City/County Management Association.
Tigue, P. (1994). Use of performance measures by GFOA
members. Government Finance Review, 10, 42-
44.
Tigue, P., & Strachota, D., (1994). The use of performance
measures in city and county budgets. Chicago:
Government Finance Officers Association.
U.S. General Accounting Office. (1993). Performance
budgeting: State experiences and implications for
the federal government. Washington, DC: Author.
U.S. General Accounting Office. (1997). Performance
budgeting: Past initiatives offer insights for GPRA
implementation (Accounting and Information Management
Division, B-275095). Washington, DC:
Author.
Wycoff, M. A., & Manning, P. K. (1983). The police and crime
control. In G. Whitaker & C. D. Phillips
(Eds.), Evaluating performance of criminal justice agencies (pp.
15-32). Beverly Hill, CA: Sage.
2014 16:41:08 PM

553
XiaoHu Wang is an assistant professor of public administration
at University of Central
Florida. His teaching and research interests are in the areas of
public budgeting, finan-
cial management, and local government management. He has
published in the Journal of
Public Budgeting, Accounting, & Financial Management.
Gerasimos A. Gianakis is an assistant professor ofpublic
administration at University of
Central Florida. His teaching and research interests include
public budgeting, financial
management, and organization theory. His research has
appeared in Public Administra-
tion Review, The American Review of Public Administration,
and Public Productivity &
Management Review.
2014 16:41:08 PM
http://www.jstor.org/page/info/about/policies/terms.jspArticle
Contentsp. 537p. 538p. 539p. 540p. 541p. 542p. 543p. 544p.
545p. 546p. 547p. 548p. 549p. 550p. 551p. 552p. 553Issue
Table of ContentsPublic Productivity & Management Review,
Vol. 22, No. 4 (Jun., 1999), pp. 427-558Volume Information
[pp. 554-558]Front MatterAbstracts [pp. 427-429]Featured
Topic: Privatization and Contracting: Managing for State and
Local ProductivityPrivatization and Contracting: Managing for
State and Local Productivity: [Introduction] [pp. 430-
434]Privatization, Contracting, and the States: Lessons from
State Government Experience [pp. 435-454]Competitive
Tendering and Contracting out: Rhetoric or Reality? [pp. 455-

469]Strategies for Avoiding the Pitfalls of Performance
Contracting [pp. 470-489]The Value of the Relationship Model
of Contracting in Social Services Reprocurements and
Transitions: Lessons from Massachusetts [pp. 490-501]Public
Agency Accountability in Human Services Contracting [pp.
502-516]Productivity in ReviewSuccessful Collaborative
Management and Collective Bargaining in the Public Sector: An
Empirical Analysis [pp. 517-536]Public Officials' Attitudes
toward Subjective Performance Measures [pp. 537-553]Back
Matter
Case Study: You must do the following:
Case Study Proposal:
• Select a peer-reviewed article in a public administration
concentration from JSTOR
(www.jstor.org) or EbscoHost (www.ebscohost.com).
o The selected peer-reviewed article
• one research question
• at least one dependent variable
• at least one independent variable
• at least one control variable
• at least one hypothesis
• at least three studies in the literature review section of the
selected
peer-reviewed article
• at least one research method (qualitative or quantitative)
• at least three findings

• Submit the selected peer-reviewed article as the case study
proposal for the instructor’s
approval in the Case Study Proposal folder under the
Assignments tab on CANVAS on
June 11, 2014.
Case Study Paper: (You will score zero points for plagiarism)
• Use the selected peer-reviewed article to address the issues in
the outline provided below
in at least 4,000 total words (12 pt. Times New roman font, one
inch margins on all
sides, left and right indentation set at zero, and before and after
spacing set at zero). The
title page and the reference page will not count toward the
minimum word requirement.
Research Question (4 points possible)
• Identify and discuss the research question and the hypotheses
in the peer-reviewed
article.
• Explain how clear and precise the research question is.
Variables (40 possible points)
• Explain the dependent and independent variables in the
selected peer-reviewed article.
• Describe how the author(s) of the selected peer-reviewed
article conceptualized
(defined) the dependent and independent variables in relation to

previous studies in the
literature review section of the selected peer-reviewed article.
• Explain how the author(s) operationalizes or measures the
dependent and independent
variables. In your explanation, be clear about whether the
measures the author used are
nominal, ordinal, interval, and/or ratio and whether the
measures are taken from previous
studies.
Identify and discuss at least two questions the author used to
operationalize the dependent
variable.
• Identify and discuss at least two questions the author used to
operationalize the
independent variable(s).
http://www.jstor.org/
http://www.ebscohost.com/
• Formulate at least three Likert scale questions with the
appropriate responses to
operationalize the dependent variable in the peer-reviewed
article.
• Formulate at least three Likert scale questions with the
operationalize one independent variable in the peer-reviewed
article.
• Formulate at least two index questions with at least 7
operationalize the dependent variable in the peer-reviewed
article.

• Formulate at least two index questions with at least 7
operationalize the independent variable in the peer-reviewed
article
Relationship of Variables, Literature, and the Research
Question (6 points possible)
• Discuss step by step how the author linked the dependent and
independent variables to
o the research question in the selected peer-reviewed article
o the hypotheses in the selected peer-reviewed article
o the previous studies in the selected peer-reviewed article.
Research Methods (9 points possible)
• Discuss the sampling method(s) used in the peer-reviewed
article and why.
• Explain the data collection method(s) used in the peer-
reviewed article and why.
• Discuss the method(s) of data analysis used in the peer-
reviewed article and why.
Research Method Assessment (6 points possible)
• Assess the appropriateness of the research method or methods
in the peer-reviewed
article.
Empirical Data and Key Argument (6 points possible)
• Discuss the extent to which the empirical data used in the

peer-reviewed article answered
the research question and supported the key argument in the
peer-reviewed article.
Findings (9 points possible)
• Describe at least three findings of the peer-reviewed article
that show that the author(s)
answered the research question and proved the key argument.
Use the following headings to organize the case study paper:
• Research Question
• Variables
• Relationship of Variables, Literature, and the Research
Question
• Research Methods
• Research Method Assessment
• Empirical Data and Key Argument
• Findings
Submit the selected peer-reviewed article in the Case Study
Proposal folder and the Case Study
paper in the Case Study Paper folder under the Assignments tab
on CANVAS. Due Dates: June
11, 2014 for Case Study Proposal; July 2, 2014 for the Case
Study Paper.

Public Officials' Views on Subjective Performance Measures

Public Officials' Views on Subjective Performance Measures

Recommended

Recommended

More Related Content

Similar to Public Officials' Views on Subjective Performance Measures

Similar to Public Officials' Views on Subjective Performance Measures (20)

More from amrit47

More from amrit47 (20)

Recently uploaded

Recently uploaded (20)

Public Officials' Views on Subjective Performance Measures