The Adoption, Use and Impacts of Performance Measures in Medium-Size Cities: Progress
                         Toward Perf...
Adoption, Use and Impacts of Performance Measures in Medium-Size Cities

                                             Abst...
2

adopt various performance measures and use them for different decision applications (de Lancer

Julnes and Holzer 2001;...
3

for understanding how chief executives perceive the usefulness, sustainability and success of

performance measurement ...
4

range. A stratified random sample of 670 cities (about 50% of the target population) and contact

data for their chief ...
5

Adoption and Use of Performance Measures

       Poister and Streib (1999) reported that larger cities were much more l...
6

       Table 3 shows the use performance measures across these three structural types. This

distribution indicates tha...
7

       Chief executives were asked to identify various features related to the organizational

culture of their cities....
8

Reasons for Adoption and Expected Results

       What reasons do chief executives identify for adopting performance me...
9

“window dressing,” perhaps as a response to pressures by peers or other groups who do not wish

for their community to ...
10

outcome measures. To help improve communications with citizens about service performance,

most cities used the result...
11

                                        Table 7 goes here

       Overall, most chief executives thought that their pe...
12

suggest that they find performance measures to be very helpful for improving the quality of

decisions related to mana...
13

indicators for decisions related to managing/evaluating programs, resource allocation/budgeting,

and improving commun...
14

Yet, almost one in five chief executives thought that performance information fell short of their

expectations and an...
15

the performance measurement. This finding is substantively important because it provides

empirical evidence that buy-...
16

change should occur when performance information suggests that policies need to be revised to

improve results and out...
17

issues within each of these three areas, chief executives have definite views about the relative

helpfulness of perfo...
18

decisions must play an important role in securing their support and buy-in. The ways and means

that localities accomp...
19

Table 1. Characteristics of the Target Population and Mail Survey Responses 2004
                              Survey ...
20

Table 4. Types of Performance Measures Used

Type Measure                       Number      Percent Using
Workload or ...
21

Figure 1. Chief Executives’ Reasons for Adoption of Performance Measures

    To improve management
                  ...
22


Table 6. Applications of Types of Performance Measures (N = 168)
                                               Type ...
23


Table 8. Perceived Impact of Performance Information on the Overall Quality of
      Decisions by City Officials

   ...
24


Table 11. Factors Influencing Chief Executive Perceptions of the Success of Performance
               Measures

    ...
Appendix
                            Selected Questions from a National Survey of
                           Municipal Per...
2

3. In your opinion, which results did city officials really expect to see after using the service or performance
      ...
3

C. IMPACTS OF PERFORMANCE MEASUREMENT
 9. How would you rate the overall helpfulness of the performance measures used i...
4

F. ORGANIZATIONAL FEATURES
15. About how long has your city used performance measures? __________years

16. Please indi...
References


Ammons, David N. 1992. Productivity Barriers in the Public Sector. In Public Productivity
    Handbook, edite...
Melkers, Julia and Katherine Willoughby. 2005. Models of Performance-Measurement Use in
      Local Governments: Understan...
Upcoming SlideShare
Loading in …5
×

Adoption and Use of Performance Measures in Medium-Sized ....doc

710 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
710
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Adoption and Use of Performance Measures in Medium-Sized ....doc

  1. 1. The Adoption, Use and Impacts of Performance Measures in Medium-Size Cities: Progress Toward Performance Management David H. Folz, Ph.D. Professor The University of Tennessee Department of Political Science 1001 McClung Tower Knoxville, TN 37996-0410 dfolz@utk.edu Ms. Reem Abdelrazek, MPA Research Associate Tennessee Advisory Commission on Intergovernmental Relations (TACIR) Reem.Abdelrazek@state.tn.us Yeonsoo Chung, Ph.D. Managing Director North American Operations Knowledge Source, Inc. ychung@knowledgesourceus.com
  2. 2. Adoption, Use and Impacts of Performance Measures in Medium-Size Cities Abstract Based on a national mail survey of chief executives in mid-sized US cities (populations between 25,000 and 250,000), this study examines the patterns of adoption, use and impacts of performance measures for the purpose of advancing understanding of the challenges involved in moving from performance measurement to performance management. This study identifies the factors that distinguish cities that adopted and used performance measures and the results that chief executives expected to derive from the use of performance measures. What chief executives thought about the helpfulness of performance measures in making various types of decisions and why they thought their use of performance measures met, fell short, or exceeded their expectations are examined. The study finds that while most chief executives thought that performance measures met or exceeded their expectations, several factors helped to explain why the use performance measures fell short of leaders’ expectations. The single most important factor that helped to explain the gap between expectations and actual experience was the extent of “buy-in” of performance measurement by line managers and administrators. The level of workforce unionization and the extent of municipal experience with performance measurement also helped to explain whether or not performance management was perceived to be successful. Keywords: performance measurement, municipal government, performance management Performance measurement in the public sector has garnered a great deal of interest among elected and appointed city officials as well as public administration scholars since at least the early 1990s (Bouckaert 1992; Wechsler and Clary 2000). Scholars have described the usefulness of tracking performance (Hatry et al 1990; Wholey 1999) and highlighted the many obstacles and unintended consequences of implementing various kinds of performance measures (Ammons 1992; Smith 1995). Several studies have examined the extent to which local officials
  3. 3. 2 adopt various performance measures and use them for different decision applications (de Lancer Julnes and Holzer 2001; Rivenbark and Kelly 2003; Poister and Streib, 1999, 2005; Melkers and Willoughby 2005; Yang and Holzer 2006). While the performance measurement literature is replete with descriptions of its potential benefits, questions remain about whether the use of performance measures makes a difference in local governance and policy making, particularly in budgeting and resource allocation decisions (de Lancer Julnes and Holzer 2001; Hatry 2002; Ho 2005; Melkers and Willoughby 2005). As Ammons and Rivenbark (2008, 304) observe, “local governments’ progress in using performance measures to influence program decisions and service delivery has lagged behind their pace in collecting and reporting basic measures.” In their survey of 277 city and county administrators, Melkers and Willoughby (2005) found that almost half of the respondents in a mixed sample of governments reported wide use of performance measures. However, respondents were “much less enthusiastic about the effectiveness of using performance measures to influence budgeting processes and outcomes in particular” (Melkers and Willoughby 2005, 188). Likewise, de Lancer Julnes and Holzer (2001) found that only a subset of local governments that collect performance measures actually use them to improve program and service decisions. While feedback about municipal service performance has been found to be helpful in informing the citizenry, Melkers and Willoughby (2005) found that few local officials expressed strong views about the value of citizen involvement in the performance measurement process. In his study of Midwestern mayors, Ho (2005) found that these officials considered performance measurement to be an important tool for helping to enhance public accountability, but only 17 percent actually involved their citizens in the process of measuring service performance. Ho (2005, 234) suggested that several political and organizational environment variables are helpful
  4. 4. 3 for understanding how chief executives perceive the usefulness, sustainability and success of performance measurement but concluded that “how performance measurement is integrated into decision making remains a black box” and merits further study. Ammons and Rivenbark (2008) examined fifteen North Carolina cities and concluded that the likelihood performance data will influence operations is enhanced by the adoption of efficiency measures, the willingness of officials to engage in performance benchmarking, and the incorporation of performance measures into key management systems. This study examines why city officials adopted performance measures, how they report using them, what impacts and results municipal chief executives realized after adoption and whether these impacts fell short or met/exceeded their expectations. We explore what city leaders perceived to be the helpfulness of particular types of measures for specific types of decisions and what these chief executives thought about the overall impact performance information had on the quality of local decision making. We are particularly interested in identifying the most salient reasons for why CEOs may perceive a gap to exist between their expectations for and actual experiences with performance measures. In other words, an inquiry that investigates why CEOs consider performance management to be successful or not offers the prospect for identifying some of the factors within local control that may affect the extent to which local officials can realize the benefits expected from performance measurement as a key component of a performance management system. Data and Methods The data for this research were collected from a mail survey and from US census data. A mail survey was sent to 670 chief executives in US municipalities with populations 25,000 to 250,000. The mayors or city managers of these mid-sized cities comprise the survey target population. There are a total of 1,339 municipalities with populations in the 25,000 to 250,000
  5. 5. 4 range. A stratified random sample of 670 cities (about 50% of the target population) and contact data for their chief executives was obtained from the International City/County Management Association (ICMA) in 2004. In addition to the names and addresses of chief executives, the ICMA data file included information on population, region, metropolitan status, and form of government for each city. The mail survey questions referenced in this study are included in the appendix. Cities with populations between the 25,000 and 250,000 were chosen because of the availability of socioeconomic data for these cities, the desire to compare findings with previous research on this population stratum (Streib and Poister 2002, 1998; Poister and Streib 1999), and survey budget resource constraints. Cities with smaller populations were excluded because the level of adoption and use of performance measures in these smaller cities is low (Rivenbark and Kelly 2003). The survey instrument was initially mailed in June 2004 followed by a subsequent second mailing to non-respondents approximately two weeks later in early July 2004. A total of 280 completed surveys were returned for a response rate of about 42 percent. Table 1 shows that the distribution of responses obtained are comparable to the distribution of cities in the sample. With respect to population class, the survey response percentages are within a few percentage points of target population. In terms of geographic region, municipalities from the northeast are somewhat under represented (-6.7%). With respect to form of government, cities with the mayor- council form of government are somewhat under-represented (-6.6%) while cities with a council- manager form of government are somewhat over-represented by 7.4%. In most respects, the profile of the cities that responded to the survey is comparable to that of all medium-sized US cities, allowing generalization to this population. Table 1 goes here
  6. 6. 5 Adoption and Use of Performance Measures Poister and Streib (1999) reported that larger cities were much more likely to adopt performance measures. They found that only 30% of cities with populations between 25,000 and 50,000 use performance measures compared to more than half of all cities with populations between 100,000 and 250,000 (Poister and Streib 1999). Our survey results also indicate a gap in the adoption level among large and small cities but we also find that the adoption and use of performance measures has grown in popularity among cities in all population ranges since the Poister and Streib (1999) survey. As Table 2 indicates, more than two-thirds (68.0%) of cities have adopted and use performance measures. Among those cities with populations between 25,000 and 50,000, 59% use performance measures while 83.7% of cities with populations between 100,000 and 250,000 use them. Table 2 goes here That more than two-thirds of all medium-sized cities use performance measures supports the conclusion reached by Melkers and Willoughby (2005, 188) that these metrics are now a “fairly pervasive” feature of local governments. The 185 cities that reported both adoption and current use of various performance measures are the main focus of this paper. Poister and Streib (1999) found that those cities with a council-manager form of government used performance measures more frequently than cities with a mayor-council form. Among the jurisdictions in our survey, council-manager cities also report using performance measures more frequently than mayor-council cities by a significant margin (72.6% v. 56.2%). When using the conceptual definitions advanced by Frederickson, Johnson, and Wood (2004) to classify municipal structures as either “political” (the traditional mayor-council form), “administrative” (the traditional council-manager form) or “adaptive” (a combination of features from the other two types), an interesting pattern emerges.
  7. 7. 6 Table 3 shows the use performance measures across these three structural types. This distribution indicates that cities with either an adapted or an administrative form are much more likely than political cities to use performance measures. In fact, about 70% of those cities Table 3 goes here served by a professional chief executive or administrative officer use performance measures compared to just 50% of those led by an elected CEO. This difference is not only statistically significant but substantively important because it suggests that cities led by professionally trained managers and administrators are much more likely to employ performance measures, regardless of whether the city has a mayor-council form or not. Following Keene, O’Neil, Portillo and Svara (2007), this finding underscores another way that professional managers add value to the communities they serve. How prevalent are particular types of performance measures among the 185 mid-sized cities that report their adoption and use? Previous research by Poister and Streib (1999) found that efficiency measures were the least frequently used type of measure and that workload or output measures were the most frequently used. Ammons and Rivenbark (2008) also report that many local governments measure performance but only with less sophisticated workload or output measures that provide little in the way of diagnostic feedback compared to higher order efficiency, effectiveness and quality of service measures. The findings in Table 4 confirm that the largest proportion of cities report using workload or output measures, but a considerably larger proportion of cities (about half) report using more sophisticated performance measures that include indicators of citizen satisfaction with services, service quality, outcome effectiveness, and efficiency. This pattern of use suggests an increasing level of sophistication in the type of performance information being collected by mid-sized cities. Table 4 goes here
  8. 8. 7 Chief executives were asked to identify various features related to the organizational culture of their cities. We examined several of these factors to determine whether any might be associated with the use of particular types of performance measures, especially in light of the previous findings by de Lancer Julnes and Holzer (2001) that variables such as management attitudes and risk taking tolerance positively influenced the actual use of performance information. Table 5 indicates that in those cities where the chief executive agreed that the managers in their jurisdiction viewed performance measurement as an important basis for making decisions, the information from outcome, efficiency, and service quality measures was much more likely to be used. The strongest single bivariate association (as measured by gamma, a frequently calculated PRE-based measure of association for ordinal data) occurred between the use of outcome measures and having an organizational culture in which managers viewed performance measurement as an important basis for making decisions. The use of workload measures was strongly correlated with a greater perceived receptivity among non-management employees to organizational change. While the correlations are in the positive direction predicted by de Lancer Julnes and Holzer (2001), we found no statistically significant connection between the use of the various types of performance information and management’s willingness to implement organizational change, the extent of support by elected officials or the presence of a system that encourages risk-taking. These findings suggest that the actual use of more types of performance measures occurs when managers understand the value of performance data for making decisions and when non-management employees are receptive rather than fearful about possible organizational change in the wake of the use of workload data in making management decisions. Table 5 goes here
  9. 9. 8 Reasons for Adoption and Expected Results What reasons do chief executives identify for adopting performance measures? In other words, what motivated local officials to invest the resources to measure and track the various aspects of service performance? Figure 1 shows that a fairly strong consensus exists among chief executives. By a large majorities, chief executives thought that adoption of performance measures would improve management decisions (81.9%), support budget recommendations and decisions (71.9%) and respond to citizen demands for greater accountability (68.6%). These findings are consistent with previous research insofar as the desire to make better management decisions also was the principal motivator reported by Poister and Streib (1999). Significant proportions of chief executives in that study also reported that performance measures were adopted in response to citizen demands for greater accountability and pressure from council members. As Figure 1 shows, the largest proportion of chief executives in our survey believed that performance measures were adopted to help improve management decisions and support budget recommendations and decisions. This suggests that local officials recognized the potential for integrating performance measures in decisions about management and budgeting decisions to an extent not reported previously. Figure 1 goes here However, what results did city officials really expect to see after using the performance measures they adopted and to what extent do these expected results correspond to the most prominent reasons why they adopted performance measures in the first place? In other words, to what extent do the actual benefits of adopting performance measures correspond to what city officials expected to achieve through their use? Within the limits of survey research, we attempt to detect whether the adoption of various performance measures might just be symbolic or
  10. 10. 9 “window dressing,” perhaps as a response to pressures by peers or other groups who do not wish for their community to seem non-progressive since other cities measure performance. Figure 2 suggests a fairly high level of correspondence among the three most frequently cited reasons for adopting performance measures and the three most frequently reported expected results. Sizeable proportions of chief executives expected that the use of performance measures would result in stronger justification for management decisions and budget requests and also to improve communication with citizens about the city service performance. Almost three of five chief executives expected that the use of performance measures would enhance the level of understanding among city council members. These findings suggest that there is very little, if any, “cognitive dissonance” with respect to the reasons offered for adopting performance measures and what local officials expected to see as a result of their implementation. In fact, the consistency between the rationale for adoption and the results expected suggests that chief executives appear to have a fairly mature, outcome-focused view of performance measurement. Figure 2 goes here Applications of Service Performance Measures Considering that chief executives expected that performance measures would help to improve management decision making, justify resource allocation decisions and improve communications with citizens about service performance, to what extent did cities actually use the different types of performance measures to advance these objectives? Table 6 indicates that for decisions related to managing/evaluating services and programs, the majority of cities used outcome and efficiency measures and just under half of cities used service quality measures. For resource allocation or budgeting decisions, the majority of cities used workload, efficiency and
  11. 11. 10 outcome measures. To help improve communications with citizens about service performance, most cities used the results from outcome measures and local surveys of citizen satisfaction. Table 6 goes here For each of the three types of decision applications, the majority of cities appear to use the types of performance measures that are most appropriate to support decisions related to each (Ammons 2001). That outcome measures are the most widely used for all three types of decision applications suggests that local officials have recognized the limits of relying primarily on workload or efficiency measures as reported earlier by Poister and Streib (1999) and Ammons and Rivenbark (2008). Defining and collecting data for organizationally relevant outcome measures and “integrating them into daily operations” is an expensive and time-consuming process and it is “critical to compute the costs (direct, indirect and intangible) associated with …PM implementation” (Frank and D’Souza 2004, 706-7). While a formal cost-benefit analysis is beyond the scope of this paper, it is possible to ascertain what chief executives think about the various impacts that these measures have had on specific types of decision outcomes, the overall quality of decisions reached, and whether actual experience matched expectations. These assessments are a useful beginning point to ascertain what chief executives think about the value of the investment in performance measurement and to identify some of the reasons why use of these measures either met, fell short or exceeded the expectations of chief executives. Perceived Impacts of Performance Measures What do chief executives think about the helpfulness of the performance measures with respect to specific types of outcomes related to managing/evaluating, resource allocation/budgeting and improving communication with citizens about service performance? Table 7 summarizes the perceptions of chief executives with respect to the helpfulness of performance measures in achieving specific types of impacts related to each function.
  12. 12. 11 Table 7 goes here Overall, most chief executives thought that their performance measures were either somewhat or very helpful in achieving outcomes related to each of the three categories. For the results related to managing/evaluating programs, many chief executives thought that performance measures were not helpful in supporting personnel performance appraisals. This finding is completely understandable since performance measures are targeted to program outcomes rather than individual accomplishments. For outcomes related to resource allocation and budgeting, chief executives were much more likely to rate performance measures as being very helpful in focusing program priorities. Almost half of all respondents perceived performance measures to be very helpful for this purpose and similarly, almost four in ten chief executives thought they were very helpful for making positive changes in program emphasis. In terms of the helpfulness of performance measures in improving communications with citizens about service performance, more than 45% of chief executives thought that they were very helpful in producing better communication between citizens and administrators/elected officials. The largest proportion of chief executives (41.4%) thought that performance measures were only somewhat helpful in improving relations with community groups. These patterns suggest that chief executives find performance measures to be most helpful when performance information is used selectively to inform particular types of decisions. Selective application of performance data that is most germane to particular types of decision applications appears to be related to how chief executives rate the helpfulness of performance information. In other words, chief executives of mid-sized cities have become more discriminating and selective about the value of certain types of performance data and perhaps less tolerant of an overload of performance information. At any rate, their collective experiences
  13. 13. 12 suggest that they find performance measures to be very helpful for improving the quality of decisions related to managing/evaluating programs, very helpful in helping to focus program priorities in decisions related to resource allocation but only somewhat helpful for improving communication with citizens. Poister and Streib (1999) and Ammons and Rivenbark (2008) acknowledged the possibility that favorable ratings of performance measures might outstrip their actual impacts. To help check for that possibility, chief executives were asked what they thought about the overall impact of performance information on the quality of decision making by the city officials that use this information. In other words, what do they perceive as the magnitude of difference that performance information has made in decisions by the city officials who use it? Table 8 shows that about 60% of CEOs think it has had a slight positive impact while another 30% think performance information has had a significant positive impact on the quality of decision making. Less than eight percent think that performance information has had no impact while about four percent are not sure about the impact of performance measures on overall decision quality. That most CEOs consider performance information to have had only a slight positive impact on the overall quality of decisions suggests that CEOs do in fact perceive a difference in the helpfulness of measures for specific types of decisions and their impact on improving the overall quality of decisions. Table 8 goes here What factors help to explain these differential perceptions about the impact of performance information among chief executives? Since chief executives perceived variations in the level of helpfulness of performance information, we hypothesized that these perceptions would be related to how they perceived the impact of performance information on the overall quality of decisions by city officials. Summary scores were computed for the helpfulness
  14. 14. 13 indicators for decisions related to managing/evaluating programs, resource allocation/budgeting, and improving communications with stakeholders. The gamma values for the crosstabulation between these variables and the perceived impact of performance information on the overall quality of decisions are shown in Table 9. These scores indicate that for each type of decision application, there is, in fact, a strong positive association between the helpfulness rating of the performance measure(s) used and the perceived impact that performance information has had on the overall quality of decisions by local officials. So, when chief executives consider performance measures to be helpful in making decisions about impacts related to each of the three broad categories, they are much more likely to perceive that the use of performance information has had a more significant positive overall impact on the quality of decisions by city officials. Table 9 goes here While the strong connection between the perceived helpfulness of performance measures and the overall positive impact of performance information on the quality of decisions is not surprising, the question of whether the actual use of performance measures fell short, met or exceeded expectations remains to be answered. If performance management is to become an enduring feature of local government decision processes, part of the political calculus of making that a reality may depend on whether chief executives think that the performance information generated falls short, meets or exceeds their expectations. Consequently, it is important to examine these perceptions and what factors may affect chief executives’ perceived success of performance measures. Table 10 indicates that most chief executives (68.7%) think that their community’s actual experience in using performance information met or exceeded their expectations. This finding augers well for the prospects of institutionalizing performance management in local governance.
  15. 15. 14 Yet, almost one in five chief executives thought that performance information fell short of their expectations and another 12.5% were unsure about it. What distinguishes those cities whose chief executives thought that performance measures met or exceeded expectations versus those whose experience fell short of expectations? Table 10 goes here Following previous research that illustrated the importance of political, cultural and community features in measuring the effects of performance measurement (Melkers & Willoughby 2005; Folz & French 2005), several municipal features were hypothesized to be helpful for explaining the variation in chief executives’ perceptions about whether performance measures either fell short or met/exceeded their expectations. Table 11 identifies four factors that together explain over one-fourth of the variation in how chief executives evaluate the “performance” of performance measures in terms of their expectations versus their actual experiences with them. The dependent variable is a dichotomous measure of whether chief executives think performance measures fell short of expectations or met/exceeded their expectations. Each of the independent variables in the model attain statistical significance at the .05 level except for the measure of employee receptivity to change which is just above this threshold. Table 11 goes here In terms of the relative importance of the independent variables in the model, what most affects whether chief executives think performance measurement has fallen short or met/exceeded their expectations is whether they believe that the performance measures employed by the city are in fact supported by the municipality’s administrative and supervisory personnel. Clearly, the support of and commitment by the supervisors responsible for managing the system of performance indicators is critical to how chief executives perceive the success of
  16. 16. 15 the performance measurement. This finding is substantively important because it provides empirical evidence that buy-in by line managers and supervisors is vital to the perceived success of performance management. When most of the administrative staff support the use of the adopted performance measures, chief executives’ expectations for performance measures are much more likely to be met or exceeded. The findings also indicate that in those cities where a larger proportion of municipal employees are unionized, chief executives are more likely to think that performance measures fell short of their expectations. We speculate that in those cities with a higher level of unionization, representatives from employee unions may have some role in negotiating which performance measures are used, the frequency of data collection, or perhaps even how performance data may impact various programmatic decisions. At any rate, higher levels of employee unionization adversely color how chief executives perceive the impacts of performance measures. There are few substitutes for experience in most local government activities and performance measurement certainly numbers among them. We find that the longer a city has used performance measures, the more likely the chief executive is to think that their city’s performance measurement system meets or exceeds their expectations. Clearly, there is a learning curve in performance measurement and a process of continual refinement in judgments about which measures are most useful for different types of decision applications. Those chief executives of cities that have more years of experience in the process are much more likely to think that performance measures have met or exceeded their expectations. Finally, when chief executives agree or strongly agree that the city’s non-managerial employees are receptive to changes in organizational policies, chief executives are more likely to think that the city’s performance measures have met or exceeded their expectations. After all,
  17. 17. 16 change should occur when performance information suggests that policies need to be revised to improve results and outcomes. A workforce that understands the need for such changes is more likely to accept them, an impression not lost upon chief executives whose perceptions of the impacts of performance measures are affected in a more positive way. Summary and Conclusion This study finds that among mid-sized US municipalities, the use of performance measures is more pervasive than ever with more than two-thirds (68%) of these cities using one or more types in 2004. Larger cities and those with appointed rather than elected chief executives are the most likely to use performance measures. Performance measurement also is particularly prevalent among cities with adapted or administrative forms of government. While workload or output measures continue to be the most widely used type of performance indicator, this study finds that outcome, service quality and citizen satisfaction measures are used more extensively than reported in earlier studies. Furthermore, these measures are most likely to be adopted among those cities where managers view performance measurement as an important basis for making decisions. Improving the decisions made by managers, supporting budget recommendations/ decisions and responding to citizen demands for greater accountability are the three reasons most commonly cited by chief executives for adopting performance measures in the first place. The high level of consistency between the reasons for adoption and the results expected from the use of performance measures indicates that chief executives have a very outcome-focused view of performance measurement and do not consider these measures to be merely symbolic. The types of performance measures adopted and used by mid-sized cities appear to be the ones most appropriate for improving various types of management decisions, supporting budget recommendations and communicating performance results to citizens and council members. For
  18. 18. 17 issues within each of these three areas, chief executives have definite views about the relative helpfulness of performance measures. Most chief executives think that performance measures are very helpful for improving the quality of decisions and decision capacity related to managing/ evaluating programs and for focusing program priorities in making resource allocations. However, they found them to be only somewhat helpful for improving communications with citizens. Overall, most chief executives think that the use of performance information has had just a slight positive impact on the overall quality of decisions by made by city officials, but almost one-third believe that it has had a significant positive impact. Only about one in ten CEO’s are not sure about or discern no impact on decision quality. In terms of whether these perceived impacts fell short, met or exceeded their expectations, most CEOs thought that their experiences with performance measures either met or exceeded their expectations, but almost one in five indicated that their experience fell short of their expectations. Regression analysis identified four factors that explained more than one-fourth of the variation in these perceptions. Among these, the single most important factor that affected chief executives’ perception of the success of performance management was whether or not they believed that most line managers and supervisors in their city supported the use of performance measures in their areas of service or program responsibility. The perceived success of performance measurement appears to depend extensively on the level of commitment by administrators and line managers to the objectives of performance measurement. While this finding is intuitive, it underscores the importance of engaging these key stakeholders in local decisions about what performance measures to use, how information for them should be collected, computed and reported, and also how this information will be used to inform various types of service and program decisions. Including line managers and administrators in these
  19. 19. 18 decisions must play an important role in securing their support and buy-in. The ways and means that localities accomplish this inclusive approach to the design and implementation of performance management merits further investigation and then replication by communities, especially since the findings from this study indicate that securing the support of these key employees is vital to the perceived success of the performance management enterprise. The success of performance measurement as a management tool also appears to be a function of experience and a trial and error process that results in refinement of both the measures used and the ways in which the information generated informs local government decisions. While the extent of employee unionization appears to present special challenges for realizing chief executives expectations for performance measurement, when both managers and employees embrace the performance measures adopted and use the information generated from these measures to improve program management, justify budget recommendations, and enhance communications with council and citizens, performance measurement is apt to meet or exceed expectations by chief executives. That the investment in and applications of performance measures has met or exceeded the expectations of chief executives augers well for the continued refinement of performance measures and indicates that chief executives recognize the specific ways that performance measurement adds value to the process of governance.
  20. 20. 19 Table 1. Characteristics of the Target Population and Mail Survey Responses 2004 Survey Target Survey responses Difference population Classification Number Percent Number Percent % Population group 100,000-249,999 88 13.1 44 15.7 2.6 50,000-99,999 197 29.4 75 26.8 -2.6 25,000-49,999 385 57.5 161 57.5 0 Geographic region Northeast 164 24.5 50 17.8 -6.7 North Central 165 24.6 70 25.0 0.4 South 162 24.2 77 27.5 3.3 West 179 26.7 83 29.7 3.0 Form of government Mayor-council 219 32.7 73 26.1 -6.6 Council-manager 422 63.0 197 70.4 7.4 Commission 11 1.6 5 1.8 0.2 Town meeting 5 .7 1 .4 -0.3 Representative 13 1.9 4 1.4 -0.5 town meeting Total Survey 670 100.0 280 100.0 Table 2. Adoption and Actual Use of Performance Measures by Population Size 25,000- 30,000- 40,000- 50,000- 100,000- 29,999 39,999 49,999 99,999 250,000 Totals N % N % N % N % N % N % Not Adopted 26 53.1 26 39.4 13 29.5 15 21.4 7 16.3 87 32.0 Adopted & Use 23 46.9 40 60.6 31 70.5 55 78.6 36 83.7 185 68.0 Totals 49 100 66 100 44 100 70 100 43 100 272 100 Gamma = .404, Sig. = .000 Table 3. Use of Performance Measures by Form of Government, 2004 Administrative Political cities Adapted Cities Cities Totals N % N % N % N % Not Adopted 11 50.0 58 29.9 16 30.8 85 31.7 Adopted & Use 11 50.0 136 70.1 36 69.2 183 68.3 Totals 22 100 194 100 52 100 268 100
  21. 21. 20 Table 4. Types of Performance Measures Used Type Measure Number Percent Using Workload or Output 156 57.4 Efficiency or Unit Cost 108 40.4 Outcome or Effectiveness 128 48.1 Service Quality 135 50.6 Citizen Satisfaction 136 50.9 Table 5. Use of Various Performance Measures and CEO Agreement with Selected Organizational Features Organizational Features Management Management views Non- Elected officials City has a willing to performance management generally reward/ implement measurement as an employees are support incentive organizational important basis for receptive to innovative system that change making decisions organizational improvements encourages change risk-taking Workload .223 .361 .452* .138 .041 Efficiency .141 .454* .066 .276 .155 Outcome .259 .566* .255 .227 .197 Service quality .266 .393* .293 .321 .259 Citizen -.027 .139 .096 .114 .122 satisfaction Notes: * Significant at the .05 level
  22. 22. 21 Figure 1. Chief Executives’ Reasons for Adoption of Performance Measures To improve management 81.9% decisions To support budget 71.9% recommendations/decisions To respond to citizen demands for greater 68.6% accountability To comply with wishes of 35.7% elected city officials To comply with state or federal reporting 14.1% requirements To respond to pressure from various community 8.6% groups Other 3.8% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Note: Based on 185 responding cities Figure 2. Chief Executives’ Expectations from Using Performance Measures Stronger justification for management 73.5% decisions Stronger justification for budget requests 72.9% Improved communication with citizens about service performance 68.0% Enhanced understanding of service performance by council members 56.9% Improvement in employee performance 48.6% Improved employee morale 11.6% Other 1.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% Note: Based on 181 responses
  23. 23. 22 Table 6. Applications of Types of Performance Measures (N = 168) Type Measure Workload Efficiency/ Outcomes/ Service Citizen Application or Output Unit Cost Effectiveness Quality Satisfaction N % N % N % N % N % Managing/ 75 44.6 87 51.8 96 57.1 82 48.8 68 40.5 Evaluating Programs Resource 99 58.9 89 53.0 92 54.8 55 32.7 43 25.6 Allocations/Budgeting Reports to 55 32.7 45 26.8 87 51.8 60 35.7 77 45.8 Citizens/Media Table 7. Chief Executives Perceptions of the Impacts of Performance Measures on Selected Dimensions Helpfulness Level (in percents) Don’t Somewhat Very Possible impacts/results N Not helpful know/ not helpful helpful sure Managing/ Evaluating Programs Improved performance among employees 177 14.7 52.0 23.2 10.2 Supported personnel performance 175 28.0 33.1 24.0 14.9 appraisals Improved quality of decisions & decision 176 5.1 33.5 53.4 8.0 capacity Resource Allocations/ Budget Requests Made positive changes in program 177 4.5 50.3 39.0 6.2 emphasis Realized some cost savings for city 177 15.8 50.8 24.9 8.5 service(s) Focused program priorities 176 13.6 31.8 49.4 5.1 Increased service quality level 176 9.1 46.0 36.9 8.0 Improved Communication Improved relations with community 174 17.8 41.4 25.3 15.5 groups Better communication between 177 6.2 39.5 45.2 9.0 administrators & elected officials
  24. 24. 23 Table 8. Perceived Impact of Performance Information on the Overall Quality of Decisions by City Officials Frequency Valid Percent No impact 12 7.4 Slight positive 96 59.3 impact Significant positive 48 29.6 impact Don't know/not sure 6 3.7 Total 162 100.0 Table 9. Associations Between Helpfulness Level of Performance Measures in Decision Applications and Perceived Impact of Performance Measures on Overall Quality of Decisions Helpfulness Level (none, somewhat, very) of Performance Measures for: Resource Allocation Managing & Improved and Budget Requests Evaluating Programs Communications (N =157) (N = 160) (N = 157) Perceived Impact on Quality of Decisions by Local Officials (none, slight, significant) .565* .457* .444* * Significant at the .05 level Table 10. Chief Executives Ratings of Performance Measures: Expectations v. Actual Experience Rating N Percent Fell short of expectations 33 18.8 Met expectations 106 60.2 Exceeded expectations 15 8.5 Don’t know/ not sure 22 12.5 Total 176 100.0
  25. 25. 24 Table 11. Factors Influencing Chief Executive Perceptions of the Success of Performance Measures Unstandardized Standardized Coefficients Coefficients t Sig. Std. B Error Beta (Constant) -.209 .194 -1.077 .284 Years of experience with .106 .035 .250 2.980 .004 performance measurement Disagree (0) Agree (1) that non-managerial employees are receptive to changes in .107 .056 .163 1.914 .058 organizational policies Disagree (0) Agree (1) that most administrators .228 .058 .339 3.966 .000 support the use of performance measures Percent municipal -.003 .001 -.253 -3.028 .003 workforce unionized Std. Error Adjusted of the R Square R Square Estimate .279 .252 .37167
  26. 26. Appendix Selected Questions from a National Survey of Municipal Performance Measurement Practices A. ADOPTION/ DEVELOPMENT OF PERFORMANCE MEASURES Cities may employ one or more of these types of measures: Workload or Output Measures - Amount of work or service provided or performed. Examples: tons of trash collected, number of calls answered. Efficiency or unit cost Measures - Dollar cost per unit of output or workload. Examples: cost per police car dispatched, cost per refuse collection account served. Outcome or Effectiveness Measures - Extent to which objectives, needs or desired impacts are achieved, met or produced. Examples: reduction in the number of commercial burglaries, reduction in substandard housing units. Service Quality Measures - A value-based assessment of services. Examples: convenience level, response time, accuracy rate, safety level, turn-around time, courtesy rating. Client or Citizen Satisfaction Measures - Extent to which clients think their needs are met; citizen ratings of programs. Examples: total complaints received, percent positive rating on a measure of service satisfaction; (information usually derived from surveys). 1. Considering these descriptions, please indicate whether your city has “Not adopted,” “Adopted but not used currently,” or “Currently use” each type of measure. (Please circle the number that applies to each type of measure). Type of Measure Not adopted Adopted, not used Currently use Workload or Output measures 1 2 3 Efficiency or Unit Cost measures 1 2 3 Outcome or Effectiveness measures 1 2 3 Service Quality measures 1 2 3 Client or Citizen Satisfaction measures 1 2 3 2. Cities adopt service measures for different reasons, some of which are listed below. In thinking about why your city adopted the measures you circled, please rank order the three most important reasons with “1” being most important. Rank _____ To improve management decisions _____ To respond to citizen demands for greater accountability _____ To comply with wishes of elected city officials _____ To respond to pressure from various community groups _____ To support budget recommendations/decisions _____ To comply with state or federal reporting requirements _____ Other (please specify):
  27. 27. 2 3. In your opinion, which results did city officials really expect to see after using the service or performance measures adopted by your city? (Please circle the numbers of all that apply). 1 Stronger justification for management decisions (e.g. personnel or resource deployment) 2 Improved communication with citizens about service performance 3 Enhanced understanding of service performance by council members 4 Stronger justification for budget requests 5 Improved employee morale 6 Improvement in employee performance 7 Other: (please specify): 4. In thinking about the above expectations city officials may have had for the impact of service performance measures, would you say your city’s actual experience with these measures generally “fell short,” “met,” or “exceeded” these expectations? (Please circle one number). 1 Fell short of the expectations 2 Met expectations 3 Exceeded expectations 4 Don’t know/ not sure B. USE & APPLICATIONS OF SERVICE/ PERFORMANCE MEASURES 7. Please circle the number of each type of measure city officials may use for each activity. Just skip any activity not relevant to your city or that is not supported by any type of performance measure. Type of Measure Activity Workload Efficiency Outcomes Quality Citizen sat. surveys Strategic Planning 1 2 3 4 5 Resource Allocation 1 2 3 4 5 (Budgeting) Managing/ Evaluating 1 2 3 4 5 Programs Internal Management 1 2 3 4 5 Reports Reports to Elected 1 2 3 4 5 Officials Reports to Citizens/ 1 2 3 4 5 Media
  28. 28. 3 C. IMPACTS OF PERFORMANCE MEASUREMENT 9. How would you rate the overall helpfulness of the performance measures used in your city with respect to each of these possible impacts? (Please circle one number for each possible impact). Helpfulness Level Not Somewhat Very Don’t know/ helpful Helpful Helpful not sure Possible Impact Made positive changes in program emphasis 1 2 3 4 Improved performance among employees 1 2 3 4 Improved quality of decisions & decision capacity 1 2 3 4 Facilitated program goal setting 1 2 3 4 Focused program priorities 1 2 3 4 Supported personnel performance appraisals 1 2 3 4 Increased service quality level 1 2 3 4 Enhanced employees’ understanding of goals 1 2 3 4 Improved relations with community groups 1 2 3 4 Realized some cost savings for city service(s) 1 2 3 4 Better communication between administrators & 1 2 3 4 elected officials Enhanced accountability of individual managers 1 2 3 4 11. Overall, what impact has the information derived from performance measures had on the quality of decision making by the city officials that use this information? (Please circle one). 1 No impact 2 Slight positive impact 3 Significant positive impact 4 Don’t know/ not sure 13. What do city administrators think about the performance measures employed? (Circle the number that best fits your opinion). City Administrators’ Stake Disagree Neither Agree nor Agree Don’t know/ not Disagree applicable The CEO supports the use of 1 2 3 4 performance measures Most department heads support the use of 1 2 3 4 performance measures Most staff administrators support the use 1 2 3 4 of performance measures Most line supervisors support the use of 1 2 3 4 performance measures Most city employees support the use of 1 2 3 4 performance measures
  29. 29. 4 F. ORGANIZATIONAL FEATURES 15. About how long has your city used performance measures? __________years 16. Please indicate the extent to which you disagree or agree with each of these statements. Strongly Disagree Agree Strongly Don’t Organizational Feature Disagree Agree Know Management is willing to implement organizational change whenever 1 2 3 4 5 appropriate. Management views performance measurement as an important basis for 1 2 3 4 5 making decisions. Non-management employees generally are receptive to change in organizational 1 2 3 4 5 policies. Elected officials generally support 1 2 3 4 5 innovative ideas for improvement. We have a reward/incentive system that 1 2 3 4 5 encourages risk-taking. G. CITY CHARACTERISTICS 17. Please indicate whether your city has any of the following features. Feature No Yes Mayor is directly elected by citizens 1 2 Mayor is selected by council 1 2 Most council members are elected by district 1 2 Most council members are elected at-large 1 2 Council members elected by a mixed district & at-large system 1 2 City has a Chief Administrative Office (CAO) position 1 2 Mayor presides over council meetings 1 2 Department heads report to the Mayor 1 2 Department heads report to a CAO 1 2 Mayor appoints and terminates CAO without consent of council 1 2 Mayor appoints and terminates CAO with consent of council 1 2 Council appoints and may terminate city manager 1 2 Statutory charter form is “Mayor-Council” form of government 1 2 Statutory charter form is “Council-Manger” form of government 1 2 Statutory charter form is “Commission” (without administrator) 1 2 18. What was your city’s total operating budget for FY 2004? $___________________________ 19. About how many full-time equivalent employees (FTEs) are employed in your city? _________ 20. About what percent, if any, of all FTEs are unionized? ___________%
  30. 30. References Ammons, David N. 1992. Productivity Barriers in the Public Sector. In Public Productivity Handbook, edited by Marc Holzer, 117-36. New York, NY: Marcel Dekker, Inc. Ammons, David N. 2001. Municipal Benchmarks: Assessing Local Performance and Establishing Community Standards. Thousand Oaks, CA: Sage Publications Inc. Ammons, David N. and William C. Rivenbark. 2008. Factors Influencing the Use of Performance Data to Improve Municipal Services: Evidence from the North Carolina Benchmarking Project. 68(2): 304-318. Bouckaert, Geert. 1992. Public Productivity in Retrospective. In The Public Productivity Handbook, ed. Mark Holzer, 5-46. New York: Marcel Dekker. de Lancer Julnes, Patria and Marc Holzer. 2001. Promoting the Utilization of Performance Measures in Public Organizations: An Empirical Study of Factors Affecting Adoption and Implementation. Public Administration Review 6(6): 693-703. Folz, David and P. Edward French. 2005. Managing America’s Small Communities: People, Politics and Performance. Lanham, MD: Rowman & Littlefield Publishers. Frank, Howard. and Jayesh D’Souza. 2004. “Twelve Years in the Performance Measurement Revolution: Where We Need to Go in Implementation Research.” International Journal of Public Administration. 27(8&9): 701-718. Frederickson, George H., Gary Alan Johnson, and Curtis H. Wood. 2004. The Changing Structure of American Cities: A Study of the Diffusion of Innovation. Public Administration Review 64(3): 320-30. Hatry, Harry P. 2002. Performance Measurement: Fashions and Fallacies. Public Performance and Management Review 25(4): 353-358. Hatry, Harry P., J. R. Fountain, J. M. Sullivan, L. Kremer. 1990. Service Efforts and Accomplishments Reporting: Its Time has Come. Norwalk, CT.: Governmental Accounting Standards Board. Ho, Alfred Tat-Kei. 2005. Accounting for the Value of Performance Measurement from the Perspective of Midwestern Mayors. Journal of Public Administration Research and Theory 16: 217-237. Keene, James; John Nalbandian, Robert O’Neill, Shannon Portillo, & James Svara. 2007. “How Professionals Can Add Value to Their Communities and Organizations,” Public Management 89, no. 2: 32-39.
  31. 31. Melkers, Julia and Katherine Willoughby. 2005. Models of Performance-Measurement Use in Local Governments: Understanding Budgeting, Communication, and Lasting Effects. Public Administration Review 65(2): 180-190. Poister, Theodore H. and Gregory Streib. 1999. Performance Measurement in Municipal Government: Assessing the State of the Practice. Public Administration Review 59(4): 325-35. Poister, Theodore H. and Gregory Streib. 2005. Elements of Strategic Planning and Management in Municipal Government: Status after Two Decades. Public Administration Review 65(1): 45-56. Rivenbark, William C. and Janet M. Kelly. 2003. Management Innovation in Smaller Municipal Government. State and Local Government Review. 35, 3: 196-205. Smith, Peter. 1995. On the Unintended Consequences of Publishing Performance Data in the Public Sector. International Journal of Public Administration. 18, 2/3: 277-310. Streib, Gregory D. and Theodore H. Poister. 2002. The Use of Strategic Planning in Municipal Governments. In The Municipal Year Book, 2002, 18-25. Washington, DC: International City/County Management Association. Wechsler, Barton and Bruce Clary. 2000. Implementing Performance Government: A Symposium Introduction. Public Productivity and Management Review 23(3): 264-66. Wholey, Joseph S. 1999. Performance-Based Management: Responding to the Challenges. Public Productivity and Management Review 22(4): 288-307. Yang, Kaifeng and Marc Holzer. 2006. The Performance-Trust Link: Implications for Performance Measurement. Public Administration Review 66(1): 114-126.

×