Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this


  1. 1. Benchmarking and Comparative Performance Measurement Don Gloo Senior Research Associate International City/County Management Association Center for Performance Measurement
  2. 2. Why Performance Measurement? <ul><li>The “World is Flat” for local governments too </li></ul><ul><li>Businesses can locate/expand anywhere </li></ul><ul><li>Employees can be anywhere </li></ul><ul><li>Governments impact quality of life </li></ul><ul><li>Measuring Performance is Key in a “Flat World” </li></ul>
  3. 3. Overview <ul><li>Center for Performance Measurement </li></ul><ul><li>How CPM adds value </li></ul><ul><li>The CPM data cycle </li></ul><ul><li>Benchmarking and linking to expectations </li></ul>
  4. 4. Center for Performance Measurement <ul><li>Established: 1994 </li></ul><ul><li>Initially, 44 city and county managers </li></ul><ul><li>Money, effort, expertise, blood, sweat, tears </li></ul><ul><li>Early support from Sloan Foundation and Urban Institute </li></ul>
  5. 5. Center for Performance Measurement <ul><li>CPM now in 12 th year </li></ul><ul><li>Open to ALL local governments in the U.S. and Canada </li></ul><ul><li>Current: 160+ participants </li></ul><ul><li>Goal: 210+ participants by 2008 </li></ul><ul><li>Subscription-based service </li></ul>
  6. 6. CPM Adds Value <ul><li>Mission: To improve the effectiveness and efficiency of local government through the collection, reporting and broad-based application of performance information </li></ul>
  7. 7. CPM Adds Value <ul><li>Collect, clean, and report participant data </li></ul><ul><li>Ensure data accuracy, validity, and reliability </li></ul><ul><li>Provide high degree of comparability </li></ul><ul><li>Identify effective management practices </li></ul><ul><li>Manage longitudinal datasets </li></ul>
  8. 8. CPM Service Areas <ul><li>Police </li></ul><ul><li>Fire & EMS </li></ul><ul><li>Code Enforcement </li></ul><ul><li>Streets </li></ul><ul><li>Libraries </li></ul><ul><li>Parks & Recreation </li></ul><ul><li>Housing </li></ul><ul><li>Refuse & Recycling </li></ul><ul><li>Facilities Management </li></ul><ul><li>Fleet Management </li></ul><ul><li>Human Resources </li></ul><ul><li>Information Tech. </li></ul><ul><li>Purchasing </li></ul><ul><li>Risk Management </li></ul><ul><li>Youth Services </li></ul>
  9. 9. Data Collection Cycle <ul><li>Five-step data cycle </li></ul><ul><li>Two cycles underway continuously </li></ul>
  10. 10. Data Collection Cycle Participants Submit Data
  11. 11. Data Collection Cycle Participants Submit Data CPM Cleans Data
  12. 12. Data Collection Cycle Participants Submit Data CPM Cleans Data Participants Validate or Correct Data
  13. 13. Data Collection Cycle Participants Submit Data CPM Cleans Data Participants Validate or Correct Data Full Datasets Posted
  14. 14. Data Collection Cycle Participants Submit Data CPM Cleans Data Participants Validate or Correct Data Full Datasets Posted Annual Report
  15. 15. Data Collection Cycle Participants Submit Data CPM Cleans Data Participants Validate or Correct Data Full Datasets Posted Annual Report Ongoing: Analysis Support & Identifying Best Practices
  16. 16. Development of Data Surveys <ul><li>“ Technical Advisory Committees” </li></ul><ul><li>CEOs, CAOs </li></ul><ul><li>Primary coordinators </li></ul><ul><li>Service-area specialists (e.g., fire chiefs, police chiefs, directors of public works) </li></ul><ul><li>Finance, accounting, IT specialists </li></ul>
  17. 17. Development of Data Surveys <ul><li>What should be measured? </li></ul><ul><li>Standardized definitions </li></ul><ul><li>What’s included, what’s excluded? </li></ul><ul><li>No estimates, no budget figures </li></ul><ul><li>“Not available” always an option </li></ul>
  18. 18. Additional CPM Services <ul><li>Training and peer-to-peer learning </li></ul><ul><li>Private web site </li></ul><ul><li>Publications </li></ul><ul><li>Analysis and research support </li></ul><ul><li>CPM Certificate program </li></ul><ul><li>Consortia and affinity groups </li></ul><ul><li>Citizen survey integration </li></ul>
  19. 19. Consortia Benefits <ul><li>All benefits of nationwide program </li></ul><ul><li>Fee waiver for training </li></ul><ul><li>More local/statewide comparisons </li></ul><ul><li>More opportunities to meet face-to-face to develop activities and share analysis </li></ul><ul><li>Virginia (29), Chicagoland (15), Puget Sound (13), Arizona (13), Oregon (10), Westchester County, NY (8), Minnesota (6) </li></ul>
  20. 20. Benchmarking <ul><li>Lake Wobegon effect—“all of the children are above average” </li></ul><ul><li>Expectations of citizens, elected officials, other stakeholders are key </li></ul>
  21. 21. Your Community’s Expectations… <ul><li>Define what should be important </li></ul><ul><li>Should focus your efforts, energy, resources </li></ul><ul><li>Should be reflected in measurements </li></ul>
  22. 22. National Citizen Survey TM <ul><li>National Research Center (NRC) </li></ul><ul><li>Uniform survey tool… </li></ul><ul><li>…but customizable too </li></ul><ul><li>Standard set of survey results </li></ul><ul><li>Normative comparisons </li></ul><ul><li>Results dovetail into CPM templates </li></ul><ul><li>100+ local governments in 35 states </li></ul>
  23. 23. Questions and, hopefully, answers… Don Gloo ICMA | Center for Performance Measurement 202.536.4418 [email_address]
  24. 24. National Citizen Survey TM <ul><li>Customizable to add relevant questions </li></ul><ul><li>Questions tested for validity and reliability </li></ul><ul><li>Requires little staff time </li></ul><ul><li>Cost effective </li></ul>
  25. 25. GASB Concept Statement No. 1 <ul><li>State and local governmental financial reports should possess these basic characteristics: understandability, reliability, relevance, timeliness, consistency, and comparability . </li></ul>
  26. 26. Core Steps <ul><li>Data collection, cleaning, and reporting are the core aspects of the Center’s work. Over the years, measures of performance collected by the Center have been developed and refined by a combination of professional city/county managers working closely with department heads and other specialists in each of the service areas. An extensive effort was made during the formative years of the program to agree on common definitions, measures, and instructions in the survey instruments, called templates. Center participants continue to update the templates each year to ensure they remain topical and relevant. Though the templates are subject to change each year, the Center endeavors to maintain consistency in the measures and definitions so that participants benefit from accurate time-trend data. Reporting jurisdictions agree to adhere to the definitions and instructions to promote data comparability among the participants. </li></ul><ul><li>After collecting data for both submission deadlines (one in the fall and one in the spring to accommodate July, September, and December fiscal year closings), Center staff begin the data cleaning process. This involves statistical outlier checks, automated logic checks (programmed to identify data points that may not relate to other responses in the templates), and comment review (CPM staff review each comment that jurisdictions submit to explain their data points). This intensive process involving thousands of data points is a crucial component of the Center’s data collection and reporting cycle. CPM staff compile results of the data cleaning and send out data verification notices to participants that have questionable data points. Participants are given several weeks to review their data and either confirm or adjust their responses. (Questionable data that are not verified are removed from the datasets.) Once Center staff receive all verifications and update the datasets, compilation of the data report begins. </li></ul><ul><li>The Center also publishes a Mid-Year data report as a precursor to the Annual Data Report; data from the Mid-Year data report is included in the Annual Data Report. The purpose of the Mid-Year report is to make available these comparative performance data at an earlier date for those jurisdictions with a fiscal year ending on or before June 30 in any given year. </li></ul>
  27. 27. Lessons Learned <ul><li>Clear statement of mission, goals, and outcomes </li></ul><ul><ul><li>Purpose </li></ul></ul><ul><ul><li>Audience </li></ul></ul><ul><ul><li>Short- and long-term use </li></ul></ul><ul><ul><li>Level of aggregation/detail </li></ul></ul><ul><li>Performance Reporting Pyramid </li></ul><ul><li>Commitment from.. </li></ul><ul><ul><li>Top executive (e.g., city manager) </li></ul></ul><ul><ul><li>Management team </li></ul></ul><ul><ul><li>Key staff </li></ul></ul><ul><li>Process for selecting services, measures, and definitions </li></ul><ul><ul><li>Steering Committee </li></ul></ul><ul><ul><li>Technical Advisory Committees </li></ul></ul><ul><ul><li>Need to achieve buy-in by users </li></ul></ul><ul><li>Collecting, cleaning, & verifying data </li></ul><ul><ul><li>Collection period (wk, mo, qtr, yr) </li></ul></ul><ul><ul><li>From hard copy & spreadsheets… </li></ul></ul><ul><ul><li>to CDs… </li></ul></ul><ul><ul><li>” Web-based templates (like “Turbo Tax”) </li></ul></ul><ul><ul><li>Participants explain discrepancies during data reporting </li></ul></ul><ul><li>Citizen surveys </li></ul><ul><ul><li>Perceptions matter </li></ul></ul><ul><ul><li>Interrelationship between measures and surveys </li></ul></ul><ul><ul><li>Sample questions, format, structure </li></ul></ul><ul><ul><li>ICMA’s National Citizens Survey </li></ul></ul><ul><ul><li>More info (including the survey itself): </li></ul></ul><ul><li>Facilitating analysis & use </li></ul><ul><ul><li>“ Use it or lose it”…. </li></ul></ul><ul><ul><li>… in budgets, team/individual goal setting, discussion of effective practices among staff </li></ul></ul><ul><ul><li>Workshops and “What Works” </li></ul></ul><ul><li>Miscellaneous tips </li></ul><ul><ul><li>No overall ranking </li></ul></ul><ul><ul><li>Assignment of a “primary coordinator” </li></ul></ul><ul><ul><li>“ Confident humility” </li></ul></ul><ul><ul><li>Need to achieve buy-in by users </li></ul></ul>
  28. 28. CPM: Challenges <ul><li>Balance of consistency vs. need for revisions vs.the needs of each consortia vs. fees </li></ul><ul><li>Covey: Quadrant II activity </li></ul><ul><li>Analysis and application of data </li></ul><ul><li>Leadership vs. delegation: Performance management is a leadership activity, not “just” a measurement activity </li></ul>
  29. 29. CPM: Next Steps <ul><li>Consortia facilitation </li></ul><ul><li>Analysis & application </li></ul><ul><li>Template improvements </li></ul><ul><li>Core measures enhancements </li></ul><ul><li>Role of performance measurement in organizational leadership: Taking CPM and ICMA to the next level </li></ul>
  30. 30. CPM: Improvements & Enhancements <ul><li>Templates: Paper, CD, FTP, Web </li></ul><ul><li>Cleaning & Verification </li></ul><ul><li>Final data: 2 yrs to 4 months (posting) </li></ul><ul><li>Two data cycles </li></ul><ul><li>Mid-year report </li></ul><ul><li>1,000s of contacts </li></ul><ul><li>PDFs of individual chapters </li></ul><ul><li>What Works </li></ul><ul><li>Electronic library of 1,000s of management documents </li></ul><ul><li>In-house training </li></ul><ul><li>ICMA University workshops at conference & state association mtgs </li></ul><ul><li>Customized workshops: all employees, elected officials </li></ul><ul><li>Core measures </li></ul><ul><li>Technical assistance: Customized graphs & reports, analysis, training in use of performance information, peer assistance </li></ul>
  31. 31. Benefits of PM <ul><li>Performance measurement is no longer a fad or a buzzword. More and more governments are seeing that a system of performance measurement aids decision makers at all levels of government, from elected officials to managers to line staff. When jurisdictions embark on a concerted effort to improve services, performance measurement can assist in determining where they are and, more important, where they may want to go. </li></ul><ul><li>Under the most desirable circumstances, a community begins by establishing its mission and then develops a strategic plan that will guide the community in fulfilling that mission. The elected council or board will then agree to short- and long-term goals to implement the community’s strategic plan. Performance measurement will then enter the picture as a way of determining progress in meeting (or maintaining) the actions required in the strategic plan. </li></ul><ul><li>Many local governments operate under the council-manager form of government. The council establishes for the manager a set of measurable goals that contribute to each of the established council goals. Working with the city manager, department heads set department objectives that will contribute to achieving the goals set for the city manager. Department heads then work with team leaders within their departments to establish goals that will contribute to departmental objectives, and so on throughout the local government. If every team and every department meets the objectives established, the city will successfully implement its strategic plan and contribute to the overall mission set forth by the council. </li></ul><ul><li>In successful performance measurement programs, the vast majority of goals and objectives are quantifiable. Some are “hard” pieces of discrete data (e.g., percentage of EMS vehicles arriving on the scene of the accident in less than five minutes from time of dispatch) while others are a bit less so (e.g., percentage of citizens rating their community parks as “excellent”). As goals and objectives move up through the organization, the measures tend to go from discrete and specific to qualitative and general. </li></ul><ul><li>If a city or county develops a set of performance measures and performance targets, individual employees, department heads, administrators, and councils can observe and measure progress. Then the organization moves as a coherent whole toward achieving community goals and/or maintaining community standards. </li></ul><ul><li>Where specific goals or targets are not achieved, council, management, and employees should work together to determine why such targets were not achieved. They may then redirect resources and redouble efforts to achieve these targets. Management and employees can analyze operations and work together to find ways of improving services. Performance measurement provides an additional decision-making tool that employees throughout the organization can utilize to make informed decisions. </li></ul><ul><li>It is rare that any government will achieve all its goals and objectives every year given the complex environment in which local governments operate (e.g., changes in the local as well as national economy, state and federal grants, state and federal mandates, weather). Nevertheless, governments engaged in any form of performance measurement should be applauded for their efforts. </li></ul>
  32. 32. Why Performance Measurement? <ul><li>Thomas Friedman: Throughout history, “place” has been dominant </li></ul><ul><ul><li>Geographic, religious, political, communications </li></ul></ul><ul><li>Technology has overcome barriers, has made the world “flat” </li></ul>
  33. 33. Why Performance Measurement? <ul><li>Evidence of a flat world: </li></ul><ul><ul><li>Silicon Valley software designers working with Chinese programmers </li></ul></ul><ul><ul><li>American companies using Indian call centers </li></ul></ul><ul><ul><li>ICMA staff telecommuting from Naples, Italy; Racine, WI; Bremerton, WA; and Leavenworth, KS </li></ul></ul><ul><li>So what? </li></ul>
  34. 34. Why Performance Measurement? <ul><li>Ironically, as “place” matters less and less for businesses… </li></ul><ul><li>It matters more and more for local governments </li></ul>
  35. 35. Why Performance Measurement? <ul><li>Businesses can and will go anywhere </li></ul><ul><li>Businesses will locate/expand where… </li></ul><ul><ul><li>Their existing employees want to live </li></ul></ul><ul><ul><li>Where they can find more skilled employees </li></ul></ul><ul><li>Government Performance  Quality of Life </li></ul>
  36. 36. Data Collection Cycle <ul><li>Participants submit data </li></ul><ul><li>Data cleaned by CPM staff </li></ul><ul><ul><li>Logic, outliers, comments, year-to-year </li></ul></ul><ul><li>Participants validate or correct data </li></ul><ul><li>Full data sets posted </li></ul><ul><li>Data report prepared </li></ul><ul><ul><li>PDFs, physical book </li></ul></ul><ul><li>Ongoing: Analysis and research support </li></ul>