Auditor Roles in Government Performance ManagementDocument Transcript
National Performance Management Advisory Commission
Issue Paper # 8
Topic: Auditor Roles in Government Performance Management
Author: Paul D. Epstein
Principal, Epstein & Fass Associates
Manager, Auditor Roles in Government Performance Measurement Project 1
The Auditor Roles Project Team may update this paper as our various project principals have an opportunity to
contribute, and as our new Advocacy Committee of expert state and local auditors begins to work.
About the Issue
There are many valuable roles and practices auditors can play, and have played, in advancing
performance management in state and local government. Our research across North America has found
numerous local, state, and provincial auditors playing one or more of these five major roles:
ROLE 1. Audit Performance or Performance Management Systems: For example, performance
audits give management valuable recommendations for improving program performance, one of
the primary goals of performance management. And by auditing a government entity’s
performance management system, auditors help management and elected officials understand how
to improve the system and use it better to improve decisions, performance, and accountability.
ROLE 2. Assess Performance Information: By auditing reliability and relevance of performance
information and how it is communicated, auditors help build confidence in the information so
managers, elected officials, and citizens are more likely to make good use of it.
ROLE 3: Define or Measure Performance (or assist management in doing so): Auditors often
help management find better measures of performance. And in a number of governments, auditors
survey citizens, providing a valuable cross-cutting function by asking about services provided by
multiple departments, and strengthening the perception of survey objectivity.
ROLE 4: Encourage or Assist Management: Auditors have “encouraged” management through
advocacy or other means (e.g., written guidance) to improve performance management. They also
have served as key arm’s-length advisors or partners with management in developing or improving
performance management systems that are stronger because of what the auditor brings to the effort.
ROLE 5: Assist Elected Officials or Citizens: Auditors have related to elected officials and
citizens concerning performance management in numerous ways. For example, they have often
helped elected officials understand the importance of improving performance management, so
legislators will provide important support for system improvements.
These five roles and fourteen related practices are well documented in a 2004 book based on research
of auditor practices. 2 The book and updated materials, including new examples from state and local
auditors and articles describing how auditor practices add value to government performance, are
available on the “Auditor Roles in Government Performance Measurement” website
(www.AuditorRoles.org), which has supporting material far beyond which this paper can provide. A
case history of auditor practices in one city government is provided at the end of this paper.
Why should auditors be involved in improving performance management? Because they add value.
Most of this paper describes how they add value, and the numerous examples documented on
www.AuditorRoles.org demonstrate that they indeed add value to performance management in many
ways. A section near the end of the paper also provides several reasons why auditors should be
involved, even though managing performance is a management function. In general, consider that in
many governments, audit offices have one of the strongest pools of quantitative skills, performance
analysis skills, and knowledge of good system controls and practices. And they can use the knowledge
and skills independently, without being distracted by the demands of day-to-day management. Their
independence also adds to the credibility of their recommended improvements, which can build broad
support for performance management improvement among elected officials and citizens as well as
managers. So, if an audit staff in a jurisdiction has such knowledge and skills to offer, it would be
foolish not to involve them in how the government improves performance management.
The “Best” Roles for Auditors to Play at Any Time is Situational
Is any one auditor role or set of practices “better” or “more appropriate” than the others? From our
research of audit organizations across North America, the answer is clearly, “No.” Or, more explicitly,
the best role for a government auditor to play at any given time depends on the maturity of the
government organization’s performance management system, and the willingness and ability of others
in the government to improve that system. One of the central findings of our research, especially of
audit offices that have at least ten years’ experience playing these roles, is that “auditors add more
value by changing practices as entities’ systems evolve.” 3 This is explained, next, by describing each
of the five roles and related practices, and explaining the circumstances in which each can be
especially valuable in advancing performance management. To give those explanations the context of
reality, this paper ends with a brief case history of the City of Austin, Texas, to show how auditors’
roles and practices in that city government have changed over the years to keep adding value in ways
that are most effective, based on how performance management has evolved over time in Austin.
ROLE 1. Audit Performance or Performance Management Systems
Practice 1a. Audit performance: Performance auditing is a well-established practice in government,
with at least a 40-year history of auditors taking an independent look at government programs and
recommending improvements. It continues to be valuable at all levels of government today.
Performance auditing has an interesting, mutually-enhancing relationship with performance
management, in which they can add value to each other. For example:
• When performance management of an agency or program is weak, a performance audit can identify
those weaknesses and recommend, for example, improvements needed in how performance is
measured, or in how performance data are used. Thus, performance auditing can help improve the
value obtained from performance management systems.
• A strong performance management system that produces relevant and reliable performance data
can, in turn, make performance auditing more valuable. This is because auditors will not have to
spend a lot of costly time and effort on primary data collection just to establish basic performance
levels, as agencies will have data for auditors to use. This value can be captured in two ways 4 :
Auditors can be more efficient and timely with performance audits. So, for the same overall
auditing cost, they can produce more performance audits and complete them sooner, thus
providing more recommendations agencies can use to improve performance each year.
With basic agency performance levels reliably established, auditors can use the opportunity
to conduct more thorough performance audits, such as deeper evaluations of impact on the
population served. These “deeper audits” can provide management or elected officials with
valuable information for improving policies, resource allocation, or program designs.
Practice 1b. Audit performance management systems: In addition to auditing a program or agency’s
performance, auditors can assess the systems and processes agencies or entire governments use to
manage performance. This can be especially valuable in the following circumstances: 5
• If an agency or entire government has no real performance management system, or a weak one, the
auditor can establish the need for implementing or improving such a system, and articulate the
benefits of doing so. This can help elected officials understand the need to invest in PM systems,
and help management find ways to use that investment to best effect.
• If a government is in the process of implementing a new PM system, or making major changes to
an existing system, the auditor can play a valuable role reviewing agencies’ implementation
progress, and recommending improvements to help agencies get up to speed faster with the new
systems. Thus, new or improved PM systems will “mature” faster and will be more useful, at an
earlier date, to both management and legislative decision makers.
ROLE 2. Assess Performance Information
Like Role 1, Role 2 is a traditional auditor role. Role 2 is “traditional” in that it involves auditors
assessing information. This auditor role increases the usefulness of information in the performance
management system, and helps build the confidence of managers, elected officials, and citizens in the
information, so they are more likely to use it to improve performance and accountability. A closer look
at the three “practices” of Role 2 suggests when each can be especially valuable.
Practice 2a. Test relevance and reliability: As with auditing PM systems, testing relevance or
reliability (or both) of performance information can be especially useful in the early stages of
implementing a new or improved PM system. At such a time, this can best be done by auditors on a
broad-scale, at a relatively fast pace (e.g., assessing selected performance measures from all agencies
over one to three years), to build management and legislative confidence in performance data as a new
or improved PM system is implemented. Once all or most agencies have made their measures relevant
and established the controls to keep their data reliable, then the audit office can perform this practice
less often. Instead of testing a large number of measures each year, auditors might “spot check”
agency data from time to time, or simply make it a regular practice to test measures and data whenever
they do program performance audits. From our entire framework, we have documented the most
auditor examples for this practice. 6 In some cases, as in the City of Austin, auditors have trained
managers in testing and improving data reliability. As management takes responsibility for data
reliability, auditors can perform this practice less often. 7
Practice 2b. Assure performance reports: Beyond testing data reliability, auditors can play a valuable
role making public performance reports more useful. In North America, Canadian provincial auditors
have taken the lead on this practice. 8 But state and local auditors in the U.S. can also use criteria such
as those suggested by the GASB to review reported performance information and recommend how it
can be more effectively communicated and made more valuable to intended users.
Practice 2c: Support external review (of performance information or reporting): Auditors have been
prominent among the reviewers the AGA has used to evaluate state and local public performance
reports. When state or local government organizations want to submit their performance reports for
such a review, or have an external review of their PM system conducted, auditors can help their
government entity prepare for such a review so they will get more from it.
Non-traditional, but Still Valuable, Auditor Roles in Performance Management
We consider Roles 3, 4, and 5 to be “non-traditional” for auditors because in performing these roles,
auditors step out of “traditional” audit processes. However, for several practices related to these roles,
some auditors have followed auditing standards and considered initiatives to be audit projects.
Alternatively, auditors can perform these practices them as “non-audit services.” Important “auditor
independence” issues arise with non-audit services. The 2007 edition of Government Audit Standards
(the “Yellow Book”) provides specific guidance to auditors on how to maintain independence while
providing non-audit services. 9 The Yellow Book specifies some services (e.g., “third party surveys”)
where no “safeguards” are needed to preserve independence. For other non-audit services, such as the
auditor serving in an advisory capacity to management, Yellow Book safeguards represent good
practice for any collaborative relationship. For example, one such safeguard involves spelling out, as in
a memorandum of understanding, what the auditor will do and not do, and what management will do,
making sure that the auditor does not perform any management functions.
ROLE 3: Define or Measure Performance (or assist management in doing so)
Practice 3a. Help choose measures or targets: The key word here is “help.” It is inappropriate for
auditors to decide what performance measures management should select, or what performance
expectations or targets should be set. But often, based on findings in audits or other studies, auditors
have found either a lack of performance measurement, performance measures that are not sufficiently
relevant, or a lack of a sound basis for setting targets. 10 Sometimes auditors have seen opportunities
for more useful comparisons if measures are changed or added, and have recommended that measures
used by other jurisdictions be adopted to enable external benchmarking, such as measures in ICMA’s
comparative performance measurement program. In all cases, the auditor’s role is to provide
recommendations or advice, and for management and elected officials to decide what the actual
measures and targets should be. Some situations where this practice can be especially helpful are:
• When either elected legislators or executive management feel that, for a particular program,
service, or agency, current measures or targets do not give them a sufficient basis for decision
making or assessing accountability.
• When a program is new—particularly a relatively new kind of program without a long history or
“industry standard” method of measurement (e.g., transit oriented development programs)—the
auditor can help the agency running the program find a useful, practical way to measure and target
the program faster than the agency might on its own;
• When an auditor finds that an agency is not taking advantage of available methods (e.g., practices
proven elsewhere) or available bases for comparison that could provide valuable information for
improving performance and accountability;
• When an auditor finds that the customers of a service, or members of the public with a particular
interest in a program, do not identify with the measures used for the quality, timeliness,
effectiveness, or results of that service or program.
Practice 3b. Collect data: All of the cases of this practice we have documented involve auditors
conducting or commissioning surveys or focus groups that cut across department lines, such as surveys
of satisfaction and perceptions that ask about multiple services as well as cross-cutting issues such as
“livability” or “business climate.” 11 Government Auditing Standards allow auditors to perform such
surveys “as an independent third party” without impairing auditor independence. 12 Of course, there are
many jurisdictions where management has conducted or commissioned surveys or focus groups. But
auditors obtaining data from citizens, interest groups, or service customers is not intrusive on
management as other kinds of data collection would be, such as collecting data that normally comes
from service employees recording information on daily operations. Situations in which it can be useful
for auditors to use this practice to improve performance management include:
• Where elected officials or management request the auditor to collect this kind of data.
• Where management was not collecting this type of data, and the auditor determined that such
information would be useful to elected officials and citizens for decision making or accountability.
• Where there is a sense of a “disconnect” between how one or more agencies are reporting on their
performance and how citizens or service customers perceive that performance;
• Where there are signs that the public may not completely trust management to ask unbiased
questions about service performance or to provide unbiased reports of results.
ROLE 4: Encourage or Assist Management
Practice 4a. Encourage management … to develop and implement performance management systems.
Practice 4b. Assist management … in designing, improving, or maintaining performance management
systems, or build the capacity of management to do so. (Examples at: http://www.auditorroles.org/exemplary-
The key words in these practices are “encourage” and “assist.” Developing, designing, implementing,
and maintaining performance management systems are clearly management functions. But across the
country, at the state and local levels, auditors have played valuable roles in encouraging or assisting
management in developing and improving performance management systems and their use—for
examples go to the web addresses linked above. Many of these examples have involved extremely
effective, productive working relationships between auditors and managers. That is the case with a
current example in King County, Washington, where the auditor convenes and facilitates a work group
of representatives from executive departments, the judiciary, the County Council, and separately
elected officials. The auditor’s role is spelled out as advisory, but has been valuable in helping the
county develop a consistent approach to strategic planning and performance measurement. The
effectiveness of the work group, and positive experiences across branches of government, are reflected
in the Performance and Accountability Act passed by the County Council in July 2008, mandating
performance management improvements recommended by the work group. 13 As in King County, such
working relationships can readily be documented and managed in ways that do not impair auditor
independence or have auditors intrude on management functions.
While it is always useful to have good working relationships between management and auditors
concerning performance management, perhaps the best times for these practices are:
• When a jurisdiction is contemplating developing a new performance management system or
making major improvements in an existing system, these practices can help bring together the best
of the different knowledge, skills, and perspectives of auditors and management to produce a better
system than either group would produce on its own.
• When new performance management systems or major system improvements are being
implemented, auditors can provide valuable training, technical assistance, or advice to help agency
managers become productive users of the system sooner than otherwise possible.
• When implementation of an existing PM system is uneven in a jurisdiction, auditors can help
executive management assist those agencies that most need to improve how they use the system.
• At any time, auditors can develop and provide guidance materials to encourage or assist managers
in making better use of performance management systems.
ROLE 5: Assist Elected Officials or Citizens
This is a potentially extensive role in which auditors assist or relate to citizens, non-executive elected
officials, or other external stakeholders, such as interest groups and the media. Five different auditor
practices are encompassed in this role, including:
• Practice 5a. External advocacy [to] external stakeholders, concerning the development,
implementation, or improvement of government performance management systems.
• Practice 5b. Report performance: Produce and issue external periodic performance reports.
• Practice 5c. Assist external decision making: Assist external stakeholders in using performance
information to make decisions.
• Practice 5d. Engage citizens … in determining performance goals, objectives, or measures.
• Practice 5e. Assess citizen engagement … related to performance management. 14
Depending on the auditor reporting relationship in a jurisdiction, this role or some of these practices
may be more or less appropriate for the auditor, for example:
• Auditors who report to management may be least likely to play this role, but sometimes managers
ask the auditor to do it (e.g., in Phoenix, the city manager asked the auditor to engage citizens to
help make performance measures and reports more understandable and useful to citizens).
• Elected auditors or those who report to a legislative body may be most likely to play this role, as
they have a responsibility to assist the constituents or elected body they report to.
• Auditors who report to independently-elected officials (e.g., Comptrollers) may not perform this
role directly, but may help their “boss” play this role, and may encourage that official to do so.
Auditors have made valuable contributions to help state and local legislators understand the need for
improving performance management, and in formulating legislative guidance or mandates to do so.
The King County example of the auditor “Assisting Management,” noted above, could also be listed
under this role, as the auditor’s office helped draft parts of the Performance and Accountability Act.
There are other examples of auditors drafting legislation or policy guidance used by legislators to
advance performance management, as well as assisting legislators and citizens in other ways. 15
For four of these five practices, there are no apparent issues involving auditor independence or auditors
playing a management role. However, Practice 5b. Report performance, involving issuing public
performance reports of a government entity, is generally considered a management function. And
independence questions persist, as auditors cannot audit their own report, especially in view of recently
tightened government audit independence standards. (Some auditors check management’s data before
reporting it. But they cannot audit their own analyses, interpretations, or communications approaches.)
However, auditors have produced some of the best public performance reports, including several that
have won certificates of achievement from the AGA. 16 Also, auditors have sometimes stepped in and
provided this important public reporting service where management has not, providing important gains
in public accountability. So, we are not quite prepared to say there are no circumstances in which
auditors should prepare external performance reports. For now, we will continue to follow
developments of this practice and audit independence standards.
In Summary, Auditors Can Add Value to Performance Management in Many Ways
State and local audit offices across the country have been helping improve performance management
by performing the roles and practices described above. Some, such as auditors in the State of Florida,
and the Cities of Austin (TX), Phoenix (AZ), and Portland (OR) have been doing it for over 15 years. 17
These and other auditors have changed their mix of practices over time as the performance
management in their jurisdictions has changed, so they, as auditors, will add as much value to
performance management as they can at any given time. They have demonstrated that there is not one
role that auditors should play in performance management, but a variety of roles and practices that
auditors can play to add value. They have also demonstrated that it is valuable for auditors to change
their mix of practices over time in order to best match what auditors do to the changing performance
management environments of their governments. That way, auditors can optimize their roles, and add
more value to improving performance management than if they always did the same things.
But Why the Auditor? Couldn’t Management Do All These Things?
If a few names of the above roles and practices are changed (e.g., change “audit” performance to
“study” or “evaluate” performance), one could easily imagine management performing all of the
above-described roles and practices. Particularly in larger jurisdictions that can afford to dedicate
separate staff to oversee performance management, management does perform many of these practices.
They even perform “encourage or assist management,” as, for example, a central performance
management team under a chief executive that assists agency managers in improving their
performance management practices. So, why involve auditors? There are a number of reasons. The
reasons that most apply will differ from one jurisdiction to another. For example:
• Auditors bring an independent perspective that adds credibility to performance management
practices. They bring a perceived extra level of objectiveness to performance management systems,
and their participation is likely to increase trust of elected officials and citizens.
• Auditors do not face day-to-day concerns of running programs and services, which can distract
from improving performance management. Managers must focus most on delivering services, but
auditors can provide a helpful extra set of eyes and ears, and often have more time and freedom to
research best practices from other sources, to find ways to improve performance management.
• In many jurisdictions, the auditor’s office is a source of some of the strongest quantitative,
analytic, and procedural skill sets, which are valuable for improving performance management.
For example, when auditors cooperate in developing or improving a performance management
system, they can help ensure that controls to assure data reliability are designed into the system.
• Sometimes management just doesn’t do it. In some jurisdictions managers do not implement
performance management systems, or they do not make the effort to improve existing systems or
make them work well. In these cases, auditors can fill the void to improve accountability and
sometimes stimulate performance gains. Should a jurisdiction and its citizens suffer from no
performance management, or weak performance management, because improving performance
management is a “management prerogative” that management chooses not to exercise?
All in all, improving performance management is not something that either managers or auditors
should do. In the best case, auditors and managers work cooperatively to bring their different
perspectives and skills to bear on developing good performance management systems and on
consistently improving those systems over time.
Case History: City of Austin, Texas 18
From 1985, when the Austin City Auditor started conducting performance audits (Practice 1a. Audit
performance), through 1992, very common audit findings across many city agencies involved weak
performance measurement practices, with measures often not relevant or non-existent, data unreliable,
and the use of data by management limited at best. So in 1992, the City Auditor encouraged the City
Council (Practice 5a. External advocacy) to demonstrate its interest in performance measurement to
city management. The City Auditor reports to the City Council. That year, the City Auditor’s Office
drafted a resolution that the Council passed encouraging city agencies to measure and report
Building on the Council resolution, the City Auditor began adding value in a new way, by assessing
the systems used by the city to measure, report, and improve performance (Practice 1b. Audit
performance management systems). The Auditor’s Office conducted entity-wide performance
management systems audits or assessments in 1994, 1996, 1998, and 2002, using results both to
continue advocating to Council (Practice 5a External advocacy) to require greater improvement, and
to help management design and improve these systems (Practice 4b. Assist management). One
approach taken to assisting management, in the late 1990s, involved the City Auditor’s Office serving
on a committee with management to oversee agency business planning, including development of new,
more relevant performance measures.
In 2002, city management enhanced citizen access to city performance information by implementing a
web-enabled searchable database called e-performance measures with access to performance
information for all city departments. In 2002, city management also took more responsibility for the
reliability of performance information by assigning Corporate Internal Audit staff (who report to
management) to develop and implement a certification process to test performance measurement
reliability (Practice 2a. Test relevance or reliability).
In 2004, city management enhanced measures certification when the Budget Office piloted a self-
assessment process in which each year, city departments self-assess the reliability of selected measures
included in their annual business plans. These self-assessments are submitted by departments to the
Budget Office for review. This self-assessment complements the formal certification process, still
conducted selectively by Corporate Internal Audit, increasing the number of performance measures
that are reviewed for accuracy, and increasing departmental ownership of measure reliability. The
certification process is defined by the Budget Office in the city’s Managing for Results Resource
Guide provided on-line for use by departments in conducting business planning and performing self-
assessments of performance measures.
Performance measurement in Austin has reached a level of maturity such that management expects and
is required to provide performance information on city operations, be able to answer questions on the
information, and be accountable for its reliability. Also, management has been upgrading systems that
provide information to support effective performance measurement, and the City Auditor’s Office is
planning projects to test the reliability of these systems. The trend in the city government to more
integration and centralization of these types of information systems and sources will likely result in
more centralized auditing.
The City Auditor continues to audit relevant performance measures during performance audits of
individual departments and issue areas, as well as assessing data reliability of selected systems that
provide information used for performance measurement and other purposes. Just as management has
been upgrading automation of performance information, the City Auditor’s Office has upgraded its
reliability testing (Practice 2a. Test relevance or reliability) to include the use of software tools for
data mining and analysis of entire data universes with less reliance on sampling, or geographic
information software for testing the completeness and accuracy of geographic information that may be
used in performance measurement.
As the quality of department performance information has improved, it has become more useful not
only to management and City Council, but also as a tool for the City Auditor. Now, the City Auditor’s
Office reviews trends in department performance measures as a regular part of the office’s citywide
risk assessment process for determining departments and programs to audit in the coming year, which
helps the City Auditor find higher-value targets for its regular program of performance auditing
(Practice 1a. Audit performance).
“Auditor Roles” is a Project of The Institute of Internal Auditors (The IIA) in partnership with the Association of Local
Government Auditors and the National Association of State Auditors, Comptrollers, and Treasurers, funded by the Alfred
P. Sloan Foundation and The IIA. The analyses, interpretations, and views in this paper are those of the author(s), and do
not necessarily represent the views of these organizations.
Epstein, Paul, Stuart Grifel, and Stephen Morgan Auditor Roles in Government Performance Measurement: A Guide to
Exemplary Practices at the Local, State, and Provincial Levels (Altamont Springs, FL: The Institute of Internal Auditors
Research Foundation, 2004)
Epstein, Grifel, and Morgan, pp. 19–23.
Epstein, Paul and Alina Simone, “Add Value to Government Performance and Performance Audits,” in GAP News, 3rd
Quarter, 2007, and at [http://www.auditorroles.org/Articles/AddValue_TestPerfMeasures.html] Also see how Florida’s
Office of Program Policy Analysis and Government Accountability (OPPAGA), has used both these approaches: using
reliable agency data to audit performance and building on the data to conduct “deeper” audits, at
For example, see Epstein, Paul, “Performance Management Systems Offer High-value Audit Opportunities,” in GAP
News, 1st Quarter, 2008, and at [http://www.auditorroles.org/Articles/AddValue_AuditPMSystems.html]. Also see
examples from eight local, state, and provincial governments at
For example, see Epstein, Grifel, and Morgan, pp. 50–61, and
See Epstein, Grifel, and Morgan, p. 51, and
See Epstein, Grifel, and Morgan, pp. 61–69, and
“Organizational Independence When Performing Nonaudit Services” in Government Auditing Standards, (Washington,
DC: U.S. Government Accountability Office, 2007), paragraphs 3.20–3.30, and Appendix I paragraphs A3.02–A3.03. For
a discussion of auditor independence, with reference to these standards, as related to specific Auditor Roles and Practices
described in this paper, see [http://www.auditorroles.org/professional-context/independence-issues.html].
For state and local examples of this practice, see
For examples, see
Government Auditing Standards (2007), Appendix I, (paragraph A3.03g).
For the King County (WA) example, see:
We have not identified auditors assessing citizen engagement that directly relates to performance management, but we
have documented auditors assessing engagement for other purposes. We see this as a “potential future practice,” because if
citizen engagement in performance management becomes more prevalent, it will be valuable for an independent auditor to
assess and recommend improvements in the effectiveness and representativeness of that engagement. Our examples are at:
For examples, see:
Among currently listed (2007) AGA Certificate of Achievement winners at [http://agacgfm.org/performance/sea/] the
Palo Alto (CA), Portland (OR), and Prince William County (VA) reports were prepared by auditors. Other auditors’ reports
have won this award in previous years.
For case histories of these audit offices performing these practices over time, see Epstein, Grifel, and Morgan, pp. 20–23,