Presented on Sept. 19, 2014 at the Public Performance Measurement and Reporting Conference, Newark, NJ.

1. 9/15/2014
1
Performance Measurement
and Evaluation in the
Obama Administration
September 19, 2014
Public Performance Measurement and Reporting Conference
Rutgers University, Newark, NJ
1
Nicholas R. Hart
nrhart@gwu.edu | www.nickhart.us | @nickrhart
Trachtenberg School of Public Policy and Public Administration
The George Washington University
Research Strategy
• Discussion Today:
– Phase I – Review of Theory
– Phase II – Review of Administration documents
• Forthcoming:
–– Phase III –– Interviews with OMB staff
– Phase IV – Interviews with Federal agency staff
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 2

2. 9/15/2014
2
Comparison of Evaluation and
Perf. Measurement Differences
Issue Program Evaluation Performance Measurement
Frequency Irregular Regular, continuing, ongoing
Coverage Done on only a few programs, issue‐specific
Covers most programs, general
purpose and use
Depth of
Information
Seeks reasons for poor/good
performance
Only tells "the score," not why
Attribution Assessed Assumed
Cost/Resources May be high for each study, resources
are targeted
Costs tend to be spread out;
resources embedded in
program infrastructure
Utility/Purpose Major program decisions, negotiated at Continuous program
initiation improvement; information use
evolves
Adapted from Hatry (2013) and McDavid and Hawthorne (TK)
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 3
Patton (2008) on synergies
• Indicators of performance can …
– Highlight projects that perform well,
– Serve as resource for accountability, and
– Contribute to impact assessments of government
policies.
• But evaluation is needed to relate indicators
to interventions AND to explain impacts on
outcomes.
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 4

3. 9/15/2014
3
Wholey’s Sequential Purchase of
Information
– Evaluability Assessment
– Rapid Feedback Evaluation
– Performance Monitoring
– Intensive Evaluation
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 5
Forms of Complementarity
– Sequential
– Information
– Organizational
– Methodological
– Hierarchical
Nielson and Hunter (2013), Boll and Hoeberg (2013), Rist (2006)
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 6

4. 9/15/2014
4
Examples of Complementarity
• Validating performance measurement data
• Source of data for evaluation
• Identify a need for evaluation
• Identify ways to improve data collection
(Hatry 2013; Hunter and Nielson 2013)
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 7
Relationships between Performance
Measurement and Evaluation
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 8

5. 9/15/2014
5
Context
• Obama Campaign ‐‐ Data analytics prioritized for campaign
donations, volunteers, and voter behavior
• New Administration aimed to develop a reframed performance
management system and macro‐level evaluation capacity building
• The two are supposedly integrated and complementary in
practice…
– Only 37% of Federal program managers reported that an evaluation
had been completed within the past 5 years of any program,
operation, or project they were involved in (GAO 2013)
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 9
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 10

6. 9/15/2014
6
Years 1 & 2 (2009‐10):
Getting off the Ground
• Context – Economic Recession & Recovery Act
• 2009 – Replacing PART, first Evaluation
Memorandum setting up “Voluntary
Evaluation Initiative” ($$) and Open
Government Directives
• 2010 – High Priority Performance Goals
(HPPGs)
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 11
Years 3 & 4 (2011‐12):
Implementing Core Changes
• Context: Gearing up for re‐election, House flip in
midterm elections
• GPRA Modernization
• Performance.gov launched to announce priority
goals
• Administrative Flexibility efforts ‐‐ Performance
Partnerships
• Second major evaluation memo:
– Recast complementary nature of performance
measurement and evaluation
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 12

7. 9/15/2014
7
Priority Goals
• Define roles for COOs, PIOs, and goal leaders
• Established goal framework:
– Federal Cross‐Agency Goals (4 years/qtr reviews)
– Agency Priority Goals (2 years/qtr reviews)
– Strategic Goals (4 years/annual reviews)
• Reported on Performance.gov which launched
in mid‐2011
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 13
CAP Goals – Veterans Example
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 14

8. 9/15/2014
8
Priority Goals – HHS Example
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 15
Evaluation Capacity Building (ECB)
• Increase use and production of “rigorous”
evaluation.
1. Increase funding (evaluation initiative)
2. Increase expertise (workshop series)
3. Improve access to data
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 16

9. 9/15/2014
9
ECB: Voluntary Evaluation Initiative
• Agencies submitted ideas for specific
evaluations – in first year administration
requested $100m for 17 agencies.
– Unclear how many were ultimately funded
• Second year – 19 evaluations, amount not
specified
– Unclear how many were ultimately funded
• No dedicated funding in subsequent years
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 17
ECB: Evaluation Workshop Series
• OMB‐led interagency workshops to discuss
capacity issues
– First workshop on how to build capacity centered
on Administration priorities
• E.g. chief evaluation offices, procurement strategies,
funding set‐asides
– One session in 2014 on integrating performance
and evaluation
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 18

10. 9/15/2014
10
ECB: “Low(er) Cost” Evaluations
• Improving access and use of administrative
data – e.g. National Directory of New Hires or
SSA annual wage data
• GSA/PIC nudge team focused on low cost
administration interventions
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 19
Second Term – Years 5 & 6 (2013‐14)
• Context: Post‐Election
•• Joint OSTP, DPC, OMB, CEA Evaluation and
Evidence Memo & CEA Chapter in ERP
• OMB dedicated staff to “Evidence and
Innovation”
• Nudge team creation at GSA (Social and
Behavioral Sciences Team)
• Administrative Data Efforts
• New Performance Management Agenda & PMAB
N. HART | nrhart@gwu.edu Management and Evaluation in the Obama Administration 20

11. 9/15/2014
11
Second Term – Years 7 & 8
• To be seen…
– Guidance for next Budget suggests similar
approach to first 2 years of 2nd term.
– Expect few changes as term comes to a close.
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 21
General Observations
• Difficulties translating talk in to actions.
• Shift from legislative priorities to more
administrative actions.
• Evaluation and performance management
infrequently integrated in substance, function
or organizational structure.
• Prioritization of ECB efforts by agencies tends
to focus disproportionately on HHS, Ed, and
Labor.
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 22

12. 9/15/2014
12
Where do we go from here?
• Need to jointly engage evaluation and performance
measurement practitioners in agencies when
developing measures, performance plans, and
evaluations.
• Need to engage senior leaders empowered to facilitate
integration – may require training for senior leaders
and line staff.
• Recognize Federal agencies begin both efforts from a
variety of starting points – initiatives must be designed
to allow agencies flexibility to do what makes sense.
– Evaluation activities might benefit from a baseline capacity
survey.
N. HART | nrhart@gwu.edu Performance Measurement and Evaluation in the Obama Administration 23