The Role of Federal Evaluation Activities in the “Evidence-Based” Policy and “Data-Driven” Government Movements
•
1 like•113 views
In early 2019, a new federal law called the Foundations for Evidence-Based Policymaking Act (Evidence Act) established new expectations for how federal agencies support and engage in evaluation activities. As part of the broader evidence movement, the new law recognizes relationships across traditional disciplinary silos and outlines and expectation that federal agencies will better collaborate to produce and use evaluations to inform key decisions. This webinar will describe the contemporaneous policy environment for federal evaluation, offer insights about the role in the broader evidence movement, and discuss opportunities for interacting with emerging data strategies and planning processes that affect evaluators. Participants will gain an appreciation for the nuances of federal evaluation policy activities, develop an understanding of how the activities relate to other data policy reforms, and be introduced to opportunities for ongoing participation from the evaluation community.
Recommended
Recommended
More Related Content
What's hot
What's hot (11)
Similar to The Role of Federal Evaluation Activities in the “Evidence-Based” Policy and “Data-Driven” Government Movements
Similar to The Role of Federal Evaluation Activities in the “Evidence-Based” Policy and “Data-Driven” Government Movements (20)
More from Nick Hart, Ph.D.
More from Nick Hart, Ph.D. (16)
Recently uploaded
Recently uploaded (20)
The Role of Federal Evaluation Activities in the “Evidence-Based” Policy and “Data-Driven” Government Movements
- 1. Role of Federal Evaluation Activities in the “Evidence-Based” Policy and “Data- Driven” Government Movements Nick Hart, Ph.D. / Nick.Hart@DataFoundation.org Webinar for American Evaluation Association on October 1, 2019
- 2. |@Data_Foundation • datafoundation.org About the Data Foundation ● Non-profit research think-tank based in DC ● Support efforts to provide research, training and education about data policy – including evaluation
- 4. |@Data_Foundation • datafoundation.org There are Success Stories!
- 5. |@Data_Foundation • datafoundation.org There are Success Stories!
- 6. |@Data_Foundation • datafoundation.org What are the movements? Evidence-Based Policy & Data-Driven Government ● Connote that analytics can be better used to inform decision-making ● Evaluation plays critical role as a form of “evidence” ● Built on numerous drivers that affect federal evaluation policy
- 8. |@Data_Foundation • datafoundation.org What’s the Role for Evaluators?
- 9. |@Data_Foundation • datafoundation.org Role for Evaluation Community Evidence ActData Linkage AppropriationsEvidence Commission
- 10. |@Data_Foundation • datafoundation.org What happened?
- 11. |@Data_Foundation • datafoundation.org What happened? ● New laws (Evidence Act, OPEN Government Data Act, Bipartisan Budget Act, individual authorizations) ● Bipartisan reports (Evidence Commission) ● Additional legislative proposals (transparency, data, evidence) ● Administrative proposals (statistical agencies, evaluation leaders, learning agendas) ● Executive Branch guidance (data inventories, open data, impact evaluation) ● Attention to people and funding
- 12. |@Data_Foundation • datafoundation.org What actually changed?
- 13. |@Data_Foundation • datafoundation.org What actually changed? ● Budget Requests Funding ? ● Guidance No. of Evaluations ? ● Enthusiasm Culture Change ? ● Leaders Leadership ? ● Dialogue Practice ? ● Better Government ?
- 15. |@Data_Foundation • datafoundation.org What does it all mean?
- 16. |@Data_Foundation • datafoundation.org What does it all mean? 1. Intense emphasis on evaluation policy government-wide is still relatively new, and will continue to be a learning process for all.
- 17. |@Data_Foundation • datafoundation.org What does it all mean? 2. While the discourse may not appear to always be about evaluation, what is often of eventual interest in policy is informed, high-quality “impact evaluation”
- 18. |@Data_Foundation • datafoundation.org What does it all mean? 3. Organizational capacity and culture change are central – biggest challenges are people, systems, and resources
- 19. |@Data_Foundation • datafoundation.org Where do we go from here?
- 20. |@Data_Foundation • datafoundation.org Where do we go from here? #1: More education about evaluation www.govDATAx.com
- 21. |@Data_Foundation • datafoundation.org Where do we go from here? #2: Better case studies of use in real-world (that made a significant, sustained difference) #3: Embrace the EBP & DDG mantras to demonstrate evaluation value #4: Integrate into government processes and cultures
- 22. |@Data_Foundation • datafoundation.org Where do we go from here? #5: Support agency leaders in roles to support effective implementation of the Evidence Act: ○ Recognize opportunities ○ Ask questions about evidence ○ Support the Learning Agenda process ○ Allocate resources where critical ○ Follow-up on Evidence Plans ○ Sustained attention
- 23. |@Data_Foundation • datafoundation.org Where do we go from here? #6: Increase participation from the evaluation community in: ○ Federal Data Strategy formulation and execution ○ Learning Agenda development ○ Establishment of evaluation units, policies, and workforce ○ Responses to information collection requests ○ Engagement in advisory committees
Editor's Notes
- Good afternoon – thanks for joining -0- sponsored by AEA with support from Moore Foundation Intro EPTF DF DC OMB Goal of this webinar is to describe the policy environment for evaluation Lots of discussion recently about terms like EBP and DDG – where does evaluation fit into this, and how does the evaluation community support policy that actually makes evaluation successful
- Since will see references throughout… [describe DF]
- Context of an environment that at least in media may be capture that we’re not always using evaluation and evidence – or may not want to. This cartoon – policy-based evidence-making. True, politics and values will always play a role But at the federal level movements underway to improve, and better address our ability to explain and understand policies and programs
- These success stories are examples of how the movement is advancing today. Attempts to better understand, learn, and improve. And its not just based on evaluation – sometimes this is performance information, descriptive statistics, descriptive regression analyses, etc. But evaluation has a big role – and is well-recognized as one of the essential forms of evidence needed in these movements. -- will come back to this point. The movements have been built on frustration about not being able to answer key questions – and a desire to see iterative improvements in program performance, even when budgets may not be changing drastically. One central point --- whether we are talking about EBP of DDG – folks are typically talking about the same thing whether they realize it or not. And they really want evaluation, whether they realize it or not. These are really multi-disciplinary, trans-disciplinary efforts. They combine individuals with variety of epistemologies. They combine folks with different backgrounds. And they will lead to more collective and actionable knowledge.
- Let’s talk about the movements and motivators: In 2012 – budget fight – Ryan Murray led to CEP CEP recs – much on data -0- specific focus and role ofr evaluation Led to FEBP In parallel excutive branch push: PMA Reform proposal re evaluation + Las Budget proposals New guidance to agencies == A-11, m-19-23 Development of resoruces on what it means to use – and how AEA Roadmap Many agencies have taken this as encouragement to push forward, even before there was a law. Looking for signals of broader support to change how we do business.
- Probably wondering – what’s the role for evaluation because you keep saying evidence?
- Evaluation Community has not waited on the sidelines. Actively involved in the movements. Again, CEP specifically advanced evaluation – and AEA was well-represented in that process. AEA Has also been part of many of the efforts since, applauding the CEP, encouraging the Evidence Act, providing expertise in areas like data linkage, and even encouraging approps for new chief evaluation officers.
- This is a lot of movement – and pieces of the puzzle. What happened? Well, as I describe this you might think it just sounds like organized chaos. You could be right. But there is method to the madness.
- The new laws provide a legal framework to operate in. Political leadership and the CEP report provides a bipartisan environment to ensure this is not politicized (even if the use may be) It’s encouraged more forward thinking and planning about how to improve the system – including proposals in Congress and the Exec branch to bolster the system It’s definitely raised attention for the need to have capacity to do this work.
- Other than enthusiasm and reports – what has really changed?
- Depends on where you sit. Depends on agency Depends on role. But I think a lot has actually changed. The real question is – are these the outcomes that we seek?
- I’ve worked in the policy environment for years – there are always optimists and pessimists I’m the former. This is an exciting time to be working on evaluation – to be thinking about how to evolve the entire system for good, and to encourage participation in these movements that are much bigger than evaluation.
- So what does this really all mean – 3 take-away points:
- While the evaluation field has been talking about evaluation for a long time – not everyone else has This is still relatively new We have pockets of programs that have historically done well, and others that are still learning As we figure out how to implement the Evidence Act – and whatever is next – there will be mistakes, and we need to do what evaluators do: learn and improve. Will be important to hold government accountable – political and career staff inclusive.
- Evaluation may not be the term everyone is using, but it’s the concept everyone is leading up to. Specifically want to be able to better articulate impacts – but we know doing that well often involves knowledge of process, may not strictly be a quantitative exercise, and that there are lots of aspects to evaluation that are relevant Must be able to continue helping others learn and understand.
- We often like to have debates about methods. It’s not our biggest problem. Systems change is. And we know generally what it takes to change organizational culture – and that it’s also really hard to do.