Uploaded on

Slides for Evaluating Advocacy Presentation

Slides for Evaluating Advocacy Presentation

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
250
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • With a real-time approach evaluators are embedded and emphasize a collaborative and participatory evaluation process. This approach is different from traditional evaluation in which the evaluator remains completely separate from the program or strategy. “Evaluators become part of a team whose members collaborate to conceptualize, design and test new approaches in a long-term, ongoing process of continuous improvement, adaptation, and intentional change. The evaluator’s primary function in the team is to elucidate team discussions with evaluative questions, data and logic, and to facilitate data-based assessments and decision-making in the unfolding and developmental processes of innovation.”  Patton, M. Q. (2006).Evaluation for the way we work. The Nonprofit Quarterly.
  • The framework contains specific types of strategies and activities, organized according to where they fall on two strategic dimensions—the audience targeted (x-axis) and the outcomes desired (y-axis). Audiences are the groups that policy strategies target and attempt to influence or persuade. They represent the main actors in the policy process and include the public (or specific segments of it), policy influencers (e.g., media, community leaders, the business community, thought leaders, political advisors, etc.), and decision makers (e.g., elected officials, administrators, judges, etc.). These audiences are arrayed along a continuum according to their proximity to actual policy decisions; the farther out they are on the continuum, the closer they are to such decisions. Naturally, decision makers are the closest to such decisions, and therefore are on the continuum’s far end. Grantmaking may focus on just one audience or target more than one simultaneously. Outcomesare the results an advocacy or policy change effort aims for with an audience in order to progress toward a policy goal. The three points on this continuum differ in terms of how far an audience is expected to engage on a policy issue. The continuum starts with basic awareness or knowledge. Here the goal is to make the audience aware that a problem of potential policy solution exists. The next point is will. The goal here is to raise an audience’s willingness to take action on an issue. It goes beyond awareness and tries to convince the audience that the issue is important enough to warrant action, and that any actions taken will in fact make a difference. The third point is action. Here, policy efforts actually support or facilitate audience action on an issue. Again, grantmaking may pursue one outcome or more than one simultaneously. Foundations can use the framework to examine how to position their public policy strategies along these two dimensions. Rather than jumping straight to decisions about which activities to fund (e.g., public awareness campaigns, polling, etc.), the framework encourages foundations to think first about which audiences they need to engage and how hard they need to “push” those audiences toward action.HYPOTHETICAL: The shading in the figure at right illustrates how this might work. The hypothetical policy goal in this example calls for an action-oriented strategy focused primarily at the public or community level. The strategy supports activities that include organizing, coalition building, and mobilization activities to generate the action needed to move the policy issue forward.RISK: It’s also important to note that foundations perceive different parts of this framework as riskier than others.

Transcript

  • 1. Evaluating Advocacy:
    Dilemmas, Tactics, and Methods
    Julia Coffman
    Center for Evaluation Innovation
    jcoffman@evaluationinnovation.org
    October 7, 2011
  • 2. Three Questions
    • How is evaluating advocacy different?
     What can we measure about advocacy?
    How can we measure it?
  • 3. How is evaluating
    advocacy different?
  • 4. Challenge: The policy environment is complex,
    and that makes attribution hard.
  • 5. Solution: If accountability is the purpose, demonstration of contribution
    is expected, not attribution.
    What is my unique
    contribution?
  • 6. Challenge: Advocacy strategies shift in
    response to the environment.
    Shifting Politics
    Shifting Economics
    New Partners
  • 7. Solution: Advocacy is a good opportunity
    to integrate or embed evaluation for learning
    (not just accountability).
    ADVOCACY
    EVALUATION
  • 8. Challenge: Timeframes can be unpredictable.
    1 Year
    1 Year
    1 Year
    Advocacy Timeframe
    Goal
    Reporting Timeframe
  • 9. Solution: Assess progress,
    not just the end result.
    Goal
    …progress…
  • 10. What can we measure about influencing?
  • 11. Measure meaningful things.
    Don’t just count what is easy to quantify.
  • 12. Have realistic expectations.
    What are other influencers doing?
    Where is your issue in the policy process?
    What’s the political
    context?
    What’s the opposition doing?
    What are you doing and who are you targeting?
  • 13. Measure the changes made along the way,
    not just the end result.
    Policy Influence
    …progress…
    INTERIM OUTCOMES
  • 14. Interim outcomes are the changes you expect as you work toward your goal.
    Think about the changes you will
    see in your audiences.
  • 15. Use the framework to think about interim outcomes.
    ACTION
    HOW will they change as a result of your work?
    WILL
    OUTCOMES
    AWARENESS
    WHO will change as a result of your work?
    DECISION MAKERS
    INFLUENCERS
    PUBLIC
    AUDIENCES
  • 16. Where are your audiences?
    How far do you need to move them?
    ACTION
    Increase quality of child care
    WILL
    Child care providers
    INTERIM OUTCOMES
    AWARENESS
    Parents of young children
    DECISION MAKERS
    INFLUENCERS
    PUBLIC
    Legislators
    AUDIENCES
  • 17. Interim Outcomes
    Awareness
    Action
    Increased knowledge
    Increased collaboration among advocates
    Increased issue visibility or recognition
    Increased media coverage
    Activities and Outputs
    Policy Goals
    Will
    Reframing of the issue
    Changed attitudes or beliefs
    New and active advocates
    Increased salience
    New and active high-profile champions
    Increased personal or collective efficacy
    New donors
    Increased willingness to act
    Increased or diversified funding
    Increased capacity
    to act
  • 18. How can we measure it?
  • 19. Traditional Evaluation Methods
    Interviews
    Surveys
    Focus Groups
    Polling
  • 20. Non-Traditional Methods
    Charting and Mapping
    Survey and Interview
    Debriefing and Rating
    Media/Messaging Tracking
    System Mapping
    Intense Period Debriefs
    Research Panels
    Media Tracking
    Media Scorecards
    Network Mapping
    Crowdsourcing
    360-Degree Critical Incident Debriefs
    Advocacy Capacity Assessment
    Snapshot Surveys
    Policy Maker Ratings
    ECCO Analysis
    Critical Incident Timelines
    Intercept Interviews
    Policy Tracking
    Champion Tracking
    Bellwether Methodology
  • 21. Bellwether Methodology
    Bellwethers are:
    “Influentials” in the public and private sectors whose positions require that they are politically informed and that they track a broad range of policy issues
    Developed by Harvard Family Research Project
  • 27. Bellwether Methodology
  • 28. Bellwether Methodology
    Can preschool help address
    the achievement gap?
    How would you address the achievement gap?
  • 29. Bellwether Methodology
    What is the likelihood that California will increase preschool investments in the next 3 years?
  • 30. Policymaker Ratings
    Ratings are completed for a whole governing body or defined group of policymakers
    Developed by Harvard Family Research Project
  • 31. Policymaker Ratings
    Developed by Harvard Family Research Project
  • 32. Policymaker Ratings
    Del Norte
    * Hypothetical Data
    Siskiyou
    Modoc
    Trinity
    Shasta
    Lassen
    Humboldt
    Tehama
    Plumas
    Mendocino
    Butte
    Glenn
    Sierra
    = Increase in Support
    Nevada
    Yuba
    Colusa
    Placer
    Lake
    Sutter
    El Dorado
    Yolo
    Sonoma
    Napa
    Alpine
    Sacramento
    Amador
    Solano
    Calaveras
    San
    Marin
    Mono
    Costa
    Tuolumne
    Joaquin
    Contra
    San Francisco
    Alameda
    Stanislaus
    San
    Mariposa
    Mateo
    Santa Clara
    Merced
    Santa
    Madera
    Cruz
    Fresno
    Inyo
    San
    Benito
    Low Support
    Tulare
    Monterey
    Kings
    San
    Medium Support
    Luis
    Obispo
    Kern
    San Bernardino
    High Support
    Santa
    Barbara
    Ventura
    Los Angeles
    Orange
    Riverside
    Imperial
    San Diego
    Developed by Harvard Family Research Project
  • 33. Policymaker Ratings
    Developed by Harvard Family Research Project
  • 34. Wrap Up
    • Stay anchored in what you can reasonably change.
    • 35. It’s okay to prioritize and focus on what is most important to assess.
    • 36. Be creative—advocacy evaluation is an emerging field.
  • Center for Evaluation Innovation
    www.evaluationinnovation.org