Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators


Published on

This presentation describes the findings from The California Endowment Clinic Consortia Policy and Advocacy Program Evaluation, including: activities to increase clinic consortia advocacy capacity; expanded decision maker support for key policies benefiting clinics and their target populations; and expanded access to health care care for vulnerable populations. Summary findings and lessons for evaluators are included.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • The 2006 media representatives reinforced what we learned 2004 - it ’s the relationship. We added a couple of questions on specific grant-funded educational activities like clinics tours and issues briefs and interestingly they were consistently rated lower than being accessible, responsive, etc.
  • Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for Evaluators

    1. 1. Evaluating The California Endowment Clinic Consortia Policy and Advocacy Program: Lessons for EvaluatorsAnnette L. Gardner, PhD, MPHSara Geierstanger, MPHClaire Brindis, DrPHPhilip R. Lee Institute for Health Policy StudiesUniversity of California, San FranciscoNovember 4, 2011 University of California San Francisco
    2. 2. Presentation Objectives UCSF evaluation design and findings by outcome  Monitoring advocacy  Assessing impact Relevance of findings for researchers/evaluators, advocates and funders
    3. 3. The California Endowment Clinic Consortia Policy and Advocacy ProgramComponent of Community Clinics Initiative(CCI)Launched in 2001 and refunded two moreRounds in 2004 and 2007, an investment of$28 million14 local/regional clinic consortia and 5statewide clinic associations representing 800+primary care clinicsGoals: 1) Increase services to underserved populations 2) Increase access through improved clinic efficiencies
    4. 4. Theory of Change:“Resources and staffing dedicated to policy advocacy and TA will increase the collective influence of clinics and strengthen a broad base of long-term support of clinic policy issues.”
    5. 5. Grantees – 18 State, Regional, Clinic ConsortiaAlameda Health Consortium (AHC)Alliance for Rural Community Health (ARCH)California Family Health Council (CFHC)California Planned Parenthood Education Fund (CPPEF)California Primary Care Association (CPCA)California Rural Indian Health Board (CRIHB)Capitol Community Health Network (CCHN)Central Valley Health Network (CVHN)Coalition of Orange County Community Clinics (COCCC)Community Clinic Association of Los Angeles County (CCALAC)Community Clinic Consortium Serving Contra Costa and SolanoCounties (3C)Community Health Partnership, Inc. (CHP)Council of Community Clinics (CCC)North Coast Clinics Network (NCCN)Northern Sierra Rural Health Network (NSRHN)Redwood Community Health Coalition (RCHC)San Francisco Community Clinic Consortium (SFCCC)Shasta Consortium of Community Health Centers (SCCHC)
    6. 6. The California Endowment Clinic Consortia Policy and Advocacy Program 2001 - 03 2004 - 06 2007 - 09Advocacycapacity Technical AssistancePartnerships
    7. 7. Evaluation Design 2001 - 03 2004 - 06 2007 - 09 MethodsEvaluationTAEvaluationPartnership
    8. 8. Policy Change Logic Model: Measures and ToolsOutcome 1 Outcomes 2, 3 Outcome 4 Outcome 5 Outcome 6 Increase Increased policymaker Strengthened services for ImprovedExpand awareness clinic underserved healthadvocacy And support operationscapacity and Outcomes for clinic policy uninsured Issues, funding* Annual Advocacy * Decision Maker Activities Worksheet Awareness Surveys (2) * Clinic Focus * Policy Case * Policy Case* Annual Grantee * Interviews with Media Groups (2) Studies (3) Studies (3) Interviews Representatives (2) * Grantee Case * Grantee Case * Grantee Case* Sustainability * Policy Outcomes Wrksht Studies (17) Studies (17) Studies (17) Survey * Funding Secured Wrksht
    9. 9. Outcome 1: Expand Advocacy CapacityOrganizational capacity gains:  Staffing  Ability to maintain advocacy  Technical acumenAdvocacy activities:  Strategically deploy advocacy “toolkit”, e.g., Working with the media Educating/working with elected officials pre/post passage Pursuing/leveraging partnerships with stakeholders Mobilizing grassroots advocates
    10. 10. Tool: Annual Assessment of Advocacy Capacity and Outcomes 13 grant- Feder State Local Effective Outcome 1: Outcome 2: Outcome 3: funded al (Y/N) (Y/N) ness (1 Increased Achieve a Increased activities (Y/N) to 4, 4= clinic policy policymaker “very funding change awareness effective” (Y/N) (Y/N) (Y/N) Media advocacy Contact policymaker Analysis: s directly 1) Determine percent activities at the federal, state and local levels, and overall Informal contacts 2) Calculate effectiveness overall and by with elected officials activity 3) Determine percent of activities that Hosting achieved each Outcome clinic tours for 4) Sort by grantee age (pre/post 1990 launch) policymaker s Helping draft regulations, rules
    11. 11. Findings: Grantee Advocacy Capacity, 2002 to 2006 Number: engaged in more grant-funded advocacy activities--increase from 73% to 86% of surveyed activities. Target: consistently targeted most activities (80%) at the local and state levels and less (60%) at the federal level. Effectiveness: fairly constant or 3.28 to 3.36 (4 equaled “very effective” Grantee type: newer (post-1990) consortia were equally effective in achieving advocacy goals by 2006.
    12. 12. Outcome 2: IncreasePolicymaker AwarenessWhoever controls the flow ofinformation dictates ourperceptions and perspectives;whoever controls the newsshapes our destiny. - George Clinton Musician and Funk Philosopher
    13. 13. Evaluating Grantee Work With the MediaEffectiveness of Advocacy Activities Worksheet: # of granteesmedia compared to using media at federal, state, local levels;other advocacy achievement of three Program outcomestacticsPartnerships with # and type of media request for information;media Number and type of media outletsCoverage secured # of stories in newspapers, radio Placement of stories, e.g., page, column Newspaper article type (op-ed, Letter to the Editor)Presence of policy Inclusion of specific issues, e.g., role ofissues in the clinics, patients servedmedia Framing of issues - content analysisOutreach to media # of press releases developed and distributed (grantee tracking)
    14. 14. Findings-- Media Perspective: Ef fectiveness of Grantee Strategies to Obtain Media Coverage Press ReleasesTiming messages Provide data 2006 Access to clinic 2004 staff/patientsPrompt responsesto media inquiries Relationships w/media 0 1 2 3 4 5 5=very effective
    15. 15. Policymaker and Community LeaderFamiliarity with Grantee and Clinic Activities 3.7 3.3 3.4 3.5 3.1 2.7 2003 Policymakers 2004 Community Leaders Grantee Grantee Clinic Activities Activities 4 = high familiarity
    16. 16. Educating AND Working with Policymakers Advocacy contributes to policymaker awareness. The challenge is to position yourself as a credible partner and work with decision makers Watch for opportunities, e.g., bipartisan support Leverage the partnership with decision makers pre/post passage of policies Maintenance of effort required - ongoing education and visibility
    17. 17. Outcome 3: Increased Policymaker Support for Clinic for Clinic Funding Tool: Annual Grantee Policy Outcomes Worksheet Policy Bill Grant-funded policy Target populations Outcomes as of targeted Number advocacy activities to be affected by 12/07. Choose from and/or undertaken in 2007. policy. Choose list: a) Passed - signed title? Choose from list: a) from list: a) clinics; in to law; b) Vetoed; c) media advocacy; b) in- b) uninsured; c) low- Pending - still being person meetings with income populations; considered by decision decision makers; c) d). clinic consortia; e) makers; d) funding drafting of rules, other special secured; e) Other: list. regulations; d) materials populations; f) Other: development; e) grass list roots activities; f) Other: listFederal Analysis:State 1. Count number policies and of grantees that mentioned each policy 2. Code advocacy activities, target populationsLocal 3. Code and count outcomes, e.g., “passed”, “vetoed”, “pending”
    18. 18. Evidence of Policymaker Support –Funding Secured by Grantees (ROI)
    19. 19. Funding Secured, by Source
    20. 20. Outcomes 4 & 5: Strengthened Clinic Operationsand Increased Services for The Underserved and UninsuredFindings: Evidence that more Californians have insurancecoverage and are using the services to which they haveaccess
    21. 21. OUTCOME 6: Improved Health Outcomes for Targeted PopulationsAchievements in three areas :
    22. 22. Summary of Evaluation FindingsAdvocacy as staffing and toolkit to be deployedstrategically – gains after 2-3 yearsStrategic involvement in policymaking process -desired policy change facilitated by maintenanceof effort, technical acumen and credibilityCrafting working relationships with decisionmakers - “pays off”Achieving a policy change – gains to memberclinics and their target populationsPartnerships with member clinics – critical toachieving a workable solution
    23. 23. Policy Change Logic Model Increase Increased Expand policymaker Strengthened services for ImprovedAdvocacy awareness clinic underserved health operations and outcomesCapacity and uninsured support
    24. 24. Suggestions for EvaluatorsAddress different information needs of advocates,funders, evaluatorsCombine rigor with creativity  Mixed methods  Triangulate  Clarify what you’re evaluating re “advocacy” and “advocacy capacity”Create a working partnership with advocates,funderCommunications are key
    25. 25. Thank you! For More InformationAnnette Gardner, PhD, MPHPhilip R. Lee Institute for Health Policy StudiesUniversity of California, San Francisco3333 California St., Suite 265San Francisco, CA 94118Phone: (415) 514-1543Email: annette.gardner@ucsf.eduhttp://ihps.medschool.ucsf.edu/News/california_endowment.aspx University of California San Francisco