Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Community-based Evaluation Methods and Practice

987 views

Published on

Community-based Evaluation Methods and Practice

Published in: Education
  • Be the first to comment

Community-based Evaluation Methods and Practice

  1. 1. Community based evaluationCommunity based evaluation methods & practicemethods & practice Program Science in PracticeProgram Science in Practice CAHR 2015 Ancillary EventCAHR 2015 Ancillary Event April 30, 2015April 30, 2015 Javier MignoneJavier Mignone Javier.mignone@umanitoba.caJavier.mignone@umanitoba.ca
  2. 2. Outline of the presentationOutline of the presentation - Clarifying the terminologyClarifying the terminology - Issues that must be addressedIssues that must be addressed - Program evaluation: a seven stepProgram evaluation: a seven step processprocess - Examples of collaborationExamples of collaboration J Mignone - University of Manitoba - 2015 2
  3. 3. What is program evaluation?What is program evaluation? ““Program evaluation is theProgram evaluation is the systematicsystematic collection of informationcollection of information about theabout the activities, characteristics, and outcomes ofactivities, characteristics, and outcomes of programs to make judgments about theprograms to make judgments about the program, improve program effectiveness,program, improve program effectiveness, and/or inform decisions about futureand/or inform decisions about future programming.” (Patton, 1997)programming.” (Patton, 1997) J Mignone - University of Manitoba - 2015
  4. 4. Why?Why? “…“…..for use by specific peoplefor use by specific people to reduceto reduce uncertainties, improve effectiveness, anduncertainties, improve effectiveness, and make decisions with regard to what thosemake decisions with regard to what those programs are doing and affecting.” Itprograms are doing and affecting.” It means, “gathering data that are meant tomeans, “gathering data that are meant to be, and actually are, used for programbe, and actually are, used for program improvement and decision making.”improvement and decision making.” (Patton, 1997)(Patton, 1997) J Mignone - University of Manitoba - 2015
  5. 5. Difference between evaluation andDifference between evaluation and researchresearch Basic scientific research is undertaken toBasic scientific research is undertaken to discover new knowledge, test theories,discover new knowledge, test theories, establish truth, and generalize across timeestablish truth, and generalize across time and space.and space. Program evaluation is undertaken toProgram evaluation is undertaken to inform decisions, clarify options, reduceinform decisions, clarify options, reduce uncertainties, and provide informationuncertainties, and provide information about programs and policies withinabout programs and policies within contextual boundaries of time, place,contextual boundaries of time, place, values, and politics.values, and politics. J Mignone - University of Manitoba - 2015
  6. 6. Difference between evaluation andDifference between evaluation and research (cont’d)research (cont’d) The difference between research andThe difference between research and evaluation has been described byevaluation has been described by Cronbach and Suppes as: Research isCronbach and Suppes as: Research is aimed at truth; Evaluation is aimed ataimed at truth; Evaluation is aimed at action.action. J Mignone - University of Manitoba - 2015
  7. 7. Core questionsCore questions Who uses the systematically producedWho uses the systematically produced information?information? How is the systematically producedHow is the systematically produced information actually used?information actually used? J Mignone - University of Manitoba - 2015
  8. 8. Community-based evaluationCommunity-based evaluation What community?What community? ◦ PlacePlace ◦ IdentityIdentity ◦ InterestInterest What does community-based mean?What does community-based mean? ◦ It depends on the purpose and the processIt depends on the purpose and the process J Mignone - University of Manitoba - 2015 8
  9. 9. Community-based evaluation (cont’d)Community-based evaluation (cont’d) Purpose of evaluationPurpose of evaluation Process of evaluationProcess of evaluation J Mignone - University of Manitoba - 2015 9
  10. 10. Some specific issues to addressSome specific issues to address Expectations about evaluationExpectations about evaluation Plural or conflicting agendasPlural or conflicting agendas TrustTrust Funding requirements and agenciesFunding requirements and agencies Evaluation plan developmentEvaluation plan development Cultural interpretations/knowledgeCultural interpretations/knowledge J Mignone - University of Manitoba - 2015 10
  11. 11. Issues to address (cont’d)Issues to address (cont’d) Appropriate methodologyAppropriate methodology TimelinesTimelines Ownership of data (e.g., OCAP)Ownership of data (e.g., OCAP) PoliticsPolitics ParticipationParticipation EthicsEthics Use/dissemination of findingsUse/dissemination of findings J Mignone - University of Manitoba - 2015 11
  12. 12. Program evaluation: A seven stepProgram evaluation: A seven step processprocess 1.1. Identify the primary intended users ofIdentify the primary intended users of the evaluationthe evaluation 2.2. Identify and focus the relevantIdentify and focus the relevant evaluation questionsevaluation questions 3.3. Make design methods and measurementMake design methods and measurement decisionsdecisions 4.4. Collect dataCollect data 5.5. Organize data for stakeholder analysisOrganize data for stakeholder analysis 6.6. Involve users in interpretation ofInvolve users in interpretation of findingsfindings 7.7. Facilitate intended use by intended usersFacilitate intended use by intended usersJ Mignone - University of Manitoba - 2015 12
  13. 13. Examples of collaborationExamples of collaboration Summer Institute on Program EvaluationSummer Institute on Program Evaluation (University of Manitoba)(University of Manitoba) Community Health Information andCommunity Health Information and Research Partnerships (CHIRP) (UniversityResearch Partnerships (CHIRP) (University of Manitoba students)of Manitoba students) J Mignone - University of Manitoba - 2015 13
  14. 14. ?????? J Mignone - University of Manitoba - 2015 14
  15. 15. ReferencesReferences  Chomsky N (2003)Chomsky N (2003) Necessary illusions: Thought control in democratic societiesNecessary illusions: Thought control in democratic societies. Toronto:. Toronto: House of Anansi Press.House of Anansi Press.  First Nations Centre at NAHO (2007) OCAP: Ownership, control, access andFirst Nations Centre at NAHO (2007) OCAP: Ownership, control, access and possession. NAHO: Ottawapossession. NAHO: Ottawa  Horne T (1995)Horne T (1995) Making a difference: Program evaluation for health promotionMaking a difference: Program evaluation for health promotion.. WellQuest Consulting: Edmonton.WellQuest Consulting: Edmonton.  Kaufman Kantor G & Kendall-Tackett K (2000)Kaufman Kantor G & Kendall-Tackett K (2000) A guide to family intervention andA guide to family intervention and prevention program evaluationprevention program evaluation. University of New Hampshire. Electronic version.. University of New Hampshire. Electronic version.  King JA, Stevahn L, Ghere G & Minnema J (2001) Toward a taxonomiy of essentialKing JA, Stevahn L, Ghere G & Minnema J (2001) Toward a taxonomiy of essential program evaluator competencies.program evaluator competencies. American Journal of Evaluation, 22American Journal of Evaluation, 22(2), 229-247.(2), 229-247.  Morra Imas LC & Rist RC (2009) The road to results: Designing and conductingMorra Imas LC & Rist RC (2009) The road to results: Designing and conducting effective development evaluations. The World Bank: Washington DC.effective development evaluations. The World Bank: Washington DC.  Patton MQ (1997)Patton MQ (1997) Utilization-focused evaluation: The new century text.Utilization-focused evaluation: The new century text. 33rdrd edition,edition, Thousand Oaks: Sage.Thousand Oaks: Sage.  Patton MQ (2008)Patton MQ (2008) Utilization-focused evaluationUtilization-focused evaluation, 4th Edition, Thousand Oaks: Sage., 4th Edition, Thousand Oaks: Sage.  Patton MQ (2012) Essentials of utilization-focused evaluation. Thousand Oaks: Sage.Patton MQ (2012) Essentials of utilization-focused evaluation. Thousand Oaks: Sage.  The Provincial Centre for Excellence for Child and Youth Mental Health at CHEOThe Provincial Centre for Excellence for Child and Youth Mental Health at CHEO (2007)(2007) Doing more with program evaluation: A toolkit for conducting program evaluationDoing more with program evaluation: A toolkit for conducting program evaluation.. Ottawa, OntarioOttawa, Ontario www.onthepoint.cawww.onthepoint.ca  Shadish WR, Cook TD, Campbell DT (2002)Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasi-experimentalExperimental and quasi-experimental designs for generalized causal inferencedesigns for generalized causal inference. Boston: Houghton Mifflin.. Boston: Houghton Mifflin.  Suchman EA (1967) Evaluative research: Principles and practice in public service andSuchman EA (1967) Evaluative research: Principles and practice in public service and social action programs. New York: Russell Sage.social action programs. New York: Russell Sage. J Mignone - University of Manitoba - 2015 15

×