Considerations, analysis of approaches and guidelines
Small Group 310/15-10/26
• Welcome to our presentation on Chapters 9-11 discussing Considerations, Analysis of Approaches, and Guidelines for evaluations.• Content for each section will be delivered in multiple media based on the most effective means for presentation.• Please review the summary content and respond to the discussion questions at the end of each section using the Blackboard discussion threads.
Other CurrentConsiderations: CulturalCompetenceand CapacityBuilding Analysis of Approaches Chapter 9 Chapter 10 Clarifying the Chapter 11 Evaluation Request and Responsibilities
Other Current Considerations: Cultural Competence and CapacityBuildingPresented by: Kaylea Howarth
• Two factors that influence evaluation practice, but transcend particular evaluation approaches: • the need to build cultural competence • evaluation capacity building or mainstreaming evaluation• We need to give consideration to the context of our evaluations because they take place in both an organizational context as well as a broader cultural context. These contexts influence, or should influence, our evaluation choices.
“A systematic, responsive inquiry that is actively cognizant,understanding, and appreciative of the cultural context inwhich the evaluation takes place; that frames and articulates theepistemology of the evaluative endeavor; that employsculturally and contextually appropriate methodology; and thatuses stakeholder-generated, interpretive means to arrive at theresults and further use of the results” (SenGupta, Hopson, &Thompson-Robinson, 2004, p. 13).
• The issue: As evaluators, we often conduct evaluations within cultural contexts that differ from our own.
Why is it important?• To identify the needs of stakeholders• To consider multiple perspectives of program success• To increase the legitimacy of the evaluation to all stakeholders• To increase the usefulness of the resultsHow can we achieve it?• Self-examination• Awareness of your own cultural norms• Make inclusion a priority• Quiet observation, respectful interactions, and reflection
Example: A pipeline company assigns an internalevaluator from head office to launch an evaluation of thecompany’s community investment program. This requirescultural competence in recognizing the context of eachrural community and responding appropriately to eachstakeholder’s needs.
• Evaluations within an organizational context have impact beyond programs, but on the organization itself.• Process Use: • “Individual changes in thinking and behavior, and program or organizational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process” (Patton, 1997c, p. 90). Process Use • Learning • Decisions Organization Evaluation • Analysis Results • Changes
• The recognition of this impact lead to the movement to mainstream evaluation in organizations. • Mainstreaming Evaluation: • “The process of making evaluation an integral part of an organization’s everyday operations. Instead of being put aside in the margins of work, evaluation becomes a routine part of the organization’s work ethic if it is mainstreamed. It is part of the culture and job responsibilities at all levels of the organization” (Sanders, 2002, p. 254). Evaluati Workflow Workflow on Evaluati onNon- MainstreamedMainstreamed
• In parallel to mainstreaming, others have maintained a focus on evaluation capacity building (ECB) in organizations.Evaluation Capacity Building (ECB) “A context-dependent, intentional action system of guided processes and practices for bringing about and sustaining a state of affairs in which quality program evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or more organizations/programs/sites” (Stockdill, Baizerman, & Compton, 2002, p. 8).
• Evaluations in organizations have two forms: • project-based, designed to provide information on the program or policy being evaluated • ongoing, with the evaluator working to sustain an environment conducive to evaluation and its use within the organization
Strategies for building evaluation capacity andmainstreaming:• Stakeholder Involvement in an Evaluation• Coaching/Mentoring• Technology• Written Materials• Training• Communities of Practice• Meetings
• Example: A financial trade company recognizes a need to evolve their programs over time to remain competitive within their sector. Yet as opposed to maintaining an evaluation group separate from the internal programs and processes to be evaluated, senior leadership pushes to mainstream the evaluation processes so that each job role and workgroup is involved in program improvement.
Limitations of mainstreaming evaluation and buildingevaluation capacity:• Need to build competence and skills within the organization• Danger of reducing quality of formal evaluations• Organization members are not evaluation professionals
1. Cultural Competence: refer to example 1 on slide 9 You are the lead evaluator for the pipeline company’s Community Investment program evaluation. Develop a strategy for how you will enact cultural competence in deal with the regional contexts within your evaluation. Use the text for examples and guidelines.2. Mainstreaming Evaluation: refer to example 2 on slide 15 You are a senior leader at the financial trade organization. Develop a strategy for how you will mainstream evaluation. Use the text as well as the Preskill presentation on slide 14 (specifically 10:14 – 14:30) for examples and guidelines. Please post your responses in the designated Chapter 9 thread on Blackboard.
A Comparative Analysis of ApproachesPresented by: Carolina Sanchez-Lopez & Reynaldo Lopez
• Please view the following Prezi for a summary of Chapter 10: Chapter 10: A Comparative Analysis of Approaches Prezi http://prezi.com/7sdnxf_99ake/a-comparative-analysis-of- approaches/?utm_source=share&utm_campaign=shareprezi&utm_medium=email
1. Should we attempt to synthesize the various approaches into one? What would be the advantages of doing so?2. Adams Elementary School has started a volunteer program in which parents are encouraged to help out in the classroom. The goal of the program is to not only provide the teacher with assistance, but also to get parents more involved in the school and their children’s education. The principal hopes to boost the learning of the children who are achieving below grade level by getting their parents more involved in their children’s education through volunteer efforts in the classroom. Contrast using a decision-oriented, participant-oriented, and program- oriented approach.Please post your responses in the designated Chapter 10 thread onBlackboard.
Clarifying the Evaluation Request and ResponsibilitiesPresented by: Carole Poche
• Please view the following Prezi for a summary of Chapter 11: Chapter 11: Clarifying the Evaluation Request & Responsibilities http://prezi.com/recommend/5q1wxgdcqfsh
• In getting down to some solid planning for an evaluation – what concerns must one address?• When might you refuse to do an evaluation? What types of reasons exist not to evaluate?• What types of research results can not be used?• What reasons are given for the creation of evaluability assessment? How does a program become evaluable?Please post your responses in the designated Chapter 11 thread onBlackboard.
• Fitzpatrick, J.L., Sanders, J.R., & Worthen, B.R. (2011). Program Evaluation: Alternative Approaches and Practical Guidelines (4th Ed.). Pearson. Allyn & Bacon.• Patton, M. Q. (1997c). Utilization-focused evaluation: The new century text. Thousand Oaks, CA: Sage.• Preskill, H. Effective Strategies for Facilitating Evaluation Capacity Building. [video] Retrieved from http://www.youtube.com/watch?v=uW-D9K0Ohl0• Sanders, J. (2002). Presidential address: On mainstreaming evaluation. American Journal of Evaluation, 23(3), 253-259.• Stockdill, S. H., Braizerman, M., & Compton, D. W. (2002). Toward a definition of the ECB process: A conversation with the ECB lliterature. In D.W. Compton, M. Braizerman, & S. H. Stockdill (Eds.), The art, craft, and science of evaluation capacity building. New Directions for Evaluation, No. 93, 7- 26. San Francisco: Jossey-Bass.• SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation: An overview. In M. Thompson-Robinson, R. Hopson, & S. SenGupta(Eds.), In search of cultural competence in evaluation: Toward principles and practices. New Directions for Evaluation No. 102, 5-20. San Francisco: Jossey-Bass.• Trevisan, M. And Huang, Y. (2003) Evaluability Assessment: A Primer, Practical Assessment, Research & Evaluation. Retrieved from http://pareonline.net/getvn.asp?v=8&n=20Images retrieved from iStock Royalty Free Images