Positive Youth Development (PYD) Framework toolkit application modules - Dr Bardini and Cassandra Jessee
•Download as PPTX, PDF•
2 likes•1,682 views
Dr Bardini and Cassandra Jessee from YouthPower hosted a workshop on Measuring Positive Youth Development (PYD) at the 8th AfrEA International Conference in Kampala, Ghana.
More Related Content
What's hot
Similar to Positive Youth Development (PYD) Framework toolkit application modules - Dr Bardini and Cassandra Jessee
Similar to Positive Youth Development (PYD) Framework toolkit application modules - Dr Bardini and Cassandra Jessee (20)
More from Khulisa Management Services
More from Khulisa Management Services (20)
Recently uploaded
Recently uploaded (20)
Positive Youth Development (PYD) Framework toolkit application modules - Dr Bardini and Cassandra Jessee
- 1. Application of the PYD Framework
- 2. Divide into Groups • Sectors: • Education/ Workforce Development • Health • Democracy & Governance • Move with your items, as you will stay in these groups for much of the training.
- 3. Five Key Phases of Program Design and Evaluation for PYD 1. Refine key desired outcomes or research questions 2. Selecting PYD features and beneficiaries of the program 3. Finalize the logic model 4. Decide what to measure, and how (study design and indicators) 5. Analyze the data, disseminate the findings and adapt your program
- 4. Read Youth Theater for Peace (YTP) Case Study
- 5. Phase One: Refine Key Desired Primary Outcomes and Research Questions Key Question How can I use the PYD Measurement Framework to help me select the key primary outcomes and research questions?
- 6. Primary outcomes: What you hope to achieve at the end of your youth program. This should be related to your overall program goal. The research question: Whether the activities and outcomes of the program lead to a change in your primary outcomes of interest?
- 7. PYD Framework Four PYD Domains: • Assets • Agency • Contribution • Enabling Environment
- 8. Using the PYD Framework to Define Primary Outcomes and Research Questions Domains • How do your outcomes/ research questions relate to the four domains? • Can you include more than one? Features • Which PYD features support your goal? • These will help ensure your activities have long-term PYD impact Constructs and Indicators • Take into account relevant constructs when developing outcomes and research questions • Consider which constructs are associated with key sectoral outcomes of interest
- 9. Case Study: YTP Phase 1 • Goal: “Empower youth and adults in conflict-prone communities to stimulate lasting conflict prevention and promote a culture of peace within their communities and beyond.” • Two outcomes: • Conflict prevention • Reducing violence
- 10. Phase Two: Selecting PYD Features and Beneficiaries Key Questions Which PYD features should be included in my program, and should be the target beneficiaries? How are features and beneficiaries connected to the key outcomes of my program?
- 11. The seven PYD features are based on the most effective elements of a PYD program. Beneficiaries can include multiple levels
- 12. Using the PYD Framework to Select Program Features and Beneficiaries Features • Consider which features support your primary outcomes of interest Domains • How do the features you’ve selected connect back to the domains? Which beneficiaries do these features relate to?
- 13. Case Study: YTP Phase 2 Features • Skill building: program aimed to give youth tools to express themselves and take action, respect diverse opinions and develop empathy, and understand conflict cycles and triggers • Safe Space: YTP created a safe space on the stage for trust and dialogue Beneficiaries • Youth • Adult mentors
- 14. Group Activity: Complete Phase One and Phase Two • What are the primary outcomes of the YTP program, specifically related to PYD? • What are the primary research questions? • What PYD features relate best to the program? • What beneficiaries is the program targeting?
- 15. Phases One and Two: Refine Key Desired Primary Outcomes and Research Questions; and Selecting PYD Features and Beneficiaries •What types of challenges did you face in this exercise? •What did you learn?
- 16. Phase Three: Finalizing the Program Logic Model Key Question How can I use the PYD Measurement Framework to develop my logic model?
- 17. Logic Model
- 18. Using the PYD Framework to Finalize a Logic Model • Connect PYD domains and features (inputs and activities) with PYD outcomes of interest, which are tied to the indicators • This helps to illuminate outputs, outcomes and impacts, and ultimately the sector-specific PYD outcomes of interest (outcomes and impacts)
- 19. Case Study: YTP Phase 3
- 20. Case Study: YTP Phase 3
- 21. Case Study: YTP Phase 3
- 22. Case Study: YTP Phase 3
- 23. Group Activity: Complete Phase 3 Connect inputs and activities with PYD outcomes of interest
- 24. Phase Three: Finalizing the Program Logic Model • What types of challenges did you face in this exercise? • What did you learn?
- 25. Phase Four: Decide What to Measure, and How Key Questions Which PYD constructs do I want to measure? Which PYD measurement sources (e.g., survey tools) do I use to measure my constructs of interest? What research design best addresses the evaluation needs of my project?
- 26. Indicators Should • Reflect the PYD constructs central to the goals and objectives of your project • Use sources appropriate to that indicator • Reflect the appropriate level or stage of the program • Be the right type of indicator • Pertain to the targeted beneficiaries of the program
- 27. Using the PYD Framework to Select PYD Indicators and Data Collection Methods Constructs and Indicators • Review Illustrative Indicators Table and find constructs relevant to your project • Select indicators most suited to beneficiaries and level of analysis Domains • Ensure you have at least one indicator for each relevant domain Features • Ensure you have at least one indicator for each construct indicated by the relevant features
- 28. Case Study: YTP Phase 4: Possible Indicators Assets: Skill Building • Increased interpersonal skills • Increased higher-order thinking skills Contribution: Youth Engagement • Youth report being participating in program’s design, implementation, or evaluation Agency: Skill Building • Increased positive beliefs about own future • Increased feelings of self-efficacy • Increased ability to plan and set goals Enabling Environment: Safe Space • Increased feeling of safety in their environment
- 29. Group Activity: Complete Phase 4 • Use the illustrative indicators table to select 3 relevant PYD indicators for the project • Consider the study design that you would develop in order to measure these indicators
- 30. Phase Four: Decide What to Measure, and How • What types of challenges did you face in this exercise? • What did you learn?
- 31. Phase Five: Analyze, Disseminate and Learn From the Data Key Questions How will I analyze the data that I collect? How will I learn from and communicate the PYD story from my program?
- 32. PYD approaches for data analysis • Qualitative data gathered through interviews, meetings, or focus groups • Prompts or questions can link directly to the PYD domains or concepts. • Data can be coded for the domains or specific PYD concepts.
- 33. PYD approaches for data analysis • Quantitative data should be collected and analyzed to detect the desired change in PYD indicators. BASELINE • SELF- EFFICACY • PROBLEM- SOLVING MIDLINE • SELF- EFFICACY • PROBLEM- SOLVING ENDLINE • SELF- EFFICACY • PROBLEM- SOLVING FOLLOW-UP • SELF- EFFICACY • PROBLEM- SOLVING
- 34. PYD approaches for data analysis • Compared to whom? PRETEST • SELF-EFFICACY • PROBLEM- SOLVING MIDLINE • SELF-EFFICACY • PROBLEM- SOLVING ENDLINE • SELF-EFFICACY • PROBLEM- SOLVING FOLLOW-UP • SELF-EFFICACY • PROBLEM- SOLVING PRETEST • SELF-EFFICACY • PROBLEM- SOLVING MIDLINE • SELF-EFFICACY • PROBLEM- SOLVING ENDLINE • SELF-EFFICACY • PROBLEM- SOLVING FOLLOW-UP • SELF-EFFICACY • PROBLEM- SOLVING Program Participants Control Group
- 35. PYD approaches for data analysis Contributing to community Positive self- image Social skills PYD constructs are interconnected but distinct. Sometimes useful to aggregate. Sometimes better not to.
- 36. PYD approaches for data analysis • Disaggregating by groups…
- 37. Using the PYD Framework to Analyze and Disseminate Data Domains • Are there ways to code the data by domain or group measures into domains? • Dissemination: Discuss program working in each PYD domain Features • How can you use data to understand how well PYD features were used and associated with improvements? • Dissemination: Discuss which features were most effective Constructs and Indicators • Ensure data analyses are appropriate to type of measure and data source • Dissemination: Represent indicators accurately
- 38. Case Study: YTP Phase 5 To disseminate the results, YTP organized town hall meetings where youth presented the results, thereby increasing their contribution and ownership of the program and research results. Materials created included write-ups and graphics.
- 39. Group Activity: Complete Phase 5 • What are some of the perceived challenges for analyzing data? How would you work around those and use the PYD framework to analyze data? • Brainstorm 2-3 ways on how you would disseminate the findings from your program, especially in a youth-friendly manner • Consider how you may adapt your program after analysis and dissemination of findings to be more effective in PYD
- 40. Phase 5: Analyze, Disseminate and Learn From the Data • What challenges did you face in this exercise? • What did you learn?
- 41. Questions?
Editor's Notes
- My goal today in this section of the training in not to teach you about M&E, which many of you already have skills in, but instead to give you a better understanding of how to use the PYD Measurement toolkit for your own projects and programs. I’ll provide ways that you can apply the PYD Framework, explain the 5 key phases of PYD program design and evaluation, provide practical examples, and also give you hands-on experience with the new PYD measurement toolkit through several group activities.
- Divide audience into sector-related interests based on their work They will work through a hypothetical USAID sector program which will include/integrate PYD features. They will work together to complete activities after each section for the rest of “Indicator Application” Will confirm sectors after receiving attendee list Democracy & Governance: EJEMPLAR program (Mexico) Health: Stepping Stones Project Education/ Workforce Development: CDA Morocco
- In the toolkit we focus on 5 key phases in which PYD measurement is relevant. These five phases are introduced on page 27 of the toolkit. You will likely recognize these although you probably have a variety of definitions that describe them. These phases are important for both Mission staff and implementers, but some of them may fall more squarely into the Mission’s role. We’re going to focus on all 5 phases and give your some practical hands-on exercises to improve your skills. Phases 2-4 may likely have more practical application for implementers, but all 5 also tie into working together with funders and donors as well. We’re going to walk through each phase together and I’m going to show you examples of how the toolkit is designed to help you. While you may not be in the position to create a program from scratch and thus create its activities and outcomes, it’s important to see how PYD fits in with your program. You can find PYD features within already existing activities, and measure PYD in addition to your objectives.
- Throughout this section, I will be talking about the Youth Theater for Peace (YTP) Program in Kyrgyzstan as an example YTP was a USAID funded project implemented by IREX. Please take a minute to read about the program in the handout. [https://www.usaid.gov/kyrgyz-republic/fact-sheets/youth-theater-peace] DCT: a participatory theater methodology in which youth develop short plays based on conflict issues and present them to their communities in interactive performances We’ve included the five phases on the bottom of this handout as a reminder to keep them in mind while considering the program.
- First key outcomes and research questions have to be refined. So, how can you use the PYD measurement framework to help select the primary outcomes and research questions specific to PYD?
- Most of the primary outcomes for youth programs are sector-specific, e.g., increased access to vocational education, increased participation in the political process, increased use of contraception. However, some programs may have PYD specific primary outcomes, for example, youth participating in civil society, or improved communication skills for youths. The research question is basically “Does the program get us closer to our goal, or more specifically, do the activities in the program have the desired impact on the primary outcomes?” While your programs most likely already have outcomes laid out, you can consider PYD outcomes that may also be related.
- Refresher on PYD Framework… we will be using this in each of these phases
- You can use the PYD Framework to work through Phase 1, reviewing how your outcomes/research questions relate to the four domains, and which features best map to your goal. Additionally, you can take into account relevant constructs when considering your outcomes. The table on page 29 of the toolkit, can help guide you. Let’s think about the Program Goal for YTP. Ask participants to state what they think the goal is… Hopefully come up with something like “sustainable conflict prevention and the community level/ youth empowerment/ etc.” Elements of this program goal clearly map against the four domains: Assets: Program is building youths’ skills in conflict resolution Agency: Youth are empowered to use those skills to engage others Contribution: Youth engage with others in the community Enabling environment: Adult mentors are trained to lead the program with youth. You can also review the seven features when considering the program goal and consider which ones support it. Those that best support the goal of the YTP program would be: Skill building; Youth Engagement/ Contribution; Belonging and Membership. More detail on these features will be discussed in the next Phase. Finally, you can review the constructs and indicators when developing your outcomes and research questions to make sure you are taking into account the most relevant constructs and indicators. We will discuss this in more detail in a later phase.
- As we’ve been discussing, this is the stated goal of the YTP program. The stated goal of the YTP program was “to empower youth and adults in conflict-prone communities to stimulate lasting conflict prevention and promote a culture of peace within their communities and beyond.” This overall goal is linked to two outcomes: conflict prevention and reducing violence. One can easily see elements of the PYD definition reflected in this goal; in fact, the goal itself is empowerment of young people. The program was designed to “facilitate community-based reconciliation and reflection about conflict stories and paths to resolution.” The goal can also be converted into the research question: How can youth and adults in conflict-prone communities be empowered to promote lasting conflict prevention and promote a culture of peace within their communities and beyond? For more specific ways in which the YTP case study could use the PYD Measurement Framework to select primary outcomes and research questions, see Figure 4 on page 29 of the toolkit.
- In phase 2 the key outcomes have been defined – probably by USAID missions, which will be included in the CDCS and the PMPs for each project or program. Now the outcomes and research questions from Phase 1 should drive the design of the program activities. This is when the FEATURES of PYD programs need to be considered, and identifying who the primary beneficiaries will be. Even if your program activities have already been designed, you can consider which PYD features they already include, and how to emphasize them.
- The seven PYD features are based on the most effective elements of a PYD program (skill building; youth engagement and contribution; healthy relationships and bonding; belonging and membership; positive norms, expectations and perceptions; safe space; access to age appropriate and youth friendly services and integration among services). Possible indicators of the proposed outcome should be linked with these features to strengthen the PYD program design. Program activities can be geared toward a variety of beneficiaries. In most cases, youth are the primary beneficiaries of youth programming. However, the PYD Framework will help you identify other beneficiaries who play an important role in supporting youth. This is particularly true for programs targeted at improving the enabling environment and community development. Beneficiaries can include youth, teachers, parents, mentors, support service staff (such as staff from a clinic), policy makers and any other stakeholders who engage with youth. For example, a program could work with parents to improve healthy relationships and bonding, or train staff from a health clinic to improve access to age appropriate and youth friendly services.
- Again, we can use the PYD Framework to complete Phase 2. There is a table on page 31 that may help guide you. For the YTP program, we would first look at the list of seven features, consider the program goal, and think about which features would support this goal. At the same time, look at how those features connect back to the domains. Remind participants the YTP goal [“to empower youth and adults in conflict-prone communities to stimulate lasting conflict prevention and promote a culture of peace within their communities and beyond.”] This goal was connected to many PYD features. For example, it clearly focused on Skill Building (which fits within Assets and Agency domains) as it aimed to give youth tools to express themselves and take action as well as skills to understand conflict cycles and triggers. Within the Contribution Domain, it hit on the Youth Engagement feature as youth helped develop participation roles, traded leadership roles during activities, and then had the opportunity to facilitate community-based reconciliation and reflection. Another feature that YTP addressed was that of a Safe Space (within the Enabling Environment Domain) as the program created a safe space on the stage for trust and dialogue, where facilitated interaction supplanted usual cultural norms. Finally, the feature of Belonging and Membership (which fits in the Enabling Environment domain) also was addressed with YTP, as the program created a sense of community in which participation is valued and engagement across groups is encouraged. Now, we would look back on the features we chose, and consider what beneficiaries they take into account. Obviously with the YTP program, they targeted youth, but there was another beneficiary group. Ask participants for ideas? It was the adult mentors.
- So, we’ve just discussed how the YTP program completes Phase 2. These features listed are just some examples. Now we’ll move on to your own case study and complete Phase 1 and 2.
- Now, in your sectoral groups, you will use the handout of a hypothetical program to complete Phase One and Phase Two. The tables on pages 29 and 31 of the toolkit may help with these processes.
- In phase 3, the primary outcomes and research questions have been selected, as well as program features and beneficiaries. Now the next step is to make them fit together. The assumptions underlying the choice of features and beneficiaries should be realistic and connected, so that the desired outcomes are accomplished and research questions can be answered. While your program may already have an underlying logic model, let’s see how we can fit PYD into the logic model.
- Many program designers and evaluators use a theoretical framework to organize how their program features will link to intermediate and longer-term outcomes. These frameworks are also the basis of M&E plans. These may be referred to as the Logic Model, Theory of Change, or Logframe. Often, these terms convey different things to different people, but the underlying objective is similar: to define results and logical strategy underlying a program or project through “if…then” statements (If I do X, then I expect Y to occur.) A logic model documents and graphically illustrates the program’s components, both those within the control of the project implementers (inputs, activities, and outputs) and those outside of the control of the implementers but still within their influence (outcomes and impact). This discussion is based on a Logic Model, defined above, but these guidelines can refer to all theoretical frameworks. Strong evidence or theory of change is needed to lay out how the program activities are linked to outputs, outcomes, and impact. A good practice for designing program logic models is to collaborate with as many key stakeholders as possible, seeking their input and addressing their concerns. Not only does this ensure their buy-in, but it also creates accountability across stakeholders for the program and its outcomes. Additionally, you can also consult the literature to better understand the effectiveness of the interventions you are interested in using.
- Now, let’s use the PYD framework to consider our logic models. Our PYD domains as well as features (which can serve as inputs or activities) should be connected with the outcomes we are interested in. In the YTP program, let’s think about our features… e.g., skill building, safe spaces, youth engagement, and belonging and membership. Will these features help lead to the outcomes the YTP program is trying to affect? How will we know if they do or don’t? What additional inputs are needed? Perhaps there are some assumptions we need to address? These questions can all be answered through a good logic model as the foundation for your M&E work. More discussions on logic models, especially related to PYD, can be found on page 33 of the toolkit.
- This figure shows an abbreviated logic model for the YTP program in Kyrgyzstan. You can see the PYD domains and features clearly represented throughout the logic model The program also incorporated assumptions necessary for its success. The first was that the intensive camps would bring young people from antagonistic groups to a neutral territory, along with adult mentors, to engage in Drama for Conflict Transformation (DCT) activities. Using the PYD features, one can select key input activities that align with specific features of importance to achieving the project goal. In the Logic Model for the YTP program, this is represented by the inputs (i.e., resources like money and time), and more specific to PYD, the activities that align with PYD features like skills building (train youth in conflict resolution and other interpersonal skills) belonging and membership, and youth engagement and contribution. The second assumption was that facilitated exercises would allow young people to engage with people from other communities and explore non-violent resolutions to conflict. Finally, the third assumption was that interactive theater presented by youth for their own communities would promote community-based reconciliation and create opportunities for community members to examine conflict situations and rehearse peaceful resolution strategies. This is first shown in the Logic Model by the short-term outcomes (youth participants recognize increased positive beliefs about their own future – agency), long-term outcomes (community and youth experience increased prosocial norms – enabling environment), and finally, the overall program goal (sustainable conflict prevention). Using the PYD Framework helps to link the program assumptions to the short and longer-term goals of the program and ensure the program is ready to measure its PYD aspects.
- This figure shows an abbreviated logic model for the YTP program in Kyrgyzstan. You can see the PYD domains and features clearly represented throughout the logic model The program also incorporated assumptions necessary for its success. The first was that the intensive camps would bring young people from antagonistic groups to a neutral territory, along with adult mentors, to engage in Drama for Conflict Transformation (DCT) activities. Using the PYD features, one can select key input activities that align with specific features of importance to achieving the project goal. In the Logic Model for the YTP program, this is represented by the inputs (i.e., resources like money and time), and more specific to PYD, the activities that align with PYD features like skills building (train youth in conflict resolution and other interpersonal skills) belonging and membership, and youth engagement and contribution. The second assumption was that facilitated exercises would allow young people to engage with people from other communities and explore non-violent resolutions to conflict. Finally, the third assumption was that interactive theater presented by youth for their own communities would promote community-based reconciliation and create opportunities for community members to examine conflict situations and rehearse peaceful resolution strategies. This is first shown in the Logic Model by the short-term outcomes (youth participants recognize increased positive beliefs about their own future – agency), long-term outcomes (community and youth experience increased prosocial norms – enabling environment), and finally, the overall program goal (sustainable conflict prevention). Using the PYD Framework helps to link the program assumptions to the short and longer-term goals of the program and ensure the program is ready to measure its PYD aspects.
- This figure shows an abbreviated logic model for the YTP program in Kyrgyzstan. You can see the PYD domains and features clearly represented throughout the logic model The program also incorporated assumptions necessary for its success. The first was that the intensive camps would bring young people from antagonistic groups to a neutral territory, along with adult mentors, to engage in Drama for Conflict Transformation (DCT) activities. Using the PYD features, one can select key input activities that align with specific features of importance to achieving the project goal. In the Logic Model for the YTP program, this is represented by the inputs (i.e., resources like money and time), and more specific to PYD, the activities that align with PYD features like skills building (train youth in conflict resolution and other interpersonal skills) belonging and membership, and youth engagement and contribution. The second assumption was that facilitated exercises would allow young people to engage with people from other communities and explore non-violent resolutions to conflict. Finally, the third assumption was that interactive theater presented by youth for their own communities would promote community-based reconciliation and create opportunities for community members to examine conflict situations and rehearse peaceful resolution strategies. This is first shown in the Logic Model by the short-term outcomes (youth participants recognize increased positive beliefs about their own future – agency), long-term outcomes (community and youth experience increased prosocial norms – enabling environment), and finally, the overall program goal (sustainable conflict prevention). Using the PYD Framework helps to link the program assumptions to the short and longer-term goals of the program and ensure the program is ready to measure its PYD aspects.
- This figure shows an abbreviated logic model for the YTP program in Kyrgyzstan. You can see the PYD domains and features clearly represented throughout the logic model The program also incorporated assumptions necessary for its success. The first was that the intensive camps would bring young people from antagonistic groups to a neutral territory, along with adult mentors, to engage in Drama for Conflict Transformation (DCT) activities. Using the PYD features, one can select key input activities that align with specific features of importance to achieving the project goal. In the Logic Model for the YTP program, this is represented by the inputs (i.e., resources like money and time), and more specific to PYD, the activities that align with PYD features like skills building (train youth in conflict resolution and other interpersonal skills) belonging and membership, and youth engagement and contribution. The second assumption was that facilitated exercises would allow young people to engage with people from other communities and explore non-violent resolutions to conflict. Finally, the third assumption was that interactive theater presented by youth for their own communities would promote community-based reconciliation and create opportunities for community members to examine conflict situations and rehearse peaceful resolution strategies. This is first shown in the Logic Model by the short-term outcomes (youth participants recognize increased positive beliefs about their own future – agency), long-term outcomes (community and youth experience increased prosocial norms – enabling environment), and finally, the overall program goal (sustainable conflict prevention). Using the PYD Framework helps to link the program assumptions to the short and longer-term goals of the program and ensure the program is ready to measure its PYD aspects.
- Using the blank logic model handout, try to create a logic model for your sectoral program. You can use the PYD Framework to create the logic model by connecting the PYD domains and features (inputs and activities) with the PYD outcomes of interest, which are tied to the indicators. This will help to better define outputs, outcomes, and impacts, and ultimately the sector-specific or PYD outcomes of interest (outcomes and impacts). The logic model documents and graphically illustrates the program’s components; both those within the control of the project implementers (inputs, activities and outputs) and those outside the control of the implementers but still within their influence (objectives). It is created by using causal if/then statements (If I do X, then I expect Y to occur). Reading a logic model means following the chain of reasoning or “If...then...” statements which connect the program’s parts.
- Now we’re going to jump to phase 4 of the phases we emphasize in the toolkit. This is where we really get to the nuts and bolts of choosing measures and designing the evaluation. After you complete the logic model and clarify the underlying assumptions of how your PYD program features lead to your outcomes, you need to decide exactly what to measure and how you will measure it. There are two steps in this phase: deciding on indicators that pertain to the constructs of interest, and selecting the right sources of those indicators and data collection strategies.
- Indicators show how and whether a program, project or activity is meeting the stated program objectives. Indicators will inform you if what you are trying to change is improving or not. The indicators you select for your PYD program should follow these guidelines on the slide. The toolkit includes a list of constructs (training, positive identity, youth engagement, etc.) by domain and for each construct a set of recommended or illustrative indicators. This table starts on page 69. As you can see the table is organized by domain. The hope is that programs will include activities to impact all four domains, and relevant indicators will be selected for each of them. But if not, if the key outcomes really hinge on a particular domain, like agency, then you can go to that domain and select the best indicators for the particular project in a particular set of local conditions. Keep the points on this slide in mind when selecting indicators. Each of them are explained in more detail in the toolkit on page 35. Let’s look there and see if you have any questions.
- We can use the PYD framework in Phase 4 to choose indicators. There is a table on page 39 of the toolkit that provides general guidance. The list of the illustrative indicators is on page 69. There are reference sheets in the annex of the toolkit that can also be used. You can use the table on page 115 (Annex G) to see which tools map to specific PYD items. For YTP, we will review the PYD Illustrative Indicators Table on page 69 (Annex E). Since the YTP program addressed all 4 domains, we can look at the entire table and find a relevant construct. I’ll pick an example, say “Interpersonal skills/ social and communication skills” which falls under the Assets domain (and Skill Building feature) and see if there is an indicator suited to the beneficiaries and level of analysis. Ask participants; “Would the indicator ‘increased interpersonal skills at the conclusion of training/ programming” be appropriate for YTP? YES Now you want to ensure that indicators of multiple PYD constructs are included as well as at least one indicator for each relevant domain and feature of the program. Then, you can select different study methodologies, such as quantitative or qualitative methods, or tapping into existing data sources, to triangulate data. The reference sheets will help you to come up with a study design. Discuss with participants a possible study design for the indicator chosen for YTP
- We discussed one sample indicator for YTP already, but here are some more. Note how there are some for each relevant domain. The YTP program identified research questions and determined appropriate indicators and study design to best address them. They focused on measuring constructs that illuminate the PYD program features: skill building, engagement and contribution, safe environments, belonging, and healthy relationships. More information can be seen in Table 8 on page 40 of the Toolkit. For the YTP program, many more indicators could have been selected to measure inputs, outputs, outcomes and impacts. The indicators shown above pertain only to the illustrative indicators presented in this toolkit (which are also available separately on the youth power web site); as previously discussed, they would also use other sectoral-based indicators not referenced in the PYD illustrative indicator table.
- Now, let’s use your group case study to choose at least 3 relevant illustrative indicators. Use the Table in Annex E to choose PYD specific indicators. Using the Illustrative Indicators table in 3 steps. Go to the section of the table for the domains that best represent your project. The assets section is 1st – in fact everything in the toolkit is ordered in this way (assets, agency, contribution, environment) Look at the lines that describe the feature in the 1st column. Consider the indicators (the definitions are provided) and choose the indicators that make the most sense. Then, decide on a study design for these indicators. Consider where you are in the project (design, implementation, close-out) and your study design (baseline, mid-term, endline). Determine what types of data you will need to collect and how you will collect them. The reference sheets in Annex F can help you with specific indicators.
- Now we’re going to talk about phase 5 Analysis and dissemination from PYD programs, in many ways, is similar to any other program. If the program is continuing, or being implemented in other countries, then the evaluation can be used to modify and improve the project design. The evaluation should include explanations that help interpret the results and recommendations for making targeted changes to the program or future program designs. However, there are a few key differences when considering analysis and dissemination of information from PYD programs…
- Data analysis and collection can be both qualitative and quantitative When considering youth and PYD, qualitative can help you to measure behaviors, attitudes, perceptions, and awareness, which are particularly relevant to youths.
- Some of you have already designed or conducted evaluation research. But I’m going to emphasize here that PYD has the word DEVELOPMENT right in it. In order to measure the impact of a program on individual development you have to take measurements at least twice, preferably more times. So at least baseline and endline. But midline assessments are very useful to catching unintended consequences early, and for understanding the course of development in relationship to the program. If all the improvement happens by the halfway point, then maybe the program can be shortened…
- I’m sure you know this. So I’m just going to emphasis that having an appropriate comparison group is especially important when you are assessing the impact of a program on individual developmental outcomes. Kids change over time no matter what! So you really have to compare the program participants to other kids who are also changing …
- Another thing I want to point out is the many of the PYD indicators are related to each other – they are interconnected. So when the program has an impact on social skills for instance, it will also likely improve self-image and contributing to their communities in some way. Implementers are going to select indicators and report the amount of change in each indicator – and that is very important information. But you should keep in mind, that the changes they report can be thought of in aggregate – as a total picture of the program impact. Understanding EXACTLY how each PYD construct influences the others over time requires frequent assessments and some pretty complicated statistical modeling. However, qualitative data might be very helpful in understanding in more detail HOW the program works.
- PYD is a very broad idea – it is for EVERYONE. Who wouldn’t benefit from more assets, stronger agency, more contributing, and a more supportive environment? PYD programs may target particular groups who are vulnerable Or They might explicitly work to be inclusive of all youth so that historically excluded youth can participate. Remember that bonding and having a sense of belonging and membership are part of the PYD strategy. But in order to understand how programs may have different effects on different groups of people, data need to be disaggregated. They need to be reported separately for relevant groups – like for males and females separately. By age group – separating the 10 to 14 year olds from the 20 to 29 year olds! Or by something of local importance like whether the youth is attending a school or not…
- When analyzing and disseminating data, you can also use the PYD framework. There is a table on page 44 to help you do so. Let’s think about the YTP program. Are there ways to code the data by domain or group measures into domain? The YTP program could have specifically looked at how its program impacted youth in terms of improving assets in comparison to agency or enabling environment or contribution. How can you use data to understand how well the PYD features were implemented and how each feature was associated with improvements? YTP specifically can look at how different age groups responded to each indicator in order to disaggregate results. When disseminating results, you can also keep the PYD framework in mind. Begin the narrative about PYD, using information on how your program worked in each of the relevant domains. For example, the YTP program’s report began: “Through these skill building activities, these participants acquired Assets and built their own Agency. Surrounded by an Enabling Environment of supportive adults who supported healthy relationships and bonding with youth, program participants were able to Contribute to their community by engaging with others to teach about peaceful conflict resolution.” This enabled them to emphasize domains and features in their reports. Additionally, it’s important to consider how to convey data and results to youth in particular. The YTP program specifically used a town hall process to do so, which also included the community.
- As we discussed, these are some of the ways YTP disseminated results.
- Now, we’ll complete Phase 5 for our group case studies. Table 9 on page 44 may be helpful to keep in mind. In your groups, think through the following: What are some of the perceived challenges for analyzing data? How would you work around those and use the PYD framework to analyze data? Brainstorm 2-3 ways on how you would disseminate the findings from your program, especially in a youth-friendly manner Consider how you may adapt your program after analysis and dissemination of findings to be more effective in PYD