Evaluating Ontario Crown Ward Education Championship Teams
•Download as PPTX, PDF•
1 like•341 views
Formative evaluation of the Crown Ward Education Championship Team (CWECT) program in Ontario (Kelly Weegar)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (11)
Similar to Evaluating Ontario Crown Ward Education Championship Teams
Similar to Evaluating Ontario Crown Ward Education Championship Teams (20)
More from Linkedin Education of Children in Care Network
More from Linkedin Education of Children in Care Network (20)
Recently uploaded
Recently uploaded (20)
Evaluating Ontario Crown Ward Education Championship Teams
- 1. uOttawa.cauOttawa.ca A Formative Evaluation of Two Crown Ward Education Championship Teams in Ontario Kelly Weegar, A. J. Hickey, Samantha Shewchuk, Mariama Mary Fall, & Robert J. Flynn crecs.uOttawa.ca Centre for Research on Educational and Community Services (CRECS)
- 2. crecs.uOttawa.ca Presentation Outline 1. Overview of the CWECT program in Ontario 2. Evaluation purpose, scope, and questions 3. Relevant literature 4. Evaluation participants and methods 5. Evaluation findings and recommendations 6. Conclusions and next steps
- 3. crecs.uOttawa.ca What is a Crown Ward Education Championship Team (CWECT)? • Supported by a tri-ministry working group • “The goal is to enable better education, training and employment opportunities for all youth with Crown wardship status by: – Developing a local community-based partnership model – Identifying youth with Crown wardship status within the community – Creating a sustainable and responsive support system for these youth to help them make the transitions from one level of education to the next”
- 4. crecs.uOttawa.ca Other (e.g., other youth organizations) Regional Ministry representation (MCYS/MED/ MAESD) Youth Employment Ontario service providers Local post- secondary institutions (colleges and universities) Local child welfare organizations Local school boards (English & French)
- 5. crecs.uOttawa.ca History of the CWECT Program Figure 1. Spread of CWECTs in Ontario, 2008-2012.
- 6. crecs.uOttawa.ca Purpose & Scope of the Evaluation • Given the relative newness of the CWECT program in Ontario, together with the lack of previous evaluations, we chose to conduct a formative evaluation a. To find out how the CWECT program was working “on the ground”, in the opinions of its stakeholders • To discover how clearly and with how much consensus the stakeholders perceived the intended purpose of the two programs, how useful they saw program activities, and how helpful they felt the programs had been b. To ultimately prepare for an outcome evaluation
- 7. crecs.uOttawa.ca Purpose & Scope of the Evaluation • Local level (two individual CWECTs) – Purpose: to assess how the program appears to be functioning and to suggest how it might be improved – Scope: Young people in care (more than current Crown Wards) • Provincial level – Purpose: to make a contribution to the functioning of the CWECT initiative province wide (i.e., to enhance the relevance and quality of the program ) 7
- 8. crecs.uOttawa.ca Evaluation Questions Addressed 1. What are the needs of the stakeholders (i.e., the local Crown Wards/youth in care, caregivers, and community organizational partners)? 2. Does the CWECT program meet the needs of its stakeholders? 3. How was the CWECT program being implemented? 4. Does the CWECT program appear to be working, in the opinion of the stakeholders?
- 9. crecs.uOttawa.ca Relevant Literature • At present, there are no evidence-based interventions for increasing post-secondary access and retention for youth in foster care. • However, a number of educational advocacy programs, like CWECTs, do aim to support the educational goals of at-risk youth. These programs have produced preliminary findings that suggest positive impacts, including: – Fewer school absences or changes in schools – Improved high school graduation rates – Successful resolution of education-related challenges
- 10. crecs.uOttawa.ca Relevant Literature • Post-secondary education graduates with bachelor’s degrees or college diplomas have higher incomes over the short-term and long-term than non-graduates. • Many young people in care experience lower levels of academic achievement than youths in the general population, including lower rates of graduation or entry to and completion of post-secondary education. • According to estimates from the OACAS Gateway to Success surveys of educational attainment, only about 46% of young people in care in Ontario currently graduate from high school within the usual four or five-year period.
- 11. crecs.uOttawa.ca Relevant Literature • Without early mastery of basic academic skills in reading and math, young people in care are at increased risk in the shorter and longer terms of: – Lower educational success than is warranted by their academic potential. – Higher levels of school dropout, involvement in criminal activity, homelessness, and mental health difficulties. • The need for effective help is urgent, in the preschool, primary school, and secondary school years, for successful transitions to post-secondary opportunities for children and youth in out-of-home care.
- 12. crecs.uOttawa.ca Evaluation Participants • Two CWECTs, representing four child welfare agencies: 1. CWECT-HS-FLA • Highland Shores Children’s Aid Society • Family and Children’s Services of Frontenac, Lennox & Addington 2. CWECT-LLG-R • Family and Children’s Services of Lanark, Leeds, & Grenville • Family and Children’s Services of Renfrew County
- 13. crecs.uOttawa.ca Evaluation Participants Participants Collection Method(s) CWECT team members (N = 23) * • Phone interviews • Observations Caregivers (N = 103) • Questionnaires Youth in care (N = 65) • Questionnaires • Phone interviews • Focus group Community partners (N = 12) • Questionnaires • Four participant groups, 203 participants total * Organizations represented: children’s aid societies, employment services, school boards, colleges, universities, counselling services, Ministry of Education.
- 14. crecs.uOttawa.ca Evaluation Methods • Mixed-methods approach – Document analysis (e.g., meeting minutes, business plans) – Literature reviews – Interviews (phone and in-person) – Questionnaires (online and paper) – Focus groups – Observation • Evaluation team met monthly to discuss progress • Evaluation team consulted with other CWECT teams (those not part of the current evaluation) to better understand the CWECT initiative
- 15. crecs.uOttawa.ca Results & Recommendations • Logic model • Findings and recommendations for each individual CWECT and overall, by each evaluation question: 1. What are the needs of the stakeholders? 2. Does the CWECT program meet the needs of its stakeholders? 3. How was the CWECT program being implemented? 4. Does the CWECT program appear to be working, in the opinion of the stakeholders?
- 16. crecs.uOttawa.ca Logic Model • One of the main objectives of formative evaluation is often to establish the program theory or framework – Connects programs activities to the intended outcomes for all stakeholders served by the program
- 17. crecs.uOttawa.ca Logic Model • During the development of logic models, a critical insight was the expansion of understanding about the clients served by the CWECT initiative – Although the program documentation for the CWECT initiative typically identifies the program clients as Crown Wards, this evaluation and the logic models have made it explicit that the CWECT initiative also intends to serve important needs youth in care more broadly, as well as caregivers and diverse community partners.
- 18. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Youth in Care • According to the online questionnaires completed by the youth for one of the CWECTs, all planned to pursue some kind of post-secondary education or training: – 17 or 70.8% aimed to go to college – 5 or 20.8% indicated wanting to pursue university – 5 or 20.8% planned to pursue an apprenticeship – 1 youth (4.2%) planned to pursue military training • For this same sample, the majority of youth also rated the importance of post-secondary education highly – On a scale from 1 (not important) to 10 (very important), the average rating was 7.83 out of 10
- 19. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Youth in Care • Nevertheless, it was consistently reported that the primary need for young people in care is increased awareness and access to information and resources. – “…help researching different colleges, help with budgeting… help me develop more skills that are related to what program I want to go into.” – “Everything is in place, but youth don’t always know about them. More direction is needed to help navigate the information online.” • Many CWECT team members also commented that positive, supportive, and consistent relationships are vital to the the educational success of youth in care
- 20. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? • Related to this, both CWECTs have consistently devoted time and resources to allow their youth in care to explore possibilities and to hear about others’ experience with post-secondary planning and transitioning to independent living, particularly in the face of adversity. For example: – College tours – University tours – Experiential learning events – Motivational speakers – Career information sessions
- 21. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Caregivers • Similarly, caregivers from both teams stated that they needed more information to be able to guide their youth in care effectively. – “I need to be educated on what the necessary steps are and how to go about them (as things have changed since I went). For example, how to apply for college, residence, OSAP, etc. Also, there are so many grants available for our youths that learning about them and how to apply would be helpful.” • CWECT members also stated that caregivers need more information about their important role in and influence on the educational success of youth in care.
- 22. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? • Information from interviews with CWECT members and program documentation confirmed that attempts have been made by both teams to share this kind of information with caregivers (e.g., evening information sessions, including guest speakers). • However, both teams commented that it is a challenge to get caregivers to attend knowledge-sharing events, and many were not sure why. – Some team members speculated that geographical factors (e.g., large, rural areas mean more time is needed for travel) remain one of the biggest challenges
- 23. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Community partners • Largely based on results from the CWECT team member interviews, the biggest need for community partners appears to be related to awareness and lacking information. – Several CWECT members noted that community partners tend to lack awareness about the needs of youth in care beyond academics (e.g., mental health needs), and could benefit from learning more about how positive school experiences and supportive relationships can significantly influence to their success.
- 24. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Recommendation: We recommend that young people in care receive more one-to-one support to help them navigate the educational and employment process. Recommendation: CWECTs should continue to expand their services of information and support to all three stakeholder groups—young people in care, and the caregivers and community partner organizations who are essential collaborators in helping the young people reach their educational and career goals. – E.g., information-sharing among caregivers and community partners through a variety of methods (e.g., newsletters, information sessions, or webinars).
- 25. crecs.uOttawa.ca Question 1. What are the needs of the stakeholders? Recommendation: To help target the needs of all clients served by the CWECT, a central website is needed to serve the informational needs of all local CWECTs and their various stakeholders. This web site would alleviate the need for local CWECTs to try to “reinvent the wheel” (which, in any event, exceeds their financial and operational capacities). The central website would be updated regularly with items about new opportunities related to the education and living situation of young people in care in Ontario. (For an example of what such a website might look like, see http://www.equalfutures.org/about/ ).
- 26. crecs.uOttawa.ca Question 2. Does the CWECT program meet the needs of its stakeholders? • All members agreed that the purpose is about helping youth in care improve educational and employment outcomes, and the belief that they can be successful – “I think we fundamentally do share all the same beliefs, I think we do believe in the importance of keeping the needs of youth in care at the forefront, and I think that we're all committed to doing the best we can and continuing to get better in serving those kids within the structures that we have—within whatever system we come from. That’s fundamentally why I believe that we do what we do.” • At this time, it appears that some needs of some youth are being met (e.g., information sharing and experiential learning events are taking place)
- 27. crecs.uOttawa.ca Question 2. Does the CWECT program meet the needs of its stakeholders? • Many members from both teams noted that there is a need to boost involvement of caregivers. Thus, the needs of caregivers are likely not being met. • Due to difficulties reaching community partners, we are unable to assess whether their needs are being met.
- 28. crecs.uOttawa.ca Question 2. Does the CWECT program meet the needs of its stakeholders? • But, are there adequate resources for CWECTs to be able to carry out program activities? – Almost all CWECT team members noted lacking guidance from the three Ministries in terms of planning effective ways to engage youth, caregivers and community partners. – Furthermore, several team members added that having a full-time child welfare representative chair the team as a side job was a limitation of the current CWECT model (this was not a reflection of the current chairs, but rather a structural issue – i.e., each CWECT needs a full-time paid coordinator).
- 29. crecs.uOttawa.ca Question 2. Does the CWECT program meet the needs of its stakeholders? • Recommendation: Work with local CWECTs to make it possible to deploy a full-time CWECT coordinator in each program, as the present model of part-time coordinators simply does not provide enough time for program planning and development and taking care of the many needs of all the stakeholders (the young people in care, caregivers, and staff of community-partner organizations). – Ideally, this individual would be familiar with all involved systems, including child welfare and the various employment and school systems (primary, secondary, college/university/apprenticeship programs) in the region.
- 30. crecs.uOttawa.ca Question 2. Does the CWECT program meet the needs of its stakeholders? • Recommendation: The three sponsoring Ontario Ministries need to provide increased and consistent support to the individual CWECTs. For instance, the three Ministries could: – Sponsor provincial or regional workshops to help local CWECTs improve their capacity to plan, implement, evaluate, and improve their services. – Compile a database of CWECT success stories, for local programs to use, and place this database on the central website for easy access.
- 31. crecs.uOttawa.ca Question 3. How is the CWECT program being implemented? • Both CWECTs have devoted many resources to activities for the youth, and attempts have been made to increase knowledge and nurture relationships with caregivers and community partners. • However, implementation of program activities has been largely affected by when they are receiving their funds; i.e., annual funds have typically been received halfway through the fiscal year, which has led to a sense of urgency to plan and implement activities.
- 32. crecs.uOttawa.ca • Recommendation: At minimum, funds for the individual CWECTs should be delivered promptly at the beginning of each fiscal year. Improvements in program implementation are also likely if the teams are able to receive funds for multiple years at once. For instance, if the CWECTs were allocated funds on a 3-5 year basis, this would allow the teams to set and plan for longer- term goals. Question 3. How is the CWECT program being implemented?
- 33. crecs.uOttawa.ca Question 4. Does the CWECT program appear to be working? • CWECT members all said they were hoping they were making a difference for youth in care and felt as though they were, but had no real data/evidence to confirm this. • However, the biggest impact seems to be in the area of fostering partnerships between child welfare and education (e.g., supporting targeted conversations, increasing the accountability for both to adhere to a plan to support Crown Wards).
- 34. crecs.uOttawa.ca Question 4. Does the CWECT program appear to be working? – “We are getting much closer to talking the same language now in terms of the educational needs of these kids.” – “I think we’re having an impact because we continue to work together, and we continue to gain more community partners. I think we gain a better understanding the longer we work together. The more we understand the perspective of others will help us in supporting kids.”
- 35. crecs.uOttawa.ca Conclusions & Next Steps • Consider funding a pilot outcome evaluation of one or more local CWECT programs as a needed follow-up to the present formative evaluation. • CWECT committee members (and the three sponsoring Ontario Ministries) should acquaint themselves, caregivers, and community partners (including schools) with the wealth of accessible online information about effective, evidence-based methods of intervening to help young people in care to improve their educational outcomes (e.g., one-to-one tutoring). Such information is provided in chapter 4 of our evaluation report.
- 36. crecs.uOttawa.ca Acknowledgements • Thank you especially to: – Highland Shores-Frontenac, Lennox & Addington CWECT – Lanark, Leeds, & Grenville-Renfrew CWECT
- 37. crecs.uOttawa.ca Contact information and to receive an e-copy copy of our final evaluation report Kelly Weegar • Email: kweeg065@uottawa.ca Robert Flynn • Email: rflynn@uottawa.ca Centre for Research on Educational and Community Services (CRECS) University of Ottawa 136 Jean Jacques Lussier Vanier Hall, Room 5002 Ottawa, Ontario, Canada K1N 6N5
Editor's Notes
- Herein we report on a formative evaluation that we conducted in 2015-2016 of two local Crown Ward Education Championship Teams (CWECTs) in Ontario: the Highland Shores-Frontenac, Lennox & Addington CWECT, and the Lanark, Leeds, & Grenville-Renfrew CWECT. We wish to thank the stakeholders of both programs and their sponsoring Children’s Aid Societies (CASs) for their excellent collaboration. We thank them also for their openness in agreeing to being publicly identified in this report. We hope our report is useful to both CWECTS, to other local CWECTS in Ontario, and to the Ministries of Children and Youth Services (MCYS), Education (MED), and Training, Colleges, and Universities (MTCU), which are jointly responsible for the provincial CWECT initiative.
- Community groups should include: All local English and French language school boards Local child welfare organization(s) Local post-secondary institutions (colleges and universities) Youth oriented Employment Ontario service provider(s) Regional Ministry (MCYS/EDU/MTCU) representation Other community representation as deemed appropriate (e.g. Aboriginal organizations, other youth organizations)
- I will add brief points here to summarize the evolution of the initiative since 2008
- **Between 2008 and 2012, the provincial program had grown from a pilot initiative involving only a few CWECTs and CASs into an Ontario-wide program of 21 CWECTs and some 40 CASs. Despite this rapid growth, no previous evaluation--whether formative or summative, process or outcome-oriented--had ever been carried out, to the best of our knowledge.
- Put differently… Scope: While the name may suggest that these the Crown Ward Education Championship Teams (CWECTs) serve only Crown Wards, we and our local CWECT and CAS partners agreed that the scope of our evaluation would be focus on young people in care, not just current Crown Wards (e.g., also former CWs over the age of 18 - Continued Care and Support for Youth, or CCYS). Namely, both teams we worked with for the current evaluation indicated serving more than just current Crown Wards with their respective CWECT resources and activities. Extending the evaluation to children living with their families in the community was seen as unfeasible for several reasons: the six-month timeline and modest resources of the evaluation, the absence of previous evaluations, and the fact that the CWECT program would no doubt struggle for some time with its new and greatly expanded mandate to serve all children and youth in child protection. Provincial Level: Besides our focus on the two local CWECT programs, we also wanted to contribute to the functioning and improvement of the CWECT program on an Ontario-wide basis. We thought that much was likely to be learned from our local findings by CWECT programs elsewhere in Ontario and by the Ministry of Children and Youth Services (MCYS) that oversees and funds them.
- Answers to questions such as these are vital to practitioners, managers, government and voluntary-sector policy-makers, and researchers who wish to improve the educational outcomes of youth in care. Clearly, interventions in Ontario such as the CWECT program that aim to improve educational outcomes, and specifically PSE access and retention, need to be accompanied by a vigorous, ongoing cycle of research, demonstration, evaluation, and program improvement.
- The programs mentioned show that jurisdictions in several countries, like Ontario, are now trying to coordinate services that already exist in communities, rather than duplicating services or providing services in silos. These jurisdictions believe that communication networks linking youth-serving agencies should be sustained to increase youth involvement and support shared efforts. Despite these promising developments, strong evidence regarding their effectiveness is still lacking. Much more attention to evaluation needs to be paid by governments and educational advocacy groups themselves to discover truly ‘best’ practices.
- This formative evaluation was undertaken at the request of two senior managers who had key roles in operating the two Crown Ward Education Championship Teams. **We commend the two local CWECTS that asked us to conduct the present evaluation for the impressive work they have accomplished to date, their sincere interest in improving their services, and their openness in agreeing to be identified publicly. **Both CWECTs consent to identifying themselves.
- Data summary considering both CWECTs combined It should be noted that the number of completed responses for the community partner and youth was less than originally hoped. Caution is thus needed in interpreting our data for these samples, as they are based on small samples that are not necessarily representative of the views of these groups.
- **Focus only on general findings We commend the Ministries of Children and Youth Services (MCYS), Education (MED), and Training, Colleges and Universities (MTCU) for launching the CWECT initiative and providing the resources that have allowed it to grow from a small pilot into a province-wide program. In formulating the following recommendations, we hope to contribute to enhancing the relevance and quality of the program…. Note that recommendations are presented as “in an ideal world…” and may seem a bit unfeasible, and certainly need to be interpreted in context and individually by the teams (e.g., considering their available resources), but we tried to present what we believe to be the best case scenario without being completely unrealistic.
- So, in sum, the sampled youth in care have aspirations. However, we need to keep in mind that aspirations do not equal success in PSE or employment – i.e. attending PSE does not mean that they stay. Furthermore, only those attending events participated in our evaluation.
- Unfortunately, it is difficult to determine whether community partners agree with these suggested needs, as only 4 individuals from 1 community organization responded to the online questionnaire that was developed to explore their needs related to the education of Crown Wards. Considering the responses of the 4 individuals who were surveyed, it appears that community partners want more information about available supports and how to assist.
- For example, this website could include resources and information (e.g., financial opportunities, application tips, etc.), as well as a space for members to send questions to the CWECT or chat with other members. This website could also be updated regularly with pertinent information.
- Following the identification of the needs for each client group as described in the results for the first evaluation question (i.e., for youth, caregivers, and community partners), program documentation and information from the interviews with the CWECT members were used to determine whether the current program activities seemed to be addressing the needs of each client group. It appears the education and employment needs of youth in care are being partially met, as the majority of CWECT activities in 2014-2015 appeared to focus on information sharing and experiential opportunities, as well as some relationship building opportunities for the youth. For example, planned colleges/university tours likely increase a youth’s awareness of what a college/university can offer, whereas motivational speaker events allow information sharing to take place, particularly about what the transition to independent living and post-secondary education or employment may look like. Note again that caregivers are typically invited to attend the planned youth events as well.
- Needs of youth are being partially met as the majority of activities have been focused on the youth including relationship building and information sharing .
- Again, it will remain important for the CWECTs to continue to expand their resources and activities to all stakeholders. However…
- We wish to thank the stakeholders of both programs and their sponsoring Children’s Aid Societies (CASs) for their excellent collaboration. We thank them also for their openness in agreeing to being publicly identified in this report. We hope our report is useful to both CWECTS, to other local CWECTS in Ontario, and to the Ministries of Children and Youth Services (MCYS), Education (MED), and Training, Colleges, and Universities (MTCU), which are jointly responsible for the provincial CWECT initiative.