The document summarizes a presentation about planning multi-site evaluations of model and promising crime prevention programs. It outlines the National Crime Prevention Centre's (NCPC) mandate to fund effective interventions and build knowledge. NCPC conducts multi-site evaluations of 2 model and 6 promising programs across provinces to increase understanding of what works. Key challenges include varying program timing, fidelity, and benchmarks across sites. Mitigation strategies involve coordination, fidelity checks, and quantifying benchmarks to strengthen multi-site analysis.
Curriculum is a concept that is understood differently by different people in different context, threfore, here I provide an overview of what curriculum is according to Stenhouse.
PAG-USBONG NG NASYONALISMO SA EUROPA AT IBA’T IBANG.pptxJemjem47
Ito'y patungkol sa pag-usbong ng nasyonalismo sa Europa at iba't ibang bansa. Isang proseso ang paglinang at pag-unlad ng nasyonalismo, hindi maaaring biglaan. Kailangan itong madama, paghirapan ng mga tao upang mahalin nila ang kanilang bansa. Sa iba, ang kahulugan nito ay damdamin ng pagiging matapat at mapagmahal sa bansa. Sa iba, kailangang isakripisyo pati ang buhay.
Ika-13 siglo, dumating ang mga Tartar o Mongol mula sa Asia at sinakop ang mga mamamayan ng Russia nang mahigit sa 200 taon. Nag-iwan ng mga bakas sa pananalita, pananamit at kaugalian ng Ruso ang nasabing panahon ng pananakop. Naging tagapagligtas ng Russia si Ivan the Great.
Curriculum is a concept that is understood differently by different people in different context, threfore, here I provide an overview of what curriculum is according to Stenhouse.
PAG-USBONG NG NASYONALISMO SA EUROPA AT IBA’T IBANG.pptxJemjem47
Ito'y patungkol sa pag-usbong ng nasyonalismo sa Europa at iba't ibang bansa. Isang proseso ang paglinang at pag-unlad ng nasyonalismo, hindi maaaring biglaan. Kailangan itong madama, paghirapan ng mga tao upang mahalin nila ang kanilang bansa. Sa iba, ang kahulugan nito ay damdamin ng pagiging matapat at mapagmahal sa bansa. Sa iba, kailangang isakripisyo pati ang buhay.
Ika-13 siglo, dumating ang mga Tartar o Mongol mula sa Asia at sinakop ang mga mamamayan ng Russia nang mahigit sa 200 taon. Nag-iwan ng mga bakas sa pananalita, pananamit at kaugalian ng Ruso ang nasabing panahon ng pananakop. Naging tagapagligtas ng Russia si Ivan the Great.
Using case-based methods to assess scalability and sustainability: Lessons fr...JSI
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
Using case-based methods to assess scalability and sustainability: Lessons fr...Barb Knittel
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
How can companies select the right software development methodology for their...Ezinnovation
Ultimately, the selection of the MVP software development methodology should be a well-informed decision based on project-specific considerations. It is also worth noting that hybrid methodologies or customized approaches can be adopted by combining the best elements from different methodologies to suit unique project requirements.
The presentation was delivered at the Canadian Evaluation Society Conference in Ottawa Canada. Using rigorous evaluations in a multi-site framework, Pawson & Tilley's realist approach is tested using Canadian evalutions of funded crime prevention projects.
This university lecture at Carleton University shares various evaluation research designs that can be used with community based organizations, especially when a comparison group cannot be identified (i.e. implicit designs and regression discontinuity)
Using case-based methods to assess scalability and sustainability: Lessons fr...JSI
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
Using case-based methods to assess scalability and sustainability: Lessons fr...Barb Knittel
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
How can companies select the right software development methodology for their...Ezinnovation
Ultimately, the selection of the MVP software development methodology should be a well-informed decision based on project-specific considerations. It is also worth noting that hybrid methodologies or customized approaches can be adopted by combining the best elements from different methodologies to suit unique project requirements.
The presentation was delivered at the Canadian Evaluation Society Conference in Ottawa Canada. Using rigorous evaluations in a multi-site framework, Pawson & Tilley's realist approach is tested using Canadian evalutions of funded crime prevention projects.
This university lecture at Carleton University shares various evaluation research designs that can be used with community based organizations, especially when a comparison group cannot be identified (i.e. implicit designs and regression discontinuity)
This is a presentation summarizing 150 years of afterlife experience presented at the Science of Mediumship Event at the Novotel Hotel in Toronto, Canada. This presentation includes survival of consciousness evidence pertaining to mental and physical mediumship, near death experiences, out of body experiences, electric voice phenomenon and past life regression therapy
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
Embracing GenAI - A Strategic ImperativePeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Biological screening of herbal drugs: Introduction and Need for
Phyto-Pharmacological Screening, New Strategies for evaluating
Natural Products, In vitro evaluation techniques for Antioxidants, Antimicrobial and Anticancer drugs. In vivo evaluation techniques
for Anti-inflammatory, Antiulcer, Anticancer, Wound healing, Antidiabetic, Hepatoprotective, Cardio protective, Diuretics and
Antifertility, Toxicity studies as per OECD guidelines
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
1. Planning Multi-site
Evaluations of
Model and Promising Programs
Presented by Donna Smith-Moncrieffe
Canadian Evaluation Society Conference (Ontario Chapter)
October 1, 2009, Ramada Hotel, Toronto
2. Presentation Outline
National Crime Prevention Centre (NCPC) mandate
Multi-site evaluations of model & promising Programs
Definitions
Why conduct multi-site evaluations of model and promising
programs?
Using Multi-site evaluations to contribute to potential program
sustainability
2
3. Presentation Outline cont’d
Planning challenges of multi-site evaluations
Challenges in the planning stages (8 multi-site
evaluations)
Key mitigation strategies for implementing multi-site
evaluations
Sample planning charts/tools:
Stop Now and Plan (SNAP®) program: Quantifying
fidelity checklists
Multisystemic Therapy (MST) program: Proposed
correlation of results with fidelity levels
Stop Now and Plan (SNAP®): Identifying
quantifiable benchmarks for between site analysis
3
4. NCPC Mandate/Core Activities
Mission statement:
To provide national leadership on effective and cost-efficient ways to
both prevent and reduce crime by addressing known risk factors in
high risk populations and places
Core activities:
Supporting targeted interventions in local communities
Building and sharing practical knowledge with policy makers and
practitioners
4
5. NCPC priorities:
Provide funding for the following target groups/crime issues:
Children and youth at risk
Crime prevention in Aboriginal communities
Prevent recidivism among high-risk groups
Priority crime issues (youth gang, drug-related crimes)
5
6. Model and Promising Programs:
NCPC Definitions
• Prevention program that meets the highest
standards (i.e. peer-reviewed evaluations with
Model Programs rigorous designs )
• Demonstrates significant sustained reduction in
problem behaviour or risk factors for criminal
offending
•Effectiveness is maintained over time
•Demonstrates some reductions in problem
behaviour or risk factors for criminal offending
Promising Programs
•Program components have been evaluated with
less rigorous designs (i.e. pre and post tests)
6
7. Definitions: Multi-site evaluations
NCPC criteria used to be considered multi-site:
√ One central evaluation for all sites (multi-center)
√ Intent to deliver the same program in at least 2 or more different
regional settings
√ Within and between site analysis is conducted and reported in one
report
√ Intent to implement sites concurrently
√ Differences in setting, target group or additional non-core
components are apparent
7
8. NCPC Planned Multi-Site Evaluations
•2 model programs and 6 promising programs (up to
Type of Multi-site Evaluation
4 sites in different Provinces)
• Randomized control trial (SNAP® program only)
Evaluation Designs
• Matched comparison group & Cost benefit analysis
(where causal links can be established and where
variables can be monetized)
Duration of Evaluations • 4 to 5 year duration with 1 year follow up
• External contracted evaluations
Contract Type
•Competitive process (posting on MERX)
8
9. Why Conduct Multi-site evaluations
with Model Programs?
Building the knowledge base of “what works” for wider
dissemination across Canada
Increases confidence that observed impacts
(effectiveness) can be attributed to the intervention
Increases knowledge about the program elements that
contribute most to its effectiveness
Increases knowledge about the contextual factors that
contribute to effectiveness
9
10. Multi-Site Evaluations: Contributing to
Potential Program Sustainability
Disseminate Results
and encourage
NCPC time-
the provinces
limited funding
to implement
effective programs
Contribute Identify and
to Treasury Board encourage the development
reporting of Model and Promising Programs
Use
multi-Site Collaborate with
evaluations the provinces and
to increase lead FPT meetings
knowledge
10
11. NCPC Challenges:
Planning Multi-Site Evaluations
Co-ordinating the timing for all sites
Ensuring program fidelity within each site is high (80% or
higher)
Identify quantifiable benchmarks for key outcomes (increases
comparability for a between site analysis)
Variation in program developer involvement
Delays in contract approvals (at least one project within the
multi-site needs to be approved before the evaluation contract
can be processed)
11
12. Mitigation Strategy:
Addressing Variation in Timing for all Sites
Call for proposals in different provinces
occurred at different intervals throughout the
Issue/Challenge year. A number of factors contribute to
variation in program development and approval
time periods.
•Within site analysis is feasible but between site
Implications analysis and pooling is not feasible until all of the
sites are at similar stages
•NCPC encouraged organizations to develop key model
and promising programs (prioritizing exercise)
Mitigation Strategy
• NCPC funded “knowledge to practice” seminars to
train organizations in the development of model and
promising programs (helps prioritize key programs)
12
13. Mitigation Strategy:
Ensuring Fidelity is High in each Site
Sponsoring organizations potentially deviate
from the intended program. Fidelity problems
Issue/Challenge result from varying interpretations of the
program, inability to attract the proper target
group or intentions to test a new component
that is not part of the model.
Implications •Between site analysis and interpretation is unreliable
when each site implements the program differently.
• Develop quantifiable fidelity check lists
• Ensure funds for frequent fidelity checks are available
Mitigation Strategy
•Ensure research and evaluation staff are available to
support program officers developing model and promising
programs
13
14. Mitigation Strategy:
Ensuring Fidelity is High in each Site (cont’d)
Sample : Quantifiable fidelity checklist (note: this is not the full checklist)
√ Target Group Population Met: 8/10
√ 12 Cognitive Behavioural group Sessions 10/12
Sessions delivered weekly within 3 months
Delivered by a trained facilitator
Approved manuals used (weekly documentation)
√ 12 Cognitive Behavioural concurrent parenting sessions 10/12
Fidelity Level: High Total Score: 28/34 or 82.4%
____________________________________________________________
Fidelity Legend
No Fidelity: 0-29%
Low Fidelity: 30-69%
Moderate Fidelity: 70-79%
High Fidelity: 80-100%
14
15. Mitigation Strategy:
Sample MST: Proposed Correlation of
Results with Fidelity Levels
Site 1 Site 1 Site 2 Site 2
(MST) (Control) (MST) (Control)
MST re-conviction 59.3% 69.2% 59.3% 58.3%,
Rates (%)
(known data)
MST 75% or Usual 55% or Usual
Levels of Fidelity Mod Level Services Low Services
(%) Level
(proposed)
Recommendation Further Further explore
exploration weak areas of
not fidelity
required
15
16. Mitigation Strategy: Identifying
Quantifiable
Benchmarks for Between Site Analysis
Data are not present for all outcomes of
interest or they vary significantly between
studies. Significance testing types also vary
Issue/Challenge
(Statistical significance, Clinical Significance
and Effect Sizes).
•The benchmark set for sites may be atypical
Implications leading to an inaccurate interpretation of
program efficacy
• Use meta-analysis results if available
(standardizes results)
Mitigation Strategy
• Set benchmarks using results that have been
produced only from programs implemented with
high levels of fidelity
16
17. Sample Planning Chart:
Identifying Benchmarks for Key Outcomes
Key Target Site Site Site Site
Outcome 1 1 2 2
(At 12 weeks only)
s SNAP Control SNAP Control
Conduct- Moderate effect sizes
Type ½ SD change
Behaviour
Stat. Sign. changes
(CBCL)
(p<0.05)
Police 100% ↓ at 6 months
Contact 80% ↓ after 6
months
70-75%↓ l (annually
after program
completion 17
18. Planning Challenges Cont’d:
Variation in Program Developer Involvement
Program developers for promising and model
programs vary in their involvement.
Issue/Challenge Licensing requirements and fee schedules are
also variable.
NCPC involvement in customizing budgets,
developing program theories, fidelity checklists
Implications and quality assurances will vary and require
increased FTE involvement
•Plan all 8 multi-site evaluations on a case by
case basis
Mitigation Strategy
• Assess licensing and quality assurance
requirements in the early stages of project
development.
18
19. Summary:
Planning High Standard Multi-site
Evaluations
√ Ensure central co-ordination approach is used (i.e. One evaluation
firm oversees all evaluations for each site)
√ Enhance fidelity and quality assurance checks
√Target population is well defined and each site adheres to the criteria
√ Core set of instruments are used for all sites (including same risk assessment)
√ Quality assurance procedures are implemented (including fidelity checklists)
√ All sites implement core components (additional non-core components are recognized
and analyzed accordingly)
√ Identify quantifiable benchmarks/targets for key outcomes
√ Correlate fidelity levels with key outcomes to explain variation
between sites
19
20. Contact Information
Donna Smith-Moncrieffe
Senior Evaluation Advisor
Public Safety Canada, National Crime Prevention Center
Telephone: 416-952-0423
E-mail: donna.smith-moncrieffe@ps.gc.ca
20