Presented at the 2015 CGIAR Evaluation Community of Practice meeitng. CGIAR is moving towards a coordinated evaluation system to comprehensively cover the programs, insitutions, and activities. The presentation offers examples of decentralized evaluaitons as approached by other agencies, and aspects for CGIAR to consider.
2. grantseeking creating a program logic modelRebecca White
Grants for beginners. Module 2 of grant seeking series. Covers how to develop a program logic model for grant development. Basic program logic models include highlighting the situation and priorities; development of overall program goal; determining program outcomes, outputs and inputs; identifying any assumptions and external factors that are in play; and developing an program evaluation plan.
Presented at the 2015 CGIAR Evaluation Community of Practice meeitng. CGIAR is moving towards a coordinated evaluation system to comprehensively cover the programs, insitutions, and activities. The presentation offers examples of decentralized evaluaitons as approached by other agencies, and aspects for CGIAR to consider.
2. grantseeking creating a program logic modelRebecca White
Grants for beginners. Module 2 of grant seeking series. Covers how to develop a program logic model for grant development. Basic program logic models include highlighting the situation and priorities; development of overall program goal; determining program outcomes, outputs and inputs; identifying any assumptions and external factors that are in play; and developing an program evaluation plan.
Determinants of Evaluation Supply at the US EPA: A Case Study of the RCRA Ha...Nick Hart, Ph.D.
What facilitators encourage the production of evaluation in EPA’s RCRA Program? What real and perceived barriers impede the production of ex media and ex post evaluation in RCRA?
Qualitative Comparitive Analysis (QCA)and Complimentary Methods 29 April 2016nds1202
In this seminar the authors share methodological learning from this complex evaluation. The innovative methodology employed a mixed-method approach designed to capture the breadth and of a large and diverse portfolio without losing the depth of analysis of complex change processes.
CSU Extension, Engagement and the Logic modelSteven Newman
Presentation delivered to graduate class Principles of Extension.
Much of the material generated in this lecture were from the extension, logic model, scholarship of engagement were taken from the University of Wisconsin-Extension, Program Development and Evaluation program.
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Learn to create a program logic model. Designed for Cooperative Extension Service professionals providing university outreach programs. Logic models are a mainstay in the program development process for community-based, outreach programs.
Developing an evaluation strategy to gain insights into the ROER4D multi-nati...SarahG_SS
Presentation at the OE Global Conference held in Banff, Canada in April 2015. This presentation introduces the development of the evaluation strategy, using Utilization Focused Evaluation (UFE), for the ROER4D Project (http://roer4d.org/).
You can access the abstract here: http://conference.oeconsortium.org/2015/presentation/developing-an-evaluation-strategy-to-gain-insights-into-a-multi-national-project-roer4d/
This presentation provides an overview of the Systematic Inquiry Cycle and Logic Modeling as tools for designing and developing a research study or project/program initiative.
This is a 2-hour presentation and workshop given to the residents at Boston University as part of the Dental Public Health program. Topic presents one of the useful tools for program planning and evaluation in any field. A list of useful websites for online courses and worksheets are provided at the end.
Utilization focused evaluation: an introduction (Part 1 - ROER4D) SarahG_SS
Introductory slides on Utilization Focused Evaluation (UFE) that I presented to the ROER4D team (http://roer4d.org/) on 22 September 2014 as part of the project's evaluation process.
Policy Compass Workshop, 1st of June 2015
The main goal of Policy Compass is to develop a research prototype of an easy-to-use, highly visual and intuitive tool for social networks and eParticipation platforms, enabling citizens and public officials to easily create, apply, share, embed, annotate and discuss causal models, charts and graphs of historical data from trusted open data sources. The aim is to develop methods and tools that facilitate more factual, evidence-based, transparent and accountable policy evaluation and analysis.
http://policycompass.eu/
Federal, state, provincial and foundation grant applications in both the United States and Canada are increasingly requiring the use of logic models in their grant applications. Depending on the level of complexity required, these can prove a major stumbling block, especially with looming deadlines. The purpose of this seminar is to unlock the mystery surrounding their development and use. At the conclusion, we will not promise that you will like them any better, just understand them and fear them less.
Ideal program for entrepreneurs, investors and managers to understand A to Z managing the coffee shop or restaurant with an In-depth approach to service industry in order to facilitate the functions of managing cafe.
Determinants of Evaluation Supply at the US EPA: A Case Study of the RCRA Ha...Nick Hart, Ph.D.
What facilitators encourage the production of evaluation in EPA’s RCRA Program? What real and perceived barriers impede the production of ex media and ex post evaluation in RCRA?
Qualitative Comparitive Analysis (QCA)and Complimentary Methods 29 April 2016nds1202
In this seminar the authors share methodological learning from this complex evaluation. The innovative methodology employed a mixed-method approach designed to capture the breadth and of a large and diverse portfolio without losing the depth of analysis of complex change processes.
CSU Extension, Engagement and the Logic modelSteven Newman
Presentation delivered to graduate class Principles of Extension.
Much of the material generated in this lecture were from the extension, logic model, scholarship of engagement were taken from the University of Wisconsin-Extension, Program Development and Evaluation program.
http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Learn to create a program logic model. Designed for Cooperative Extension Service professionals providing university outreach programs. Logic models are a mainstay in the program development process for community-based, outreach programs.
Developing an evaluation strategy to gain insights into the ROER4D multi-nati...SarahG_SS
Presentation at the OE Global Conference held in Banff, Canada in April 2015. This presentation introduces the development of the evaluation strategy, using Utilization Focused Evaluation (UFE), for the ROER4D Project (http://roer4d.org/).
You can access the abstract here: http://conference.oeconsortium.org/2015/presentation/developing-an-evaluation-strategy-to-gain-insights-into-a-multi-national-project-roer4d/
This presentation provides an overview of the Systematic Inquiry Cycle and Logic Modeling as tools for designing and developing a research study or project/program initiative.
This is a 2-hour presentation and workshop given to the residents at Boston University as part of the Dental Public Health program. Topic presents one of the useful tools for program planning and evaluation in any field. A list of useful websites for online courses and worksheets are provided at the end.
Utilization focused evaluation: an introduction (Part 1 - ROER4D) SarahG_SS
Introductory slides on Utilization Focused Evaluation (UFE) that I presented to the ROER4D team (http://roer4d.org/) on 22 September 2014 as part of the project's evaluation process.
Policy Compass Workshop, 1st of June 2015
The main goal of Policy Compass is to develop a research prototype of an easy-to-use, highly visual and intuitive tool for social networks and eParticipation platforms, enabling citizens and public officials to easily create, apply, share, embed, annotate and discuss causal models, charts and graphs of historical data from trusted open data sources. The aim is to develop methods and tools that facilitate more factual, evidence-based, transparent and accountable policy evaluation and analysis.
http://policycompass.eu/
Federal, state, provincial and foundation grant applications in both the United States and Canada are increasingly requiring the use of logic models in their grant applications. Depending on the level of complexity required, these can prove a major stumbling block, especially with looming deadlines. The purpose of this seminar is to unlock the mystery surrounding their development and use. At the conclusion, we will not promise that you will like them any better, just understand them and fear them less.
Ideal program for entrepreneurs, investors and managers to understand A to Z managing the coffee shop or restaurant with an In-depth approach to service industry in order to facilitate the functions of managing cafe.
Whole systems change across a neighbourhood
How can we collaborate with people to help them build their resilience? Get under the skin of the culture and the lives people live. Identify people’s feelings and experiences of community and understand what people think is shaped by different values and by the environment and infrastructure around them. The future of collaboration could bring many opportunities but people find it more difficult to live and act together than before. How can we help people…and communities build their resilience? Understand people’s different situations and capabilities to develop pathways that help them build resilient relationships. Help people experience and practice change together. Help people grow everyday practices into sustainable projects. Turn people’s everyday motivations into design principles. Support infrastructure that connects different cultures of collaboration. Build relationships with people designing in collaboration for the future…now.
SUBJECT: SOCIO EDUCATIONAL PROJECT
TUTOR: DR. MIGUEL PONCE
THEME: PART III. PROJECT EVALUATION
STUDENTS:
MONTESDEOCA BENITEZ DIANA PRISCILA
PACHACAMA SIMBAÑA DAYSI ALEXANDRA
Using case-based methods to assess scalability and sustainability: Lessons fr...JSI
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
An institutional perspective on analytics that focusses on a particular tool developed using an agile methodology to visualise learner behaviours in MOOCs via Sankey diagrams.
A presentation for my Ed. D. Degree Program relating to Program Evaluation Models: Developers of the Management-Oriented Evaluation Approach and their Contributions;
How the Management-Oriented Evaluation Approach Has Been Used; Strengths and Limitations of the Management-Oriented Evaluation Approach; Other References, Questions for Discussion
Using case-based methods to assess scalability and sustainability: Lessons fr...Barb Knittel
Overview of the SC4CCM project and end-line evaluation questions focused on scalability and sustainability. Methodological approaches including case selection strategies, mixed method approaches, within-case and cross-case analysis processes. (Sangeeta Mookherji, GWU)
This recorded Accountable Health Communities Model webinar provides an overview of the learning system and implementation plan guide.
- - -
CMS Innovation Center
http://innovation.cms.gov
We accept comments in the spirit of our comment policy:
http://newmedia.hhs.gov/standards/comment_policy.html
CMS Privacy Policy
http://cms.gov/About-CMS/Agency-Information/Aboutwebsite/Privacy-Policy.html
International Food Policy Research Institute (IFPRI) organized a three days Training Workshop on ‘Monitoring and Evaluation Methods’ on 10-12 March 2014 in New Delhi, India. The workshop is part of an IFAD grant to IFPRI to partner in the Monitoring and Evaluation component of the ongoing projects in the region. The three day workshop is intended to be a collaborative affair between project directors, M & E leaders and M & E experts. As part of the workshop, detailed interaction will take place on the evaluation routines involving sampling, questionnaire development, data collection and management techniques and production of an evaluation report. The workshop is designed to better understand the M & E needs of various projects that are at different stages of implementation. Both the generic issues involved in M & E programs as well as project specific needs will be addressed in the workshop. The objective of the workshop is to come up with a work plan for M & E domains in the IFAD projects and determine the possibilities of collaboration between IFPRI and project leaders.
Assignment 2 Designing a Training ProgramDue Week 8 and worth 3.docxsherni1
Assignment 2: Designing a Training Program
Due Week 8 and worth 300 points
Create a written proposal in which you detail the complete design of an employee training program.
Write a six to eight (6-8) page paper in which you:
1. Design a two (2) day training program for a group of twenty (20) employees.
2. Identify two to three (2-3) training needs though a Training Needs Analysis (TNA) and justify an
approach for this training.
3. Develop the training objective for this program based on an analysis of the business.
4. Determine the training cost for the training program you are proposing. Include a detailed
breakdown of time allotted for each piece, the subsequent cost analysis, and the total cost for the
project as a whole.
5. Select key training method (s) to deliver the program to employees, such as an e-Learning
module or a one-day face-to-face training program.
6. Create an agenda of activities for the training program.
7. Use at least five (5) quality academic resources in this assignment. Note: Wikipedia and other
Websites do not qualify as academic resources.
Your assignment must follow these formatting requirements:
Be typed, double spaced, using Times New Roman font (size 12), with one-inch margins on all
sides; citations and references must follow APA or school-specific format. Check with your
professor for any additional instructions.
Include a cover page containing the title of the assignment, the student’s name, the professor’s
name, the course title, and the date. The cover page and the reference page are not included in
the required assignment page length.
The specific course learning outcomes associated with this assignment are:
Identify when to conduct a Training Needs Analysis (TNA).
Analyze the various approaches to performing a Training Needs Analysis.
Develop strategies for training design.
Use technology and information resources to research issues in training and development.
Write clearly and concisely about training and development using proper writing mechanics.
Grading for this assignment will be based on answer quality, logic / organization of the paper, and
You are part of a behavior consulting team providing consultation to local service agencies in your area.
Choose one of the following scenarios and write a comprehensive plan for program design and evaluation You are being asked to apply what you have learned about how Evidence-Based Program Planning applies to organization-wide behavior interventions.
· Just Like Home nursing home wants to assess the possible need for a new behavior management program to prevent Alzheimer’s patients from wandering unsupervised and endangering themselves.
· VHab vocational training program for developmentally disabled adults wants to assess the possible need for a new program to train clients as potential volunteers for various community services.
· Stars ABA program, providing clinic-based ABA interventions for children with autism, wants to assess ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
2. Agenda
1. Performance Management and Program Evaluation Overview
2. Introduction to Case Study: New Roads for New Visions
3. Conducting the Needs Assessment
4. Creating the Logic Model
5. Building the Performance Management System
6. Linking to Program Evaluation
7. Conclusion
3. Defining PM and PE
PERFORMANCE
“an organization’s ability to achieve its goals and objectives
measurably, reliably, and sustainably through intentional actions”
PERFORMANCE MANAGEMENT
“the set of self-correcting processes,
grounded in real-time data measuring,
monitoring, and analysis, that an
organization uses to learn from its work and
to make tactical and strategic adjustments
to achieve its goals and objectives.”
PROGRAM EVALUATION
“the use of social research
procedures to systematically
investigate the effectiveness of
social intervention programs.”
4. Distinctions Between PM and PE
Differing Objectives
PERFORMANCE MANAGEMENT
-Decide what data to collect
-Determine how to convert performance
data into actionable information to support
both tactical and strategic decision making
PROGRAM EVALUATION
-Determine how and to what extent goals
and objectives are fulfilled.
-Judge efficiency, effectiveness, impact, and
sustainability
5. PM and PE Compliments
PERFORMANCE
MANAGEMENT
Cannot properly attribute
actions to outcomes or
confirm data validity
PROGRAM
EVALUATION
Often does not build internal
organizational capacity or
provide consistent updates
needed for ongoing
improvement
8. NRNV’s Needs Assessment Using GIS and
Community Data
1. Siting the NRNV
Students First Center
2. Community Data
3. Spatial Analysis and
Geoprocessing with
GIS
4. Proximity and Access
5. Community Assets
and Capacity
6. Risk Factors
Purpose
• Community center in a high reentry
neighborhood in LA to serve NRNV
youth
• Transition from in camp and reentry
services, reduce risk of recidivism
• Target and serve only probation youth
Youth Reentry
Community-Based
and Youth Specific
Probation Services
Youth
Outcomes
(Reduced Risk of
Recidivism)
9. Community Data
• Lennox and Athens Park target
areas
• Juvenile reentry rates per
1,000 children
• Location of probation specific
youth services from the
Rainbow Resource directory
(anti-gang, at risk, probation
services)
10. NRNV’s Needs Assessment Using GIS and
Community Data
1. Siting the NRNV
Students First Center
2. Community Data
3. Spatial Analysis and
Geoprocessing with
GIS
4. Proximity and Access
5. Community Assets
and Capacity
6. Risk Factors
Resources and Skills
• ArcGIS, QGIS
• Geocoding (point locations)
• Thematic Symbology (shading)
• Clip
• Buffer
• Dissolve
• Select by location and attribute
• Graduated symbology
11. Proximity and Access
• Existing NRNV youth reentry
locations and .25 mile buffer
• Accessibility of youth to
target areas by highways and
streets
12. Community Assets and
Capacity
• Schools, churches, parks
and recreation centers, bus
routes
• Youth locations and youth
probation resources
• Services to youth ratio
13. Risk Factors
• Violent crimes for
previous 6 months
• Increasing violent
crime by 20 incidents
per graduated symbol
15. The Logic Model
Actions are Linked to Outcomes
As a result…
Logic Model = Blueprint
“defines a strategy with operational, measurable outcomes and
objectives that can be used to set expectations and then drive
performance,” (118)
16. New Roads for New Visions Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
Framing the Logic Model for the Client:
• Explain what the Logic Model is and what it is used
for.
• Present the Logic Model in the context of the entire
PM project.
• Define Logic Model vocabulary
• Provide several examples
Need Resources? See handout
17. New Roads for New Vision Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
Provide Client with brainstorm template and two or three suggestions for
each category
Staff:
1 FTE Counselor
4hrs of
Employment
Training
Improved Pro-
Social Behavior
% Graduated
from High
School
18. New Roads for New Vision Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
Based on Client brainstorm, design logic model first draft
19. New Roads for New Vision Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
Typical Feedback Process:
• Send first draft to client
• Client has a chance to review and respond with comments
• Evaluator discusses potential changes with client to make sure
edits fit the logic model format and accurately reflect the
client’s program.
• If necessary, change column labels and add any visual cues
such as arrows, bold words, etc…
• Make changes and return to client
20. New Roads for New Vision Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
Finalizing the Draft:
1. Send revised draft back to client for any additional revisions
2. Ask client if the organization/program has any key
stakeholders who should also review the logic model
3. If applicable, send logic model to stakeholders for review
4. Incorporate any additional edits and return to client for a
final review
21. New Roads for New Vision Logic Model Process
1. Investment
2. Brainstorm
3. Draft
4. Feedback
5. Final Draft
6. Ongoing
Revision
The Logic Model Should Always be a ‘Living Document’
• Emphasize to the client that the logic model can change as the
program naturally evolves
• As the PM process continues, it is normal for the client to
want to revise outputs/outcomes. These changes should be
adjusted in the logic model.
• Save each logic model version separately; this way, the client
(and evaluator) can have a record of the changes made.
23. The Measuring and Monitoring System
”Such systems are the
means for keeping track of
performance against a few
key indicators that show
whether (and how well)
and organization is doing
what it should, and the
levels it should, with the
quality it should, at the
cost levels it should – and
in doing so achieves the
results that it should.” (14)
24. Setting up the NRNV System
Extract the Outcomes and/or Outputs from the Logic Model and organize
into a ‘Data Outline.’
The Data Outline can be organized as a table with the following columns
(example):
• Outcome
• Data Source
• Measure
See NRNV Example for Further Details….
25. Setting up the NRNV System
Create any new data collection forms/fields: NRNV needed to create a
‘Post-Detention Survey’ in an electronic format capable of storing
responses to answers. Additionally, existing forms needed to be
updated and modified.
Create a data collection and storage plan: See NRNV Example
Select system platform: NRNV had started to use Microsoft Access, so it
made sense to continuing expanding upon this system.
26. A Note about PM Data Platforms –
Do you need tech support?
Selecting the PM Data Platform should depend on your own expertise as well
as your client’s resources.
Points to Consider:
1) Can the client afford to purchase a ‘built’ platform such as Efforts to
Outcomes or Microsoft Access?
2) Should you seek help and/or hire a developer capable of assisting with
coding for programs such as Visualforce and Salesforce?
See handout detailing commonly used PM Data Platforms
27. Reporting–
Ask again: Do you need tech support?
Establish the client expectations for PM reporting
• Who is the report audience?
• Does the client have a budget for outsourced reporting?
Option 1: Create the report template on your own
• Depending on your own skills, create a report template using the fields
established in Data Analysis Plan – Many PM platforms (such as Access and
Salesforce) have built in reporting abilities that are easy to use
• See Reporting Resources for webinars to enhance your skills
Option 2: Outsource reporting to a data dashboard provider or tech savvy
consultant
• See reporting resources for reporting platform examples
28. NRNV’s Year 1 Evaluation Plan
1. Year 1 Evaluation
Plan
2. Program Impact
Theory
3. Complementarities
with PM
4. Formative
Evaluation
5. Summative
Evaluation
Purpose
• Culturally relevant, utilization-focused
• Document program implementation
• Measure programmatic outcomes
• Identify ways to improve the program
29. NRNV’s Year 1 Evaluation Plan
1. Year 1 Evaluation
Plan
2. Program Impact
Theory
3. Complementarities
with PM
4. Formative
Evaluation
5. Summative
Evaluation
Theory of Change
• Based on Logic Model
• Informs evaluation questions
• Leads to more sensitive and valid evaluation
designs
• Cause and effect sequences that link program
services and activities to short-term and long-
term outcomes
30. Students First Center: Program Impact Theory
Students
First
Center
School and/or
Vocational Attainment
Employment
Improved Criminal
Justice Status
Pro-Social Behavior
Achieve Career Goals
Realize Full Human
Potential
Enroll/Complete High
School
Enroll in Post-
Secondary Education
Enroll in Vocational
Certificate Program
Complete
Employment Training
Obtain Legal
Documents
Reduced Contact with
the Law
Complete Probation
Seal Juvenile Records
Identify Hobbies and
Personal Interests
Improved
Psychological Status
31. NRNV’s Year 1 Evaluation Plan
1. Year 1 Evaluation
Plan
2. Program Impact
Theory
3. Complementarities
with PM
4. Formative
Evaluation
5. Summative
Evaluation
Complementaries with Performance Management
• Information-use same data to answer different
questions based on different analyses
o NRNV Database
o NRNV Post-Probation Survey
o Student Service Plans
o Student Transition Plans
• Methodical-similar processes and tools to
collect and analyze data and convert data into
actionable information
• Evaluator role=interpret data, feedback loop
33. NRNV’s Year 1 Evaluation Plan
1. Year 1 Evaluation
Plan
2. Program Impact
Theory
3. Complementarities
with PM
4. Formative
Evaluation
5. Summative
Evaluation
Implementation Evaluation
• Often combined with performance
management
• Process and Progress Questions and Measures
o Who is being served?
o Was the program implemented as expected?
o Type and volume of service
• Provides information to summative evaluation
and helps to explain findings
34. NRNV’s Year 1 Evaluation Plan
1. Year 1 Evaluation
Plan
2. Program Impact
Theory
3. Complementarities
with PM
4. Formative
Evaluation
5. Summative
Evaluation
Intermediate Outcomes
• How well did the program work?
• What change occurred?
o Achievement
o Attainment
o Status
o Behavior
35. Measuring Impact
• Long Term Outcomes
• Consequences of program on a broader scale
• Demonstrate Causality
• Counterfactual- What would have happened in the
absence of the program?
36. Examples of Impact Evaluation
Pre/post measurements
• NRNV post-detention survey
• Adapt pre and post NRNV surveys for control
group
Random assignment
• No treatment
• Different intensity or treatment
Treatment - SFC and not other reintegration programs
Control group - Other youth exiting Camp David
Gonzalez who didn’t participate in C2C or attend SFC
Randomized Control Trial
X X X
37. Wrap Up
• An effective, meaningful, and utilized performance management
system informs the program evaluation.
• Evaluators have the skills and knowledge to help programs build
evaluation capacity.
• Thus, evaluators can play a pivotal role in improving program
evaluation by promoting continuous quality improvement and
overall program effectiveness.
38. References
• Hunter, D.E.K. & Nilsen, S.B. (2013). Performance management and evaluation:
Exploring complementarities. In S.B. Nielsen & D.E.K. Hunter (Eds.), Performance
management and evaluation. New Directions for Evaluation, 137, 7-17.
• Hatry, H.P. (2013). Sorting the relationships among performance measurement,
program evaluation, and performance management. In S.B. Nielsen & D.E.K.
Hunter (Eds.), Performance management and evaluation. New Directions for
Evaluation, 137, 19-32.
• Nielsen, S.B. & Hunter, D.E.K. (2013). Challenges to and forms of complementarity
between performance management and evaluation. In S.B. Nielsen & D.E.K.
Hunter (Eds.), Performance management and evaluation. New Directions for
Evaluation, 137, 115-123.