1. Extending Success:
Defining and Evaluating
Excellence in The Met
Community School
Extended Day Programming
Mateus Baptista, Lauren Combs, Jesse DeLauder, Gretchen Mills
Wednesday, July 30, 14
3. MCS
The Met Community School
Mandatory extended day for 9th
grade students
Diverse programming
Experiential learning
Founded in 2003
Wednesday, July 30, 14
4. Community
Learning Centers
“In 1994, Congress authorized the 21st Century
Community Learning Centers (21st Century)
program to open up schools for broader use by
their communities. In 1998, the program was
refocused on providing academic, enrichment, and
recreational activities in public schools during
the after-school hours (centers also could be open
before school, on weekends, and during the
summer). The program grew from an appropriation
of $40 million in 1998 to $1 billion in 2002, where it
has remained.” (U.S. DOE, 2005)
Wednesday, July 30, 14
5. Research Question
How can the MCS
extended day program
measure and track
progress towards
their internal goals?
Wednesday, July 30, 14
6. Q1:What are the
perceived goals of MCS?
Q2: Comparison of MCS
program qualities with
those of high-performing
after school programs
Q3: What are available
indicators of success
within MCS?
How can the
MCS extended
day program
measure and
track progress
towards their
internal
goals?
Wednesday, July 30, 14
7. Literature
Qualities of similar high-
performing after-school and
extended day program
Successful evaluation of
after-school and
extended day programs
Methods of research
and evaluation
The Met in
Context
Research Design
Analysis
Findings
Recommendations
Wednesday, July 30, 14
9. Methods
Interviews Questionnaire
N=11 (Two admin, five student, four provider)
Convenience Sample
Purpose:
1) Address how extended day
goals are perceived
2) Compare MCS qualities to
those of existing programs
3) Identify available indicators
of success
Purpose:
Measure perceived program
success towards three MCS
goals
Wednesday, July 30, 14
10. Goals
MCS Goals According to Director
Safety
College and
Career Readiness
Social and
Emotional Learning
Wednesday, July 30, 14
11. Methods
Interview Protocol - Providers and Administrators
Purpose: We are a research team interested in helping the MCS design an internal evaluation system. To do this, we
would like to hear about your personal experience with the extended day program.
You will not be identified by your responses, and they will remain confidential—only to be accessed by our four-person
research team.
Is it ok if we record this interview?
1. Please describe your role in the extended day program.
2. What do you see as the goals of the extended day program?
3. How do you know if the extended day program is meeting these goals?
5. What do you see are the strengths and weaknesses of the program?
6. How do you communicate to other staff/teachers within the program? How do you record challenges or successes for
yourself?
7. Are there formal or informal opportunities for parental engagement?
8. How long have you worked at MCS?
9. What ethnicity do you identify with?
10. Is there anything else you would like to share that you have not yet had a chance to share?
Wednesday, July 30, 14
12. Methods
Interview Protocol - Student
Purpose: We are a research team interested in helping the MCS design an internal evaluation system. To do this, we would
like to hear about your personal experience with the extended day program.
You will not be identified by your responses, and they will remain confidential—only to be accessed by our four-person
research team.
Is it ok if we record this interview?
1. Tell us about your experience with the extended day program at MCS.
2. Do you think it is important to have a program like this here in your community?
3. Did you continue on in the extended day program past 9th grade? Why did you do so?
4. How has the program affected your academics? Social life?
5. How is the program similar to what you do at school? How is it different?
6. What would you be doing after school if you weren’t coming to the program?
7. What is your favorite and least favorite part of the extended day program?
8. Do you have any ideas to make the extended day program better?
9. Do people listen when you have an opinion or concern about the extended day program?
10. Is there anything else you would like to tell us?
11. What ethnicity do you identify with?
12. What grade are you in?
13. How many years have you attended the extended day program?
Wednesday, July 30, 14
13. Methods
Follow-Up Questionnaire
The Met Community School strives to provide its students with a safe
environment, opportunities for social/emotional learning, and the skills
necessary to succeed in college and career. To what extent do you think the
extended day program is successful in achieving these goals?
Safety
College and Career
Readiness
Social/Emotional
Learning
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
Not at all
successful
Somewhat
successful
Very
successful
Completed in private by ten of twelve interviewees and returned
Wednesday, July 30, 14
14. Analysis
5) Questionnaire: measures of
central tendency
Qualitative
Quantitative
Subgoals
Qualities
Indicators
Q1
Q2
Q3
Subgoals
Qualities
Indicators
Subgoals
Qualities
Indicators
Safety
College/Career
Readiness
Social/Emotional
Learning
4) Post Coding: Frequency counts
1) Transcribe Interviews
2) Initial Coding - General
3) Final Coding - Categorical
Wednesday, July 30, 14
15. Codes
Room for Improvement
Low participation
Disconnect between provider & admin
Disorganization
Add programs
Mandatory for all grades
No student input
College/Career
Readiness
Multiple Pathways
Extend Learning Times
Academic Programming
Build Job Skills
Networking
MCS Connection to Met
School
Internships
Need for MCS Administration & Organization
Professional Development Opportunities
Internal Assessments
Staff Supports
Diversity of Programs
Formal/Informal Family Engagement
Staff Accountability
School/Community Partnerships
Provider Anxiety
Safety
Keep Kids Out of Trouble
Participation
Attendance
Safe Environment
Curb Delinquent
Behavior
Social/Emotional
Learning
Formal Student Leadership
Opportunities
Emotional Growth
Peer Relationships
Mentoring
Student Engagement
Finding your Passion
Informal Student Leadership
Opportunities
Enrichment Opportunities
Wednesday, July 30, 14
16. Findings
Subquestion #1: What are the perceived goals of
MCS?
When asked “what do you feel are the goals of the
extended day program?” administrators and providers
responded...
*Students were asked, “Do you think it’s important to have a program like MCS in your community? If so, why?”
Q1
GoalCodes
Wednesday, July 30, 14
17. Findings
MCS Qualities and Indicators by Goal
Q1
0
10
20
30
40
50
60
70
Frequency
Talking About MCS Goals
SAFETY
Participation
Attendance
Safe Environment
Curb Delinquent
Behavior
COLLEGE AND CAREER
READINESS
Academic Programming
Build Job Skills
Networking
MCS Connection to Met
Vocational Training/
Internships
SOCIAL-EMOTIONAL
LEARNING
Student Leadership
Emotional Growth
Peer Relationships
Mentoring
Student Engagement
Enrichment Opportunities
Wednesday, July 30, 14
18. Findings
Perceived Success by Goal
Q1
4.7
4
4.5
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Safety College and Career Readiness Social-Emotional Learning
Perceived Success Toward MCS Goals
Wednesday, July 30, 14
19. Quality:
A feature or component of
a high performing after-
school program.
Subquestion #2: Comparison of MCS
program qualities with those of high-
performing after school programs
AnalysisQ2
Wednesday, July 30, 14
21. Findings
Comparison of MCS program qualities
with those of high-performing after-school
programs
0
5
10
15
20
25
30
SAFETY
COLLEGE & CAREER READINESS
SOCIAL-EMOTIONAL LEARNING
N/A
Q2
Wednesday, July 30, 14
22. Indicator:
“A set of statements about activities
and behavior that can be used to
measure the extent to which after-
school programs are providing high-
quality opportunities for
participants.” (Providence After
School Alliance)
Subquestion #3: What are available
indicators of success within MCS?
AnalysisQ3
Wednesday, July 30, 14
23. Analysis
What are available indicators of success
within MCS?
When asked about measuring success and positive outcomes,
administrators, students, and providers mentioned...
Attendance
Delinquent behavior
Mentorship
Student showcase
Feelings towards
school
Networking
Building job skills
Peer relationships
Emotional growth
Student empowerment
Q3
Wednesday, July 30, 14
25. Recommendations
Goals for MCS
MCS should set
clear goals that
are communicated
program-wide.
Recommendation:
Mission statement
Finding:
Emergent goals
were informed
by the three
stated goals, but
interviewees did
not have a clear
goal framework.
Wednesday, July 30, 14
26. Recommendations
Stated Goals from
Director
Emergent Goals
Qualities
Indicators
Evaluation of MCS
programming
should be
informed by
emergent goals
and best practices,
using the
appropriate stated
indicators.
Wednesday, July 30, 14
27. Recommendations
Emerging
Goals
Qualities Indicators
Social/Emotional
Learning
Find your passion
Enrichment
opportunities
College/Career
Readiness
Extend classroom
learning
Safety
Foster relationships
Diversity of enrichment
programs
Student Leadership
(formal and informal)
Academic programming
Internships
Connection to Met
Trying something new
Mentoring
Peer relationships
Emotional growth
Student engagement
Networking
Job skills
Feelings towards
schools
Student showcase
Participation
Safe environment
Delinquent behavior
Attendance
Evaluation
Measurements
Expectancy of success
Effectance motivation
Evaluation of students
by provider and mentor
Focus groups
Longitudinal research of
college enrollment, job
placement, and program
satisfaction
Drop out rates
Graduation rates
Student pre- and post- tests
Assessment of student
showcase
Attendance records
Student pre- and
post- tests
Focus groups
Wednesday, July 30, 14
28. Limitations
Self- and organization-preservation bias
Limits of internal validity
Limited pre-existing data
Convenience sampling
Limits of external validity: unique
situation
Summer Program
Small sample size
Wednesday, July 30, 14
29. Next Steps
3)Study on the overarching
community effects
1)
Study
performed
during
extended day
programming
2) Design
evaluation tools
Wednesday, July 30, 14
30. “So the strengths are just opening up so many
doors of opportunity for these kids [who] may
come from a background where they felt they
wouldn’t amount to anything. Or they always
could but they didn’t know how. It’s opening
these doors so they can see which direction
they may want to go, or figure out something
about themselves they didn’t know.
It’s just self-discovery.”
-Provider & Former Student
Wednesday, July 30, 14
