The DESSA Comprehensive
System – A Data-Driven Approach
to SEL
Paul LeBuffe
VP of Research and Development
Aperture Education
Aperture Education
• New social venture formed in 2017 by The
Devereux Foundation and Apperson, Inc.
• Sole mission is the promotion of social and
emotional competence in children, youth and
adults
• Currently offers The DESSA Comprehensive
System delivered via the Evo
Social/Emotional Assessment and
Intervention System (Evo SEL)
Principles Underlying the DESSA
Comprehensive System
• SEL should be strength-based
• SEL should be data-driven
• Responsive to the actual needs of your students
• Support CQI and Accountability
• SEL should encourage local decision making
(variation within form)
• Teachers and staff know their students best
How Can S/E Assessment Improve
Practice?
• High-quality assessment can:
• Enhance awareness of each student’s unique S/E
strengths and needs
• Guide instruction to address the needs that actually
exist
• Enable teachers and pupil personnel staff to honor,
maintain, and leverage student strengths
• Assist in developing rapport with parents and students
• Identify meaningful strengths for IEPs
• Makes possible true prevention
3 Rs of Quality
Assessment
Rigor
Reasonableness
Relevance
What are the Needs &
Components?
• Universal Screening
• Detailed Assessment
• SEL Instruction
• Progress Monitoring
• Outcome Evaluation &
Quality Improvement
• DESSA-mini
• DESSA
• DESSA Strategies
• Ongoing Progress
Monitoring Form
(OPM)
• Advanced
Interpretation
Techniques
The DESSA-mini
• Completed by Teachers, OST Staff, Parents
• Four equivalent 8-item forms
• Completed in 1-2 minutes per student; one planning period
per class
• Yields one score – Social-Emotional Total Score (SET)
• Online via Evo SEL & paper and pencil
• English, Spanish, Dutch, and Italian
How Aperture Describes Social
and Emotional Skills
NB: not a deficit.
“Burden” placed on
adults to provide
instruction.
DESSA-mini Results
• One Score – Social Emotional Total (SET)
• T-Scores
• Mean of 50, SD of 10
• Percentiles
• Descriptive Terms for T-Score Ranges
• > 60 = Strength (≈ 16%)
• 41-59 = Typical (≈ 68%)
• < 40 = Need for Instruction (≈16%)
DESSA-mini
• DESSA mini normative group
• Standardization data for Teacher Raters (N =
1,249)
• Region: NE = 24.6%; South = 39.1%; Midwest =
22.3%; West = 14.0%
• 50.8% Males
• Grades: Kindergarten through 8
• Representative with respect to Race, Hispanic
ethnicity, and SES
DESSA-mini Psychometrics
• Reliability and SEM
DESSA-mini Psychometrics
Key Findings from Allentown
Study
• Sensitivity
• Criterion > .50 (Glascoe)
• Obtained .63
• Specificity
• Criterion > .75
• Obtained .98
• Consistent Classification 87% of the time
Relationship Between Academic Achievement
and Social-Emotional Competence
38.3
6.4
28.6
12.5
24.7
38.2
8.4
42.9
.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Need Strength
%ofStudents
DESSA-mini Category
% of Elementary Students by PSSA Math
and DESSA-mini Categories
Advanced
Proficient
Basic
Below Basic 59.0
15.2
18.2
15.9
21.1
48.9
1.7
20.1
.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Need Strength
%ofStudents
DESSA-mini Category
% of Elementary Students by PSSA
Reading and DESSA-mini Categories
Advanced
Proficient
Basic
Below Basic
Relationship Between Academic Achievement
and Social-Emotional Competence
38.6
10.6
24.7
19.0
27.1
28.2
9.6
42.3
.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Need Strength
%ofStudents
DESSA-mini Category
% of Middle School Students by PSSA
Math and DESSA-mini Categories
Advanced
Proficient
Basic
Below Basic
50.3
14.1
23.3
16.2
20.9
35.2
5.5
34.5
.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Need Strength
%ofStudents
DESSA-mini Category
% of Middle School Students by
Reading and DESSA-mini Categories
Advanced
Proficient
Basic
Below Basic
3rd Graders SEC (N=148)
Variance Explained
SBA
Reading
Low
Income
SEC
53%
22% 9%
16%
• When social-emotional
competence (SEC) is added, an
additional 22% of SBA variance
is uniquely predicted.
• Variance attributable to
poverty alone reduced to 9%
• Together income and social
emotional competence account
for 47% of variance in reading.
• Which variable can we better
impact in our schools?
Explaining the Variance in Academic
Achievement Scores - Reading
8.3%
16.5%
75.2%
Reading PSSA
Scores
Income
Social Emotional
Competence
24.8%
Social emotional
competence explains
an additional 16.5% of
the variance in
reading scores
Social-Emotional Competency of Students at the Beginning
of School Year and Disciplinary Infractions
23.1%
59.1%
17.8%
Social-Emotional Categories
(DESSA-mini SET) Fall Baseline
Grades K-8
Strength
Typical
Need
These students are 450% (4.5X)
more likely to be suspended or
expelled by the end of the year
Can this change our paradigm from
reactive to proactive? Intervention
to prevention?
DESSA-mini Summary
• Appropriate Uses:
• Universal screening
• RTI approach - repeated “probes” of social and
emotional growth
• Needs assessment for SEL at the school or
district level
• Limitations
• No information about specific S/E competencies or
needs
• Doesn’t inform selection of strategies
The Devereux Student
Strengths Assessment
• One, 72-item form
• Completed thoughtfully by parents, teachers and
OST staff in about 5-8 minutes
• Yields 9 scores – 8 specific scales and the Social
and Emotional Composite (SEC)
• The DESSA allows for
• Comprehensive assessment of key S/E skills
• Guiding instruction
• Outcome evaluation & CQI
DESSA Scale Structure
• Social Emotional Composite
• Eight Scales
• Self Awareness
• Self-Management
• Social-Awareness
• Relationship Skills
• Goal-Directed Behavior
• Personal Responsibility
• Decision Making
• Optimistic Thinking
DESSA Scales and CASEL Alignment
Purposes of the DESSA
• Identify social-emotional strengths and needs of
elementary and middle school children. (high
school coming 2018-19 School Year)
• Tier I – (if used at universal level) Produce
classroom profiles that guide classroom-wide
prevention/promotion strategies.
• Tier II - Assess at-risk children so that targeted
small group or individual interventions can occur.
• Tier III - For special education students, identify
important strengths that can be incorporated into
IEPs and leveraged in practice.
Additional Purposes of the
DESSA
• Foster collaboration between parents and
teachers
• Document outcomes for individual students,
classrooms, and communities
DESSA Raters
• Raters provide the ratings
• Teachers, After School Staff
• Parents
• Read at about the 6th grade level
• Sufficient opportunity to observe the child
• The DESSA-HSE will have a youth self-report form
as well
Minimizing the Minimal Teacher
Rater Bias
• Teachers are very good observers of student
behavior
• Problem behavior oriented scales have used
for years
• Shapiro et al – teacher rater bias accounts for
16% of variance in scores
• Can be reduced to 10% through preservice
training
DESSA Results
• T-Scores
• Mean of 50, SD of 10
• Percentiles
• Descriptive Terms for T-Score Ranges
• > 60 = Strength
• 41-59 = Typical
• < 40 = Need for Instruction
• Individual Student Profile
• Classroom Profile
Individual Student Profile
Strength
Typical
Need for Instruction
How many ways can this
be used?
Individual Item Analysis
Significant Item Score - Strength
#18 – ask to take on additional
work or responsibilities?
Never Rarely Occasionally Frequently
Very
Frequently
#37 – follow the example of a
positive role model?
Nonsignificant Item Score - Typical
Significant Item Score - Need
#69 – use available resources
(people or objects) to solve a
problem?
Never Rarely Occasionally Frequently
Very
Frequently
Never Rarely Occasionally Frequently
Very
Frequently
DESSA Strategies
• Provided as part of Aperture Ed’s Evo SEL web-based
platform
• 5 different levels of strategies for each of the eight
DESSA scales
– Teacher Reflection & Action
– Universal
– Group
– Individual Student
– Home
• 3 different age groupings: primary, intermediate
elementary, and middle school
DESSA Strategies
• Strand 1 Evidence-based and field-tested strategies
• Strand 2 – Select strategies from SEL curricula
– Second Step, 4 Rs, Caring Classroom Community, Open Circle
• Strand 3 – Foundational SEL practices
– Learning agreements - Gratitude
– Greeting rituals - Movement
– Trauma-informed Practices - Student Voice
– Peer Coaching
Response to Intervention
• Core components of RTI
• Universal screening
• Classroom-based interventions
• Frequent monitoring of progress
• Has been very successful in preventing
academic failures
• Why wouldn’t we take the same approach
with social and emotional competency?
38
Pretest-Posttest Comparisons
Pretest-Posttest
Comparison
Time 1 T-Score Posttest
Confidence
Range
Time 2 T-
Score
Outcome – Check One
Significant
Decline
No Change Significant
Increase
Personal Responsibility 58 50-65 62 X
Optimistic Thinking 39 31-49 46 X
Goal-Directed Behavior 51 44-58 50 X
Social Awareness 36 28-49 52 X
Decision Making 48 40-56 38 X
Relationship Skills 58 51-64 62 X
Self-Awareness 40 32-50 57 X
Self-Management 53 45-60 59 X
Social-Emotional
Composite
47 44-50 54 X
Rater 1 Name: ________________
Rater 2 Name: ________________
Date of Rating: ________________
Date of Rating: ________________
Mary Smith
Mary Smith
10/10/14
2/28/15
Footer
School-Wide Outcome Evaluation
School Name: __________________________
James Madison Elementary Date Range: _____________9/1/12 – 6/15/13
Outcome
Scale # in Need
at Pretest
% in Need
at Pretest
# (%) Significant
Decline at
Posttest
# (%)
No Change at
Posttest
# (%) Significant
Improvement at
Posttest
Comments
Self-Awareness 72 20% 7 (10%) 29 (40%) 36 (50%) Moderate issue, moderate
effectiveness. Target for QI
Self-Management 36 10% 6 (17%) 24 (67%) 6 (17%) Smaller issue, limited
effectiveness. Target for QI.
Relationship Skills 72 20% 0 (0%) 15 (21%) 57 (79%) High effectiveness
Optimistic Thinking 120 33% 6 (5%) 30 (25%) 84 (70%) Major issue for school; High
effectiveness
Social-Emotional
Composite
94 26% 10 (11%) 25 (26%) 59 (63%) Nearly 2/3 of students showed
overall improvement during year
Total Number of Students: ___360
Utilization of the DESSA Comprehensive
System with a Tiered Approach
• Tier I – Universal - Primary Prevention
• Universal Screening with DESSA-mini
• Consider Universal Assessment with the DESSA
• Classroom Profile
• Identifies common areas of strength and concern
• Leads to classroom-wide strategies
• Tier II – Targeted - Secondary Prevention
• Identification of children at-risk due to S/E Needs
• Individual Child Profile
• Targeted Strategies
Utilization of the DESSA (cont.)
• Tier III – Indicated - Tertiary Prevention
• Identification of strengths and needs in identified
children
• Incorporated into IEPs
• Strong basis for collaboration with parents
• Program Evaluation
• Evaluate progress at the scale, child, and
classroom level
• Basis for quality improvement and continuing
education efforts
Applications of DESSA
Teachers & School Psychologists
• Teachers
• Screen & Assess
• Guide Selection of Strategies
• Enhance SEL Curricula
• School Psychologists
• Complement deficit oriented assessments
• Set Goals and Objectives for pull-out groups
• Item Level Analysis to identify targets & strengths
• Classroom Consultation Tool
Ways the DESSA adds Value
• Meet requirements in states with social-
emotional learning standards
• Assess competencies related to academic
achievement
• Support implementation of SEL and similar
programs
• Provides a wealth of competence building
strategies
• Meet requirements to do meaningful strength-
based assessment, especially with IEPs
• Contribute to our understanding of disorders
Thank You!
Contact Information:
Paul LeBuffe, VP of Research & Development
Aperture Education
plebuffe@ApertureEd.com
(704) 644-8676
www.ApertureEd.com

DESSA Overview - NASP 18

  • 1.
    The DESSA Comprehensive System– A Data-Driven Approach to SEL Paul LeBuffe VP of Research and Development Aperture Education
  • 2.
    Aperture Education • Newsocial venture formed in 2017 by The Devereux Foundation and Apperson, Inc. • Sole mission is the promotion of social and emotional competence in children, youth and adults • Currently offers The DESSA Comprehensive System delivered via the Evo Social/Emotional Assessment and Intervention System (Evo SEL)
  • 3.
    Principles Underlying theDESSA Comprehensive System • SEL should be strength-based • SEL should be data-driven • Responsive to the actual needs of your students • Support CQI and Accountability • SEL should encourage local decision making (variation within form) • Teachers and staff know their students best
  • 4.
    How Can S/EAssessment Improve Practice? • High-quality assessment can: • Enhance awareness of each student’s unique S/E strengths and needs • Guide instruction to address the needs that actually exist • Enable teachers and pupil personnel staff to honor, maintain, and leverage student strengths • Assist in developing rapport with parents and students • Identify meaningful strengths for IEPs • Makes possible true prevention
  • 5.
    3 Rs ofQuality Assessment Rigor Reasonableness Relevance
  • 6.
    What are theNeeds & Components? • Universal Screening • Detailed Assessment • SEL Instruction • Progress Monitoring • Outcome Evaluation & Quality Improvement • DESSA-mini • DESSA • DESSA Strategies • Ongoing Progress Monitoring Form (OPM) • Advanced Interpretation Techniques
  • 7.
    The DESSA-mini • Completedby Teachers, OST Staff, Parents • Four equivalent 8-item forms • Completed in 1-2 minutes per student; one planning period per class • Yields one score – Social-Emotional Total Score (SET) • Online via Evo SEL & paper and pencil • English, Spanish, Dutch, and Italian
  • 9.
    How Aperture DescribesSocial and Emotional Skills NB: not a deficit. “Burden” placed on adults to provide instruction.
  • 10.
    DESSA-mini Results • OneScore – Social Emotional Total (SET) • T-Scores • Mean of 50, SD of 10 • Percentiles • Descriptive Terms for T-Score Ranges • > 60 = Strength (≈ 16%) • 41-59 = Typical (≈ 68%) • < 40 = Need for Instruction (≈16%)
  • 11.
    DESSA-mini • DESSA mininormative group • Standardization data for Teacher Raters (N = 1,249) • Region: NE = 24.6%; South = 39.1%; Midwest = 22.3%; West = 14.0% • 50.8% Males • Grades: Kindergarten through 8 • Representative with respect to Race, Hispanic ethnicity, and SES
  • 12.
  • 13.
  • 15.
    Key Findings fromAllentown Study • Sensitivity • Criterion > .50 (Glascoe) • Obtained .63 • Specificity • Criterion > .75 • Obtained .98 • Consistent Classification 87% of the time
  • 16.
    Relationship Between AcademicAchievement and Social-Emotional Competence 38.3 6.4 28.6 12.5 24.7 38.2 8.4 42.9 .0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Need Strength %ofStudents DESSA-mini Category % of Elementary Students by PSSA Math and DESSA-mini Categories Advanced Proficient Basic Below Basic 59.0 15.2 18.2 15.9 21.1 48.9 1.7 20.1 .0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Need Strength %ofStudents DESSA-mini Category % of Elementary Students by PSSA Reading and DESSA-mini Categories Advanced Proficient Basic Below Basic
  • 17.
    Relationship Between AcademicAchievement and Social-Emotional Competence 38.6 10.6 24.7 19.0 27.1 28.2 9.6 42.3 .0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Need Strength %ofStudents DESSA-mini Category % of Middle School Students by PSSA Math and DESSA-mini Categories Advanced Proficient Basic Below Basic 50.3 14.1 23.3 16.2 20.9 35.2 5.5 34.5 .0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0 Need Strength %ofStudents DESSA-mini Category % of Middle School Students by Reading and DESSA-mini Categories Advanced Proficient Basic Below Basic
  • 18.
    3rd Graders SEC(N=148) Variance Explained SBA Reading Low Income SEC 53% 22% 9% 16% • When social-emotional competence (SEC) is added, an additional 22% of SBA variance is uniquely predicted. • Variance attributable to poverty alone reduced to 9% • Together income and social emotional competence account for 47% of variance in reading. • Which variable can we better impact in our schools?
  • 19.
    Explaining the Variancein Academic Achievement Scores - Reading 8.3% 16.5% 75.2% Reading PSSA Scores Income Social Emotional Competence 24.8% Social emotional competence explains an additional 16.5% of the variance in reading scores
  • 20.
    Social-Emotional Competency ofStudents at the Beginning of School Year and Disciplinary Infractions 23.1% 59.1% 17.8% Social-Emotional Categories (DESSA-mini SET) Fall Baseline Grades K-8 Strength Typical Need These students are 450% (4.5X) more likely to be suspended or expelled by the end of the year Can this change our paradigm from reactive to proactive? Intervention to prevention?
  • 22.
    DESSA-mini Summary • AppropriateUses: • Universal screening • RTI approach - repeated “probes” of social and emotional growth • Needs assessment for SEL at the school or district level • Limitations • No information about specific S/E competencies or needs • Doesn’t inform selection of strategies
  • 23.
    The Devereux Student StrengthsAssessment • One, 72-item form • Completed thoughtfully by parents, teachers and OST staff in about 5-8 minutes • Yields 9 scores – 8 specific scales and the Social and Emotional Composite (SEC) • The DESSA allows for • Comprehensive assessment of key S/E skills • Guiding instruction • Outcome evaluation & CQI
  • 24.
    DESSA Scale Structure •Social Emotional Composite • Eight Scales • Self Awareness • Self-Management • Social-Awareness • Relationship Skills • Goal-Directed Behavior • Personal Responsibility • Decision Making • Optimistic Thinking
  • 25.
    DESSA Scales andCASEL Alignment
  • 26.
    Purposes of theDESSA • Identify social-emotional strengths and needs of elementary and middle school children. (high school coming 2018-19 School Year) • Tier I – (if used at universal level) Produce classroom profiles that guide classroom-wide prevention/promotion strategies. • Tier II - Assess at-risk children so that targeted small group or individual interventions can occur. • Tier III - For special education students, identify important strengths that can be incorporated into IEPs and leveraged in practice.
  • 27.
    Additional Purposes ofthe DESSA • Foster collaboration between parents and teachers • Document outcomes for individual students, classrooms, and communities
  • 28.
    DESSA Raters • Ratersprovide the ratings • Teachers, After School Staff • Parents • Read at about the 6th grade level • Sufficient opportunity to observe the child • The DESSA-HSE will have a youth self-report form as well
  • 29.
    Minimizing the MinimalTeacher Rater Bias • Teachers are very good observers of student behavior • Problem behavior oriented scales have used for years • Shapiro et al – teacher rater bias accounts for 16% of variance in scores • Can be reduced to 10% through preservice training
  • 30.
    DESSA Results • T-Scores •Mean of 50, SD of 10 • Percentiles • Descriptive Terms for T-Score Ranges • > 60 = Strength • 41-59 = Typical • < 40 = Need for Instruction • Individual Student Profile • Classroom Profile
  • 31.
  • 32.
    How many wayscan this be used?
  • 33.
    Individual Item Analysis SignificantItem Score - Strength #18 – ask to take on additional work or responsibilities? Never Rarely Occasionally Frequently Very Frequently #37 – follow the example of a positive role model? Nonsignificant Item Score - Typical Significant Item Score - Need #69 – use available resources (people or objects) to solve a problem? Never Rarely Occasionally Frequently Very Frequently Never Rarely Occasionally Frequently Very Frequently
  • 34.
    DESSA Strategies • Providedas part of Aperture Ed’s Evo SEL web-based platform • 5 different levels of strategies for each of the eight DESSA scales – Teacher Reflection & Action – Universal – Group – Individual Student – Home • 3 different age groupings: primary, intermediate elementary, and middle school
  • 36.
    DESSA Strategies • Strand1 Evidence-based and field-tested strategies • Strand 2 – Select strategies from SEL curricula – Second Step, 4 Rs, Caring Classroom Community, Open Circle • Strand 3 – Foundational SEL practices – Learning agreements - Gratitude – Greeting rituals - Movement – Trauma-informed Practices - Student Voice – Peer Coaching
  • 37.
    Response to Intervention •Core components of RTI • Universal screening • Classroom-based interventions • Frequent monitoring of progress • Has been very successful in preventing academic failures • Why wouldn’t we take the same approach with social and emotional competency?
  • 38.
  • 39.
    Pretest-Posttest Comparisons Pretest-Posttest Comparison Time 1T-Score Posttest Confidence Range Time 2 T- Score Outcome – Check One Significant Decline No Change Significant Increase Personal Responsibility 58 50-65 62 X Optimistic Thinking 39 31-49 46 X Goal-Directed Behavior 51 44-58 50 X Social Awareness 36 28-49 52 X Decision Making 48 40-56 38 X Relationship Skills 58 51-64 62 X Self-Awareness 40 32-50 57 X Self-Management 53 45-60 59 X Social-Emotional Composite 47 44-50 54 X Rater 1 Name: ________________ Rater 2 Name: ________________ Date of Rating: ________________ Date of Rating: ________________ Mary Smith Mary Smith 10/10/14 2/28/15
  • 40.
    Footer School-Wide Outcome Evaluation SchoolName: __________________________ James Madison Elementary Date Range: _____________9/1/12 – 6/15/13 Outcome Scale # in Need at Pretest % in Need at Pretest # (%) Significant Decline at Posttest # (%) No Change at Posttest # (%) Significant Improvement at Posttest Comments Self-Awareness 72 20% 7 (10%) 29 (40%) 36 (50%) Moderate issue, moderate effectiveness. Target for QI Self-Management 36 10% 6 (17%) 24 (67%) 6 (17%) Smaller issue, limited effectiveness. Target for QI. Relationship Skills 72 20% 0 (0%) 15 (21%) 57 (79%) High effectiveness Optimistic Thinking 120 33% 6 (5%) 30 (25%) 84 (70%) Major issue for school; High effectiveness Social-Emotional Composite 94 26% 10 (11%) 25 (26%) 59 (63%) Nearly 2/3 of students showed overall improvement during year Total Number of Students: ___360
  • 41.
    Utilization of theDESSA Comprehensive System with a Tiered Approach • Tier I – Universal - Primary Prevention • Universal Screening with DESSA-mini • Consider Universal Assessment with the DESSA • Classroom Profile • Identifies common areas of strength and concern • Leads to classroom-wide strategies • Tier II – Targeted - Secondary Prevention • Identification of children at-risk due to S/E Needs • Individual Child Profile • Targeted Strategies
  • 42.
    Utilization of theDESSA (cont.) • Tier III – Indicated - Tertiary Prevention • Identification of strengths and needs in identified children • Incorporated into IEPs • Strong basis for collaboration with parents • Program Evaluation • Evaluate progress at the scale, child, and classroom level • Basis for quality improvement and continuing education efforts
  • 43.
    Applications of DESSA Teachers& School Psychologists • Teachers • Screen & Assess • Guide Selection of Strategies • Enhance SEL Curricula • School Psychologists • Complement deficit oriented assessments • Set Goals and Objectives for pull-out groups • Item Level Analysis to identify targets & strengths • Classroom Consultation Tool
  • 44.
    Ways the DESSAadds Value • Meet requirements in states with social- emotional learning standards • Assess competencies related to academic achievement • Support implementation of SEL and similar programs • Provides a wealth of competence building strategies • Meet requirements to do meaningful strength- based assessment, especially with IEPs • Contribute to our understanding of disorders
  • 45.
    Thank You! Contact Information: PaulLeBuffe, VP of Research & Development Aperture Education plebuffe@ApertureEd.com (704) 644-8676 www.ApertureEd.com