Your SlideShare is downloading. ×
Final msa nais merged presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Final msa nais merged presentation

2,065
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,065
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Mission Skills Assessment Assessing Character Traits in Middle School Students Independent Schools Data Exchange 1 INDEX Mission Skills Assessment www.indexgroups.org 617-413-8382
  • 2. Mission Skills Assessment  Mission Skills Assessment: A Tool To Alter The Way Schools Think About EducationMSA measures six character strengths—Teamwork, Creativity, Ethics,Resilience, Curiosity, Time Management—that have proven essentialfor success in school and in life. The assessment gives each school a morescientific way to measure its curriculum’s success at meeting the goalsoutlined in its mission. INDEX Mission Skills Assessment 2
  • 3. Who We Are•  INDEX Elementary Schools: National benchmarking and best practices group of 28 Independent PS – 8/9 co- educational day schools with 400+ students.•  Center for Academic and Workforce Readiness and Success, Educational Testing Services (ETS). INDEX Mission Skills Assessment 3
  • 4. MSA SchoolsDuke School (NC) Old Trail School (OH)Episcopal Day School (GA) The Pike School (MA)Far Hills Country Day School (NJ) Rippowam Cisqua School (NY)The Foote School (CT) Rodeph Sholom School (NY)Grace-Saint Luke’s Episcopal School (TN) Rumson Country Day School (NJ)Green Vale School (NY) Saint Martin’s Episcopal School (GA)Greenwich Country Day School (CT) Shady Hill School (MA)The Lexington School (KY) Shore Country Day School (MA)Marin Country Day School (CA) St. Patrick’s Episcopal School (DC)New Canaan Country School (CT) Trinity Episcopal School (NC) INDEX Mission Skills Assessment 4
  • 5. What is the Mission Skills Assessment The MSA is a longitudinal assessment measuring and benchmarking student achievement and improvement in core mission skills. •  Teamwork •  Creativity •  Ethics •  Resilience •  Curiosity •  Time Management INDEX Mission Skills Assessment 5
  • 6. Roadmap•  R&D •  Cognitive Labs •  Fall Test •  Fall Test •  Fall Test•  ETS •  Pilot Test •  Spring Test •  2nd Report •  INDEX Schools•  Constructs •  Horizons •  3rd Report • 1st Report INDEX Mission Skills Assessment 6
  • 7. The Assessment•  Web-based.•  Student self-assessment •  2 30-minute tests for approximately 60 minutes total. •  Assess once per year. •  SJT and other student-completed performance measures. •  6th, 7th, and 8th graders. •  The test is fun! There are no wrong answers.•  Teacher-rater assessment •  Teacher rates each student individually•  Outcome data (e.g. test scores, grades, absences, etc.)•  Institutional focus. No tracking of individual performance. INDEX Mission Skills Assessment 7
  • 8. Noncognitive Skills Research INDEX Mission Skills Assessment 8
  • 9. What are Noncognitive skills?•  Not traditionally “tested” by standardized assessments•  Not new but newly important; labeled “21st century skills”•  Essential capacities necessary for success in school and in life•  Often taught implicitly, but can be taught explicitly INDEX Mission Skills Assessment 9
  • 10. Social &Attitudes & Habits & Emotional Personality Beliefs Processes Qualities Attitudes Emotion Time Openness to(evaluations) Management Management Experience Intelligence Test Anxiety Goal Setting Conscientiousness Theories Test-takingSelf-Efficacy Self-Regulation Extraversion Strategies Coping withSelf-Concept Study Skills Agreeableness Stress Emotional Confidence Teamwork Self-Monitoring Stability Lipnevich, MacCann, & Roberts (2011) INDEX Mission Skills Assessment 10
  • 11. Disposition Toward School GPA Class Discipline Infirmary Absences VisitsSSAT .27 -.04 -.05 -.04Conscientiousness .21 -.31 -.23 -.22Teacher ratings .38 -.35 -.42 -.26(noncognitive) INDEX Mission Skills Assessment 11
  • 12. Grades rho Primary Secondary TertiaryConscientiousness .28 .21 .23Openness .24 .12 .07Agreeableness .30 .05 .06Emotional Stability .20 .01 -.01Extraversion .18 -.01 -.03Cognitive ability .58 .24 .23 Poropat, A. (2009). Psychological Bulletin INDEX Mission Skills Assessment 12
  • 13. Academic SuccessNoncognitive skills correlate positivelywith achievement on NAEP and PISA.Noncognitive factors predict college gradesas strongly as cognitive variables do.ETS, (2008); Poropat, (2009) INDEX Mission Skills Assessment 13
  • 14. Life TrajectoryEarly childhood interventions do notraise IQ, but improve noncognitiveskills which affect education,employment, earnings, and crime.Heckman et al. (2010) INDEX Mission Skills Assessment 14
  • 15. Life SatisfactionNoncognitive variables…have beendemonstrated to predict happiness,health, marital satisfaction, and peerrelationships.Diener & Lucas (1999); Bogg & Roberts (2004); Watson, Hubbard, & Wiese, (2000); Jensen-Campbell et al. (2002) INDEX Mission Skills Assessment 15
  • 16. DemographicsThe benefits of higher noncognitive skillsare demonstrable across IQ, socio-economic class, gender, and race.Tough, Paul. How Children Succeed: Grit, Curiosity and the Hidden Power of Character.(2012) INDEX Mission Skills Assessment 16
  • 17. Ethics Love of Learning Resilience 100% 73% 67% Analysis of mission statements of 20 independent middle schoolsCreativity Teamwork 67% 40% INDEX Mission Skills Assessment 17
  • 18. Analysis of university mission statements (Oswald et. al., 2004) Intellectual Interpersonal IntrapersonalKnowledge and mastery Appreciation for Social responsibility and of general principles diversity citizenship Continuous learning, Leadership Physical andintellectual interest and psychological health curiosity Artistic and cultural Interpersonal skills Career orientation appreciation Adaptability and life skills Perseverance Ethics and integrity INDEX Mission Skills Assessment 18
  • 19. * * ** “Very Important”Percent Rating * * The Conference Board, Corporate Voices for Working Families, Partnership for 21st Century Working Skills, and the Society for Human Resource Management. INDEX Mission Skills Assessment 19
  • 20. The Character Imperative1) Research shows that these skills lead topositive educational and life outcomes;2) Colleges, universities, and employers areseeking these skills in students andworkers; and3) We must demonstrate the value we add. INDEX Mission Skills Assessment 20
  • 21. Guiding Children to Achieve their Full PotentialStudents are provided with a distinctadvantage if their schools excel in trainingnoncognitive skills. Independent Schoolscan lead the way in designing innovativecharacter assessments and intentionalclassroom strategies. INDEX Mission Skills Assessment 21
  • 22. Constructs in MSA INDEX Mission Skills Assessment 22
  • 23. TeamworkModels of teamwork include 100 •  Cooperation 90 •  Influence 80 •  Conflict resolution •  Guiding others 702nd most valued work 60attribute according to Are they 50Really Ready to Work? report Ma th IT Wr E E C R W L C L O T itte thics nglis ritic ead ritin ifelo reat eade ral C eam ork nC h al Th ing g ng Le ivit y rsh W om wor Eth om ink Com arn ip . k ic . ing p ingIn high school students, teacher reports of teamwork were correlated with gradesacross several subjects (Wang, MacCann, Zhuang, Liu, & Roberts, 2009) INDEX Mission Skills Assessment 23
  • 24. Creativity•  Originality, progressiveness, or imagination •  Dictionary.com•  Tasks measuring creativity, “require examinees fairly quickly to think of…a series of responses fitting the requirements of the task….” •  Carroll, 1993; p. 428 INDEX Mission Skills Assessment 24
  • 25. Creativity 100 905th most valued workattribute according to Are 80they Really Ready to Work?report 70 60 50 Ma IT Wr E E C R Wr Lif C L Or Te Wo th itte thics nglis ritic ead i elo rea ead a a nC h al Th ing ting ng tivi ty ers l Co mwo rk E thi Le hip m. rk om ink Com a rn c . ing p ing Creativity predicts graduate school outcomes beyond GRE verbal and mathematics scores (Frederickson & Ward, 1975; Bennett & Rock, 1995; Bennett & Rock, 1998) INDEX Mission Skills Assessment 25
  • 26. Ethics 100Defined in our study ascheating and concern forothers 90One of the most valued work 80attribute according to Are theyReally Ready to Work? report 70 60 50 Ma IT Wr E E C R W L C L O T W th itte thics nglis ritic ead ritin ifelo reat eade ral C eam ork nC h al ing g ng ivit rsh om wor Eth Th Le y om ink Com a rn ip . k ic . ing p ing INDEX Mission Skills Assessment 26
  • 27. Resilience•  A level of adaptability that allows individuals to survive and thrive in adverse conditions •  Dent and Cameron (2003)•  One way we measure resilience in the MSA is to measure core self-evaluation – or how capable, worthy, and effective a person feels. INDEX Mission Skills Assessment 27
  • 28. 2002 Income 1979 Core Self Evaluation Judge and Hurst (2007) INDEX Mission Skills Assessment 28
  • 29. Intrinsic Motivation / Love of Learning / Curiosity•  What people will do without external inducement (Malone & Lepper, 1987)•  In a study of 3rd – 8th graders, intrinsic motivation was positively correlated with: •  Grades: r = .34 •  Standardized Test Scores = r = .27•  However… INDEX Mission Skills Assessment 29
  • 30. (Lepper, Corpus, & Iyengar, 2005) INDEX Mission Skills Assessment 30
  • 31. Time Management Determining one’s needs, setting goals to meet needs, and prioritizing and planning to meet goals (Lakein, 1973). Planning OrganizationGrades .21** .38**Grades .25** .42**(6 months later) Liu, Rijmen, MacCann, & Roberts (2009) INDEX Mission Skills Assessment 31
  • 32. How are we doing it? INDEX Mission Skills Assessment 32
  • 33. Our GPS: Multi-Trait Multi-Method (MTMM) Design Self Ratings “Triangulation” Not directly observable Measurement True True Method Resiliency Resilience Alternative Teacher Method RatingsThe underlying view of measurement in the MTMM analysis is that to measure a theoretical construct,different measures, each with its own bias, are selected. Bias that is due to method effects is reduced through atriangulation process. Baron & Kashy, 1992, p. 170 INDEX Mission Skills Assessment 34
  • 34. Item TypesINDEX Mission Skills Assessment 35
  • 35. Self-Report INDEX Mission Skills Assessment 36
  • 36. Situational Judgment Test INDEX Mission Skills Assessment 37
  • 37. Biographical Data INDEX Mission Skills Assessment 38
  • 38. Creativity Performance Test INDEX Mission Skills Assessment 39
  • 39. Teacher Ratings INDEX Mission Skills Assessment 40
  • 40. Resilience: Representative Items Self-ReportSituational Judgment Test I overcome challenges and setYou are feeling stressed about the amount of backs.homework that you have been given by yourteacher(s). Below are some ways that youmight think, feel, or act in this situation, rightat the time that you feel stressed-out. Rate how Resilienceoften you do each activity when you feelstressed. How do you think, feel, or act whenyou are stressed from having too muchhomework to do? Teacher Ratings1.  I take control and say to myself: I can do This student is easily this! discouraged. INDEX Mission Skills Assessment 41
  • 41. What did we find? INDEX Mission Skills Assessment 42
  • 42. Participation•  Fall 2011 •  18 INDEX schools •  2,081 students•  Spring 2012 (optional assessment) •  13 INDEX schools •  1,677 students•  Fall 2012 •  20 INDEX schools •  > 2,600 students •  Data being analyzed INDEX Mission Skills Assessment 43
  • 43. AverageConstructScoresByGender 103 102 101AverageConstructScore 100 99 98 Male 97 Female 96 95 94 2011 2012 2011 2012 2011 2012 2011 2012 2011 2012 2011 2012 Creativity Ethics Intrinsic9Motivation Resilience Teamwork Time9Management Construct 44 INDEX Mission Skills Assessment
  • 44. The MSA DemonstratesEvidence of Reliability INDEX Mission Skills Assessment 45
  • 45. Reliability 2011 2012 Construct! Reliability! Reliability!Time Management! 0.86! 0.86! SAT ReliabilityResilience! 0.84! 0.85! Math .91Intrinsic Motivation! 0.89! 0.90!Creativity! 0.88! 0.88! Critical Reading .91Teamwork! 0.77! 0.85!Ethics! 0.90! 0.85! INDEX Mission Skills Assessment 46
  • 46. The MSA Demonstrates Evidence of Validity INDEX Mission Skills Assessment 47
  • 47. The MSA Predicts•  Grades•  Teacher Ratings of Student Quality •  “How would you rate this student compared to other students in your school?” INDEX Mission Skills Assessment 48
  • 48. r = .56 Creativity and Student Quality Rating 42012 Student Quality Rating 3.5 3 2.5 2 1.5 1 0.5 0 E - Bottom D - 20%-39% C - 40%-59% B - 60%-79% A - Top 20% 20% 2011 Creativity INDEX Mission Skills Assessment 49
  • 49. How “Big” Are Our Correlations?Examples from other research (Meyer et al, 2001) Variable 1 Variable 2 CorrelationSugar Consumption Child’s Behavior .00 Aspirin Heart Attack Death .02 Parental Divorce Child Well-Being .09 Lead Exposure Child IQ .12 Ibuprofen Pain Reduction .14 Alcohol Aggressive .23 Behavior Viagra Sexual Functioning .38 Gender Height .67 INDEX Mission Skills Assessment 50
  • 50. 6th r = .29 7th r = .59 8th r = .58 Time Management and GPA by Grade 4 3.8 3.6 3.4Spring 2012 GPA 3.2 3 6 2.8 7 2.6 2.4 8 2.2 2 E - Bottom D - 20%-39% C - 40%-59% B - 60%-79% A - Top 20% 20% 2011 Time Management INDEX Mission Skills Assessment 51
  • 51. The MSA Predicts•  Student Well Being • My#life#is#going#well# • My#life#is#just#right# • I#would#like#to#change#many#things#in#my#life# • I#wish#I#had#a#different#kind#of#life# • I#have#a#good#life# • I#have#what#I#want#in#life# • My#life#is#be;er#than#most#kids’# INDEX Mission Skills Assessment 52
  • 52. r = .33 Resilience!and!Well!Being! 5! 4.9! 4.8!2012!Well!Being! 4.7! 4.6! 4.5! 4.4! 4.3! 4.2! 4.1! A!-!Top!20%! B!-!60%-79%! C!-!40%-59%! D!-!20%-39%! E!-!Bo7om!20%! 2011!Resilience! INDEX Mission Skills Assessment 53
  • 53. r = .31 Ethics and Well Being 5 4.9 4.82012 Well Being 4.7 4.6 4.5 4.4 4.3 4.2 4.1 A - Top 20% B - 60%-79% C - 40%-59% D - 20%-39% E - Bottom 20% 2011 Ethics INDEX Mission Skills Assessment 54
  • 54. There is initial evidence that the MSA predicts just as well as standardized tests INDEX Mission Skills Assessment 55
  • 55. MSA and ERB Scores: Correlations with Student Quality RatingsCorrelation with Student Quality 0.6 0.50 0.5 0.46 0.45 0.44 0.4 0.38 0.31 0.3 0.2 0.16 0.15 0.1 0 INDEX Mission Skills Assessment 56
  • 56. MSA and ERB Scores: Correlations with GPA 0.7 0.59 0.6 0.56Correlation with GPA 0.52 0.49 0.5 0.41 0.4 0.28 0.27 0.3 0.21 0.2 0.1 0 INDEX Mission Skills Assessment 57
  • 57. MSA and ERB Scores: Correlations with Well Being 0.3 0.28Correlation with Well Being 0.25 0.23 0.22 0.22 0.19 0.2 0.14 0.15 0.1 0.04 0.05 0.01 0 INDEX Mission Skills Assessment 58
  • 58. Current and Next Steps•  Continual refinement of MSA assessment (adding interests)•  Assessing underserved populations with MSA•  Gathering interest from additional schools INDEX Mission Skills Assessment 59
  • 59. Implications for Schools INDEX Mission Skills Assessment 60
  • 60. Implications for Schools•  Wide-ranging and deep:•  Why?•  What?•  How? INDEX Mission Skills Assessment 61
  • 61. Why? Mission and Vision•  Reexamine school mission and how measured•  Strength of mind and strength of character•  Develop a Vision for Character INDEX Mission Skills Assessment 62
  • 62. How? Programs- intentional and well-designedHigh-quality Evidence-training and based support Evaluation INDEX Mission Skills Assessment 63
  • 63. What? ProgramsHow can we teach these skills better and how will we know weare succeeding?•  Deliberate, explicit and implicit instruction•  Proven interventions•  Performance Character Audit•  Articulated, spiraled, integrated curriculum•  Grade-level goals•  Updated and analyzed curriculum maps INDEX Mission Skills Assessment 64
  • 64. Programs•  Social Emotional Learning, advisories, personal awareness, and co-curricula•  Shared lexicon, common language-Glossary of Terms for Character•  Best Practices-follow through, evaluation, refinement INDEX Mission Skills Assessment 65
  • 65. What? Training and Support•  Ongoing education–school leaders, faculty, parents•  Shared vision, vocabulary, and resources•  Participation in regional and national educator character summits•  Ongoing partnerships and dialogue INDEX Mission Skills Assessment 66
  • 66. What? Evidence-based Evaluation•  Updated Progress Reports•  Parent and student conferences•  Faculty and parent education summits•  Greater emphasis: Self-reflection and meta-cognition•  Surveys INDEX Mission Skills Assessment 67
  • 67. Evidence-based Evaluation•  Integrated assessments•  Participation in new assessments (MSA)•  Benchmarking•  Data for national accreditation and new criterion INDEX Mission Skills Assessment 68
  • 68. Measuring what we value: Criterion 13:The Standards require a school to provide evidenceof a thoughtful process, respectful of its mission,for the collection and use in school decisionmaking of data (both external and internal) aboutstudent learning. INDEX Mission Skills Assessment 69
  • 69. Improvement PhasesEnvision Implement Evaluate and RefineCommitment tied to Needs and resources Professionalmission assessment Development & PartnershipsCreate clear vision Action Plans, Communication and integration, and engagement enhancementsBuild engagement Evidence-based •  What is working? strategies and programs •  How do we know? •  How can we improve teaching and learning? INDEX Mission Skills Assessment 70
  • 70. Community Responses•  Excitement!•  Shared understanding•  Partnership•  Shared commitment to studying results and asking the next questions… INDEX Mission Skills Assessment 71
  • 71. Your School, Your MissionLook to your mission, look to your alumni, lookto your purpose and your school’s view of whatour society needs the most: creative andinnovative thinkers, people of fine judgment,leaders, collaborators, and…then select or designthe instruments which will help you make thesepredictions.Martin, Jonathan. Lessons from Law School Admissions: More from USC Attributes that Matter, 21K12Blog, January 23, 2013. INDEX Mission Skills Assessment 72
  • 72. Thank you! Questions? Comments? FOR MORE INFORMATION, PLEASE CONTACT: INDEX INDEPENDENT SCHOOL DATA EXCHANGE LISAPULLMAN@INDEXGROUPS.ORG WWW.INDEXGROUPS.ORGMSA Steering Committee:Chuck Baldecchi Dave Michelman Lisa PullmanHead of School Head of School Executive Director, INDEXThe Lexington School (KY) Duke School (NC) lisapullman@indexgroups.orgCBaldecchi@thelexingtonschool.org Dave.Michelman@dukeschool.org   Rich RobertsTim Bazemore Ned Murray Managing Principal Research ScientistHead of School Head of School ETSNew Canaan Country School (CT) Episcopal Day School (GA) rroberts@ets.orgTBazemore@countryschool.net nmurray@edsaugusta.comJeremy Burrus Jennifer PhillipsResearch Scientist Director of Educational AdvancementETS Far Hills Country Day School (NJ)jburrus@ets.org jphillips@fhcds.org INDEX Mission Skills Assessment 73