Your SlideShare is downloading. ×
0
Measuring Grit  Do Non-Cognitive Attributes Impact AcademicSuccess, Engagement, Satisfaction and Retention?              D...
Measuring Grits• We recommend a glass beaker for  measuring grits. We are from Alabama,  we know about these things.
What is Grit?• Why does one student who had straight  A’s in high school drop out of college after  one year, and another ...
What is Grit?• Grit is that elusive quality that prompts one  student to stick with it while others quit.• For over ten ye...
Some Students Seem To Have   More Grit Than Others
Three Approaches to         Measuring Grit• Stick your head in the sand.• Use a brief, non-prescriptive survey.• Use Smart...
What is SmarterMeasure?• A 124-item online skills test and attributes  inventory that measures a student’s level  of readi...
What Are The Ingredients     In Our Grits?
What Does The Assessment Measure?   INTERNAL             EXTERNAL                 SKILLS   INDIVIDUAL         LIFE FACTORS...
How Does The Beaker Work?
How Do You Spot a Bad Grit?
Adjusting Readiness RangesAdjusting the cutpoints can makethe reporting amore accuratepredictor ofsuccess.
How Do Schools Use It?•   Orientation Course•   Enrollment Process•   Information Webinar•   Public Website•   Class Parti...
Thermometer Analogy• More important than taking your child’s  temperature is taking appropriate action  based on their tem...
Measuring the Grits
Progression of SmarterMeasure Data Utilization                  Predictive                  Correlation                 Co...
ResearchIdeas on theResearchPage of theWebsite
Approaches to Research Projects Internally   Company    ProfessionallyConducted     Assisted     Assisted
Middlesex Community College• 6% to 13% more students failed online  courses than on-ground courses.• Intervention Plan  - ...
Research Findings• Analyzed 3228 cases over two years• Significant positive correlation between  individual attributes and...
Results of Middlesex Research Before SmarterMeasure™ was implemented, 6% to 13% more students failed online courses than s...
Results of Middlesex Research Failure rates reduced by as much as 10%
Action Plan• Empower eLearning staff, faculty advisors,  and academic counselors with student  data  Three areas of      M...
Project Summary“In summary, theimplementation ofSmarterMeasurehas helped studentsto achieve betteracademic successby ident...
CEC - The Need• We need to know which students to advise  to take online, hybrid or on-campus  courses.• We need to know w...
The Analysis• What is the relationship between  measures of student readiness and  variables of:  – Academic Success - GPA...
The Analysis• Phase One – Summer 2011   – Included data from all three delivery systems – online,     hybrid and on-campus...
The Findings• Academic Achievement  – The scales of Individual Attributes, Technical    Knowledge, and Life Factors had st...
The Findings• Retention  – The measure of Learning Styles produced a    statistically significant mean difference    betwe...
The Findings• Engagement  – The scales of Individual Attributes and Technical    Competency had statistically significant ...
The Findings• Satisfaction  – Structural equation modeling was used to create    a hypothesized theoretical model to deter...
The Findings• Statistically Significant Relationships                     Academic      Engagement   Retention            ...
The Findings• Student Categorizations  – Enrollment Status    • Positive – active/graduated (34.3%)    • Negative – withdr...
The Findings - Correlates Readiness Domain                                  Readiness Domain Subscales                    ...
The Findings - Predictors Readiness Domains               GPA               F       p     Life Factor            Place and...
The Findings - Predictors Readiness Domains      Credit Hours Earned         F       p     Life Factor                Plac...
The Recommendations• We need to know which students to advise  to take online, hybrid or on-campus  courses.  – A profile ...
The Recommendations• We need to know which students to direct  to which student services to help them  succeed.  – An onli...
The Recommendations• We need to know how to best design our  courses so that new students are not  overwhelmed.  – Limit a...
Argosy University• Required in Freshman Experience course• Students reflect on scores and identify  areas for improvement ...
Argosy University -             COMPARE• Compared the traits, attributes, and skills of the online  and hybrid students.• ...
Argosy University - EXPLORE   • Correlational analysis between SmarterMeasure scores     and student satisfaction, retenti...
Argosy University - TREND• Aggregate analysis of SmarterMeasure data to  identify mean scores for students.• Comparison ma...
Argosy University - APPLY• Findings were shared with the instructional  design and student services groups and  improvemen...
J. Sargeant Reynolds           Community College• Required as admissions  assessment                           Attributes•...
Findings• Statistically significant correlations:  - Dedicated place, support from employers    and family, access to stud...
Academic Success Rates70605040                                               High Score                                   ...
Five SchoolsWhat is the relationship between measures of online student readiness and measures        of online student sa...
Methodology     Incoming vs OutgoingData from 1,611 students who completed both theSmarterMeasure Learning Readiness Indic...
Findings• There were statistically significant  relationships between factors of readiness  and satisfaction.
Comparison to  Compass ScoresNorth Central Michigan College - Petoskey, MI
National Data• 2012 Student Readiness Report• Data from 690,927 students from 324  colleges and universities
Online Learner Demographics•   70% were female•   59% were Caucasian/White•   54% had never taken an online course before•...
Online Learner Demographics•   Dominant Social learning style•   Highly motivated•   Moderate reading skills•   Pressed fo...
Profile of a  Successful Online Student• Four demographic variables have had a  statistically significant higher mean for ...
Profile of a Successful        Online Student• Caucasians have had the highest means  for four years in Technical Knowledg...
Conclusion• Statistically significant relationships exist  between measures of online student  readiness and measures of a...
Conclusion• Students individually benefit and schools  collectively benefit from measuring learner  readiness and appropri...
SmarterMeasure.com
SmarterMeasure ReferencesMiddlesex, Argosy, J. Sargeant Reynolds:http://smartermeasure.com/research/research-results/Noel ...
“Live as if you were to dietomorrow. Learn as if you wereto live forever.” Mahatma Gandhi
For More Info   SmarterServices.com    (877) 499-SMARTERinfo@SmarterServices.com
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Measuring Grit - 2013 SmarterMeasure Presentation
Upcoming SlideShare
Loading in...5
×

Measuring Grit - 2013 SmarterMeasure Presentation

381

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
381
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Measuring Grit - 2013 SmarterMeasure Presentation"

  1. 1. Measuring Grit Do Non-Cognitive Attributes Impact AcademicSuccess, Engagement, Satisfaction and Retention? Dr. Mac Adkins, PresidentProvided by SmarterServices
  2. 2. Measuring Grits• We recommend a glass beaker for measuring grits. We are from Alabama, we know about these things.
  3. 3. What is Grit?• Why does one student who had straight A’s in high school drop out of college after one year, and another one excel?• Why does one single mom with three children graduate Summa Cum Laude and another one drop out?
  4. 4. What is Grit?• Grit is that elusive quality that prompts one student to stick with it while others quit.• For over ten years we have measured levels of grit in over 2,000,000 students at over 500 colleges and universities.• Today I want to share with you the results of research related to the impact that grit has on student success.
  5. 5. Some Students Seem To Have More Grit Than Others
  6. 6. Three Approaches to Measuring Grit• Stick your head in the sand.• Use a brief, non-prescriptive survey.• Use SmarterMeasure Learning Readiness Indicator.
  7. 7. What is SmarterMeasure?• A 124-item online skills test and attributes inventory that measures a student’s level of readiness for studying online• Used by over 500 Colleges and Universities• Taken by over 2,000,000 students
  8. 8. What Are The Ingredients In Our Grits?
  9. 9. What Does The Assessment Measure? INTERNAL EXTERNAL SKILLS INDIVIDUAL LIFE FACTORS TECHNICAL ATTRIBUTES Availability of Time Technology Usage Motivation Dedicated Place Life Application Procrastination Reason Tech VocabularyTime Management Support from Family Computing Access Help Seeking Locus of Control TYPINGLEARNING STYLES Rate Accuracy Visual Verbal ON-SCREEN Social READING Solitary Physical Rate Aural Recall Logical
  10. 10. How Does The Beaker Work?
  11. 11. How Do You Spot a Bad Grit?
  12. 12. Adjusting Readiness RangesAdjusting the cutpoints can makethe reporting amore accuratepredictor ofsuccess.
  13. 13. How Do Schools Use It?• Orientation Course• Enrollment Process• Information Webinar• Public Website• Class Participation• Facebook
  14. 14. Thermometer Analogy• More important than taking your child’s temperature is taking appropriate action based on their temperature.• More important than measuring student readiness is taking appropriate action based on the scores.
  15. 15. Measuring the Grits
  16. 16. Progression of SmarterMeasure Data Utilization Predictive Correlation Comparison Descriptive Student Service
  17. 17. ResearchIdeas on theResearchPage of theWebsite
  18. 18. Approaches to Research Projects Internally Company ProfessionallyConducted Assisted Assisted
  19. 19. Middlesex Community College• 6% to 13% more students failed online courses than on-ground courses.• Intervention Plan - Administer SmarterMeasure - Identify which constructs best predicted success - Provide “Success Tips” as identified Distributed by website, email, orientation course, records office, library, posters, and mail
  20. 20. Research Findings• Analyzed 3228 cases over two years• Significant positive correlation between individual attributes and grades Motivation Impacts Grades
  21. 21. Results of Middlesex Research Before SmarterMeasure™ was implemented, 6% to 13% more students failed online courses than students taking on-ground courses. After the implementation, the gaps were narrowed: 1.3% to 5.8% more online students failed than on- ground students.
  22. 22. Results of Middlesex Research Failure rates reduced by as much as 10%
  23. 23. Action Plan• Empower eLearning staff, faculty advisors, and academic counselors with student data Three areas of Motivation Self Time Discipline Management focus
  24. 24. Project Summary“In summary, theimplementation ofSmarterMeasurehas helped studentsto achieve betteracademic successby identifying their In essence, with various strategies implemented to promotestrengths and SmarterMeasure™, a “culture” wasweaknesses in created during advising and registration for students, faculty, and support staff toonline learning.” know that there is a way for students to see if they are a good fit for learning online.
  25. 25. CEC - The Need• We need to know which students to advise to take online, hybrid or on-campus courses.• We need to know which students to direct to which student services to help them succeed.• We need to know how to best design our courses so that new students are not overwhelmed.
  26. 26. The Analysis• What is the relationship between measures of student readiness and variables of: – Academic Success - GPA – Engagement – Survey (N=587) – Satisfaction – Survey (Representative Sample based on GPA and number of courses taken per term) – Retention – Re-enrollment data
  27. 27. The Analysis• Phase One – Summer 2011 – Included data from all three delivery systems – online, hybrid and on-campus – Analyzed data at the scale level• Phase Two – Fall 2011 – Focused the research on online learners only – Analyzed data at the sub-scale level• A neutral, third-part research firm (Applied Measurement Associates) used the following statistical analyses in the project: – ANOVA, Independent Samples t-tests, Discriminant Analysis, Structural Equation Modeling, Multiple Regression, Correlation.
  28. 28. The Findings• Academic Achievement – The scales of Individual Attributes, Technical Knowledge, and Life Factors had statistically significant mean differences with the measures of GPA.
  29. 29. The Findings• Retention – The measure of Learning Styles produced a statistically significant mean difference between students who were retained and those who left. • A 73% classification accuracy of this retention measure was achieved. – The scales of Individual Attributes and Technical Knowledge were statistically significant predictors of retention as measured by the number of courses taken per term.
  30. 30. The Findings• Engagement – The scales of Individual Attributes and Technical Competency had statistically significant relationships with the four survey items related to Engagement. – The scales of Life Factors, Individual Attributes, Technical Competency, Technical Knowledge, and Learning Styles were used to correctly classify responses to the survey questions related to engagement and satisfaction with up to 93% classification accuracy.
  31. 31. The Findings• Satisfaction – Structural equation modeling was used to create a hypothesized theoretical model to determine if SmarterMeasure scores would predict satisfaction as measured by the survey. – Results indicated that prior to taking online courses, student responses to the readiness variables were statistically significant indicators of later student satisfaction. – Therefore, the multiple SmarterMeasure assessment scores are a predictor of the Career Education survey responses.
  32. 32. The Findings• Statistically Significant Relationships Academic Engagement Retention Achievement Individual X X X Attributes Technical X X X Knowledge Learning X X Styles Life Factors X X Technical X Competency
  33. 33. The Findings• Student Categorizations – Enrollment Status • Positive – active/graduated (34.3%) • Negative – withdrew/dismissed/transfer (65.7%) – Academic Success Status • Passing – A, B or C (48.9%) • Failing – D, F or Other (21.1%) – Transfer Credit – (21.8%) – Not reported – (8.2%)
  34. 34. The Findings - Correlates Readiness Domain Readiness Domain Subscales Positive vs. Negative Pass vs. Fail Life Factor Place, Reason, and Skills Place Learning Styles Social and N/A Logical Personal Attributes Academic, Help Seeking, Procrastination, Time Management Time Management, and Locus of ControlTechnical Competency Internet Competency Internet Competency and Computer CompetencyTechnical Knowledge Technology Usage and Technical Vocabulary Technical Vocabulary
  35. 35. The Findings - Predictors Readiness Domains GPA F p Life Factor Place and Skills 12.35 .0001 Learning Styles Verbal a and Logical 3.95 .02 Personal Attributes Help Seeking, Time Management, and Locus of 21.11 .0001 ControlTechnical Competency Computer and Internet 22.75 .0001 CompetencyTechnical Knowledge Technology Vocabulary 38.76 .0001
  36. 36. The Findings - Predictors Readiness Domains Credit Hours Earned F p Life Factor Place 12.37 .0001 Learning Styles Visual 6.81 .01 Personal Attributes Academic Attributes, Help Seeking, and Locus of 13.40 .0001 ControlTechnical Competency Computer Competency 12.23 .0001 and Internet CompetencyTechnical Knowledge Technology Usage and Technology Vocabulary 26.97 .0001
  37. 37. The Recommendations• We need to know which students to advise to take online, hybrid or on-campus courses. – A profile of a strong online student is one who: • Has a dedicated place to study online • Possesses strong time management skills • Demonstrates strong technical skills • Exhibits a strong vocabulary of technology terms
  38. 38. The Recommendations• We need to know which students to direct to which student services to help them succeed. – An online student who should be directed toward remedial/support resources is one who: • Has a weak reason for returning to school • Has weak prior academic skills • Is not likely to seek help on their own • Is prone to procrastinate • Has low, internal locus of control • Has weak technology skills
  39. 39. The Recommendations• We need to know how to best design our courses so that new students are not overwhelmed. – Limit advanced technology in courses offered early in a curriculum – Foster frequent teacher to student interaction early in the course – Require milestones in assignments to prevent procrastination – Clearly provide links to people/resources for assistance
  40. 40. Argosy University• Required in Freshman Experience course• Students reflect on scores and identify areas for improvement in their Personal Development Plan• Group reflection with others with similar levels of readiness
  41. 41. Argosy University - COMPARE• Compared the traits, attributes, and skills of the online and hybrid students.• Substantial differences between the two groups existed.• Changes were made to the instructional design process for each delivery system. Online Hybrid
  42. 42. Argosy University - EXPLORE • Correlational analysis between SmarterMeasure scores and student satisfaction, retention, and academic successStatistically Significant Motivation Factors: Technical TimeTechnical Competency Motivation Satisfaction Availability of Time. Retention Success
  43. 43. Argosy University - TREND• Aggregate analysis of SmarterMeasure data to identify mean scores for students.• Comparison made to the national mean scores from the Student Readiness Report. National Scores Argosy Scores
  44. 44. Argosy University - APPLY• Findings were shared with the instructional design and student services groups and improvements in processes were made.For example, since technicalcompetency scores increase asthe students take more onlinecourses, the instructionaldesigners purposefully allowedonly basic forms of technology tobe infused into the first coursesthat students take.
  45. 45. J. Sargeant Reynolds Community College• Required as admissions assessment Attributes• Integral part of their QEP• Computed correlations with grades and Life SmarterMeasure Factors Grades Technical sub-scales of over 4000 students.• P Learning Styles
  46. 46. Findings• Statistically significant correlations: - Dedicated place, support from employers and family, access to study resources, and academic skills (Life Factors) - Tech vocabulary (Technical Knowledge) - Procrastination (Individual Attributes) Scores Grades
  47. 47. Academic Success Rates70605040 High Score Low Score302010 0 Skills Resources Time Less than 10% of students with low scores experienced academic success.
  48. 48. Five SchoolsWhat is the relationship between measures of online student readiness and measures of online student satisfaction?
  49. 49. Methodology Incoming vs OutgoingData from 1,611 students who completed both theSmarterMeasure Learning Readiness Indicatorand the Priority Survey for Online Learners wereanalyzed.
  50. 50. Findings• There were statistically significant relationships between factors of readiness and satisfaction.
  51. 51. Comparison to Compass ScoresNorth Central Michigan College - Petoskey, MI
  52. 52. National Data• 2012 Student Readiness Report• Data from 690,927 students from 324 colleges and universities
  53. 53. Online Learner Demographics• 70% were female• 59% were Caucasian/White• 54% had never taken an online course before• 35% were traditional aged college students• 52% were students at an associate’s level institution
  54. 54. Online Learner Demographics• Dominant Social learning style• Highly motivated• Moderate reading skills• Pressed for time• Fast typists• Increasing technical skills
  55. 55. Profile of a Successful Online Student• Four demographic variables have had a statistically significant higher mean for four years in a row.Females higher inIndividualAttributes, AcademicAttributes, and TimeManagement.Males higher in TechnicalKnowledge.
  56. 56. Profile of a Successful Online Student• Caucasians have had the highest means for four years in Technical Knowledge.• Students who have taken five or more online courses have had the highest means for four years in Individual Attributes, Technical Knowledge, and Procrastination.
  57. 57. Conclusion• Statistically significant relationships exist between measures of online student readiness and measures of academic success, engagement, satisfaction and retention.
  58. 58. Conclusion• Students individually benefit and schools collectively benefit from measuring learner readiness and appropriately responding.
  59. 59. SmarterMeasure.com
  60. 60. SmarterMeasure ReferencesMiddlesex, Argosy, J. Sargeant Reynolds:http://smartermeasure.com/research/research-results/Noel Levitz Study: https://www.noellevitz.com/papers-research-higher-education/2011/2011-adult-and-online-learner-satisfaction-priorities-reportsStudent Readiness Report:http://smartermeasure.com/smartermeasure/assets/File/Online-Student-Readiness-Report.pdf
  61. 61. “Live as if you were to dietomorrow. Learn as if you wereto live forever.” Mahatma Gandhi
  62. 62. For More Info SmarterServices.com (877) 499-SMARTERinfo@SmarterServices.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×