Users’ Group 2013
Best Practices
Julie Owen, VP Marketing & Operations
SmarterServices
Provided by
Remember When?
DID YOU KNOW?
• SmarterMeasure has helped over 2.4
MILLION students!
• We have a k12 version that serves 7-12
graders.
• You can modify your readiness ranges to
make the tool more predictive.
Best
•
•
•
•
•
•
•

Practices

Popularize – buy in
Plan – strong implementation model
Personalize – use custom features
Petition - Call to Action
Pursue - Follow Through
Produce – Conduct Research
Present– co-present with us at a
conference or webinar
POPULARIZE/PROMOTE
• Advertise & Promote
• Educate with our Training
• Get Everyone Involved
PLAN
• Integrate SmarterMeasure into a required
orientation course
• 68% of client schools administer the
assessment to all students, not just
eLearning students
J. Sargeant Reynolds
Community College
• Required as admissions
assessment
• Integral part of their QEP
• Computed correlations
with grades and
Life
Factors
SmarterMeasure
sub-scales of over 4000
students.
• P

Attributes

Grades

Learning
Styles

Technical
Findings
• Statistically significant correlations:
- Dedicated place, support from employers
and family, access to study resources, and
academic skills (Life Factors)
- Tech vocabulary (Technical Knowledge)
- Procrastination (Individual Attributes)
Scores

Grades
Academic Success Rates
70
60
50
40

High Score
Low Score

30
20
10
0
Skills

Resources

Time

Less than 10% of students with low scores
experienced academic success.
PERSONALIZE
• Utilize custom features
• Modify Readiness Ranges
Student Login Page
Adjusting Readiness Ranges
Adjusting the cut
points can make
the reporting a
more accurate
predictor of
success.
PETITION
Add call to action in text of student report
Read Feedback on report
Write reflection paper on results
Work through appropriate resources for
support
PETITION
Add call to action in text of student report
Read Feedback on report
Write reflection paper on results
Work through appropriate resources for
support
Discuss results with administrator
Discussion or chat about learning style
PURSUE
• After measuring student
readiness, what action will you take
based
on the scores?
Coaching
Assign mentor
Ongoing communication
Provide faculty member with results
Evaluate & Provide remedial
resources
PRODUCE
Progression of SmarterMeasure Data Utilization
Predictive
Correlation
Comparison
Descriptive

Student Service
Research
Ideas on the
Research
Page of the
Website
CEC - The Need
• We need to know which students to advise
to take online, hybrid or on-campus
courses.
• We need to know which students to direct
to which student services to help them
succeed.
• We need to know how to best design our
courses so that new students are not
overwhelmed.
The Analysis
• What is the relationship between
measures of student readiness and
variables of:
– Academic Success - GPA
– Engagement – Survey (N=587)
– Satisfaction – Survey (Representative Sample
based on GPA and number of courses taken
per term)
– Retention – Re-enrollment data
The Findings
• Statistically Significant Relationships
Academic
Achievement

Engagement

Retention

Individual
Attributes

X

X

X

Technical
Knowledge

X

X

X

X

X

Learning
Styles
Life Factors
Technical
Competency

X

X
X
The Recommendations
• We need to know which students to advise
to take online, hybrid or on-campus
courses.
– A profile of a strong online student is one
who:
•
•
•
•

Has a dedicated place to study online
Possesses strong time management skills
Demonstrates strong technical skills
Exhibits a strong vocabulary of technology terms
The Recommendations
• We need to know which students to direct
to which student services to help them
succeed.
– An online student who should be directed toward
remedial/support resources is one who:
• Has a weak reason for returning to school
• Has weak prior academic skills
• Is not likely to seek help on their own
• Is prone to procrastinate
• Has low, internal locus of control
• Has weak technology skills
The Recommendations
• We need to know how to best design our
courses so that new students are not
overwhelmed.
– Limit advanced technology in courses offered early in
a curriculum
– Foster frequent teacher to student interaction early in
the course
– Require milestones in assignments to prevent
procrastination
– Clearly provide links to people/resources for
assistance
Middlesex Community College
• 6% to 13% more students failed online
courses than on-ground courses.
• Intervention Plan
- Administer SmarterMeasure
- Identify which constructs best predicted success
- Provide “Success Tips” as identified
Distributed by website, email, orientation
course, records office, library, posters, and
mail
Research Findings
• Analyzed 3228 cases over two years
• Significant positive correlation between
individual attributes and grades

Motivation

Impacts

Grades
Results of Middlesex Research
Before SmarterMeasure™ was implemented, 6%
to 13% more students failed online courses than
students taking on-ground courses. After the
implementation, the gaps were narrowed: 1.3%
to 5.8% more online students failed than onground students.
Results of Middlesex Research
Failure rates reduced by as much as 10%
PRESENT
• http://www.smarterservices.com/conferences.cfm
• http://www.smarterservices.com/webinars.cfm
Getting Better
Best is defined as better
than all others: of the
highest quality or
standard or the most
excellent type

Sloan C Users' Group Best Practices

  • 1.
    Users’ Group 2013 BestPractices Julie Owen, VP Marketing & Operations SmarterServices Provided by
  • 2.
  • 3.
    DID YOU KNOW? •SmarterMeasure has helped over 2.4 MILLION students! • We have a k12 version that serves 7-12 graders. • You can modify your readiness ranges to make the tool more predictive.
  • 5.
    Best • • • • • • • Practices Popularize – buyin Plan – strong implementation model Personalize – use custom features Petition - Call to Action Pursue - Follow Through Produce – Conduct Research Present– co-present with us at a conference or webinar
  • 6.
    POPULARIZE/PROMOTE • Advertise &Promote • Educate with our Training • Get Everyone Involved
  • 7.
    PLAN • Integrate SmarterMeasureinto a required orientation course • 68% of client schools administer the assessment to all students, not just eLearning students
  • 8.
    J. Sargeant Reynolds CommunityCollege • Required as admissions assessment • Integral part of their QEP • Computed correlations with grades and Life Factors SmarterMeasure sub-scales of over 4000 students. • P Attributes Grades Learning Styles Technical
  • 9.
    Findings • Statistically significantcorrelations: - Dedicated place, support from employers and family, access to study resources, and academic skills (Life Factors) - Tech vocabulary (Technical Knowledge) - Procrastination (Individual Attributes) Scores Grades
  • 10.
    Academic Success Rates 70 60 50 40 HighScore Low Score 30 20 10 0 Skills Resources Time Less than 10% of students with low scores experienced academic success.
  • 11.
    PERSONALIZE • Utilize customfeatures • Modify Readiness Ranges
  • 12.
  • 13.
    Adjusting Readiness Ranges Adjustingthe cut points can make the reporting a more accurate predictor of success.
  • 14.
    PETITION Add call toaction in text of student report Read Feedback on report Write reflection paper on results Work through appropriate resources for support
  • 16.
    PETITION Add call toaction in text of student report Read Feedback on report Write reflection paper on results Work through appropriate resources for support Discuss results with administrator Discussion or chat about learning style
  • 18.
    PURSUE • After measuringstudent readiness, what action will you take based on the scores? Coaching Assign mentor Ongoing communication Provide faculty member with results Evaluate & Provide remedial resources
  • 19.
    PRODUCE Progression of SmarterMeasureData Utilization Predictive Correlation Comparison Descriptive Student Service
  • 20.
  • 21.
    CEC - TheNeed • We need to know which students to advise to take online, hybrid or on-campus courses. • We need to know which students to direct to which student services to help them succeed. • We need to know how to best design our courses so that new students are not overwhelmed.
  • 22.
    The Analysis • Whatis the relationship between measures of student readiness and variables of: – Academic Success - GPA – Engagement – Survey (N=587) – Satisfaction – Survey (Representative Sample based on GPA and number of courses taken per term) – Retention – Re-enrollment data
  • 23.
    The Findings • StatisticallySignificant Relationships Academic Achievement Engagement Retention Individual Attributes X X X Technical Knowledge X X X X X Learning Styles Life Factors Technical Competency X X X
  • 24.
    The Recommendations • Weneed to know which students to advise to take online, hybrid or on-campus courses. – A profile of a strong online student is one who: • • • • Has a dedicated place to study online Possesses strong time management skills Demonstrates strong technical skills Exhibits a strong vocabulary of technology terms
  • 25.
    The Recommendations • Weneed to know which students to direct to which student services to help them succeed. – An online student who should be directed toward remedial/support resources is one who: • Has a weak reason for returning to school • Has weak prior academic skills • Is not likely to seek help on their own • Is prone to procrastinate • Has low, internal locus of control • Has weak technology skills
  • 26.
    The Recommendations • Weneed to know how to best design our courses so that new students are not overwhelmed. – Limit advanced technology in courses offered early in a curriculum – Foster frequent teacher to student interaction early in the course – Require milestones in assignments to prevent procrastination – Clearly provide links to people/resources for assistance
  • 27.
    Middlesex Community College •6% to 13% more students failed online courses than on-ground courses. • Intervention Plan - Administer SmarterMeasure - Identify which constructs best predicted success - Provide “Success Tips” as identified Distributed by website, email, orientation course, records office, library, posters, and mail
  • 28.
    Research Findings • Analyzed3228 cases over two years • Significant positive correlation between individual attributes and grades Motivation Impacts Grades
  • 29.
    Results of MiddlesexResearch Before SmarterMeasure™ was implemented, 6% to 13% more students failed online courses than students taking on-ground courses. After the implementation, the gaps were narrowed: 1.3% to 5.8% more online students failed than onground students.
  • 30.
    Results of MiddlesexResearch Failure rates reduced by as much as 10%
  • 31.
  • 32.
  • 33.
    Best is definedas better than all others: of the highest quality or standard or the most excellent type