This document summarizes a webinar about using data from SmarterMeasure, an assessment tool, to help students and institutions. It discusses three levels of benefit: students learning about themselves, schools learning how to support students, and leaders analyzing data. It then describes how two community colleges currently use SmarterMeasure data in student advising and orientation. The webinar covered adjusting SmarterMeasure's readiness ranges and considering custom questions. It concluded by discussing five ways institutions can maximize their use of SmarterMeasure data: configuring the assessment, addressing bias, customizing questions, classifying data, and creating analytical datasets.

1. WOW, I Did Not Know
I Could Do That!
Tricks for configuring and analyzing
SmarterMeasure Data
Cynthia Pascal, Northern Virginia Community College
Lou Ann Martin, Tri-County Technical College
Courtney Tillett, Tri-County Technical College
Dr. Mac Adkins, President, SmarterServices

2.  All attendee lines are muted.
 Please submit questions via chat.
 Q & A at the end.
 The session is being recorded and an archive will be
available.
Housekeeping

3.  Introduction – Making Sense of your Data
 Northern Virginia Community College
 Tri-County Technical College
 Five C’s to Maximize SmarterMeasure
Agenda

4. Level One – Students learn about themselves
•Aha moment
•The tiger looking in the mirror
Level Two – Schools learn how to best support students
•Integration into orientation courses, first year experience programs
•Recommendation of appropriate remedial / support resources
Level Three – Leaders learn how to maximize data
•Computing retention gains, comparing populations
•Fulfilling accreditation requirements, justifying costs
Levels of Benefit

5. Level One – Students learn about themselves
•Aha moment
•The tiger looking in the mirror
Level Two – Schools learn how to best support students
•Integration into orientation courses, first year experience programs
•Recommendation of appropriate remedial / support resources
Level Three – Leaders learn how to maximize data
•Computing retention gains, comparing populations
•Fulfilling accreditation requirements, justifying costs
Levels of Benefit

6. Which level would you say best describes the highest
level of how your school is using SmarterMeasue?
Level One – Using it as a student service
Level Two – Integrated into your software and processes
Level Three – Conducting statistical analysis of your data
Poll

7. Progression of SmarterMeasure
Data Utilization
Predictive
Correlation
Comparison
Descriptive
Student Service Metrics

8. Approaches to Research Projects
Internally
Conducted
Company
Assisted
Professionally
Assisted

9.  Cynthia Pascal
 Coordinator of Academic Advising & Retention
Northern Virginia Community College

10. Northern Virginia Community College
Northern Virginia Community College’s Extended
Learning Institute
Current Use of
SmarterMeasure
• Website
• Orientation
• Welcome Messages
• Student Planners
• Embedded in Bb
• Freshman Seminar/
College Skills
Assignment
• Advising partner
colleges of students’
“goodness of fit”

11. Northern Virginia Community College
Tool to assess incoming students
Northern Virginia Community College’s
Extended Learning Institute

12. Northern Virginia Community College
Northern Virginia Community College’s Extended
Learning Institute
Data used to have conversations
with students who are on
academic probation or taking a
class for the third time.
Data used to have
conversations with students
who are on academic
probation or repeating a class
for the third time.

13. Northern Virginia Community College
Northern Virginia Community College’s Extended
Learning Institute

14.  Lou Ann Martin – Title III Activity Director for the
Academic Support Network
 Courtney Tillett – Information Tech-instruction-
applications Analyst
Tri-County Technical College

15.  Current use
 Completed voluntarily by incoming students
 Completion encouraged through acceptance letter
 Results captured in Banner
 Results reviewed briefly with students by advisors
 Students enrolled in Freshman Seminar/College Skills classes
use results as springboard for discussions and journal entries
 Goal
 Would like to be able to identify students at-risk due to non-
academic issues
 Initial research found no specific correlations
 Tried different approach after consulting with
SmarterMeasure
 Preliminary results are promising and suggest the need to
tweak our readiness ranges
SmarterMeasure Results at TCTC

16. Acceptance
Letter

17. Results in Banner

18. Readiness
Ranges
Analysis
Default
Cut
Points
N p-value F
Adjusted
Cut
Points
p-
value
F
Mean
GPA
Personal
Attributes
70 / 85 1,457 0.2723 1.30 80 /90 0.2327 1.46 Fail:
2.15
Question:
2.30
Pass:
2.34
Life
Factors
70 / 85 1,482 0.2453 1.41 80 / 90 0.1217 2.11 Fail:
2.21
Question:
2.24
Pass:
2.56
Reading
Recall
30 / 65 1,380 0.1237 2.09 70 / 90 0.0002 8.52 Fail:
2.21
Question:
2.24
Pass:
2.56
Technical
Competen
cy
80 / 90 1,362 0.0011 6.82 90 / 100 0.0367 3.31 Fail:
1.84
Question:
2.26
Pass:
2.34
Technical
Knowledg
e
50 / 75 1,352 0.0039 5.56 70 / 80 0.0057 5.18 Fail:
2.03
Question:
2.26
Pass:
2.45
Typing
Rate
21 / 31 1,346 0.0303 3.51 25 / 40 0.1226 2.10 Fail:
2.17
Question:
2.36
Pass:
2.36

19.  Notes:
 Based on “Making SmarterMeasure a Strong Predictor of
Academic Success”
 Used ANOVA to determine the strength of the relationships
between the readiness ranges and first-semester GPA
 6,931 FTPS students from the fall and spring semesters of
the last three academic years (2011-2012 – 2013-2014)
 1,593 matched to SmarterMeasure results
 Any test taken after the first day of class for each semester
was excluded from the analyses
Readiness Ranges Analysis

20.  How can the ranges for Personal Attributes and Life
Factors be adjusted?
 Would any of the individual subscales in either section
be more predictive?
 What is the practical significance of the findings?
Questions Raised

21.  Configure the assessment engine
 Control for self-selection bias
 Consider adding your own questions
 Classify using your own demographic factors
 Create a sanitized data set
Five C’s to Maximize
SmarterMeasure

22.  Your Account Manager is eager to train you on any of
these configuration options.
Configure the Assessment Engine
Julie Owen Tara McLaughlin

23.  Login Tutorial
Video
 Add Google
Analytics
 Integrate into
LMS or SIS
 Require
student email
 Turn on/off
demographics
School
Level

24.  More
demographics
 Adjust time
zone
 Custom
Remedial
Resources
 Custom
Remedial Text
 Require
student email
 Renewal alert
School
Level

25.  Integrate into
third party
software such
as an LMS, SIS
or Advising
suite.
 Single Sign On
 Automated
data transfer
 Gradebook
integration
School
Level

26.  Re-order
sections
 Skip sections
 Add Math and
Writing
 Adjust
Readiness
Ranges
 Select
Versions
Group
Level

27.  Change
scoring type –
4 point or
percentage
 Assessment
Mode –
Sequential or
Disaggregate
 Show National
Averages
 Results
options –
Report,
Custom Text
or URL
Group
Level

28. Custom
Interface
Text

29. Select each of the ways that your
school has configured SmarterMeasure.
Re-order or delete sections
LMS integration
Adding custom questions
Custom introductory or score report
text
Adjusting Readiness Ranges
Poll

30. Control for Self-Selection Bias

31. Consider Adding Your Own Questions

32. Classify Using Your Own
Demographic Factors

33. Create a Sanitized Data Set

34. Based on the webinar today how likely is
your school to enhance the way it uses
SmarterMeasure?
Very likely – I learned some actionable
ideas.
Likely – This has me thinking.
Not likely – This is interesting but there is
no need to change what we are doing.
Poll

35. Let Us Know If We Can Help
Julie Owen
Julie@SmarterServices.com
Tara McLaughlin
Tara@SmarterServices.com

36. Please Give Us Your Feedback
You will receive a post-webinar email with a link to the survey.