Exploring the practice of predictive analytics through the lens of an urban-serving, "new traditionals" campus.

1. Civitas Learning, Inc.
The Million More Mission
Driving insight & action to help students
learn well and finish strong

2. Civitas Learning, Inc.
CIVITASLEARNING.COM
University
of
Washington
Sea2le,
Tacoma,
Bothell

3. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
By
the
Numbers
• Three
campuses
• ~55,000
students
• Sea2le:
High-­‐Profile
Research
1
(45,000)
• Bothell:
Technology
Corridor
(5,000)
• Tacoma:
Urban-­‐Serving
(4,500)

4. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
By
the
Federal
ReporMng
Measures
• College
Scoreboard
(First
Year
–>
6
year
GraduaMon)
• Sea2le:
81%
• Bothell:
68%
• Tacoma:
49%

5. Civitas Learning, Inc.
THAT
CONNECTS
DISPARATE
SYSTEMS,
DATA
&
INITIATIVES
TO
GAIN
INSIGHT
INTO
WHAT’S
WORKING
&
WHAT’S
NOT
&
FOR
WHOM
ANALYTICS
PLATFORM

6. Civitas Learning, Inc.
ROBUST
TESTING
LAYER
Test.
Measure.
OpMmize.
Know
what
is
and
is
not
working.
Measure
program
effecMveness.
DYNAMIC
PREDICTIVE
FLOW
MODELS
Turn
the
lights
on.
See
trends.
Understand
success
and
risk
drivers.
Leverage
powerful
predictors.
The
infrastructure
to
support
beGer
decisions
Available
Data:
INSTITUTIONAL
DATA,
WORKFORCE
DATA,
CENSUS
DATA,
K-­‐12
DATA,
SYSTEMS
INSIGHT
&
ACTION
APPS
Deliver
acMonable
insight
to
users
for
greatest
impact
to
raise
the
floor
&
the
ceiling.
Data
feedback
stream
into
the
predicMve
flow
models.
Tap
into
all
available
data
sources
across
the
enMre
student
journey.
UNIFIED
DATA
LAYER

7. Civitas Learning, Inc.
Comprehensive Data
PRECISE DECISIONS
Multiple sources & systems. Derived features.
Institution specific models down to student level.
BLUNT DECISIONS
Point Data
Few data points from single systems.
B R I N G I N G D A T A T O G E T H E R F O R
BETTER DECISION-MAKING

8. Civitas Learning, Inc.
E A R L Y L E A R N I N G S
STUDENT SUCCESS SCIENCE
Broad
Deep
Practical
Specific
Diverse
Sustainable
Intuitive
Accessible
Humanizing
Cross-System (SIS, LMS, CRM, etc.)
Insightful features (GPA trend, Patterns)
Real-world quality, consistency
To the college and each student
Fluent in many modeling approaches
Model re-fresh & rapid feedback/testing
Intuitive UI to drive adoption
Meet users where they are
Personal, empowering, dynamic
Data
Science
People

9. Civitas Learning, Inc.
Promote student persistence
Front
Line
Revolutionize degree planning
INSTITUTIONAL LEADERSHIP
ADVISORS
STUDENTS
FACULTY
Advance social learning
Maximize student engagement
Turn your lights on
CIVITAS
LEARNING
PLATFORM
DATA
PROCESSING
&
VALIDATION
+
PREDICTIVE
MODELING
INSIGHT
+
ACTION
ANALYTICS

10. Civitas Learning, Inc.
illumeTM
ADMINISTR ATORS

14. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
Finding
change
in
pockets
• Increase
retenMon
from
82%
to
90%
by
2020.
• 8%
=
2,840
Students.
$36M
in
at
risk
tuiMon.
• Low
and
mid
hanging
fruit
already
picked.
Where
to
look
for
change?
• Used
Civitas
Insight
App
Illume
to
explore
their
data.

15. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
More
than
pocket
change
• 660
Non-­‐Resident
Female
students
with
GPAs
between
2.4
–
3.0
at
risk
to
not
re-­‐enroll
for
Fall
2014.
• Not
the
typical
risk
target.
• 10%
of
total
at
risk
populaMon.
$22M
in
at
risk
tuiMon.
“Civitas
is
helping
us
iden1fy
cohorts
of
at
risk
students
that
we
may
never
have
been
able
to
see
otherwise.
This
makes
it
possible
for
our
interven1ons
to
be
highly
targeted.”

16. COURSE
/
DEGREE
INSIGHTS

17. Pathway
insights
Drop
out
flows
GraduaMon
flows

18. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
UW
Tacoma
• Urban-­‐Serving
• Majority:
• Pell-­‐Eligible
• First
GeneraMon
• Under-­‐represented
Minority
• Work
more
than
30
hours

19. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
Lower
Division
• ~30%
First
Year
A2riMon
• ~15%
Second
Year
A2riMon
• Strong
upper
division
retenMon
(lose
~5%,
50%
6-­‐year
graduaMon)

20. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
The
Challenge
• Understand
Student
Needs
• Respond
Quickly
with
JIT
Support
• Smart-­‐Target
At-­‐Risk
Students

21. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
Knowing
Who,
When
and
Why
• Aggressive
Advising
• Peer
Mentoring
• Faculty
Awareness
and
Alerts
• Nudges/Engagement/ConnecMon
• Personalized
Learning
IniMaMves

22. inspireS TU DENT
TM
ACTION

23. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

24. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
inspireA DV I S O R
TM
ADVISOR

25. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

26. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

27. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

29. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

30. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

31. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

32. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION

35. CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
PUBLIC 4-YEAR
PUBLIC 2-YEAR
PROPRIETARY
PRIVATE NON-PROFIT
SYSTEMS
75
PARTNER
INSTITUTIONS
&
SYSTEMS

36. 875
Campuses
2.7M
Students
14M
Student
Records
150M
Enrollment
Records
1000+
PredicZve
models
in
producZon
OUR CURRENT REACH

37. Impact
on
student
performance
2.42
percentage
points
• Rigorous
Experimental
Design
• SophisMcated
Infrastructure
Persistence
7.30
percentage
points
3.21
percentage
points
Persistence
Persistence
3.0
percentage
points
Course
CompleMon
on
average
(36K
pairs)
with
repeated
use
when
app
was
used
in
online
classroom
studying
68,000
enrollments
and
up
to
2.42
percentage
points
7.62
percentage
points
on
average
(16K
pairs)
for
new
students
and
up
to

38. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
WHAT
WE
ARE
Learning
together®
2.42
percentage
point
increase
in
persistence.
Increasing
by
up
to
7.30
percentage
points
when
used
5+
Mmes.
Fidelity
to
plan
resulted
in
an
even
higher
rate
of
persistence.
Students
with
GPAs
between
2.4
–
3.0
were
at
risk
of
non-­‐
retenMon.
660
non-­‐resident
female
students
who
could
benefit
from
intervenMons.
Students’
engagement
score
visible
in
applicaMon
“heat
map”
enabled
faculty
to
personalize
outreach
to
students.
Aver
email
exchanges
and
phone
calls,
one
students
was
able
to
increased
his
grade
by
approximately
10
percentage
points
h2p://civitaslearningspace.com

39. Civitas Learning, Inc.
Civitas
learning
labs:
research
agenda
• Loss,
Mpping,
and
momentum
points
on
student
pathways
to
a
degree
or
credenMal
• Deeper
learning
strategies
• Late
enrollment
dilemmas
• Financial
aid
vs.
self-­‐pay
dynamics
• IntervenMon
and
inspiraMon
science
about
what
works
in
the
“AcMvaMon-­‐Zone,”
when
early
indicators
show
the
need
for
support
• Part-­‐Mme
a2endance
dynamics
• Online,
on-­‐ground,
and
blended
modality
differences
and
synergies
• Gender
gaps
–
men
overall,
women
in
STEM
• Incoming
student
success
• Undeclared
major
dangers
and
strategies
to
help
students
find
purpose
• Gatekeeper
and
milestone
courses
• Online
and
blended
course
success
strategies
• Right-­‐sizing/opMmizing
financial
aid
packages,
helping
students
make
good
choices
with
financial
aid
packages
• Transfer
student
dynamics,
including
strategies
to
incent
“producMve
transfer”
• Developmental
educaMon
innovaMon
• Recovery
from
failure,
tenacity,
grit,
mindset:
the
family
of
non-­‐cog
research
• STEM
compleMon
iniMaMves
• STEAM
iniMaMves,
combining
arts
with
STEM
• HBCUs
and
HSIs
–
the
vision,
opportunity
and
challenges
• Semester
shock
aver
transfer
• ExpectaMons
shock
for
high
school,
transfer,
or
returning
students
• Distance
learning
challenges
and
opportuniMes
• First-­‐generaMon
student
success

40. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
“Students
decide
if
they’ll
come
back
to
school
within
the
first
4-­‐6
weeks
they’re
on
campus,
when
they
barely
have
their
feet
under
them.
The
window
of
opportunity
to
help
make
things
come
together
for
them
is
small.”
Dr.
Jeff
Orgera
Senior
Assistant
VP
Student
Affairs,
Success
IniMaMves
University
of
Arizona

41. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
“Many
of
our
students
are
first-­‐genera1on
so
the
world
of
higher
educa1on
is
new
to
them.
We
have
to
be
very
inten1onal
about
the
advising
learning
outcomes
and
pedagogies
we
use
to
help
students
learn
about
their
learning.”
Joyce
Romano
VP
Student
Affairs
Valencia
College

42. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
“Many
of
our
first-­‐genera1on
college
students
have
no
exposure
to
academic
or
careers
op1ons
prior
to
joining
ACC.
We
wanted
to
spend
more
1me
talking
about
their
voca1onal
and
academic
goals
to
find
beJer
fits,
and
iden1fy
the
stumbling
blocks
and
challenges
they
may
face
along
the
way.”
Dr.
Virginia
Fraire
VP
Student
Success
AusMn
Community
College

43. Civitas Learning, Inc.
Maximizing
data
availability
Data-­‐Availability
SegmentaZon
• Divide
and
conquer
• Efficiently
evaluates
pa2erns
of
data
availability
to
idenMfy
segments
• Typically,
10-­‐20
data-­‐availability
segments
within
an
insMtuMon
• Maximizes
potenMal
of
available
data
• Improves
model
accuracy
and
robustness
as
well
as
providing
greater
insights

44. Civitas Learning, Inc.
Science:
Segments
&
Clusters
16
Segments,
32
Clusters
=
32
retenZon
models
for
one
insZtuZon
Segment
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Cluster
1
1
1
1
1
1
1
1
1
2
2
3
3
4
4
5
#
of
students
(K)
1.1
1.2
1.3
1.4
1.6
1.7
1.8
1.3
2.0
2.5
3.0
4.4
5.2
6.6
9.8
46
Term
Sequence
1
2
2
3
4
4
4
1
1
3
4
2
3
2
1
3
4
Term
Sequence
2
2
2
3
8
6
6
1
1
6
6
2
6
2
1
3
8
RetenZon
Rate
.36
.55
.84
.81
.79
.77
.46
.75
.58
.84
.67
.49
.81
.86
.81
.89
Using
top
segment
features,
clustering
idenMfies
and
groups
similar
students
within
segments
to
enable
simpler,
but
more
robust
and
powerful
models

45. Civitas Learning, Inc.
CIVITAS
LEARNING,
INC.
–
CONFIDENTIAL
INFORMATION
Science:
Model
CompeMMon
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
Boundary
decision
• Neural
networks
• SVM
• Bayesian
regularizaMon
network
Parametric
• MulMvariate
Gaussian
• Generalized
Linear
Models
• Naïve
Bayes
Non-­‐parametric
• CART/Random
Forest/GBM/BART
• Gaussian
mixture
model
• kNN
with
VQ
• LVQ