Predictive analysis, Data Science

1. Predictive Analytics 101:
An overview of how to create a dataset and
model to identify students at risk of attrition
Karen DeSantis
Senior Analyst
Office of Planning, Assessment and Institutional Research
Pace University
Pace’s Inaugural Retention Conference
June 16, 2017

2. Data Types and Sources
• Demographic
• Economic
• High school specific
• Pace specific
• Dates and deadlines
• Census
• Applications (Pace University and Financial Aid)
• Orientation
– BCSSE (Beginning College Survey of Student Engagement)
– Placement tests
• Historical data

3. Variables
• Demographic
– Gender, Age, Race, International, Underrepresented Minority
• Economic
– Financial Aid package, Tuition, Unmet need, Grants
• High school specific
– GPA, test scores (SAT, ACT, etc.)
– BCSSE responses, Placement data (from Orientation)
• Pace specific
– School, Campus, Residence, Major, CAP or Honors, Legacy, Athlete
• Dates and commitment
– Deposit Date, Attended orientation
• End of Semester Data:
– Starfish, Event attendance, End of semester GPA

4. Models
• Identified Dependent variable: Prediction of which students
will leave the University
– One semester (Fall to Spring semesters) – only a small percentage leave
– One year (Fall to Fall semesters) – up to 25% leave
• Gathered historical data for 2013, 2014, and 2015 First Year,
Full Time class cohorts
• Gathered data for the 2016 First Year, Full time cohort
• Data cleaning takes more time than you expect
– Variables may be missing
– Some students did not take BCSSE, SATs or complete FAFSA forms
– Recoding of variables into binary variables (0,1)
– Computing variables to be on a scale rather than absolute
values such as financial aid

5. Model – Variable selection
• Which variables correlated with the Dependent variable for
the historical data?
o SAT scores
o High School GPA
o Placement scores
o Undecided majors

6. Analysis
• Binary Logistic analysis
– Binary selected because there are two outcomes: Return or Attrite
• Statistical package selected affects analysis
– SPSS requires all variables to have a value to include a case (student) in the
analysis
• If a case has one variable empty, it will not be included in the SPSS analysis
– Created a binary “Dataset” variable so the analysis was run on the complete
dataset with an Attrition variable (students from 2013 to 2015) and used the
variables for the 2016 students without an Attrition value
• Saved Predicted values
– Analysis provided a predicted value for all students in the model
• Compared predicted values for each of the 2013 to 2015
cohorts to see how well the model fit with the students who
already left

7. Lists of Students
• Students with the highest predicted value for attrition were
identified for the 2016 cohort
• List of top 500 students was isolated and shared with the
Division of Student Success
• Using financial aid variables as well as the predicted attrition
variable, identified students who had highest financial need
within the 2016 cohort
• List of top 500 students with highest financial need shared
with Financial aid

8. Assessment of Model
• Identify 2016 cohort students who attrite from Fall to Spring
• Assessed identified students predicted scores from the two
models
• Identifying top predicted students in each cohort year and
comparing attrition rates for the two models
• Comparing top predicted students from 2016 to the top
predicted students attrition rates for the previous years
• Future: After Fall 2017 census, compare attrition of 2016
students who were contacted with attrition of the whole
class.

9. Outreach Feedback
• Feedback from DSS and Financial Aid
– How many students were actually contacted?
• What were their difficulties contacting some students?
– Comments and suggestions by those who performed the outreach
• Were students already on advisors/counselors radar?
– How outreach was performed and by whom
– What outcomes happened after DSS outreach?
– Did FA outreach result in additional financial aid awards for the
following year?

10. Next steps
• Remove 2013 data from analysis
– BCSSE data is more complete beginning with the 2014 cohort when it was
included in orientation
• Plans for Fall 2017 cohort

11. Additional Ideas
• What new variables can we add to the model?
– Grades from Math Courses or first Course in major
– Blackboard engagement
Concerns?
Suggestions?
Questions?

12. Thank you
Karen DeSantis
kdesantis@pace.edu