This document discusses using institutional research to monitor student progress at universities in South Africa. It provides examples of analyses that can be conducted, such as dropout and completion rates by qualification type, student mobility between institutions, and the relationship between matric scores and first year performance. The document outlines the approaches, tools, and process used to conduct cohort tracking analyses at a university, including defining cohorts of students enrolled in the same qualification and tracking their progression over multiple years. Areas for improving institutional research on student progress are also identified, such as integrating the work of different university departments and adding more student variables to the analyses.

1. Institutional Research for
student progress
•Marthie Cronje
•HEQC workshop, TUT
•4-5 June 2008
Why?
• Moral responsibility by institutions
• Skills needs
• Better throughout = more student places
• Performance indicator
– DoE: Planning and Funding
– HEQC: Efficiency
• Funding: Teaching Output
2

2. SA context
• Low retention rates
• Low completion rates
• Equity of success
• Graduation ratios vs cohort tracking
– SAAIR project
– DoE project
– HERD-SA project
3
HERD-SA 2006/7
Purpose:
• to standardise the measurement of retention rates in terms of SA
definitions,
• to develop the appropriate methodology for reviewing and
presenting retention data and
• to develop a profile of student characteristics from the data
collected/identified and determine what, if any, relationships the
data may have to student retention.
Completed Dec 2007.
Phase 1 CD: all institutions for capacity development.
4

3. HERD-SA 2008
• Follow-up project under auspices of HESA
– continuing the development and dissemination of relevant
definitions and standardisation of concepts related to student
retention;
– continuing research on student retention and graduation data with the
objective of refining existing approaches to the collection,
analysis and presentation of such data and developing new
approaches;
– researching and developing ways of promoting the effective use of
student retention and graduation data for improving student
performance in the institutions; and
– assisting with building institutional capacity to analyse and
interpret student progression data.
5
STUDENT PROGRESS @ UJ
6

4. Types of analysis
• Policy-making
– Admissions Policy
• Enrolment Planning
• Monitoring improvement
– DVC: Academic, Deans, etc.
• Informing re-admission decisions
– Deans
• Programme reviews
• Problem identification and intervention planning
– Faculties, Departments, Student support services
• Researchers
7
Who?
• Institutional Research function
– Cohort analysis
– Monitoring/Reporting
– Enrolment Planning
– Policy-making
– Projects/ad hoc
• Researchers
– Admission/Placement Policy/Testing
– Identifying at risk students/ factors
– Evaluating interventions
8

5. Examples of analyses
9
Dropout rates –
Undergraduate
Qualification
type
Value Cohort
2003 2004 2005 2006 2007
National
Diploma
Baseline enrolment 4404 3887 3671 3287 3356
% cancel year1 3% 3% 3% 3% 4%
% dropout year 2 31% 29% 27% 27% 24%
General 1st B
degrees
Baseline enrolment 3560 3961 3356 3429 4029
% cancel year1 2% 3% 3% 3% 2%
% dropout year 2 19% 23% 22% 21% 15%
Professional
1st B degrees
Baseline enrolment 641 740 666 678 832
% cancel year1 2% 3% 4% 5% 4%
% dropout year 2 11% 19% 19% 22% 15%
10

6. Completion rates - UG
Qualification type Values 2000 2001 2002 2003 2004 2005
National Diploma
% grad yr3 (cumulative) 15% 17% 14% 14% 17% 18%
% grad yr4 (cumulative) 27% 29% 27% 28% 29%
% grad yr5 (cumulative) 32% 36% 34% 34%
% grad yr6 (cumulative) 36% 40% 38%
% grad yr7 (cumulative) 38% 42%
General 1st B
degree
% grad yr3 (cumulative) 34% 33% 34% 30% 28% 32%
% grad yr4 (cumulative) 50% 51% 51% 50% 45%
% grad yr5 (cumulative) 60% 59% 61% 57%
% grad yr6 (cumulative) 63% 62% 64%
% grad yr7 (cumulative) 65% 63%
Professional 1st B
degree
% grad yr4 (cumulative) 32% 36% 37% 35% 26%
% grad yr5 (cumulative) 46% 56% 54% 52%
% grad yr6 (cumulative) 54% 63% 61%
% grad yr7 (cumulative) 57% 68% 11
12
Student mobility

7. M-score and dropout
13
Matric vs year 1
performance
14

8. Enrolment modeling
2007 2008 2009 2010
National Diploma
New intake 163 135 160 180
Remaining students 350 307 317 321
Total 513 442 477 501
General 1st B
degree
New intake 389 477 500 520
Remaining students 1030 953 993 1051
Total 1419 1430 1493 1571
15
Progress to further studies
Qual type Grad year n
Years since graduating
1 2 3 4 5
N.Dip
proceeding
to B.Tech
2003 1495 23% 27% 29% 31% 32%
2004 1540 24% 31% 33% 35%
2005 1858 23% 28% 31%
2006 2058 24% 28%
2007 2100 24%
B.Tech
proceeding
to Master’s
2003 564 2% 4% 4% 4% 4%
2004 554 3% 4% 4% 4%
2005 660 3% 4% 4%
2006 718 2% 3%
2007 796 2%
16

9. HOW?
17
18
Approaches to monitoring
student progress
Monitoring student
progress
•Module pass rate
•Degree credit
success rate
Qualification
progress
Tracking:
•Retention
•Graduation rate
Non-tracking:
•Graduation ratio
(DoE definition)
Module/subject
success
•Brochure: “Monitoring student progress…”

10. Cohort data
• Track students enrolling for a specific qualification type for
the first time, for seven years.
• Events: Enrol (yr2-7), cancel (yr1-3) and graduate (yr1-7).
• In same qualification type/group.
• Primary qualification in year 1.
• Operational data.
• Database record for each student:
– Easy to add additional information.
– Sections of the database can be extracted for specific purposes,
e.g. Faculties, service units.
19
Current variables
• Faculty and qualification
• Gender
• Race
• Age in cohort year
• Previous activity
• Nationality
• Full-time/part-time in year 1
• Contact/Distance in year 1
• Result in year 1 (where
available)
• Entrance category
• M-score
20

11. Process
1100+ qualifications x 9 cohorts = 143792 records
Access queries (via macros).
Excel Pivot table with 79 calculated fields:
• Retention (#, %, Any, Same Faculty)
• Cancel (#, %: Year 1 to year 3)
• Graduation (#, %, Any, Same Faculty, specific year only or
cumulative)
• Dropout (#, %)
• Stopout (Year 2)
21
22

12. Tools used
• Data extraction
– MS Access via HEDA ODBC, Oracle Discoverer
• Data analysis
– MS Excel, SPSS
23
Relationships
• Create awareness of availability
– Reports
– Presentations
– Project participation
– Volunteer additional information where suitable
– Maintain availability and credibility
• Determine question to be answered
• Define!
• Educate!
24

13. Areas for improvement
• Integration of work in IR and other areas.
– AD: interventions.
– Academic research: Researching reasons, solutions, frameworks.
• Real time identification of at risk students.
• Add variables
– Socio-economic indicators.
– Placement profile.
– Admission details.
– Interventions.
– Satisfaction/engagement…
• Formalise definitions and educate!
25
The end
Marthie Cronje
cronjem75@gmail.com
26