This document describes techniques for data-driven curriculum analysis using learning analytics. It analyzes course data from a computer science program to identify which courses are most difficult, how courses are related based on student performance, groups of related courses, and paths that often lead to student dropout. The techniques included difficulty estimation using GPA, dependence estimation using correlation, curriculum coherence using factor analysis, dropout paths using sequence mining, and analyzing student load vs performance. The analysis provided insights into rethinking prerequisites, course groupings, factors influencing dropout, and how to better present the curriculum workload. The goal is to apply these techniques to other institutions' data to provide practitioners insights for curriculum improvement.

1. Techniques for Data-Driven
Curriculum Analysis
Gonzalo Mendez,
Xavier Ochoa & Katherine Chiluiza

2. http://www.slideshare.net/xaoc
h

3. Siemens, George, and Phil Long. "Penetrating the fog: Analytics in learning
and education." Educause Review 46.5 (2011): 30-32.

4. Siemens, George, and Phil Long. "Penetrating the fog: Analytics in learning
and education." Educause Review 46.5 (2011): 30-32.

5. Which are the hardest/more difficult
courses?
What lead our students to
success/failure?
How courses are
related?
Are there courses that could be
eliminated?
Is the work-load adequate for our
students?
??

6. How can Learning Analytics help?
Which tools could it provide to
curriculum-designers?

7. Our goals

8. Use readily available data
Grades are always collected and
historically stored

9. Create discussion starters
Metrics for evaluation are evil, but
metrics for insight could be useful

10. Easy to apply and understand
Could be integrated into a Learning
Analytics toolbox

11. Eat your own dog-food
Apply them to our own data to obtain
insight
(12-year historical data on CS program)

12. Let’s start

13. (1) Difficulty Estimation
How difficult a course is, not how
good the students are

14. Technique
Difficulty metrics

15. Two estimation metrics

16. GPA - Course grade
Course grade >
GPA
Course grade <
GPA
0
Course grade =
GPA
Three scenarios:
Differences between
GPA and course
grade
> 0< 0

17. Real examples

18. But…
They are not normal!

19. Three Two estimation metrics

20. Difficult Classes (Top 10)
Perceive
d
Estimated (first
5)Algorithms Analysis
Operating Systems
Physics A
Differential Equations
Linear Algebra
Programming Fundamentals
Object-Oriented Programming
Differential Calculus
Data Structures
Statistics
Operating Systems
Statistics
Differential Equations
Linear Algebra
Programming Languages
Electrical Networks I
Artificial Intelligence
Programming Fundamentals
Data Structures
Hardware Architecture and Organization

21. Perception != Estimation
What makes a course difficult then?

22. (2) Dependance Estimation
How well I do a student does in a
course affects how well he/she does
in another

23. CORE - CS CURRICULUM
Basic Physics
Integral Calculus
Multivariate Calculus
Electrical Networks
Digital Systems I
Hardware Architectures
Operative Systems
General Chemistry
Programming
Fundamentals
Object-oriented
Programming
Data Structures
Programming
Languages
Database Systems I
Software Engineering I
Software Engineering II
Oral and Written
Communication Techniques
Computing and Society
Discrete Mathematics
Algorithms Analysis
Human-computer
Interaction
Differential Calculus
Linear Algebra
Differential Equations
Ecology and
Environmental Education
Statistics
Economic Engineering I
Artificial Intelligence
PROFESSIONAL TRAINING HUMANITIES BASIC SCIENCE

24. Technique
Pearson product-moment
correlation coefficient
(A lot of it)

25. DEPENDANCE ESTIMATION
Programming
Fundamentals
Data Structures
(0.321)
Object Oriented
Programming
(0.309)

26. DEPENDANCE ESTIMATION
Computing
and Society
Operating Systems
(0.582)
Discrete Mathematics
(0.614)
Human-Computer Interaction
(0.6226)

27. Maybe we should rethink our
prerequisites
Why Programming Fundamentals does not correlates?
Why Computers and Society correlates with a lot of
courses?

28. (3) Curriculum Coherence
How courses group together

29. CORE - CS CURRICULUM
Basic Physics
Integral Calculus
Multivariate Calculus
Electrical Networks
Digital Systems I
Hardware
Architectures
Operative
Systems
General Chemistry
Programming
Fundamentals
Object-oriented
Programming
Data Structures
Programming
Languages
Database Systems I
Software Engineering I
Software Engineering II
Oral and Written
Communication
Techniques
Computing and Society
Discrete Mathematics
Algorithms Analysis
Human-computer
Interaction
Differential Calculus
Linear Algebra
Differential
Equations
Ecology and
Environmental
Education
Statistics
Economic Engineering I
Artificial Intelligence
PROFESSIONAL TRAINING HUMANITIES BASIC SCIENCE

30. Technique
Exploratory Factor
Analysis
(EFA)

31. 31

32. UNDERLYING STRUCTURE
Electrical
Networks
Differential
Equations
Software Engineering II
Software Engineering I
HCI
Oral and Written
Communication
Techniques
General Chemistry
Programming
Languages
Object-Oriented
Programming
Data Structures
Artificial Intelligence
Operative Systems
Software Engineering
Object-Oriented
Programming
Economic Engineering
Hardware Architectures
Database Systems
Digital Systems I
HCI
Differential and Integral Calculus
Linear Algebra
Multivariate Calculus
Digital Systems I
Basic Physics
Programming Fundamentals
Discrete Mathematics
General Chemistry
Statistics
Data Structures
Computing and Society
Algorithms Analysis
Differential Equations
Ecology and Environmental Education
Object-Oriented Programming
FACTOR 1: The basic training
factor
FACTOR 2: The advanced
CS topics factor
FACTOR 3: The client
interaction factor
FACTOR 4: The
programming
factor
FACTOR 5: The ?
factor

33. Grouping is also off
Fundamental Programming is not in the Programming factor?
What to do with Electrical Networks and Differential Equations?

34. (4) Drop-out Paths
What courses lead the students to
drop-out

35. DROPOUT AND ENROLLING PATHS
Time
(semesters)
0
1
2
3
4
Dropout
They are all happy, but as time goes
by…

36. Technique
Sequence Mining
(Sequential PAttern Discovery using
Equivalence classes - SPADE)

37. DROPOUT PATHS
Sequence Support
<Physics A, Dropout> 0.6081967
21
<Differential Calculus , Dropout> 0.5704918
03
<Programming Fundamentals , Dropout> 0.5327868
85
<Integral Calculus , Dropout> 0.4967213
11
<Physics A, Differential Calculus , Dropout> 0.4344262
3
<Linear Algebra , Dropout> 0.4327868
85
<Differential Calculus, Integral Calculus ,
Dropout>
0.3852459
02
<Physics C , Dropout> 0.3475409

38. Most drop-outs fail basic courses
Should students start with CS topics?
Too much pressure in engineering courses?

39. (5) Load/Performance Graph
What students think they can manage
vs. what they can actually manage

41. Technique
Simple Visualisation:
Density Plot of
Difficulty taken vs. Difficulty approved

42. LOAD/PERFORMANCE GRAPH

43. LOAD/PERFORMANCE GRAPH

44. LOAD/PERFORMANCE GRAPH

45. Unrealistic Suggested Load
How to the present the Curriculum in a better way?
How we can recommend students the right load?

46. Our goals?

47. Which are the hardest/more difficult
courses?
What lead our students to
success/failure?
How courses are
related?
Are there courses that could be
eliminated?
Is the work-load adequate for our
students?
??

48. ??
What makes a course difficult
then?
Why Programming Fundamentals does not
correlates?
Why Computers and Society correlates with a lot of
courses?
Fundamental Programming is not in the Programming
factor?
Should students start with CS topics?
Too much pressure in engineering
courses?
How to the present the Curriculum in a better
way?
How we can recommend students the right
load?
What to do with Electrical Networks and Differential
Equations?

49. Our ambitious goal?
Apply these techniques at your own
data in your own institution

50. Our more ambitious goal?
Make you think about LA techniques
that can be easily transferred to
practitioners

51. Gracias / Thank you
Xavier Ochoa
xavier@cti.espol.edu.ec
http://ariadne.cti.espol.edu.ec/xavier
Twitter: @xaoch