Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

- Data mining to predict academic per... by Ranjith Gowda 2030 views
- STUDENT PERFORMANCE ANALYSIS USING ... by Akshay Jain 683 views
- USING LEARNING ANALYTICS TO PREDICT... by African Virtual ... 430 views
- Analysis of student performance by ... by Auricle Technolog... 349 views
- Attendance and student performance ... by Cindy Paynter 382 views
- Data mining by Akannsha Totewar 157761 views

576 views

Published on

data mining field with recommender systems techniques for predicting

student performance. Additionally, we study the importance of including

students’ attempt time sequences of parameterized exercises.

The approaches we use are Bayesian Knowledge Tracing (BKT), Performance

Factor Analysis (PFA), Bayesian Probabilistic Tensor Factorization

(BPTF), and Bayesian Probabilistic Matrix Factorization (BPMF).

The last two approaches are from the recommender system’s field.We approach

the problem using question-level Knowledge Components (KCs)

and test the methods using cross-validation. In this work, we focus on

predicting students’ performance in parameterized exercises. Our experiments

shows that advanced recommender system techniques are as accurate

as the pioneer methods in predicting student performance. Also, our

studies show the importance of considering time sequence of students’

attempts to achieve the desirable accuracy.

Published in:
Data & Analytics

No Downloads

Total views

576

On SlideShare

0

From Embeds

0

Number of Embeds

6

Shares

0

Downloads

37

Comments

0

Likes

3

No embeds

No notes for slide

Explain the sequence: order of attempts

Add Some figures

- 1. Predicting Student Performance in Solving Parameterized Exercises 2Shaghayegh Sahebi (Sherry) • One question template with multiple parameter sets – One template generates many questions – Each question be repeated multiple times by the same student – Makes cheating difficult – The student can learn by practicing over time
- 2. Predicting Student Performance in Solving Parameterized Exercises 3Shaghayegh Sahebi (Sherry)
- 3. Predicting Student Performance in Solving Parameterized Exercises 4Shaghayegh Sahebi (Sherry) • Unproductive repetitions – Students who are not good in managing their learning [Hsiao et. al, 2009] • How to avoid this? – Personalized e-learning system – Predict the success of students’ future attempts the same way as recommender systems – Predicting students’ performance (PSP)
- 4. Predicting Student Performance in Solving Parameterized Exercises 5Shaghayegh Sahebi (Sherry) • In static questions, the student solves a problem once – No attempt sequence on each question – Time-ignorant methods work well • Collaborative filtering approaches • Assumption in parameterized questions: the student can learn by practicing over time – Attempt sequence for each student on each question
- 5. Predicting Student Performance in Solving Parameterized Exercises 6Shaghayegh Sahebi (Sherry) • To study the – recommender systems approaches – effect of attempt sequence in PSP for parameterized questions • Approaches: – Bayesian Knowledge Tracing (BKT) – Performance Factor Analysis (PFA) – Bayesian Probabilistic Matrix Factorization (BPMF) – Bayesian Probabilistic Tensor Factorization (BPTF) – Max baseline
- 6. Predicting Student Performance in Solving Parameterized Exercises 7Shaghayegh Sahebi (Sherry) • Markov Model with two states • Models attempt sequence explicitly K K K Q Q Q Initial knowledge Learning P(T) P(G), P(S)
- 7. Predicting Student Performance in Solving Parameterized Exercises 8Shaghayegh Sahebi (Sherry) • Regression model • No attempt sequencing but implicitly models attempt history m(i, j Î KCs ,k Î items,s,f) = bk + (gj si,j + rj fi,j ) jÎKCs å
- 8. Predicting Student Performance in Solving Parameterized Exercises 9Shaghayegh Sahebi (Sherry) • From collaborative filtering • No attempt sequence modeling • We use Bayesian Probabilistic Matrix Factorization (BPMF) [Xiong et al., 2010] • Other models were used for static questions [Thai-Nghe et al., 2011] 1 0 0 0 1 1 0 1 0 0 1 1 0 0 0 1 Students Questions/ topics 0.9 0 1.5 0.4 0 1.4 0 0.9 Students KCs 0.8 0.5 0 0.3 0 0 0.5 0.8 KCs Questions/ topics
- 9. Predicting Student Performance in Solving Parameterized Exercises 10Shaghayegh Sahebi (Sherry) • Adds attempt sequence modeling to BPMF • We use Bayesian Probabilistic Tensor Factorization (BPTF) • Other models used for static questions Students Questions/ topics …
- 10. Predicting Student Performance in Solving Parameterized Exercises 11Shaghayegh Sahebi (Sherry) • Predicting success (majority class) for every attempt
- 11. Predicting Student Performance in Solving Parameterized Exercises 12Shaghayegh Sahebi (Sherry) • From QuizJET system • Java Programming Questions • Six semesters • 166 Students • 103 questions • 69.04% success records (majority class)
- 12. Predicting Student Performance in Solving Parameterized Exercises 13Shaghayegh Sahebi (Sherry) • Time-aware methods: – BKT: explicitly – PFA: counting previous success/failure – BPTF: student’s performance changes smoothly over time • Time-ignorant methods: – Matrix factorization (BPMF) – Max baseline • Collaborative filtering approaches: – Tensor factorization (BPTF) – Matrix factorization (BPMF) • Knowledge component: question • 5-Fold user-stratified cross validation – 80% of users in train data, rest in test data
- 13. Predicting Student Performance in Solving Parameterized Exercises 14Shaghayegh Sahebi (Sherry)
- 14. Predicting Student Performance in Solving Parameterized Exercises 15Shaghayegh Sahebi (Sherry) 64 66 68 70 72 74 76 78 Accuracy
- 15. Predicting Student Performance in Solving Parameterized Exercises 16Shaghayegh Sahebi (Sherry) 900 950 1000 1050 1100 1150 1200 BKT PFA BPTF False Positive
- 16. Predicting Student Performance in Solving Parameterized Exercises 17Shaghayegh Sahebi (Sherry) 0 50 100 150 200 250 300 350 400 450 BKT PFA BPTF False Negative
- 17. Predicting Student Performance in Solving Parameterized Exercises 18Shaghayegh Sahebi (Sherry) 0 20 40 60 80 100 120 140 BKT PFA BPTF BPMF Minority Recall Majority Precision
- 18. Predicting Student Performance in Solving Parameterized Exercises 19Shaghayegh Sahebi (Sherry) 0 20 40 60 80 100 120 140 160 180 BKT PFA BPTF BPMF Majority Recall Minority Precision
- 19. Predicting Student Performance in Solving Parameterized Exercises 20Shaghayegh Sahebi (Sherry) • Attempt sequence is important in PSP for parameterized questions • Recommender systems approaches are as good as the pioneers PSP methods – if they consider attempt sequence – Do not need to know the exact Knowledge Components – Encourages more research on applying more recommendation techniques in PSP
- 20. Predicting Student Performance in Solving Parameterized Exercises 21Shaghayegh Sahebi (Sherry) • Other collaborative filtering approaches • Ensemble of approaches • Effect of knowledge structure (our AIEDCS paper) • Personalize students’ experience according to our results
- 21. Predicting Student Performance in Solving Parameterized Exercises 22Shaghayegh Sahebi (Sherry) Thank You!
- 22. Predicting Student Performance in Solving Parameterized Exercises 23Shaghayegh Sahebi (Sherry) • EM algorithm for BKT and set the initial parameters as follows: p(L0) = 0:5 , p(G) = 0:2 , p(S) = 0:1 , p(T) = 0:3 . For running PFA, we use • the implementation of logistic regression in WEKA [3]. • For BPTF and BPMF: Matlab code prepared by Xiong et. al. We experimented with different latent space dimensions for BPTF and BPMF (5, 10, 20 and 30) and chose the best one, which has the latent space dimension of 10
- 23. Predicting Student Performance in Solving Parameterized Exercises 24Shaghayegh Sahebi (Sherry) • Predicting the student’s capability to solve a problem or perform an educational task, mostly based on her performance in the past • Predicting success/failure in solving a question • Questions can be related to topics (Here, each topic can have multiple questions and each question is related to one topic)
- 24. Predicting Student Performance in Solving Parameterized Exercises 25Shaghayegh Sahebi (Sherry) No significant accuracy difference between all methods except BPMF and Max Baseline (P<0.05)
- 25. Predicting Student Performance in Solving Parameterized Exercises 26Shaghayegh Sahebi (Sherry)
- 26. Predicting Student Performance in Solving Parameterized Exercises 27Shaghayegh Sahebi (Sherry) PFA tends to predict more failures for the students.
- 27. Predicting Student Performance in Solving Parameterized Exercises 28Shaghayegh Sahebi (Sherry) If BKT predicts a failure for a student, this prediction is more likely to be true compared to the other methods
- 28. Predicting Student Performance in Solving Parameterized Exercises 29Shaghayegh Sahebi (Sherry) if PFA predicts a success for a student, this prediction is more likely to be true compared to the other methods
- 29. Predicting Student Performance in Solving Parameterized Exercises 30Shaghayegh Sahebi (Sherry)
- 30. Predicting Student Performance in Solving Parameterized Exercises 31Shaghayegh Sahebi (Sherry) • Maj. Prec: TP/(TP+FP) • Min Prec: TN/(TN+FN) • Maj. Recall: TP/(TP+FN) • Min Recall: TN/(TN+FP) • Accuracy: (TP+TN)/all

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment