A cluster-based analysis to diagnose students’ learning achievements


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

A cluster-based analysis to diagnose students’ learning achievements

  1. 1. A cluster-based analysis to diagnose students’ learning achievements Luz Stella Robles Pedrozo, U. Tecnológica (Cartagena, Colombia) Miguel Rodríguez Artacho, UNED University (Madrid) IEEE EDUCON 2013 (Berlin)
  2. 2. Content1.  General Objectives2.  Background and Motivation3.  Proposed Diagnostic Test Methodology4.  Conclusions5.  Future Work IEEE EDUCON 2013 (Berlin)
  3. 3. General  Objec,ves    ScopeRecognizing Learning Disabilities trough Testing: Clustering Based Methodology and ReliabilityGeneral ObjectiveThe design and implementation of a methodology for learning disabilities diagnosis andassessment based on:û  Adaptive feedback to the students in order to individually identify learning weaknesses andmisconceptions about a topic right after assessment through testing.û  Classification of the students via clustering of the detected learning disabilities, as a supportfor the design of feedback strategies and activities for improving their academic performance. IEEE EDUCON 2013 (Berlin)
  4. 4. Background  and  Mo,va,on  û  Problems with prior knowledge diagnostic assessment using standardized tests with manual scoring: Type I ICFES multiple choice questions with only one correct answer. This kind of questions are used for: Midterm exams, SABER 5th, 9th, 11th and SABER PRO mandatory state tests in Colombia.û  The traditional education system uses pass/fail scoring scale based written exams for assessment à The score does not provide enough information about learning that can be used for performance improving.û  The recognition of learning disabilities and misconceptions is key and complex process that has to be manually performed. IEEE EDUCON 2013 (Berlin)
  5. 5. Background  and  Mo,va,on:  tests  û  Disadvantages of traditional tests : The same test, with a fixed number of items, is given to all test takers. They have limited answer choices. The test is long in order to make it more accurate.û  The assessment uses traditional methodologies which do not allow : −  Identification of systematic misconceptions and weak understanding of concepts in order to plan strategies to improve their academic performance. −  The classification and grouping of the students to undertake a re- orientation of the reinforcement activities. −  The individual recognition of the level of learning disabilities and misconceptions. IEEE EDUCON 2013 (Berlin)
  6. 6. Background  and  Mo,va,on:  feedback  A diagnostic assessment methodology that provides a classification score, identifieslearning disabilities, misconceptions and weak understanding of concepts, allowing togroup the students with similar problems in clusters, is required.Structure of the proposed diagnostic assessment methodology:û  Item Response Theory (IRT) is used as the method to obtain the skill level ofeach concept.û  The use of a system of interrelated concepts and dependences to identifycognitive disabilities (misconceptions and weak understanding of concepts)û  The use of Clustering to classify the students in groups with similar disabilities IEEE EDUCON 2013 (Berlin)
  7. 7. Proposed  Diagnos,c  Assesment  Methodology   Item Response Theory (ITR) [Thurstone, 1925], [Lord, 1952, 1968] ITR allows invariant measured variables that are independent with respect to the examinees and the used test instruments.CTT ITRLack of invariance in the properties of the Different tests can be comparable, astests with respect to the test subjects. The the skill level trend to be the samecharacteristics of the items depend on the between different item setsgroup of persons.Asumes the same error level for all subjects, Similar level of assessment accuracyor the test liability is the same for all the for all different participants.participants (as a property of the test) IEEE EDUCON 2013 (Berlin)
  8. 8. Proposed  Diagnos,c  Assesment  Methodology  ITR Modelsû  1, 2 and 3 parameters unidimensional logistic modelsû  Dichotomous answer format (only one answer)û  Performance and skills assessmentITR – Model proofingThe test instrument, with the items containing the object variable, is applied toû  Validate the ITR assumptionsû  Select the optimum models based on statistical analysisITR – Once the model is selected …û  Estimate the parameters of the selected modelû  Calculate the skill or proficiency level of the test subjectsû  Identify learning disabilities in the test subjects IEEE EDUCON 2013 (Berlin)
  9. 9. Proposed  Diagnos,c  Assesment  Methodology  Diagnostic Methodology : Item selectionû  At least one assessment item assigned to each node of the framework.û  The knowledge domain to be evaluated, categorized into sub-topics and pre-requisites.û  The dependences between the items and the concepts (concepts for the assessment ineach item).û  The weight of the concepts in each item. IEEE EDUCON 2013 (Berlin)
  10. 10. Proposed  Diagnos,c  Assesment  Methodology  An inference example (probability and statistics) IEEE EDUCON 2013 (Berlin)
  11. 11. Proposed  Diagnos,c  Assesment  Methodology   IEEE EDUCON 2013 (Berlin)
  12. 12. Proposed  Diagnos,c  Assesment  Methodology   Diagnostic MethodologyTool  used:  R  h,p://www.r-­‐project.org/   IEEE EDUCON 2013 (Berlin)
  13. 13. Proposed  Diagnos,c:  Learning  Paths  Diagnostic Methodology IEEE EDUCON 2013 (Berlin)
  14. 14. Clustering  Cluster Generation IEEE EDUCON 2013 (Berlin)
  15. 15. Clustering  Cluster Generationû  List of weakly-understood concepts per each examineeû  Total weight of each weakly-understood concept in the test (TP CI d)û  Calculate the total weight of the weakly-understood concepts in the test (PTcd) per each examinee, as in : IEEE EDUCON 2013 (Berlin)
  16. 16. Clustering  Cluster Generation IEEE EDUCON 2013 (Berlin)
  17. 17. Clustering  Cluster Generation IEEE EDUCON 2013 (Berlin)
  18. 18. Conclusions  Psychometric aspectsû  The Item Response Theory (IRT) was selected for this work after a proper understanding of itsadvantages with respect to the Classical Test Theory (CTT).û  An statistical procedure was proposed to select and validate the optimum model to use with theobtained data from the tests used in this work. A computer program was designed on the Rlanguage for analysis purposes .û  A comparative studied was performed between the score for the skills level of a group ofexaminees obtained with the classical test theory (TCT, average score) and that obtained with theIRT model (unidimensional 3 parameters logistic model 3PL) IEEE EDUCON 2013 (Berlin)
  19. 19. Conclusions  Regarding the Diagnostic MethodologyA software for diagnostic was implemented: •  Process answers of the examinees ( Deficient and Minimum) to generate the weakly- understood concepts per student •  Represent the suggested leaning paths for each examinee. •  An index representing the total weight (or total sum of weigths) of the weakly-understood concepts in the test per examinee is generated.Regarding the ClusterA computer program was implemented in R in order to generate a list classifying the examinees ingroups with similar misconceptions or learning disabilities.à Data mining tools can be as useful as Intelligent Systems in certain domains to diagnosestudent models. IEEE EDUCON 2013 (Berlin)
  20. 20. Conclusions  û  This work is useful for public education institutions in Colombia because it serves as a solution for the efficient diagnostic of the learning disabilities in students by using a test.û  The design and implementation of the diagnostic procedure, suppported with IRT and clustering procedures, allow to perform a comprehensive diagnostic of the learning disabilities, misconceptions and weak understanding of concepts in students.û  The work provides the students with a tool for the easy identification of their learning and cognitive disabilities, and the suggested self-learning path to improve their academic performance à Provide feedback IEEE EDUCON 2013 (Berlin)
  21. 21. A cluster-based analysis to diagnose students’ learning achievements THANKS! Luz Stella Robles Pedrozo, U. Tecnológica (Cartagena, Colombia) Miguel Rodríguez Artacho, UNED University (Madrid) Learning Technologies and Collaborative Systems http://ltcs.uned.es IEEE EDUCON 2013 (Berlin)