Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A cluster-based analysis to diagnose students’
learning achievements
Miguel Rodríguez Artacho
Comp. Science School
Univers...
Content
1. General Objectives
2. Background and Motivation
3. Proposed Diagnostic Test Methodology
4. Conclusions
5. Futur...
GENERAL OBJECTIVES
EADTU Webinar 2016
Scope
Recognizing Learning Flaws trough Testing: Clustering Based Methodology and Re...
BACKGROUND AND MOTIVATION
EADTU Webinar 2016
 Problems with prior knowledge diagnostic assessment using standardized test...
BACKGROUND AND MOTIVATION: TESTS
EADTU Webinar 2016
 Disadvantages of traditional tests : The same test, with a fixed
num...
BACKGROUND AND MOTIVATION: FEEDBACK
EADTU Webinar 2016
A diagnostic assessment methodology that provides a classification ...
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016
Traditional Test ITR
Lack of invariance in the properties of ...
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016
ITR Models
 1, 2 and 3 parameters unidimensional logistic mo...
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016
Diagnostic Methodology : Item selection
 At least one assess...
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016
An inference example (probability and statistics)
10
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016 11
PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY
EADTU Webinar 2016
Diagnostic Methodology
Tool used: R
http://www.r-project.org/...
PROPOSED DIAGNOSTIC: LEARNING PATHS
EADTU Webinar 2016
Diagnostic Methodology
13
CLUSTERING
EADTU Webinar 2016
Cluster Generation
14
CLUSTERING
EADTU Webinar 2016
Cluster Generation
 List of weakly-understood concepts per each examinee
 Total weight of ...
CLUSTERING
EADTU Webinar 2016
Cluster Generation
16
CLUSTERING
EADTU Webinar 2016
Cluster Generation
17
CONCLUSIONS
EADTU Webinar 2016
Psychometric aspects
 The Item Response Theory (IRT) was selected for this work after a pr...
CONCLUSIONS
EADTU Webinar 2016
Regarding the Diagnostic Methodology
A software for diagnostic was implemented:
• Process a...
CONCLUSIONS
EADTU Webinar 2016
 This work is useful for public education institutions in Colombia because it serves as a
...
ACKNOWLEDGEMENTS
EADTU Webinar 2016
Luz Stella Robles Pedrozo and Miguel Rodríguez-Artacho, "A cluster-based
analisys to d...
A cluster-based analysis to diagnose students’
learning achievements
THANKS!
Miguel Rodríguez Artacho
Comp. Science School...
Upcoming SlideShare
Loading in …5
×

A cluster-based analysis to diagnose students’ learning achievements

229 views

Published on

Miguel Rodríguez Artacho from UNED gave a presentation about A cluster-based analysis to diagnose students’ learning achievements as part of the online events by expert pool Assessment within EMPOWER.

Published in: Education
  • Be the first to comment

  • Be the first to like this

A cluster-based analysis to diagnose students’ learning achievements

  1. 1. A cluster-based analysis to diagnose students’ learning achievements Miguel Rodríguez Artacho Comp. Science School Universidad Nacional de Educación a Distancia (UNED) (Spain) miguel@lsi.uned.es @martacho EADTU Webinar 2016 1
  2. 2. Content 1. General Objectives 2. Background and Motivation 3. Proposed Diagnostic Test Methodology 4. Conclusions 5. Future Work EADTU Webinar 2016 2
  3. 3. GENERAL OBJECTIVES EADTU Webinar 2016 Scope Recognizing Learning Flaws trough Testing: Clustering Based Methodology and Reliability General Objective The design and implementation of a methodology for learning weakness diagnosis and assessment based on:  Adaptive feedback to the students in order to individually identify learning weaknesses and misconceptions about a topic right after assessment through testing.  Classification of the students via clustering of the detected learning disabilities, as a support for the design of feedback strategies and activities for improving their academic performance. 3
  4. 4. BACKGROUND AND MOTIVATION EADTU Webinar 2016  Problems with prior knowledge diagnostic assessment using standardized tests with manual scoring: Type I ICFES multiple choice questions with only one correct answer. This kind of questions are used for: Midterm exams, SABER 5th, 9th, 11th and SABER PRO mandatory state tests in Colombia.  The traditional education system uses pass/fail scoring scale based written exams for assessment  The score does not provide enough information about learning that can be used for performance improving.  The recognition of learning disabilities and misconceptions is key and complex process that has to be manually performed. 4
  5. 5. BACKGROUND AND MOTIVATION: TESTS EADTU Webinar 2016  Disadvantages of traditional tests : The same test, with a fixed number of items, is given to all test takers. They have limited answer choices. The test is long in order to make it more accurate.  The assessment uses traditional methodologies which do not allow : − Identification of systematic misconceptions and weak understanding of concepts in order to plan strategies to improve their academic performance. − The classification and grouping of the students to undertake a re- orientation of the reinforcement activities. − The individual recognition of the level of learning disabilities and misconceptions. 5
  6. 6. BACKGROUND AND MOTIVATION: FEEDBACK EADTU Webinar 2016 A diagnostic assessment methodology that provides a classification score, identifies learning disabilities, misconceptions and weak understanding of concepts, allowing to group the students with similar problems in clusters, is required. Structure of the proposed diagnostic assessment methodology:  Item Response Theory (IRT) is used as the method to obtain the skill level of each concept.  The use of a system of interrelated concepts and dependences to identify cognitive disabilities (misconceptions and weak understanding of concepts)  The use of Clustering to classify the students in groups with similar disabilities 6
  7. 7. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 Traditional Test ITR Lack of invariance in the properties of the tests with respect to the test subjects.The characteristics of the items depend on the group of persons. Different tests can be comparable, as the skill level trend to be the same between different item sets Asumes the same error level for all subjects,or the test liability is the same for all the participants (as a property of the test) Similar level of assessment accuracy for all different participants. Item Response Theory (ITR) [Thurstone, 1925], [Lord, 1952, 1968] ITR allows invariant measured variables that are independent with respect to the examinees and the used test instruments. 7
  8. 8. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 ITR Models  1, 2 and 3 parameters unidimensional logistic models  Dichotomous answer format (only one answer)  Performance and skills assessment ITR – Model proofing The test instrument, with the items containing the object variable, is applied to  Validate the ITR assumptions  Select the optimum models based on statistical analysis ITR – Once the model is selected …  Estimate the parameters of the selected model  Calculate the skill or proficiency level of the test subjects  Identify learning disabilities in the test subjects 8
  9. 9. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 Diagnostic Methodology : Item selection  At least one assessment item assigned to each node of the framework.  The knowledge domain to be evaluated, categorized into sub-topics and pre- requisites.  The dependences between the items and the concepts (concepts for the assessment in each item).  The weight of the concepts in each item. 9
  10. 10. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 An inference example (probability and statistics) 10
  11. 11. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 11
  12. 12. PROPOSED DIAGNOSTIC ASSESMENT METHODOLOGY EADTU Webinar 2016 Diagnostic Methodology Tool used: R http://www.r-project.org/ 12
  13. 13. PROPOSED DIAGNOSTIC: LEARNING PATHS EADTU Webinar 2016 Diagnostic Methodology 13
  14. 14. CLUSTERING EADTU Webinar 2016 Cluster Generation 14
  15. 15. CLUSTERING EADTU Webinar 2016 Cluster Generation  List of weakly-understood concepts per each examinee  Total weight of each weakly-understood concept in the test (TP CI d)  Calculate the total weight of the weakly-understood concepts in the test (PTcd) per each examinee, as in : 15
  16. 16. CLUSTERING EADTU Webinar 2016 Cluster Generation 16
  17. 17. CLUSTERING EADTU Webinar 2016 Cluster Generation 17
  18. 18. CONCLUSIONS EADTU Webinar 2016 Psychometric aspects  The Item Response Theory (IRT) was selected for this work after a proper understanding of its advantages with respect to the Classical Test Theory (CTT).  An statistical procedure was proposed to select and validate the optimum model to use with the obtained data from the tests used in this work. A computer program was designed on the R language for analysis purposes .  A comparative studied was performed between the score for the skills level of a group of examinees obtained with the classical test theory (TCT, average score) and that obtained with the IRT model (unidimensional 3 parameters logistic model 3PL) 18
  19. 19. CONCLUSIONS EADTU Webinar 2016 Regarding the Diagnostic Methodology A software for diagnostic was implemented: • Process answers of the examinees ( Deficient and Minimum) to generate the weakly- understood concepts per student • Represent the suggested leaning paths for each examinee. • An index representing the total weight (or total sum of weigths) of the weakly-understood concepts in the test per examinee is generated. Regarding the Cluster A computer program was implemented in R in order to generate a list classifying the examinees in groups with similar misconceptions or learning disabilities.  Data mining tools can be as useful as Intelligent Systems in certain domains to diagnose student models. 19
  20. 20. CONCLUSIONS EADTU Webinar 2016  This work is useful for public education institutions in Colombia because it serves as a solution for the efficient diagnostic of the learning disabilities in students by using a test.  The design and implementation of the diagnostic procedure, suppported with IRT and clustering procedures, allow to perform a comprehensive diagnostic of the learning disabilities, misconceptions and weak understanding of concepts in students.  The work provides the students with a tool for the easy identification of their learning and cognitive disabilities, and the suggested self-learning path to improve their academic performance  Provide feedback 20
  21. 21. ACKNOWLEDGEMENTS EADTU Webinar 2016 Luz Stella Robles Pedrozo and Miguel Rodríguez-Artacho, "A cluster-based analisys to diagnose students' learning achievements,” Global Engineering Education Conference (EDUCON), 2013 IEEE, Berlin, 2013, pp. 1118-1123. doi: 10.1109/EduCon.2013.6530248 http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6530248&is number=6530074 21
  22. 22. A cluster-based analysis to diagnose students’ learning achievements THANKS! Miguel Rodríguez Artacho Comp. Science School http://ltcs.uned.es Universidad Nacional de Educación a Distancia (UNED) (Spain) miguel@lsi.uned.es @martacho EADTU Webinar 2016 22

×