2013 03-14 (educon2013) emadrid urjc mining student repositories to gain learning analytics an experience report

645 views

Published on

2013 03-14
(educon2013)
emadrid
urjc
mining student repositories to gain learning analytics an experience report

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
645
On SlideShare
0
From Embeds
0
Number of Embeds
107
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

2013 03-14 (educon2013) emadrid urjc mining student repositories to gain learning analytics an experience report

  1. 1. Mining student repositories to gain learning analytics An experience report Gregorio Robles, Jes´s M. Gonz´lez Barahona u a {grex,jgb}@gsyc.urjc.es GSyC/LibreSoft, Universidad Rey Juan Carlos, Madrid, Spain Berlin, Germany, March 14th, 2013Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  2. 2. c 2013 Gregorio Robles, Jes´s M. Gonz´lez-Barahona u a All figures are ours, except when the original source is specified. Some rights reserved. This presentation is distributed under the “Attribution-ShareAlike 3.0” license, by Creative Commons, available at http://creativecommons.org/licenses/by-sa/3.0/Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  3. 3. Summary: What this talk is about Engineering students often have to deliver small computer programs in many engineering courses Instructors have to evaluate these assignments according to the learning goals and their quality, but ensure as well that there is no plagiarism We report the experience of using mining software repositories techniques Related efforts (please, see paper) Procedure Tools Links and ideas Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  4. 4. Context 3rd-year Telecommunication Engineering students at URJC Course on multimedia networks Assignments: small programs that exchange multimedia set-up and content information using standardized network protocols: SIP, SDP, RTP, UDP, IP... Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  5. 5. Technologies used by students Python git: distributed versioning system pep8 (a script that checks if the code follows coding guidelines) wireshark: network protocol analyzer Scope. The assignment includes: Program with communication among clients and servers using standardized protocols A live capture with the result of a scenario Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  6. 6. 2. Preprocess Cloning of the repository Checking if the files with the assignment exist are have been correctly named Checking if the style guide has been followed (with pep8) Evaluating the quality of the code (in our case, Pylint) Retrieving of the git log and analysis (analyzed with CVSAnalY) Analysis of the wireshark network exchange capture Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  7. 7. 3. Plagiarism detection 4. Functional assessment A note about ”plagiarism” Black box testing We use the non-open-source Domain-specific free web service MOSS In computer networks: (from Stanford University) standards-oriented Figure: Black-box testing. Source: goo.gl/3e1Fq Figure: Plagiarism Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  8. 8. 5. Post-process 6. Personalized exam Final grades for the Three type of questions: assignment are calculated Code snippets (own and Creates file with feedback external) for the student, with input Black box questions information from all the Questions about specific steps scenarios Instructors get a report of Personalized exams take from 10 the whole process, including to 20 minutes and can be done assignments suspicious of simultaneously by many students. plagiarism, and errors Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  9. 9. 6. Personalized exam (example question) Figure: Personalized exams Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  10. 10. Experience reportStudents Instructors Welcome feedback and We promised ourselves become better in following complete automatizing... assignments really not possible Automatizing allows to We have a lot of data... but better understand standards maybe not enough of the one we want! Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  11. 11. Conclusions/summary Not fully automatable, but scalable (continuous evaluation possible) Offers large and good feedback to students Domain of the program is very important! Group assessment could be easily introduced Discussion: we have a wealth of data, but not always the one we would really like Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics
  12. 12. Mining student repositories to gain learning analytics An experience report Gregorio Robles, Jes´s M. Gonz´lez Barahona u a {grex,jgb}@gsyc.urjc.es GSyC/LibreSoft, Universidad Rey Juan Carlos, Madrid, Spain Berlin, Germany, March 14th, 2013Gregorio Robles, Jes´s M. Gonz´lez Barahona u a Mining student repositories to gain learning analytics

×