EVALUATING THE STUDENT              ACTIVITY METER:             TWO CASE STUDIES.                         Sten Govaerts, K...
OVERVIEW    • Problem             statement    • Objectives    • Design             & Implementation    • Evaluations    •...
PROBLEM                                   3Thursday 8 December 11
PROBLEM                                   3Thursday 8 December 11
OBJECTIVES   • self-monitoring           for learners   • awareness            for teachers   • time           tracking   ...
STUDENT ACTIVITY METER                  (SAM): DEMO.                                      5Thursday 8 December 11
THREE EVALUATIONS   • with          CS students        •   ICWL 2010 paper: “Visualizing Activities for            Self-re...
2010 PAPER   • usability and user satisfaction evaluation       with 12 CS students   • in   general, people understand th...
2010 PAPER   • usability and user satisfaction evaluation       with 12 CS students   • in   general, people understand th...
2010 PAPER   • usability and user satisfaction evaluation       with 12 CS students   • in   general, people understand th...
2010 PAPER   • usability and user satisfaction evaluation       with 12 CS students   • in   general, people understand th...
METHODOLOGY.   • design-based    research: give students       & teachers a tool and evaluate   • usability            stu...
CASE STUDY   • 20        persons: 3 CGIAR, 2 Law, 8 CS teachers & 7 CS TA’s.   • An    online survey about usefulness, tea...
CGIAR CASE STUDY                                                       issue for                                          ...
CGIAR CASE STUDY                                                       issue for                                          ...
CGIAR CASE STUDY                           Bar chart              ✔                           Parallel coordinates   ✔    ...
CGIAR CASE STUDY                                            12Thursday 8 December 11
CGIAR CASE STUDY        want to search       detect outliers         for students                                 more det...
CGIAR CASE STUDY        want to search       detect outliers         for students                                 more det...
CASE STUDY   • open course on learning and knowledge analytics, http://bit.ly/       dWYVbX   • 12        visual analytics...
CASE STUDY   • open course on learning and knowledge analytics, http://bit.ly/       dWYVbX   • 12        visual analytics...
LAK CASE STUDY                                                       issue for                                            ...
LAK CASE STUDY                                                       issue for                                            ...
LAK CASE STUDY                          Bar chart              ?                          Parallel coordinates   ?        ...
LAK CASE STUDY                                          16Thursday 8 December 11
LAK CASE STUDY                                              more                                             metrics      ...
LAK CASE STUDY                                              more                                             metrics      ...
IDEA’S FROM EXPERTS                  the used resource types                  detailed information per student            ...
IDEA’S FROM EXPERTS          2       the used resource types          5       detailed information per student          4 ...
CONTINUED WORK...                                             18Thursday 8 December 11
CONTINUED WORK...                                             18Thursday 8 December 11
FUTURE WORK   • extend              SAM with feedback of       experts   • evaluate  the usefulness in a larger       grou...
YOUR DATA IN SAM ?!                          ask me!                           sten.govaerts@cs.kuleuven.be               ...
2         slides will appear on http://www.slideshare.net/stengovaerts         slides will appear on http://www.slideshare...
Upcoming SlideShare
Loading in...5
×

Evaluating the Student Activity Meter: Two Case Studies.

492

Published on

presented at ICWL 2011, Hong Kong, 8-9 December 2011.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
492
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Evaluating the Student Activity Meter: Two Case Studies.

  1. 1. EVALUATING THE STUDENT ACTIVITY METER: TWO CASE STUDIES. Sten Govaerts, Katrien Verbert & Erik Duval http://www.role-project.eu 1Thursday 8 December 11
  2. 2. OVERVIEW • Problem statement • Objectives • Design & Implementation • Evaluations • Conclusion & Future work 2Thursday 8 December 11
  3. 3. PROBLEM 3Thursday 8 December 11
  4. 4. PROBLEM 3Thursday 8 December 11
  5. 5. OBJECTIVES • self-monitoring for learners • awareness for teachers • time tracking • learningresource use and recommendations 4Thursday 8 December 11
  6. 6. STUDENT ACTIVITY METER (SAM): DEMO. 5Thursday 8 December 11
  7. 7. THREE EVALUATIONS • with CS students • ICWL 2010 paper: “Visualizing Activities for Self-reflection and Awareness” • with CGIAR courses and teachers • with Learning and Knowledge Analytics course participants 6Thursday 8 December 11
  8. 8. 2010 PAPER • usability and user satisfaction evaluation with 12 CS students • in general, people understand the visualizations well! • solved some issues and requests sinceThursday 8 December 11
  9. 9. 2010 PAPER • usability and user satisfaction evaluation with 12 CS students • in general, people understand the visualizations well! • solved some issues and requests sinceThursday 8 December 11
  10. 10. 2010 PAPER • usability and user satisfaction evaluation with 12 CS students • in general, people understand the visualizations well! • solved some issues and requests sinceThursday 8 December 11
  11. 11. 2010 PAPER • usability and user satisfaction evaluation with 12 CS students • in general, people understand the visualizations well! • solved some issues and requests sinceThursday 8 December 11
  12. 12. METHODOLOGY. • design-based research: give students & teachers a tool and evaluate • usability studies • paper mockups • surveys 8Thursday 8 December 11
  13. 13. CASE STUDY • 20 persons: 3 CGIAR, 2 Law, 8 CS teachers & 7 CS TA’s. • An online survey about usefulness, teacher issues and how the tool can resolve these. • on average: 40 mins are spent using SAM. 9Thursday 8 December 11
  14. 14. CGIAR CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ? Being aware of what students are doing ✔ ✔ Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ✔ Knowing how and when online tools have been used ? ✔ Finding the students who are not doing well ✔ ? Finding the best students ? ? Knowing how much time students spent ? ✔ Knowing if external learning resources are used ✔ ? 10Thursday 8 December 11
  15. 15. CGIAR CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ? Being aware of what students are doing ✔ ✔ Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ✔ Knowing how and when online tools have been used ? ✔ Finding the students who are not doing well ✔ ? Finding the best students ? ? Knowing how much time students spent ? ✔ Knowing if external learning resources are used ✔ ? 10Thursday 8 December 11
  16. 16. CGIAR CASE STUDY Bar chart ✔ Parallel coordinates ✔ Line chart ✔ 11Thursday 8 December 11
  17. 17. CGIAR CASE STUDY 12Thursday 8 December 11
  18. 18. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 12Thursday 8 December 11
  19. 19. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 12Thursday 8 December 11
  20. 20. CASE STUDY • open course on learning and knowledge analytics, http://bit.ly/ dWYVbX • 12 visual analytics enthousiasts + experts (who also teach) • almost identical survey to CGIAR case. 13Thursday 8 December 11
  21. 21. CASE STUDY • open course on learning and knowledge analytics, http://bit.ly/ dWYVbX • 12 visual analytics enthousiasts + experts (who also teach) • almost identical survey to CGIAR case. 13Thursday 8 December 11
  22. 22. LAK CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ✔ Being aware of what students are doing ✔ ? Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ? Knowing how and when online tools have been used ✗ ? Finding the students who are not doing well ✔ ? Finding the best students ? ✗ Knowing how much time students spent ? ✔ Knowing if external learning resources are used ? ? 14Thursday 8 December 11
  23. 23. LAK CASE STUDY issue for teacher addressed Provide feedback to the students ✔ ✔ Being aware of what students are doing ✔ ? Knowing about collaboration and communication ✔ ✗ Knowing which documents are used and how much ✔ ? Knowing how and when online tools have been used ✗ ? Finding the students who are not doing well ✔ ? Finding the best students ? ✗ Knowing how much time students spent ? ✔ Knowing if external learning resources are used ? ? 14Thursday 8 December 11
  24. 24. LAK CASE STUDY Bar chart ? Parallel coordinates ? Line chart ? 15Thursday 8 December 11
  25. 25. LAK CASE STUDY 16Thursday 8 December 11
  26. 26. LAK CASE STUDY more metrics verify the status of the classroom activity chronological course dwell time self-reflection to measure progress and increase motivation find students experiencing problems and low more data engagement 16Thursday 8 December 11
  27. 27. LAK CASE STUDY more metrics verify the status of the classroom activity chronological course dwell time self-reflection to measure progress and increase motivation find students experiencing problems and low more data engagement 16Thursday 8 December 11
  28. 28. IDEA’S FROM EXPERTS the used resource types detailed information per student detailed information of 2 students detailed usage stats of resources stats or vis. on content creation 17Thursday 8 December 11
  29. 29. IDEA’S FROM EXPERTS 2 the used resource types 5 detailed information per student 4 detailed information of 2 students 3 detailed usage stats of resources 1 stats or vis. on content creation 17Thursday 8 December 11
  30. 30. CONTINUED WORK... 18Thursday 8 December 11
  31. 31. CONTINUED WORK... 18Thursday 8 December 11
  32. 32. FUTURE WORK • extend SAM with feedback of experts • evaluate the usefulness in a larger group of students • evaluate with teachers during and after a course. 19Thursday 8 December 11
  33. 33. YOUR DATA IN SAM ?! ask me! sten.govaerts@cs.kuleuven.be 20Thursday 8 December 11
  34. 34. 2 slides will appear on http://www.slideshare.net/stengovaerts slides will appear on http://www.slideshare.net/stengovaerts 21Thursday 8 December 11
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×