Downloaded 36 times
![OBJECTIVES
• self-monitoring for learners
• awareness for teachers
• time tracking
• learning
resource use and
recommendations
http://free-extras.com/images/obama_hope_progress-2848.htm
• part of Learning Analytics research [ACM LAK conf., Siemens 2011, Duval 2011]
4](https://image.slidesharecdn.com/chi2012-sam-120518031239-phpapp01/75/The-Student-Activity-Meter-for-Awareness-and-Self-reflection-9-2048.jpg)
![METHODOLOGY.
• design-basedresearch [Obrenović 2011]: design a tool, give
students & teachers and evaluate
• iterative process of design & evaluation
• paper mockups
• usability studies
• surveys on usefulness
http://funnytogo.com/pictures/evolution/timeline.htm
9](https://image.slidesharecdn.com/chi2012-sam-120518031239-phpapp01/75/The-Student-Activity-Meter-for-Awareness-and-Self-reflection-14-2048.jpg)
![USABILITY & SATISFACTION
• average SUS score: 69,69%
[Bangor et al. 2008]
20](https://image.slidesharecdn.com/chi2012-sam-120518031239-phpapp01/75/The-Student-Activity-Meter-for-Awareness-and-Self-reflection-35-2048.jpg)
![USABILITY & SATISFACTION
• average SUS score: 69,69%
all: want to continue using it
9/11: give it to students
[Bangor et al. 2008]
20](https://image.slidesharecdn.com/chi2012-sam-120518031239-phpapp01/75/The-Student-Activity-Meter-for-Awareness-and-Self-reflection-36-2048.jpg)
More Related Content
Similar to The Student Activity Meter for Awareness and Self-reflection
The Student Activity Meter for Awareness and Self-reflection
- 1. THE STUDENT ACTIVITY METERFOR AWARENESS AND SELF-REFLECTION Sten Govaerts, Katrien Verbert, Erik Duval, Abelardo Pardo. SigCHI 2012, Long Case Study. 1
- 2. our team: Human-Computer Interaction •technology enhanced learning • music • research 2.0 2
- 3.
- 4.
- 5. OBJECTIVES • self-monitoring for learners http://free-extras.com/images/obama_hope_progress-2848.htm 4
- 6. OBJECTIVES • self-monitoring for learners • awareness for teachers http://free-extras.com/images/obama_hope_progress-2848.htm 4
- 7. OBJECTIVES • self-monitoring for learners • awareness for teachers • time tracking http://free-extras.com/images/obama_hope_progress-2848.htm 4
- 8. OBJECTIVES • self-monitoring for learners • awareness for teachers • time tracking • learning resource use and recommendations http://free-extras.com/images/obama_hope_progress-2848.htm 4
- 9. OBJECTIVES • self-monitoring for learners • awareness for teachers • time tracking • learning resource use and recommendations http://free-extras.com/images/obama_hope_progress-2848.htm • part of Learning Analytics research [ACM LAK conf., Siemens 2011, Duval 2011] 4
- 10. STUDENT ACTIVITY METER (SAM): DEMO. http://dilbert.com/strips/comic/2000-02-24/ 5
- 11.
- 12. GISMO http://www.noe-kaleidoscope.org/pub/researcher/resources/video/aied/mazza.html
- 13. The Participation Tool http://www.cs.uu.nl/docs/vakken/uem/Janssen.pdf
- 14. METHODOLOGY. • design-basedresearch [Obrenović2011]: design a tool, give students & teachers and evaluate • iterative process of design & evaluation • paper mockups • usability studies • surveys on usefulness http://funnytogo.com/pictures/evolution/timeline.htm 9
- 15. iter. evaluation goal methodology demo-graphics data usability, satisfaction, task-based interview 12 CS students timetracking I. preliminary with think-aloud, SUS at KULeuven via Twitter usefulness & MSDT assessing teacher online survey with 19 teachers & II. needs, use & Likert items, yes/no TA’s with Moodle logs usefulness and open questions CGIAR courses assessing teacher online survey with 12 participants needs, expert III. Likert items, yes/no from LAK Moodle logs feedback, use & and open questions MOOC usefulness structured face-to-face 11 teachers & VM with use, usefulness & IV. interview with tasks & TA’s at Carlos III extensive satisfaction open questions Uni. Madrid tool tracking 10
- 16.
- 17.
- 18. demogra evaluation design negative positive phics goal changes •↑learnability usability, •↓errors 12 CS satisfaction, small usability I. 1st iteration •good satisfaction students preliminary issues •usefulness usefulness positive
- 19. ITERATION TWO & THREE • CGIAR: 19 teachers - LAK: 12 • An online survey about usefulness, teacher issues and how the tool can resolve these. 13
- 20. ITERATION TWO & THREE • CGIAR: 19 teachers - LAK: 12 • An online survey about usefulness, teacher issues and how the tool can resolve these. 13
- 21. CGIAR LAK 1 2 3 4 issue for2 add- issue for add- 5 1 3 4 5 teacher ressed teacher ressed Provide feedback to the Provide feedback to the students students ✔ ? ✔ ✔ Being aware of what Being aware of whatdoing students are students are doing ✔ ✔ ✔ ? ✗ ✗ Knowing about collaboration Knowing and communication * and communication about collaboration ✔ ✔ ? Knowing which documents Knowing are used and how much used and how much which documents are ✔ ✔ ✔ ? ✗ ? Knowing how and when Knowing online and whenbeen used have been used how tools have online tools ✔ Finding the students who are not doing well Finding the students who are not doing well ✔ ? ✔ ? Finding the best students Finding the best students Knowing how much time ? ? ? ✗ ? ✔ ? students spent Knowing Knowing if externalstudents how much time learning spent ✔ ? ? ? resources are used Knowing if external learning resources areIssues used ✔Addressed by SAM 14
- 22. CGIAR LAK issue for add- issue for add- teacher ressed teacher ressed Provide feedback to the students ✔ ? ✔ ✔ Being aware of what students are doing ✔ ✔ ✔ ? Knowing about collaboration and communication ✔ ✗ ✔ ✗ Knowing which documents are used and how much ✔ ✔ ✔ ? Knowing how and when online tools have been used ? ✔ ✗ ? Finding the students who are not doing well ✔ ? ✔ ? Finding the best students ? ? ? ✗ Knowing how much time students spent ? ✔ ? ✔ Knowing if external learning resources are used ✔ ? ? ? 14
- 23. CGIAR LAK issue for add- issue for add- teacher ressed teacher ressed Provide feedback to the students ✔ ? ✔ ✔ Being aware of what students are doing ✔ ✔ ✔ ? Knowing about collaboration and communication ✔ ✗ ✔ ✗ Knowing which documents are used and how much ✔ ✔ ✔ ? Knowing how and when online tools have been used ? ✔ ✗ ? Finding the students who are not doing well ✔ ? ✔ ? Finding the best students ? ? ? ✗ Knowing how much time students spent ? ✔ ? ✔ Knowing if external learning resources are used ✔ ? ? ? 14
- 24.
- 25. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 15
- 26. CGIAR CASE STUDY want to search detect outliers for students more details on good indicator for effort student understand the workload progress evolution compare students obtain course overview increase awareness want better 1 to 1 comparison tool use for course more metrics design optimization wanted 15
- 27. LAK CASE STUDY 16 http://maranash.blogspot.com/2011/01/conference-shout-out.html
- 28. LAK CASE STUDY find students experiencing problems and low verify the status of the engagement classroom activity more chronological course dwell time metrics self-reflection to measure progress and increase motivation more data 16 http://maranash.blogspot.com/2011/01/conference-shout-out.html
- 29. LAK CASE STUDY find students experiencing problems and low verify the status of the engagement classroom activity more chronological course dwell time metrics self-reflection to measure progress and increase motivation more data 16 http://maranash.blogspot.com/2011/01/conference-shout-out.html
- 30. demogra evaluation design negative positive phics goal changes usability, •↑learnability 12 CS satisfaction, small usability •↓errors I. 1st iteration students preliminary issues •good satisfaction usefulness •usefulness positive assessing •provides awareness 19 resource teacher •all vis. useful II. teachers help function recomm. not needs, use & •many uses & TA’s useful usefulness •90% want it •provides awareness assessing re-orderable most and feedback 12 teacher parallel addressed •many uses III. participan needs, expert coordinates needs are •66% want it ts feedback, use with indecisive •recomm. can be & usefulness histograms useful
- 31. ITERATION FOUR •a CS course on C++ programming • 11 people: 7 teachers, 2 TA’s & 1 course planner • richer data set: tracking from programming environment • qualitative study using a structured face-2-face interview with 25 open questions. 18
- 32. ITERATION FOUR •a CS course on C++ programming • 11 people: 7 teachers, 2 TA’s & 1 course planner • richer data set: tracking from programming environment • qualitative study using a structured face-2-face interview with 25 open questions. 18
- 33. ACHIEVABLE TASKS conflicting visionson get a time identify students doing students doing well summary well and at risk and at risk better course monitor tool and overview resource use understand time spent better detect students course planning for assistance http://www.tdpri.com/forum/telecaster-discussion- forum/48431-ic503z-cap-52-ri-blackguard-book.html 19
- 34. ACHIEVABLE TASKS conflicting visionson get a time identify students doing students doing well summary well and at risk and at risk better course monitor tool and overview resource use understand time spent better detect students course planning for assistance http://www.tdpri.com/forum/telecaster-discussion- forum/48431-ic503z-cap-52-ri-blackguard-book.html 19
- 35. USABILITY & SATISFACTION •average SUS score: 69,69% [Bangor et al. 2008] 20
- 36. USABILITY & SATISFACTION •average SUS score: 69,69% all: want to continue using it 9/11: give it to students [Bangor et al. 2008] 20
- 37. demo- evaluation design negative positive graphics goal changes usability, •↑learnability 12 CS satisfaction, small usability •↓errors I. 1st iteration students preliminary issues •good satisfaction usefulness •usefulness positive •provides awareness 19 assessing teacher resource •all vis. useful II. teachers needs, use & help function recomm. not •many uses & TA’s usefulness useful •90% want it •provides awareness and assessing teacher most 12 re-orderable feedback needs, expert addressed III. participan PC with •many uses feedback, use & needs are ts histograms •66% want it usefulness indecisive •recomm. can be useful filter & •provides time overview search, conflicting •provides course 11 icons, visions of overview use, usefulness IV. teacher zooming in students •PC assist with detecting & satisfaction s & TA’s line chart, doing well problems editing PC or at risk •many uses & insights axes •100% want it
- 38. CONCLUSION • SAM is perceived useful and contributes to better awareness. 22
- 39. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: 22
- 40. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk 22
- 41. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview 22
- 42. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending 22
- 43. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights 22
- 44. CONCLUSION • SAM is perceived useful and contributes to better awareness. • SAM enables: • to detect potential students doing well and at risk • a better course overview • understanding student time spending • to find a wide variety of new insights • almost all want to continue using SAM 22
- 45. LESSONS LEARNED • worked well: 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 46. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 47. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 48. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings • Visualisations are powerful tool for creating awareness 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 49. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings • Visualisations are powerful tool for creating awareness • worked less well: 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 50. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings • Visualisations are powerful tool for creating awareness • worked less well: • finding real-world settings 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 51. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings • Visualisations are powerful tool for creating awareness • worked less well: • finding real-world settings • getting tracking data 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 52. LESSONS LEARNED • worked well: • Evaluate visualisations iteratively and early • Use real-world settings • Visualisations are powerful tool for creating awareness • worked less well: • finding real-world settings • getting tracking data • longitudinal study would be useful 23 http://slurmed.com/fanart/dexus5/005_professor.png
- 53. http://www.fitbit.com/ http://media.cnbc.com/i/CNBC/Sections/ News_And_Analysis/_News/_SPECIAL_REPORTS/ BetterYourBusiness_2012/BYB-Digital%20- http://quantifiedself.com/ %20April/Slideshows/Digital_Appliances_ab/10- Smart-Dig-Appliances-Wifi-scale-.jpg http://nikerunning.nike.com/nikeos/p/nikeplus/en_EMEA/sportband 24
- 54. LAK13: Learning Analyticsand Knowledge Conf. 8-12 April 2013 Leuven, Belgium http://lakconference.org/
- 55. 2 contact: sten@cs.kuleuven.be slides will appear on http://www.slideshare.net/stengovaerts @erikduval slides will appear on http://www.slideshare.net/stengovaerts 26
- 56. If you areinterested in visualisations and 2 tabletops, we have a PhD position available. contact: sten@cs.kuleuven.be slides will appear on http://www.slideshare.net/stengovaerts @erikduval slides will appear on http://www.slideshare.net/stengovaerts 26