Your SlideShare is downloading. ×
The feedback loop revisited
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

The feedback loop revisited

376
views

Published on

My PhD presentation for the learn3 project

My PhD presentation for the learn3 project

Published in: Education, Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
376
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Literature review and then found an article about experiences of using a chat tool in the classroom\n
  • Literature review and then found an article about experiences of using a chat tool in the classroom\n
  • Literature review and then found an article about experiences of using a chat tool in the classroom\n
  • \n
  • \n
  • Assessing student learning in unscripted, open-ended environments.\n\n
  • Assessing student learning in unscripted, open-ended environments.\n\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • Record student interaction and use learning analytics to provide teacher with infovis:\nTo better orchestrate the session in enactment: provide feedback in an efficient way\nTo better understand what happened in the session: summative component and improvement of the following sessions\nRecord student interaction and use learning analytics to provide students with infovis:\n To engage them in the sessions\n To make them reflect about the session, knowledge awareness, and the next step they should take (reaction)\n Use other engagement artifacts: badges, etc.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • E.g., publish datavis every week at a certain time\n
  • E.g., publish datavis every week at a certain time\n
  • E.g., publish datavis every week at a certain time\n
  • E.g., publish datavis every week at a certain time\n
  • E.g., publish datavis every week at a certain time\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. The feedback loop revisited Israel Gutiérrez Universidad Carlos III de Madrid
    • 2. Context photo: uc3mcom at flickr
    • 3. ContextLab session practices in computer rooms photo: uc3mcom at flickr
    • 4. Context Lab session practices in computer roomsActive learning activities: guided and open ended photo: uc3mcom at flickr
    • 5. Scenario photo: candicejeanl at flickr
    • 6. ScenarioMultimedia Applications (AMM) photo: candicejeanl at flickr
    • 7. Scenario Multimedia Applications (AMM)Lab sessions are 50% of the lectures photo: candicejeanl at flickr
    • 8. Scenario Multimedia Applications (AMM) Lab sessions are 50% of the lecturesWork in pairs and in a proposed practice photo: candicejeanl at flickr
    • 9. Problems photo: berkeleylab at flickr
    • 10. ProblemsFeedback to students: effective but inefficient photo: berkeleylab at flickr
    • 11. ProblemsFeedback to students: effective but inefficient Issues in the orchestration of the session photo: berkeleylab at flickr
    • 12. Problems photo: hackny at flickr
    • 13. ProblemsFeedback to the class: efficient but ineffective photo: hackny at flickr
    • 14. ProblemsFeedback to the class: efficient but ineffective Students engagement photo: hackny at flickr
    • 15. Possible solutions
    • 16. Possible solutionsUse a backchannel chat to improve communication
    • 17. Possible solutions Use a backchannel chat to improve communication Conole G., and Alevizou P (2010) “A literature review of the use of Web 2.0 tools in HigherEducation”. A report commissioned by the Higher Education Academy.
    • 18. Possible solutions Use a backchannel chat to improve communication Conole G., and Alevizou P (2010) “A literature review of the use of Web 2.0 tools in HigherEducation”. A report commissioned by the Higher Education Academy. Yardi, S. “Whispers in the Classroom." Digital Youth, Innovation, and the Unexpected. Edited byTara McPherson. The John D. and Catherine T. MacArthur Foundation Series on Digital Media andLearning. Cambridge, MA: The MIT Press, 2008. 143–164.
    • 19. Possible solutions
    • 20. Possible solutionsImprove orchestration at enactment by means of lanterns for distributed awareness
    • 21. Possible solutions Improve orchestration at enactment by means of lanterns for distributed awareness Alavi, H. S., Dillenbourg, P., & Kaplan, F. (2009). Distributed awareness for class orchestration.Learning in the Synergy of Multiple Disciplines -- Proceedings of the 4th European Conference onTechnology Enhanced Learning (pp. 211-225). Springer.
    • 22. !" Possible solutions Paulo Blikstein" ! #$%&(")*"+,("-./("012$%15.6"7,$8,"1==.74"(4(18,(4"5."%."<18?"19/"@.5,"$9"5$; 2 1 ( 1 ;("518?$9%"
    • 23. !" Possible solutions Paulo Blikstein" Learning analytics for assessment ! #$%&(")*"+,("-./("012$%15.6"7,$8,"1==.74"(4(18,(4"5."%."<18?"19/"@.5,"$9"5$; 2 1 ( 1 ;("518?$9%"
    • 24. !" Possible solutions Paulo Blikstein" Learning analytics for assessment ! #$%&(")* P. (2011). Using learning analytics to assess students ’ behavior in open ‐ ended programming Blikstein, "+,("-./("012$%15.6"7,$8,"1==.74"(4(18,(4"5."%."<18?"19/"@.5,"$9"5$; 2 1 ( 1 ;("518?$9%" tasks. Proceedings of the 1st International Conference on Learning Analytics and Knowledge 2011
    • 25. Our solution Impact for Impact forMeasures students teachers
    • 26. Our solutionOne-way websocket notifications Impact for Impact for Measures students teachers
    • 27. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedback Impact for Impact for Measures students teachers
    • 28. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Impact for Impact for Measures students teachers
    • 29. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Impact for Impact for Measures students teachers
    • 30. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Impact for Impact for Measures students teachers
    • 31. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Impact for Impact for Measures students teachers
    • 32. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
    • 33. Our solutionOne-way websocket notifications Improve attention and effectiveness of general feedbackclassONRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (support of other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
    • 34. classON
    • 35. classONaCtive Learning ASSessmentOrchestration & Notifications
    • 36. classON aCtive Learning ASSessment Orchestration & NotificationsEnactment phase: on class
    • 37. classON aCtive Learning ASSessment Orchestration & NotificationsEnactment phase: on class Activate the class: improve orchestration
    • 38. classON aCtive Learning ASSessment Orchestration & NotificationsEnactment phase: on class Activate the class: improve orchestration Activate the class: students’ engagement
    • 39. classON
    • 40. classON Formativeassessment and quick feedback
    • 41. classON Formative HTML5 &assessment web and quick technologies feedback
    • 42. classON Formative HTML5 & Attention &assessment web notification and quick technologies system feedback
    • 43. classON Formative HTML5 & Attention &assessment web notification and quick technologies system feedback Learning analytics
    • 44. classON Formative HTML5 & Attention &assessment web notification and quick technologies system feedback Orchestrating Learning learning at analytics enactment
    • 45. classON Formative HTML5 & Attention &assessment web notification and quick technologies system feedback Orchestrating Learning Data learning at analytics visualization enactment
    • 46. Formative assessment and feedback
    • 47. Formative assessment and feedbackQuick &efficient
    • 48. Formative assessment and feedback 21st Century skillsQuick &efficient
    • 49. HTML5 & web technologies
    • 50. HTML5 & web technologies• Web notifications for forefront messages
    • 51. HTML5 & web technologies• Web notifications for forefront messages• Web sockets for fast/real-time communication (datavis in real time)
    • 52. HTML5 & web technologies• Web notifications for forefront messages• Web sockets for fast/real-time communication (datavis in real time)• Key-value databases for a fast and efficient backend: MongoDB
    • 53. HTML5 & web technologies• Web notifications for forefront messages• Web sockets for fast/real-time communication (datavis in real time)• Key-value databases for a fast and efficient backend: MongoDB• JavaScript libraries for datavis: d3, protovis...
    • 54. Attention & notification system
    • 55. Attention & notification system• Web notifications in order to make info available without changing attention (supported by websockets)
    • 56. Attention & notification system• Web notifications in order to make info available without changing attention (supported by websockets)• One-click subscription to the notifications channel from the wording of the session
    • 57. Attention & notification system• Web notifications in order to make info available without changing attention (supported by websockets)• One-click subscription to the notifications channel from the wording of the session• Simple interface for the teacher to send notifications to all the connected students
    • 58. Experience
    • 59. Experience• AMM 2011
    • 60. Experience• AMM 2011• 2 sessions
    • 61. Experience• AMM 2011• 2 sessions• Increasing complexity
    • 62. Experience• AMM 2011• 2 sessions• Increasing complexity• Increasing interest
    • 63. Experience• AMM 2011• 2 sessions• Increasing complexity• Increasing interest
    • 64. Future works
    • 65. Future works• Attention improvement delaying notification delivery
    • 66. Future works• Attention improvement delaying notification delivery• Intelligent Tutoring System based on common problems (notifications log)
    • 67. Future works• Attention improvement delaying notification delivery• Intelligent Tutoring System based on common problems (notifications log)• Collection of more student events and create real time visualizations
    • 68. Keywords of the ECTEL’11 short paper
    • 69. Learning analytics
    • 70. Learning analyticsLearning analytics is the measurement, collection,analysis and reporting of data about learners andtheir contexts, for purposes of understanding andoptimising learning and the environments inwhich it occurs. CfP of Conference on Learning Analytics and Knowledge 2011
    • 71. What toinstrumentalize?
    • 72. What to instrumentalize?• Lab session wording containing event collection in JavaScript
    • 73. What to instrumentalize?• Lab session wording containing event collection in JavaScript• Login info from LMS in order to associate events to a student id
    • 74. Event collection interface !
    • 75. Event collection interface ! Buttons to start, finalize andindicate a problem in an activity
    • 76. Event collection interface ! Buttons to start, finalize and Todo listindicate a problem in an activity of tasks
    • 77. Recorded Events
    • 78. Recorded Events• Access to the wording
    • 79. Recorded Events• Access to the wording• Start working on a section
    • 80. Recorded Events• Access to the wording• Start working on a section• Finish working in a section
    • 81. Recorded Events• Access to the wording• Start working on a section• Finish working in a section• I have a problem / I solved the problem
    • 82. Recorded Events• Access to the wording• Start working on a section• Finish working in a section• I have a problem / I solved the problem• How the problem was solved: individual effort, peer interaction or teacher feedback
    • 83. Recorded Events • Access to the wording • Start working on a section • Finish working in a section • I have a problem / I solved the problem • How the problem was solved: individual effort, peer interaction or teacher feedbackHow to translate that into learning outcomes achievement?
    • 84. Contextual data• Physical distribution of the labs
    • 85. Contextual data• Physical distribution of the labs !
    • 86. Orchestrating learning at enactment
    • 87. Orchestrating learning at enactmentThe metaphor of “orchestrating learning” hasbeen frequently used to address the challengesof coordinating the complexity of conductinglearning activities. Stellar D1.1
    • 88. Teacher interface !
    • 89. Teacher information
    • 90. Teacher information• Progress
    • 91. Teacher information• Progress• Stuck in a problem
    • 92. Teacher information• Progress• Stuck in a problem• Slow progress
    • 93. Teacher information• Progress• Stuck in a problem• Slow progress• Consumed tutoring time
    • 94. Teacher information• Progress• Stuck in a problem• Slow progress• Consumed tutoring time• Teacher location
    • 95. Teacher information• Progress• Stuck in a problem• Slow progress• Consumed tutoring time• Teacher location• Next step
    • 96. Teacher information• Progress 3• Stuck in a problem• Slow progress• Consumed tutoring time• Teacher location• Next step
    • 97. Teacher information• Progress 3• Stuck in a problem 3• Slow progress• Consumed tutoring time• Teacher location• Next step
    • 98. Teacher information• Progress 3• Stuck in a problem 3• Slow progress• Consumed tutoring time• Teacher location• Next step
    • 99. Teacher information• Progress 3• Stuck in a problem 3• Slow progress• Consumed tutoring time 2’• Teacher location• Next step
    • 100. Teacher information• Progress 3• Stuck in a problem 3• Slow progress• Consumed tutoring time 2’• Teacher location• Next step
    • 101. Teacher information• Progress 3• Stuck in a problem 3• Slow progress• Consumed tutoring time 2’• Teacher location• Next step
    • 102. Teacher interface !
    • 103. Workflow
    • 104. Workflow• Use classON app to monitor student problems and state in real time
    • 105. Workflow• Use classON app to monitor student problems and state in real time• In a normal situation teacher decides to
    • 106. Workflow• Use classON app to monitor student problems and state in real time• In a normal situation teacher decides to • Directly help one of the teams
    • 107. Workflow• Use classON app to monitor student problems and state in real time• In a normal situation teacher decides to • Directly help one of the teams • Make one team help another one
    • 108. Workflow• Use classON app to monitor student problems and state in real time• In a normal situation teacher decides to • Directly help one of the teams • Make one team help another one • Send a notification (since it’s a common problem)
    • 109. Data visualization
    • 110. Data visualizationInformation visualization is the interdisciplinarystudy of the visual representation of large-scale collections of information. Michael Friendly (2008). "Milestones in the history of thematic cartography, statistical graphics, and data visualization"
    • 111. IntelligenceAmplification(W. Ross Ashby, 1956) Visualization Amplifies Cognition (@infosthetics, last Monday)
    • 112. IntelligenceAmplification(W. Ross Ashby, 1956) rs ! h e a c r te l fo Visualization o o t Amplifies Cognition A (@infosthetics, last Monday)
    • 113. Critiques to datavis
    • 114. Critiques to datavis• Lack of usability and usefulness
    • 115. Critiques to datavis• Lack of usability and usefulness• Not created for the average people
    • 116. Critiques to datavis• Lack of usability and usefulness• Not created for the average people• Noise
    • 117. Datavis for students
    • 118. Datavis for students• Aesthetically attractive
    • 119. Datavis for students• Aesthetically attractive• Fluent (straightforward and simple interaction)
    • 120. Datavis for students• Aesthetically attractive• Fluent (straightforward and simple interaction)• Make them reflect (in and out of class)
    • 121. Datavis for students• Aesthetically attractive• Fluent (straightforward and simple interaction)• Make them reflect (in and out of class) • Awareness of their knowledge and relate it to the average
    • 122. Datavis for students• Aesthetically attractive• Fluent (straightforward and simple interaction)• Make them reflect (in and out of class) • Awareness of their knowledge and relate it to the average • React with an designed (re)action
    • 123. Summing up: classONOne-way websocket notifications Improve attention and effectiveness of general feedbackRecord student interactions and use learning analytics Improve orchestration in enactment and provision of efficient feedback Improve orchestration design for the following session and acquiring evidences for summative assessment Engagement in lab sessions (also other artifacts) Awareness, reflection and reaction Impact for Impact for Measures students teachers
    • 124. Next steps
    • 125. Next steps• Define datavis for teachers’ and students’ interactions
    • 126. Next steps• Define datavis for teachers’ and students’ interactions• Development of a classON prototype
    • 127. Next steps• Define datavis for teachers’ and students’ interactions• Development of a classON prototype • Save events for every interaction with the system (even events manually annotated)
    • 128. Next steps• Define datavis for teachers’ and students’ interactions• Development of a classON prototype • Save events for every interaction with the system (even events manually annotated)• Use dataset to test the prototype
    • 129. Evaluation
    • 130. Evaluation• AMM 2012
    • 131. Evaluation• AMM 2012• Control and experiment groups OR competition between groups
    • 132. Evaluation• AMM 2012• Control and experiment groups OR competition between groups• How can we measure engagement
    • 133. Evaluation• AMM 2012• Control and experiment groups OR competition between groups• How can we measure engagement • Qualitative feedback
    • 134. Thank you very much! Any questions?