Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

EMPOWER E-Assessment week: Viewbrics


Published on

Viewbrics: mirroring and mastering complex generic skills with video enhanced rubrics through a technology-enhanced formative assessment methodology by Ellen Rusman (OUNL).

Published in: Education
  • Be the first to comment

  • Be the first to like this

EMPOWER E-Assessment week: Viewbrics

  1. 1. Viewbrics: mirroring and mastering complex generic skills with video enhanced rubrics through a technology-enhanced formative assessment methodology Dr. Ellen Rusman Welten Institute, Open University of the Netherlands & Viewbrics project team Webinar during EMPOWER e-Assesssment week, 28th of June 2018, online: ‘Like master, like man’
  2. 2. Complex generic (transversal) skills Complex skills consist of constituent subskills which concertation require high cognitive effort and concentration [1][2], and prolonged repetitive practice in order to master them. Complex generic (also ‘transversal’ or ‘21st century’) skills are not specific for a domain, occupation or type of task, but important for all kinds of work, education and life in general. These skills are applicable in a broad range of situations and many subject domains [3]. => many schools are struggling with how to teach and evaluate students’ performance of complex generic/transversal skills in their daily educational practice [4].
  3. 3. Who’s interested in this webinar? Are you a: A. Teacher in Higher Education B. Manager in Higher Education C. Researcher of Higher Education D. Advisor/trainer in Higher Education E. Otherwise, namely: ………………………
  4. 4. Your opinion: Agree/disagree/neutral? It is important for students to learn complex generic skills I know how I can support students’ learning complex generic skills in my lessons I have enough educational methods and material available to support students while learning complex generic skills
  5. 5.
  6. 6. New Zealand
  7. 7. However…. • many aspects of complex skills mastery refer to motoric activities, time-consecutive operations and processes that are hardly captured in text (e.g. body posture or use of voice during a presentation) • context in which the skill is practiced is important for its learning, as it implies and generates implicit knowledge (tacit knowledge, ‘knowing how/’knowing why’), which is interwoven with practical activities, operations and behavior in the physical world [11]. • text supposedly leaves more space for personal interpretation of the performance indicators of a complex skill than video => text-based rubrics only have a restricted capacity to clarify the targeted mastery level of a skill and to assess shown behaviour [12].
  8. 8. Towards Video-Enhanced Rubrics (VER) A Video-Enhanced Rubric (VER) is the synthesis of video modelling examples and a text-based analytic rubric in a digital formative assessment format. We expect that: • video-enhanced rubrics can foster learning from observation of (good/bad) video modelling examples [11] [12][13], thus supporting (more concrete and consistent) mental model formation • improved quality of feedback (concreteness, consistency & perceived quality) given by teachers or peers during skills training, => more effective skill mastery
  9. 9. Financed by:
  10. 10. Two main ‘research phases’: 1. Design research (ended) 2. Semi-experimental research (now in data collection phase) Design research for development of: - Ecologically valid rubrics for 3 skills - Video-modelling examples - Feedback-and reflection methodology
  11. 11. Design-based research approach
  12. 12. Presenting Information literacyCollaborating
  13. 13. Evaluation of rubrics with teachers in Viewbrics project team
  14. 14. Evaluation of language & feasibility of rubrics with pupils
  15. 15. Skills hierarchy ‘Collaborating’ Complex skill Skill cluster Sub-skills
  16. 16. Skills hierarchy and rubric ‘Collaborating’ Mastery level descriptions
  17. 17. Skills hierarchies for ‘Information literacy’ & ‘Presenting’ skills
  18. 18. Design based research
  19. 19. Online feedback & reflection tool • Make rubrics and video modeling examples for complex generic skills available to support practicing skills • Prevent piles of ‘paper work’ • Convenient, regular and structured feedback in a ‘set’ format while practicing skills: • Video-enhanced rubrics • Dashboard with automatically summarized, immediate and timely personal feedback • Insight in what goes well and what may still be improved by means of ‘reflection wheel’ visualization and (essential) tips and tops • Individual reflection support on performances and mastery of skills • Automatic and well-organized registration and archiving of performances and growth in mastery of skills
  20. 20. Evaluation of Viewbrics methodology with pupils (Sint Jan)
  21. 21. Pilot evaluation of Viewbrics with pupils - Grotius
  22. 22. Pilot evaluation of Viewbrics with pupils - Grotius
  23. 23. Answering questionnaires - Grotius
  24. 24. Assessing a ‘dummy student’s’ presentation performance - Porta Mosana
  25. 25. Pilot evaluation of Viewbrics with teachers and pupils – Porta Mosana
  26. 26. Describing experiences with Viewbrics tool – pupils Porta Mosana
  27. 27. A positive evaluation of the Viewbrics tool – pupils Porta Mosana
  28. 28. The result: feedback- and reflection process in online tool 1. Watch video-enhanced rubrics 2. Practice skills 3. Assess own performance 4. Look at and analyze feedback of teacher and peers 5. Determine (next) learning objectives
  29. 29. c Messages with tasks
  30. 30. Looking at the video-modeling examples with processing support and ‘linking’ questions (linking video modeling examples to sub-skills in rubric)
  31. 31. Processing support and ‘linking’ question ‘Linking’ video fragments to sub-skills
  32. 32. Answering the ‘linking’ questionAnswering the ‘linking’ question
  33. 33. Providing rubric description of the highest mastery level as answer model to pupils
  34. 34. • Watch complete video without questions • Reviewing video fragments organized by sub-skill
  35. 35. Practicing a complex generic skill: ‘Presenting’
  36. 36. Self-assessment on performance
  37. 37. Skill-cluster Sub-skill
  38. 38. Providing feedback to other pupils
  39. 39. Research is still ongoing, first results on effectiveness expected October 2018
  40. 40. Your opinion: Agree/disagree/neutral? Viewbrics can help students to provide insight in their growth of complex generic skills. I think Viewbrics would also be applicable in the context of higher education I would like to know more about Viewbrics
  41. 41. Video:
  42. 42. Also interested? Contact us! Project leader/principal investigator @EllenRusman More info on Viewbrics: website: Do you want to stay updated? Register for updates via:
  43. 43. Thanks for your attention! We would like to gratefully acknowledge the funding of our project by the practice-oriented research programme of the Netherlands Initiative for Education Research (NRO), part of The Netherlands Organisation for Scientific Research (NWO).
  44. 44. References [1] P. Kirschner and J. Van Merriënboer, “Ten Steps to Complex Learning A New Approach to Instruction and Instructional Design,” in In T. L. Good (Ed.), 21st century education: A reference handbook, Thousand Oaks, CA: Sage., 2008, pp. 244–253. [2] D. Galligan, Frank; Maskery, Colin; Spence, Jon; Howe, David; Barry, Tim; Ruston, Andy; Crawford, Advanced PE for Edexcel. Oxford: Heinemann Educational Publishers, 2000. [3] K. Bowman, “Background paper for the AQF Council on generic skills,” Canberra, 2010 [4] E. Rusman, A. Martínez-monés, J. Boon, M. J. Rodríguez-triana, and S. Villagrá-sobrino, “Gauging Teachers ’ Needs with Regard to Technology- Enhanced Formative Assessment ( TEFA ) of 21 st Century Skills in the Classroom,” in Computer Assisted Assessment. Research into E-Assessment. Proceedings of the International Computer Assisted Assessment (CAA) Conference, 2014, pp. 1–14. [6] C. Sluijsmans, Dominique; Joosten-ten Brinke, Désirée; Van der Vleuten, “Toetsen met leerwaarde,” Heerlen, Maastricht, 2013. [7] J. J. G. Van Merriënboer, Training Complex Cognitive Skills. Educational Technology, 1997. [11] W. Westera, “On the Changing Nature of Learning Context: Anticipating the Virtual Extensions of the World.,” Educ. Technol. Soc., vol. 14, no. 2, pp. 201–212, 2011. [12] O. Berry, M. Price, C. Rust, B. O. Donovan, M. Price, and C. Rust, “Teaching in Higher Education: Know what I mean ? Enhancing student understanding of assessment standards and criteria,” Teach. High. Educ., no. March 2015, pp. 37–41, 2007. [13] H. Rohbanfard and L. Proteau, “Live versus video presentation techniques in the observational learning of motor skills,” Trends Neurosci. Educ., vol. 2, no. 1, pp. 27–32, 2013.