Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

EMPOWER E-Assessment week: Viewbrics

Download to read offline

Viewbrics: mirroring and mastering complex generic skills with video enhanced rubrics through a technology-enhanced formative assessment methodology by Ellen Rusman (OUNL).

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

EMPOWER E-Assessment week: Viewbrics

  1. 1. Viewbrics: mirroring and mastering complex generic skills with video enhanced rubrics through a technology-enhanced formative assessment methodology Dr. Ellen Rusman Welten Institute, Open University of the Netherlands & Viewbrics project team Webinar during EMPOWER e-Assesssment week, 28th of June 2018, online: https://empower.eadtu.eu/events/100-empower-events ‘Like master, like man’
  2. 2. Complex generic (transversal) skills Complex skills consist of constituent subskills which concertation require high cognitive effort and concentration [1][2], and prolonged repetitive practice in order to master them. Complex generic (also ‘transversal’ or ‘21st century’) skills are not specific for a domain, occupation or type of task, but important for all kinds of work, education and life in general. These skills are applicable in a broad range of situations and many subject domains [3]. => many schools are struggling with how to teach and evaluate students’ performance of complex generic/transversal skills in their daily educational practice [4].
  3. 3. Who’s interested in this webinar? Are you a: A. Teacher in Higher Education B. Manager in Higher Education C. Researcher of Higher Education D. Advisor/trainer in Higher Education E. Otherwise, namely: ………………………
  4. 4. Your opinion: Agree/disagree/neutral? It is important for students to learn complex generic skills I know how I can support students’ learning complex generic skills in my lessons I have enough educational methods and material available to support students while learning complex generic skills
  5. 5. https://commons.wikimedia.org/wiki/File:Rubric.jpg
  6. 6. New Zealand
  7. 7. However…. • many aspects of complex skills mastery refer to motoric activities, time-consecutive operations and processes that are hardly captured in text (e.g. body posture or use of voice during a presentation) • context in which the skill is practiced is important for its learning, as it implies and generates implicit knowledge (tacit knowledge, ‘knowing how/’knowing why’), which is interwoven with practical activities, operations and behavior in the physical world [11]. • text supposedly leaves more space for personal interpretation of the performance indicators of a complex skill than video => text-based rubrics only have a restricted capacity to clarify the targeted mastery level of a skill and to assess shown behaviour [12].
  8. 8. Towards Video-Enhanced Rubrics (VER) A Video-Enhanced Rubric (VER) is the synthesis of video modelling examples and a text-based analytic rubric in a digital formative assessment format. We expect that: • video-enhanced rubrics can foster learning from observation of (good/bad) video modelling examples [11] [12][13], thus supporting (more concrete and consistent) mental model formation • improved quality of feedback (concreteness, consistency & perceived quality) given by teachers or peers during skills training, => more effective skill mastery
  9. 9. Financed by:
  10. 10. Two main ‘research phases’: 1. Design research (ended) 2. Semi-experimental research (now in data collection phase) Design research for development of: - Ecologically valid rubrics for 3 skills - Video-modelling examples - Feedback-and reflection methodology
  11. 11. Design-based research approach
  12. 12. Presenting Information literacyCollaborating
  13. 13. Evaluation of rubrics with teachers in Viewbrics project team
  14. 14. Evaluation of language & feasibility of rubrics with pupils
  15. 15. Skills hierarchy ‘Collaborating’ Complex skill Skill cluster Sub-skills
  16. 16. Skills hierarchy and rubric ‘Collaborating’ Mastery level descriptions
  17. 17. Skills hierarchies for ‘Information literacy’ & ‘Presenting’ skills
  18. 18. Design based research
  19. 19. Online feedback & reflection tool • Make rubrics and video modeling examples for complex generic skills available to support practicing skills • Prevent piles of ‘paper work’ • Convenient, regular and structured feedback in a ‘set’ format while practicing skills: • Video-enhanced rubrics • Dashboard with automatically summarized, immediate and timely personal feedback • Insight in what goes well and what may still be improved by means of ‘reflection wheel’ visualization and (essential) tips and tops • Individual reflection support on performances and mastery of skills • Automatic and well-organized registration and archiving of performances and growth in mastery of skills
  20. 20. Evaluation of Viewbrics methodology with pupils (Sint Jan)
  21. 21. Pilot evaluation of Viewbrics with pupils - Grotius
  22. 22. Pilot evaluation of Viewbrics with pupils - Grotius
  23. 23. Answering questionnaires - Grotius
  24. 24. Assessing a ‘dummy student’s’ presentation performance - Porta Mosana
  25. 25. Pilot evaluation of Viewbrics with teachers and pupils – Porta Mosana
  26. 26. Describing experiences with Viewbrics tool – pupils Porta Mosana
  27. 27. A positive evaluation of the Viewbrics tool – pupils Porta Mosana
  28. 28. The result: feedback- and reflection process in online tool 1. Watch video-enhanced rubrics 2. Practice skills 3. Assess own performance 4. Look at and analyze feedback of teacher and peers 5. Determine (next) learning objectives
  29. 29. c Messages with tasks
  30. 30. Looking at the video-modeling examples with processing support and ‘linking’ questions (linking video modeling examples to sub-skills in rubric)
  31. 31. Processing support and ‘linking’ question ‘Linking’ video fragments to sub-skills
  32. 32. Answering the ‘linking’ questionAnswering the ‘linking’ question
  33. 33. Providing rubric description of the highest mastery level as answer model to pupils
  34. 34. • Watch complete video without questions • Reviewing video fragments organized by sub-skill
  35. 35. Practicing a complex generic skill: ‘Presenting’
  36. 36. Self-assessment on performance
  37. 37. Skill-cluster Sub-skill
  38. 38. Providing feedback to other pupils
  39. 39. Research is still ongoing, first results on effectiveness expected October 2018
  40. 40. Your opinion: Agree/disagree/neutral? Viewbrics can help students to provide insight in their growth of complex generic skills. I think Viewbrics would also be applicable in the context of higher education I would like to know more about Viewbrics
  41. 41. Video: https://www.youtube.com/watch?v=nnQb7zSPkmg
  42. 42. Also interested? Contact us! Ellen.Rusman@ou.nl Project leader/principal investigator @EllenRusman More info on Viewbrics: website: www.viewbrics.nl Do you want to stay updated? Register for updates via: Mieke.Haemers@ou.nl
  43. 43. Thanks for your attention! We would like to gratefully acknowledge the funding of our project by the practice-oriented research programme of the Netherlands Initiative for Education Research (NRO), part of The Netherlands Organisation for Scientific Research (NWO).
  44. 44. References [1] P. Kirschner and J. Van Merriënboer, “Ten Steps to Complex Learning A New Approach to Instruction and Instructional Design,” in In T. L. Good (Ed.), 21st century education: A reference handbook, Thousand Oaks, CA: Sage., 2008, pp. 244–253. [2] D. Galligan, Frank; Maskery, Colin; Spence, Jon; Howe, David; Barry, Tim; Ruston, Andy; Crawford, Advanced PE for Edexcel. Oxford: Heinemann Educational Publishers, 2000. [3] K. Bowman, “Background paper for the AQF Council on generic skills,” Canberra, 2010 [4] E. Rusman, A. Martínez-monés, J. Boon, M. J. Rodríguez-triana, and S. Villagrá-sobrino, “Gauging Teachers ’ Needs with Regard to Technology- Enhanced Formative Assessment ( TEFA ) of 21 st Century Skills in the Classroom,” in Computer Assisted Assessment. Research into E-Assessment. Proceedings of the International Computer Assisted Assessment (CAA) Conference, 2014, pp. 1–14. [6] C. Sluijsmans, Dominique; Joosten-ten Brinke, Désirée; Van der Vleuten, “Toetsen met leerwaarde,” Heerlen, Maastricht, 2013. [7] J. J. G. Van Merriënboer, Training Complex Cognitive Skills. Educational Technology, 1997. [11] W. Westera, “On the Changing Nature of Learning Context: Anticipating the Virtual Extensions of the World.,” Educ. Technol. Soc., vol. 14, no. 2, pp. 201–212, 2011. [12] O. Berry, M. Price, C. Rust, B. O. Donovan, M. Price, and C. Rust, “Teaching in Higher Education: Know what I mean ? Enhancing student understanding of assessment standards and criteria,” Teach. High. Educ., no. March 2015, pp. 37–41, 2007. [13] H. Rohbanfard and L. Proteau, “Live versus video presentation techniques in the observational learning of motor skills,” Trends Neurosci. Educ., vol. 2, no. 1, pp. 27–32, 2013.

Viewbrics: mirroring and mastering complex generic skills with video enhanced rubrics through a technology-enhanced formative assessment methodology by Ellen Rusman (OUNL).

Views

Total views

340

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×