Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Y1Feedback Presentation at EdTech2015


Published on

Y1Feedback Presentation at EdTech2015

Published in: Education
  • Be the first to comment

  • Be the first to like this

Y1Feedback Presentation at EdTech2015

  1. 1. Assessment Feedback Practice In FirstYear Using Digital Technologies – A Baseline Review (Preliminary Findings) Lisa O'Regan, Mark Brown, Moira Maguire, Nuala Harding, Elaine Walsh, Gerry Gallagher, Geraldine McDermott EdTech 2015 – May 28th
  2. 2. @y1feedback Follow us!
  3. 3. Presentation Overview o Why Feedback? o Project Overview o Planning the Current Practice Review o Current Practice Review – Part 1 – Students o Preliminary Findings o Current Practice Review Part 2 – Staff o Preliminary Findings o Initial Conclusions & Next Steps
  4. 4. 2013 67.4%  The Irish Survey of Student Engagement (ISSE) 2013, found that nationally, 22.3% of first year undergraduate students never, and 45.1% only sometimes, received timely written or oral feedback from teachers on academic performance.
  5. 5. 2014 68.2%  The Irish Survey of Student Engagement (ISSE) 2014, found that nationally, 23.3% of first year undergraduate students never, and 44.9% only sometimes, received timely written or oral feedback from teachers on academic performance.
  6. 6. Worrying because …  Feedback is one of the most powerful influences on learning and achievement ( Hattie andTemperely 2007)  Feedback is particularly important in supporting the transition to Higher Education (Nicol 2009)  Regular and frequent formative feedback in first year is associated with student success (Tinto 2005, Nicol 2009)
  7. 7. The Project – @Y1Feedback SupportingTransition: Enhancing (Assessment) Feedback in FirstYear Using DigitalTechnologies
  8. 8. Y1FeedbackTeam  Lisa O’Regan MU (Project Manager)  Nuala Harding & Geraldine McDermott AIT  Moira Maguire & Gerry Gallagher DKIT  Mark Brown & ElaineWalsh DCU
  9. 9. Y1Feedback ProjectAim The project will develop and case study technology enhanced feedback approaches to facilitate feedback, feed-forward and peer- feedback in first year to support student transition to higher education.
  10. 10. An Design Based ResearchApproach “a series of approaches, with the intent of producing new theories, artifacts, and practices that account for and potentially impact learning and teaching in naturalistic settings.” (Barab and Squire, 2004) “research about education” rather than “research for education” (Alghamdi & Li, 2013)
  11. 11. Characteristics of Design Based Research (Reeves 2006)
  12. 12. What is Assessment Feedback? “Includes all feedback exchanges generated within assessment design, occurring within and beyond the immediate learning context, being overt or covert (actively and/or passively sought and/or received), and importantly, drawing from a range of sources.” (Evans, 2013)
  13. 13. Facets of Feedback Feedback – an amalgamation of a number of elements (Evans, 2013) including:  Information on performance or understanding (Hattie &Timperley, 2007)  Information to influence subsequent work and improve performance (Boud & Molloy, 2012)  Action to alter the gap – “information used, rather than information transmitted” (Boud & Molloy, 2012)  Student responsibility - dialogue (Carless et al., 2011), self- regulation (Nicol, 2009) for “sustainable feedback” (Hounsell, 2007)
  14. 14. Feedback Mediated by DigitalTechnologies Includes:  Written comments  Track changes, comments, annotations  Statement banks and checklists  Assessment criteria and rubrics  Audio  Video  Quizzes
  15. 15. Y1Feedback - Project Phases Phase 1 • Current Practice Review (WIP) • Literature Summary Phase 2 • Pilot feedback, feed-forward and peer feedback approaches and technologies Phase 3 • Develop case studies of best practice • Disseminate via National Symposium
  16. 16. Current Practice Review Planning Why?  To get a landscape snapshot of current assessment feedback practices across the four partner institutions from a staff and student perspective. How?  Staff Survey  Student Focus Groups  Initial EthicalApproval at MU  Ethical approval at AIT, DKIT and DCU  Eight Ethics Applications in total!
  17. 17. Current Practice Review Focus  How assessment feedback is provided in first year undergraduate programmes  The timing and frequency of assessment feedback  The types of assessment feedback provided to students  The digital technologies used in the provision of assessment feedback  Lecturer views on assessment feedback processes and practices  Student views on assessment feedback in first year
  18. 18. Current Practice Review – Part 1  First year student focus groups  MU, AIT, DKIT, DCU  Purposive sample of first year undergraduate class representatives  Completed late April 2015  Four focus groups with 36 participants in total *Note: Preliminary findings - analysis is ongoing
  19. 19. The FirstYear Class Representative Response
  20. 20. How is Assessment Feedback Provided?  ‘She [lecturer] will go through the answers and even after the exam she will sit down with you to go through it all and tell you exactly where you went wrong’  ‘We do get written feedback on the written assignments'.  ‘But tests, like continuous assessment tests we wouldn’t get feedback, we’d just get your percent’.  ‘With regards to … are just told that is your grade, that is that...there is no feedback whatsoever’.
  21. 21. When is Assessment Feedback Provided?  ‘It probably depends on how big the group is’.  ‘It depends on the tutor because the tutor I have now, she’s really quick’.  ‘Most of the time it is, it just depends on like the lecturer’.  ‘For one of the five if it comes, but for the other four it doesn’t come on time’ …(for the next assignment)  ‘It will be at least a month and a half before you get anything back’.  ‘It would take a while now I’d say for, to get your grade back, but whenever you do get it back, there is usually a good amount of feedback on it’.
  22. 22. WhatType of Assessment Feedback is Provided?  ‘You might just get a comment and it’d be like “well done” and you could have got seven out of ten and there’s no comment like “well you could have done this”, it’s just “well done”.’  'We got one aural one before’.  “Once on video”  'Ours would mostly be grades like 80% of ours would be just our grade, no written, no oral'.  ‘We do get written feedback on the written assignments, nothing really on the online because it is all netted and you can get oral feedback or a dialogue I suppose if you engage your instructor’.
  23. 23. WhatTechnologies are Used to Provide Feedback?  ‘It’s like a comment box that they have underneath your assignment'.  ‘By email and hard copy’.  ‘Yeah there is a feedback online…Because you get like an email saying you have feedback on your assignment, and then you can click into it and then…’  ‘You get the video recordings but you also just get to see what he is talking about and he is able to visually show you’.  ‘We got one (feedback) last semester’  'We get nothing (feedback) online at all’.  'Like the facility is obviously there for online feedback but I don’t know whether lecturers don’t want to do it or they don’t know how or how to put it across'.
  24. 24. StudentViews on Feedback in FirstYear  “It [feedback] is the foundation of the next however many years you are going to be here”.  “The consensus around all of the college is that we are not getting feedback at all. And that it would be highly recommended if we could. For first year students coming in after us like”.  ”It’s [first year] one of the most important years for your feedback because it’s completely different from anything you had before”.
  25. 25. Suggested Changes …  'It would be nice to have some kind of a continuance between (modules), there's no point in us getting feedback in one module, it'd be nice to have…Consistency'.  'I've emailed and emailed and emailed and now it's like she's a part-time lecturer so she's not here so I think it should be made clear to them, you're not going to see them again at least give them a little bit of feedback'.  Instead of just getting grades the whole time I think we should get more comments’.
  26. 26. Current Practice Review – Part 2  Online survey  Staff who teach undergraduate first year programmes  MU,AIT, DKIT & DCU  Anonymous  Integrative  Status: Ongoing closes 5th June 2015  Current Responses 156 *Preliminary analysis –Ongoing work
  27. 27. What are Staff Saying about Assessment Feedback in FirstYear Undergraduate Programmes?
  28. 28. Respondent Profile: 156 Numbers ofYearsTeaching Experience
  29. 29. How often is assessment feedback prepared for students on continuous assessment work?
  30. 30. How often is Assessment Feedback Provided 53% of respondents always provide feedback 32% frequently provide feedback 9% Occasionally provide feedback 3% Rarely or Never provide feedback
  31. 31. Who provides assessment feedback?
  32. 32. When is feedback provided in first year undergraduate programmes?
  33. 33. In what format is assessment feedback typically provided?
  34. 34. How do students access feedback?
  35. 35. Typical Feedback Provided
  36. 36. Technologies Most Highly Used to Deliver Feedback 1. MSWord 2. Email 3. Moodle Gradebook Feedbak Comments 4. Moodle Gradebook Feedback Files 5. Moodle Quiz 6. PDF Annotation &Turnitin Grademark 7. Rubric 8. Audio, video, wiki, social media, socrative
  37. 37. Influences on Extent of Feedback 1. Own beliefs about value of feedback (52%) 2. Own workload (41%) 3. Nature of assessment (33% ) 4. Level of involvement in the module (29%)
  38. 38. StaffViews On Feedback 98% of respondent Strongly agreed (80%) or agreed that feedback is an integral part of student learning 79% of respondents strongly agreed (32%) or agreed (43%) that students value timely feedback 71% of staff strongly agreed (23%) or agreed (48%) felt that students seem to be only interested in the grade
  39. 39. Staff Recommended FeedbackApproaches 1. Face –to- Face/ One to One (30% of responses) 2. Detailed written feedback and one-to one discussion 3. Group/class discussions 4. Rubrics/Marking Schemes/Structured feedback sheets 5. MCQs
  40. 40. Challenges to Providing Assessment Feedback in FirstYear 1. Numbers 2.Time 3.Workload 4. Student Engagement with Feedback 5. Attendance/Absenteeism 6.Technology
  41. 41. Numbers –Time -Workload “The sheer numbers 300…” “Large class sizes hinder feedback” “Workload …Too many students, too much work, not enough time to grade assignments quickly.Tensions between different deadlines e.g. research vs teaching”
  42. 42. Student Engagement with Feedback  “Students seem to want too much detail in some cases and think the role of feedback is to remediate ALL problems with the essay in solution form as written feedback. In other words – they expect to have a golden formula for guaranteed success outlined in practical and concrete terms”  “Lack of interest in deeper feedback based on mentality that their grades don’t count in first year so that passing is the only goal”  While I provide feedback on all the assignments I set, not many students engage with it. However, those that do seem to benefit from it
  43. 43. Factors preventing use of technology for feedback practices Technology  “Some aspects of grade administration in Moodle are clunky”  “Highly problematic, no support or backup as local level plus extremely time consuming process annotating in Moodle”  “Generally poor quality of IT Service”  “Student familiarity with technology”  “Don’t see the value in technology”  “One-to-one discussions can’t be replaced by technology” Time  “It is more time consuming”  “Time taken to set up technology feedback would be a major factor”
  44. 44. Lecturer Suggested Solutions  “To be able to provide effective feedback then you need to reduce the class size significantly”  “Unless modules are adequately resourced in terms of staffing, it is going to be difficult to improve the situation”  “Impact could be improved by providing feedback closer to the student submissions and introducing feedback in the first three 3 weeks”  “More training on and availability of new technologies”  “Training students on feedback might help”  “There should be some agreement between lecturers on purpose and type of feedback provided”
  45. 45. Some Initial Conclusions Student  Recognise value of feedback  Widely varied experience of assessment feedback in first year within and across disciplines  Depends on the lecturer  Low levels of technology enhanced feedback approaches Staff  Highlight importance of feedback for student learning  F2F No1 Recommendation  Technology  MoreTime Consuming  Clunky  Numbers/Large Classes  Time/Workload
  46. 46. InitialTake Aways to Address  The need for a conversation about feedback and a common understanding of what feedback is  A common process that be built into all feedback approaches  Using technology for feedback is actually more time consuming?  Myth or Reality? A question we can potentially address in the case studies  The biggest problem with technology, is the technology!  Can we improve the user experience of assessment/feedback interfaces in Moodle?  Numbers, Numbers, Numbers  Our approaches and case studies will need a focus on large numbers, be scalable and realistic  Can we replicate F2F feedback?
  47. 47. Questions?
  48. 48. @y1feedback Follow us!