Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TEAM at NUIG - 06 April 2017 - R Bree


Published on

A presentation on the TEAM project at NUIG as part of a National Seminar Series (funded by the National Forum). 6th April 2017.
3 mini-talks in this master deck.

Published in: Education
  • Be the first to comment

  • Be the first to like this

TEAM at NUIG - 06 April 2017 - R Bree

  1. 1. Dr. Ronan Bree ; @Breebio Dept. of Applied Sciences, Dundalk IT Thursday April 6th, 2017
  2. 2. Outline • Quality of teaching • Practicals – Powerful Learning Environments • Assessment of practical sessions • Introduction of digital technologies • TEAM project Overview and status update • Technologies to assist in assessing practical sessions
  3. 3. Learning & Teaching • Scholarship of learning and teaching • Recommends those in a teaching capacity to research and focus on the quality of their students’ learning and understanding • Encourage learner-focused conceptions with complementary associated practices. • Important not to confine this approach to just the classroom (Boyer 1991; Light et al. 2001)
  4. 4. Powerful Learning Environment Teacher- vs. Student-centered • ….. learners take full responsibility for the construction of their knowledge in a comfortable context, combined with targeted support from educators ensuring their approaches/activities prove effective Teacher-centred learning Student-centred learning Low level of student choice High level of student choice Student Passive Student active Power is primarily with the teacher Power primarily with the student (Elen et al. 2007)
  5. 5. Attitude Behaviour Communication skills The Practical • A place of learning; where students regularly engage with numerous aspects of education; • provide opportunities for students “to engage in processes of investigation and inquiry” (Hofstein and Lunetta 1982, 2004; Biggs 1996; Boud and Falchikov 2006; Pickford and Brown 2006; Prades and Espinar 2010; Bree et al. 2014). Academic Writing Practical laboratory skills Clinical Skill Competencies Group work Knowledge Understanding Data acquisition Data analysis/interpretation Data presentation Formulating hypotheses Peer, self-assessment Roth (Roth 1994) : potential of practical sessions to yet to be realised
  6. 6. Concerns with Practicals obtaining the correct result and completing the practical as quickly as possible time for mental engagement to relate other learning experiences to practical work can lead to a greater understanding time for interpretation, discussion, elaboration and application of one’s learning no clear purpose of the investigation and the interconnections between all the elements limited elements of metacognitive activities Become only interested in the grade (Gunstone and Champagne 1990; Gunstone 1991 Hart et al. 2000; Hofstein and Lunetta 2004). OSCEs test just a few skill sets Encourage learners to suggest hypotheses, ask questions and even design investigations with “minds-on as well as hands-on”.
  7. 7. Designing the Format Style Outcome Approach Procedure Expository Predetermined Deductive Provided Inquiry Undetermined Inductive Student Generated Discovery Predetermined Inductive Provided Problem Based Predetermined Deductive Student Generated (Bennett and O’Neale 1998; Domin 1999; Mc Donnell et al. 2007; Dunne and Ryan 2012)
  8. 8. The Expository Problem • Expository – ‘cookbook’, ‘recipe’…known procedure, known outcome..understand a phenomenon. • Repeat lecturer’s instructions, follow defined, provided protocol • The students are regularly not required to think, reflect, reconcile results or do not encounter challenges – “predictable” Follow the protocol exactly You should be getting this result Read your manual (Pickering 1987; Gunstone and Champagne 1990; Lagowski 1990; Johnstone and Al-Shuaili 2001)
  9. 9. Building Inquiry • increases the level of student involvement as there usually less direction provided by the instructor • there is an undetermined outcome and students are required to design and create their own procedure • This format provides the student with more ownership over the practical activity (Domin 1999)
  10. 10. “The key is to allow the students to design their own experiments, working through any problems they encounter along the way. It is important to allow the students to experience some frustration while doing this and to refrain from “rescuing” the situation” (Roberts 2001)
  11. 11. Assessment
  12. 12. nature of implemented assessments have the potential to determine students’ learning style, motivate and build self-confidence while also influencing the way in which students view the learning process Assessment • “Assessment is at the heart of the student experience” • Early 90s – present: “the assessment era” • Practical can represent an ideal avenue for Assessment of, for and as learning. (Sadler 1998; Nicol and Macfarlane-Dick 2006; Brown and Knight 1994; Black and Wiliam 1998; Miller et al. 1998; Prades and Espinar 2010; Broadfoot and Black 2004 )
  13. 13. Power of Assessment • Students have become very assessment driven in the current culture • that if assessment is neglected, with less assessment given to certain elements such as practical skills, students assign less value to such approaches and believe they do not warrant attention • Vital to consider what is being assessed and how this will be evaluated • Feedback essential to reap benefits (Hofstein and Lunetta 2004; Pickford and Brown 2006; Mc Donnell et al. 2007)
  14. 14. Feedback • “oil that lubricates the cogs of understanding” • Highly sought after but often misinterpreted • For students to engage with feedback, they may require guidance/advice on understanding and using it • Incremental marking system • Realise more feedback does not always equate to more learning • Content-focused vs Facilitator-focused feedback (Kulhavy et al. 1985; Weaver 2006; Brown 2007; Carnell and Lodge 2002; Watkins et al. 2002; Carnell 2007; Bree et al., 2014; Y1 Feedback, 2016) Dialogue learning-brain-stimulation-2259/
  15. 15. Traditional Modes of Assessment in Practicals • The Practical Report • Overused; lack of focus • Traditional – common headings • High workload for staff and students • Writing introductions “a sham” • Different standards • …but there are some benefits (Hughes 2004; Pickford and Brown 2006; Mc Donnell et al. 2007; Aurora 2010; Moskovitz and Kellogg 2011 ;Hunt et al. 2012; Bree et al. 2014; Whitworth and Wright 2015)
  16. 16. Traditional Modes of Assessment in Practicals • The OSCE • Gain clinical skills directly related to practice • Knowledge, practical skills & attitude • Ability to perform a skill in an OSCE environment is translatable to competency in handling a real case in a real situation • Stress/anxiety/resource intensive (Miller 1990; Linn et al. 1991; Newble 2004; Davis et al. 2006; Rushforth 2007) Knows Knows How Shows How DoesBehaviour Cognitive Assessment of performance Assessment of competence Content based assessment Factual recall
  17. 17. Alternative Format & Assessment Strategies • Move to experience learning base • Migrate from expository to Inquiry – “fun” solving real-life problems; students engaged, gain hands on experience, in charge more, develop critical thinking skills. (Johnstone and Al-Shuaili 2001; Caspers and Roberts-Kirchhoff 2003; Weaver et al. 2008; Henkel et al. 2015 ) 3/05/25/investors-show-strong- commitment-to-alternative-st • What do we want undergraduate students to learn in the practical sessions? • How might our assessment choices/design facilitate this?
  18. 18. Self- and peer- assessment Assessment as learning Rubrics Oral & Poster presentation Pre-Practical Work Improving Feedback dialogue & uptake The practical manual & Quizzes Alternative Practical Reporting Format Skill set / competency tests (e)Portfolios Walters and Adams 2002; Seery 2009; Sevian & Robinson 2011; Walgren 2011, Dunne & Ryan 2012; Eisenkraft 2003; Crowe et al. 2008; Whatley & Ahmad 2007; Chan 2010 Heyborne et al. 2011; Seery and Donnelly 2012; Karsten 2012; Qu and Lu 2012; Kihlén & Waligorski 2003; Machina & Wild 2013; Johnston et al. 2014; Dervan 2014; Boylan 2014; Hazzard 2014; Zwickl et al. 2012; Meade et al. 2015; Framp et al. 2015; Parker 2015; Boylan and Boland 2015; Dunne et al. 2015; Donaldson 2016; Forbes et al. 2016
  19. 19. Science Writing Heuristic (SWH) The Modified Science Writing Heuristic A Beginning Ideas & Questions What are my questions about this experiment? B Tests & Procedures What will I do to help answer my questions? C Observations What did I see when I completed my tests and procedures? D Claims What can I claim? E Evidence What evidence do I have for my claims? How do I know? Why am I making these claims? F Reading and Discussion How do my ideas compare with others? (Hand and Keys 1999; Keys et al. 1999; Rudd et al. 2002; Carnduff and Reid 2003; Bennett et al. 2009) “to change the experience, you don’t have to change the experiment just what you do with it”
  20. 20. Areas to consider Pre-practical Resources Assessment & Feedback Self- Assessment & Reflection Building Digital Capacity and Literacy Design, Format & Practical Learning Environment
  21. 21. Design, Format and Practical Learning Environment Learning Environment Minds-on design Inquire in some way Assessment strategies Re-design practical manualLearning style
  22. 22. Pre-practical Resources Preparation 2nd to 3rd level transition Customised videos Quiz / Questioning level Clear purpose/Context Learning objectives
  23. 23. Assessment & Feedback Assessment options / value Practical Report Rubrics as learning tools Self-marking presentations Moving up levels from MCQs Clinical/practical skill Feedback ; uptake Facilitator vs Content focused feedback Feedback review & dialogue time
  24. 24. Self-Assessment & Reflection assessing-distance-learning-538-post-1/ Self-assessment strategies Self- & Peer Assessment Training Grading Exemplars – High Quality
  25. 25. Building Digital Capacity and Literacy Embrace Development Consider programme view Digital Capacity Training
  26. 26. Supporting Assessment Methods with Technology • Digital capacity development • support and enrich vibrant learning strategies • must be embraced and utilised more • “enrich teaching, improve learning experiences, support personalised learning, facilitate access through distance learning” (European Commission 2011 ; Teaching and learning in Irish higher education: A roadmap for enhancement in a digital world 2015-2017) National Forum for the enhancement of Teaching & Learning in Higher Education
  27. 27. Quizzes Video Engagement & Collaboration Practical & Clinical Skills Electronic Reporting & Feedback
  28. 28. “Tell me, and I will forget Show me, and I may remember Involve me, and I will understand” (Confucius around 450 BC - read in Henkel et al. 2015)
  29. 29. Dr. Ronan Bree ; @Breebio Dept. of Applied Sciences, Dundalk IT Thursday April 6th, 2017
  30. 30. Background • B.Sc Hons Biochemistry NUIG 95-99 • PhD NUIG 99-03 (Byrnes & Grealy lab) • Post-doc NUIG 03-07 (Lowndes lab) • Icon Clinical Research 07-09 • Dundalk Institute of Technology Lecturer 09-present • MA Learning & Teaching 2013 DkIT • Practical assessment (Bree et al., 2014, AISHE-J) • 2017 - Retired amateur cyclist!
  31. 31. Outline • TEAM Project Steering Committee • Project concept to successful application • Practicals in the IoTs • Project Phases • Introduction of digital technologies • Current Status
  32. 32. School of Health & Science Edel Healy Moira Maguire Ronan Bree Don Faller Nuala Harding Anne Mulvihill David Dowling Gina Noonan Dina Brazil Yvonne Kavanagh Jeremy Bird Akinlolu Akande David Doyle
  33. 33. MALT Research Project Concept Research Application Developed HoS National Forum Establishment of project team, external advisor & steering committee Stakeholder involvement Student Partner & Employer engagement Ethical Approval x4 Student-Staff Workshop Lit. Review & Student Survey Thematic Analysis Staff recruitment Pilot implementation Ethical approval (x4) & Pilot evaluation
  34. 34. Practicals in the IoTs • Typically 4 x 3-hour sessions per week for approximately 10 weeks of term • Used to be 1 assessment per practical…...240 • 18 students per practical session (over 18 = 1 demonstrator) • Lecturer designs the practical session and develops the associated practical manual • Lecturer submits requirements lists 48-hours in advance to technical team • Lecturer moderates session, sets assessment and corrects assessment
  35. 35. Overassessment can represent a block to deep learning High student and staff workload Format of session? “powerful learning environment” Can Technology assist students and staff?
  36. 36. Importance of practical element of Health & Science programmes PROJECT OVERVIEW KEY QUESTION 1: How can practical classes be re-aligned to reduce over assessment and develop essential knowledge, skills and competence? KEY QUESTION 2: How can we use digital technologies to promote assessment for learning in practical settings?
  37. 37. Project Stakeholders TEAM Project Group Partner lecturing teams Senior Management Team (Heads of School) Student Partners Industry/Employer Partners Library and IT Learning and Teaching Units
  38. 38. Project Approach: Phase 1 PHASE 1 Baseline Review LITERATURE REVIEW International Best Practice STAKEHOLDER INPUT Student Needs STAKEHOLDER INPUT Employer Needs STAKEHOLDER INPUT Staff expertise and feedback
  39. 39. Project Approach: Phase 2 PHASE 2 Piloting & Evaluation of technology- enhanced assessment practices’ across all partner institutions via academic peer network DIGITAL TECHNOLOGIES Monitored ASSESSMENT APPROACHES Monitored ASSESSMENT of IMPACT
  40. 40. Project Approach : Phase 3 PHASE 3 Dissemination & Resource Sharing PROJECT WEBSITE Assessment Resource Sharing DISSEMINATION Master Classes/Workshop Publications COMMUNITY OF PRACTICE
  41. 41. Project Workshop Student Partners Keynote Speaker: Dr. Michael Seery (University of Edinburgh) ‘Learning in the 21st Century Laboratory’ Presentations on Literature findings & practical assessments in action from frontline academics Networking & Breakout Group Work Session: Staff & Students Project Workshop: 13th June 2016
  42. 42. TEAM Staff Workshop questions: 1. What do you want your students to be able to do as a result of their practical/clinical/laboratory experience? 2. How well do you think that current assessment practices help them to achieve this? 3. Reflecting on today’s session, what are the top 3 things that have made an impact and why? Please work as a group to agree and priortise these. 4. How might you use this information to enhance your practical assessment? TEAM Student Workshop questions: 1. What do you want to be able to do as a result of their practical/clinical/laboratory experience? 2. How well do you think that current assessment practices help you to achieve this? 3. Reflecting on today’s session, what are the top 3 things that have made an impact and why? How would you like to see this taken forward? Project Workshop • 50 lecturers • 17 students • Four Partner IOTs TEAM STORIES FROM THE FRONTLINE 1. AIT “Is the pen mightier than the mouse” 2. DKIT “Using customised videos to develop veterinary nursing skills”. 3. IT Sligo “Online Grading with Rubrics on Moodle?” 4. IT Carlow “Assessing Practical Skills: Can Student Videos Help?” 5. AIT “Enriching the Chemical Experimental Experience – The Potential of Developing Pre and Post – Lab Exercises” 6. DKIT "Adopting the Electronic Laboratory Notebook for Reporting in Science". 7. IT Sligo “Using Moodle Quizzes for Formative and Summative Assessment” 8. IT Carlow “Physical Physics"
  43. 43. Literature Review Student Partners Employers Academic Staff Priority themes identified for pilot studies Pre-practical Videos/Quizzes ELNs & ePortfolios Digital Feedback Rubrics Student Survey (n=651)
  44. 44. 14 Pilots 9 Pilots 20 Pilots 16 Pilots 59 Pilots - 50 academic staff - 1,481 unique students
  45. 45. 24 19 4 12 Videos/QuizzesELNs/ePortfolios Digital Feedback Rubrics
  46. 46. Dundalk IT Athlone IT IT Carlow IT Sligo B.Sc Pharmaceutical Science B.Sc Athletic and Rehabilitation Therapy Higher Cert in Pharm Tech Studies BSc in Health Science & Physiology B.Sc Agriculture B.Sc. Applied Bioscience B.Sc Analytical Science B.Sc. Biomedical Science B.Sc Veterinary Nursing Higher Cert in Science (Bioscience/Chemistry), B.Sc (Honours) Biosciences with Biopharmaceuticals. B.Sc in Human Nutrition and Human Health B.Sc (Hons) Environmental Bioscience B.Sc Biotechnology B.Sc Sport and Exercise Science, BSc Health Science & Physiology B.Sc (Hons) in General Nursing Higher Cert in Bus and Equine studies B.Sc TV/Media Production B.Sc Medical Biotechnology, B.Sc (Hons) in Intellectual Disability Nursing B.Sc. Pharmaceutical Science Higher Cert Physiology and Health Science B.Sc Pharmaceutical Science B.Sc Environmental Bioscience, B.Sc (Ord) in Forensic Toxicology, B.Sc Bioscience, B.Sc. Health Promotion Public Health B.Sc Applied Bioscience Higher cert in Dental Studies B.Sc Sport Rehabilitation Higher Cert Science Higher Certificate in Agriculture B.Sc (Hons) in Bioveterinary Science, B.Sc Strength and Conditioning B.Sc (Hons) in Midwifery Higher Certificate Science, BSc Toxicology B.Sc Analytical Science B.Sc in Health Science & Nutrition Higher Cert in Pharm Tech Studies B.Sc in Veterinary Nursing B.Sc in Bioscience. Higher Certificate Science, BSc Toxicology B.Sc (Hons) in General Nursing, B.Sc (Hons) in Psychiatric Nursing 45 Programmes
  47. 47. Large piloting team recruited DkIT AIT ITC ITS Ronan Bree Anne Mulvihill Dina Brazil Declan Shelly Karen Dunne Anne Marie O'Brien Gary Cahill Tom Patton Sinead Loughran Anne Friel Carloalberto Petti Maire McCallion Valerie McCarthy Therese Montgomery Rosemary OHara Mary Heneghan Bridget Kelly Noreen Morris Niamh Spratt Eimear Donlon Tony Lennon Carmel Kealey Kieran Germaine Padraig McGourty Suzanne Linnane Marese Gilhooly Ann-Marie Enright Orla Warren Joe Treacy Brian Murphy Yvonne Kavanagh Ann Reynolds Sinead Devery Paula Fitzpatrick Jean Carragher Ann O'Malley Myles Hackett Eileen Lane Brid Delahunt; Mary Booth Margot Mc Nelis; Siobhán Kavanagh Tonya Philips Denise Kennedy Ann Marie Fitzpatrick Barbara Grouden Olivia Corcoran Lisa Kerr
  48. 48. Local launch event Evaluations Student surveys Student Focus Groups Staff Focus Groups Pilot implementations Data analysis
  49. 49. Capacity Building Cross disciplinary through identification of priority themes Identifying and sharing best practice Peer network very well established Active student & staff engagement: Ongoing recruitment Project Website: Dissemination of project outputs Presentation at national and international conferences National Impact
  50. 50. Project Website:
  51. 51. Dr. Ronan Bree ; @Breebio Dept. of Applied Sciences, Dundalk IT Thursday April 6th, 2017
  52. 52. Dr. Ronan Bree ; @Breebio Dept. of Applied Sciences, Dundalk IT Thursday April 6th, 2017
  53. 53. Digital Technologies Pre-practical Videos/Quizzes ELNs & ePortfolios Digital Feedback Rubrics
  54. 54. Videos & Quizzes • Preparation for practicals • Video assessment • Cognitive load • New subject areas for some • Benefit of video Video Quiz Fill the gaps Commence
  55. 55. Pre-Practical Preparation Video Assessment (Ronan Bree, Karen Dunne, Dina Brazil, Yvonne Kavanagh, Ann Reynolds, Joe Treacy)
  56. 56. Equipment Purchased
  57. 57. Video slide
  58. 58. Online Quizzes • VLEs • Apps
  59. 59. Socrative
  60. 60. Electronic Lab Notebooks (ELNs) • Note-taking & Report generation • Observations, experimental results and data = traditionally paper format LabArchives OneNote • Sharing of data • Data searching • Back up possibilities • User defined templates • Security • Metadata • Employability (Sinead Loughran, Anne Mulvihill)
  61. 61. Sinead Loughran @STLoughran
  62. 62. Sinead Loughran @STLoughran
  63. 63. Sinead Loughran @STLoughran
  64. 64. Sinead Loughran @STLoughran
  65. 65. Sinead Loughran @STLoughran
  66. 66. Microsoft OneNote • Part of Microsoft package • Free for students • Apps and stand alone • Auto save • Digital notebook facility • Enroll your registered students • See their work in progress • Provide formative feedback • Submission deadline • Provide feedback (multiple options)
  67. 67. Digital Feedback • In order for assessment to be even considered effective, the role of feedback must not be underestimated. • Often not engaged with or acted on • Must be effective • Delivery important
  68. 68. Turnitin
  69. 69. Rubrics • Can evaluate & provide feedback on student work. • Permits standardisation of an assessment process across different lecturers in one module, or different modules. • Can represent complete teaching resources, versus solely being scoring tools, as they could become reflection objects during meetings between lecturers and students. • Student involvement in design David Doyle, Akinlolu Akande, Dina Brazil
  70. 70. Skills Assessment
  71. 71. Sinead Loughran @STLoughran
  72. 72. Sinead Loughran @STLoughran
  73. 73. Sinead Loughran @STLoughran
  74. 74. Digital Technologies Pre-practical Videos/Quizzes ELNs & ePortfolios Digital Feedback Rubrics
  75. 75. Scalability • Demonstrators • Technical Groups • Training • Assessment literacy
  76. 76. Further contact • DkIT, Head of School of Health & Science: Dr. Edel Healy • AIT, Dean of Faculty of Science & Health: Dr. Don Faller • IT Sligo, Head of School of Science: Dr. Jeremy Bird • IT Carlow, Head of School of Science: Dr. David Dowling • Dr. Ronan Bree • • General enquiries •
  77. 77. Acknowledgements • National Forum for Enhancement of Teaching and Learning in Higher Education • Project Steering committee, CELT teams, External advisor, Academic piloting team, student partners and employer contributions. • Simon, Michael and….. • …..and thank you for your time this afternoon
  78. 78. Dr. Ronan Bree ; @Breebio Dept. of Applied Sciences, Dundalk IT Thursday April 6th, 2017