TEAM project workshop presented at Letterkenny IoT by Dr. Ronan Bree on behalf of the TEAM steering committee, 31st Oct 2017.

1. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
TEAM Project Workshop at Letterkenny Institute of Technology
31st October, 2017
www.teamshp.ie

2. Agenda & Take Home Messages
1. Problems & Solutions in Practical
Assessment in Science
2. An overview of TEAM; The
Project, The Goals, The Vision
3. Tea/Coffee & Networking
4. TEAM Technologies; seeing them
in action
5. Workshop, Discussion &
Feedback session
• Consider your assessment approach
• Consider what you want students to
learn in the session
• Reflect on the assessment approaches
you have in place
• Reflect on alternatives and which may
suit your needs, and those of your
students

3. Identifying problems and Creating
Solutions in Practical Assessment –
Embracing Technology
Dr. Ronan Bree / Twitter: @Breebio
Dept. of Applied Sciences, Dundalk IT
TEAM Project Workshop at Letterkenny Institute of Technology
31st October, 2017

4. Outline
• Review of Literature findings
• Practical design & format
• Assessment of practical sessions
• Alternative assessment forms
• Introduction of digital technologies
• Recommendations
mindsharing.info

5. Learning & Teaching
• Scholarship of learning and teaching
• Recommends those in a teaching capacity to
research and focus on the quality of their students’
learning and understanding
• Encourage learner-focused conceptions with
complementary associated practices.
• Important not to confine this approach to just the
classroom
(Boyer 1991; Light et al. 2001)

6. Powerful Learning Environment
Teacher- vs. Student-centered
• ….. learners take full responsibility for the construction of their knowledge in a
comfortable context, combined with targeted support from educators ensuring
their approaches/activities prove effective
Teacher-centred learning Student-centred learning
Low level of student choice High level of student choice
Student Passive Student active
Power is primarily with the teacher Power primarily with the student
(Elen et al. 2007)

7. Attitude
Behaviour
Communication
skills
The Practical
• A place of learning; where students regularly engage with numerous aspects of
education;
• provide opportunities for students “to engage in processes of investigation and inquiry”
(Hofstein and Lunetta 1982, 2004; Biggs 1996; Boud and Falchikov 2006; Pickford and Brown 2006; Prades and Espinar 2010; Bree et al. 2014).
Academic Writing
Practical laboratory skills
Clinical Skill Competencies
Group work
Knowledge
Understanding
Data acquisition
Data analysis/interpretation
Data presentation
Formulating
hypotheses
Peer, self-assessment
Roth (Roth 1994) reported the potential of practical sessions to assist learning of
concepts and skills was yet to be realised

8. Concerns with Practicals
obtaining the correct
result and completing
the practical as
quickly as possible
time for mental
engagement to relate other
learning experiences to
practical work can lead to
a greater understanding
time for interpretation,
discussion, elaboration
and application of one’s
learning
no clear purpose of the
investigation and the
interconnections
between all the
elements
limited elements of
metacognitive
activities
Become only
interested in the
grade
(Gunstone and Champagne 1990; Gunstone 1991 Hart et al. 2000; Hofstein and Lunetta 2004).
OSCEs test just
a few skill sets
Encourage learners to suggest hypotheses,
ask questions and even design
investigations with
“minds-on as well as hands-on”.

9. Designing the Format
Style Outcome Approach Procedure
Expository Predetermined Deductive Provided
Inquiry Undetermined Inductive Student Generated
Discovery Predetermined Inductive Provided
Problem Based Predetermined Deductive Student Generated
(Bennett and O’Neale 1998; Domin 1999; Mc Donnell et al. 2007; Dunne and Ryan 2012)

10. The Expository Problem
• Expository – ‘cookbook’, ‘recipe’…known procedure, known
outcome..understand a phenomenon.
• Repeat lecturer’s instructions, follow defined, provided protocol
• The students are regularly not required to think, reflect, reconcile results or
do not encounter challenges – “predictable”
Follow the
protocol exactly
You should be
getting this result
Read your
manual
(Pickering 1987; Gunstone and Champagne 1990; Lagowski 1990; Johnstone and Al-Shuaili 2001)
www.creativereview.co.uk

11. Building Inquiry
• increases the level of student involvement
as there usually less direction provided by the
instructor
• there is an undetermined outcome and
students are required to design and create
their own procedure
• This format provides the student with more
ownership over the practical activity
(Domin 1999)
www.mheducation.ca

12. “The key is to allow the students to design their own
experiments, working through any problems they encounter along
the way. It is important to allow the students to experience some
frustration while doing this and to refrain from “rescuing” the
situation”
(Roberts 2001)

13. Assessment

14. nature of implemented assessments have the potential to determine
students’ learning style, motivate and build self-confidence while also
influencing the way in which students view the learning process
Assessment
• “Assessment is at the heart of the student experience”
• Early 90s – present: “the assessment era”
• Practical can represent an ideal avenue for Assessment of, for and as learning.
(Sadler 1998; Nicol and Macfarlane-Dick 2006; Brown and Knight 1994; Black and Wiliam 1998; Miller et al. 1998; Prades and Espinar 2010;
Broadfoot and Black 2004 )

15. Power of Assessment
• Students have become very assessment driven in the current culture
• that if assessment is neglected, with less assessment given to certain
elements such as practical skills, students assign less value to such
approaches and believe they do not warrant attention
• Vital to consider what is being assessed and how this will be evaluated
• Feedback essential to reap benefits
(Hofstein and Lunetta 2004; Pickford and Brown 2006; Mc Donnell et al. 2007)

16. Feedback
• “oil that lubricates the cogs of understanding”
• Highly sought after but often misinterpreted
• For students to engage with feedback, they may require guidance/advice
on understanding and using it
• Incremental marking system
• Realise more feedback does not always equate to more learning
• Content-focused vs Facilitator-focused feedback
(Kulhavy et al. 1985; Weaver 2006; Brown 2007; Carnell and Lodge 2002; Watkins et al. 2002; Carnell 2007; Bree et al., 2014; Y1 Feedback, 2016)
Dialogue
http://neurosciencenews.com/tacs-
learning-brain-stimulation-2259/

17. Traditional Modes of Assessment in Practicals
• The Practical Report
• Overused; lack of focus
• Traditional – common headings
• High workload for staff and students
• Writing introductions “a sham”
• Different standards
• …but there are some benefits
(Hughes 2004; Pickford and Brown 2006; Mc Donnell et al. 2007; Aurora 2010; Moskovitz and Kellogg 2011 ;Hunt et al. 2012; Bree
et al. 2014; Whitworth and Wright 2015)
http://www.somervillenjk12.org/Page/4719

18. Traditional Modes of Assessment in Practicals
• The OSCE
• Gain clinical skills directly related to
practice
• Knowledge, practical skills & attitude
• Ability to perform a skill in an OSCE
environment is translatable to
competency in handling a real case in
a real situation
• Stress/anxiety/resource intensive
(Miller 1990; Linn et al. 1991; Newble 2004; Davis et al. 2006; Rushforth 2007)
Knows
Knows How
Shows How
DoesBehaviour
Cognitive
Assessment of performance
Assessment of competence
Content based assessment
Factual recall

19. Alternative Format & Assessment Strategies
• Move to experience learning base
• Migrate from expository to Inquiry – “fun” solving real-life
problems; students engaged, gain hands on experience, in
charge more, develop critical thinking skills.
(Johnstone and Al-Shuaili 2001; Caspers and Roberts-Kirchhoff 2003; Weaver et al. 2008; Henkel et al. 2015 )
http://www.thinkadvisor.com/201
3/05/25/investors-show-strong-
commitment-to-alternative-st
• What do we want undergraduate students
to learn in the practical sessions?
• How might our assessment choices/design
facilitate this?

20. Self- and peer-
assessment
Assessment as
learning
Rubrics
Oral & Poster
presentation
Pre-Practical Work
Improving Feedback
dialogue & uptake
The practical manual
& Quizzes
Alternative Practical
Reporting Format
Skill set /
competency tests
(e)Portfolios
Walters and Adams 2002; Seery 2009; Sevian & Robinson 2011; Walgren 2011, Dunne & Ryan 2012; Eisenkraft 2003; Crowe et al. 2008; Whatley & Ahmad 2007; Chan 2010
Heyborne et al. 2011; Seery and Donnelly 2012; Karsten 2012; Qu and Lu 2012; Kihlén & Waligorski 2003; Machina & Wild 2013; Johnston et al. 2014; Dervan 2014; Boylan
2014; Hazzard 2014; Zwickl et al. 2012; Meade et al. 2015; Framp et al. 2015; Parker 2015; Boylan and Boland 2015; Dunne et al. 2015; Donaldson 2016; Forbes et al. 2016

21. Science Writing Heuristic (SWH)
The Modified Science Writing Heuristic
A Beginning Ideas &
Questions
What are my questions about this experiment?
B Tests & Procedures What will I do to help answer my questions?
C Observations What did I see when I completed my tests and
procedures?
D Claims What can I claim?
E Evidence What evidence do I have for my claims? How do I
know? Why am I making these claims?
F Reading and Discussion How do my ideas compare with others?
(Hand and Keys 1999; Keys et al. 1999; Rudd et al. 2002; Carnduff and Reid 2003; Bennett et al. 2009)
“to change the experience, you don’t have to change the experiment just what you do with it”

22. Supporting Assessment Methods with
Technology
• Digital capacity development
• support and enrich vibrant learning strategies
• must be embraced and utilised more
• “enrich teaching, improve learning experiences, support
personalised learning, facilitate access through distance
learning”
(European Commission 2011 ; Teaching and learning in Irish higher education: A roadmap for enhancement in a digital world 2015-2017)
National Forum for the enhancement of
Teaching & Learning in Higher Education

23. Quizzes
Video
Engagement
&
Collaboration
Practical &
Clinical Skills
Electronic
Reporting &
Feedback

24. Areas to consider
Pre-practical
Resources
Assessment &
Feedback
Self-
Assessment &
Reflection
Building
Digital
Capacity and
Literacy
Design,
Format &
Practical
Learning
Environment

25. Design, Format and Practical Learning Environment
Learning Environment
Minds-on design
Inquire in some way
Assessment strategies
Re-design practical manualLearning style

26. Pre-practical Resources
Preparation
2nd to 3rd level transition
Customised videos
Quiz / Questioning level
Clear purpose/Context
Learning objectives

27. Assessment & Feedback
Assessment options / value
Practical Report
Rubrics as learning tools
Self-marking presentations
Moving up levels from MCQs
Clinical/practical skill
Feedback ; uptake
Facilitator vs Content focused
feedback
Feedback review & dialogue time

28. Self-Assessment & Reflection
http://partnerinedu.com/2007/05/26/
assessing-distance-learning-538-post-1/
Self-assessment strategies
Self- & Peer Assessment Training
Grading Exemplars – High Quality

29. Building Digital Capacity and Literacy
Embrace Development
Consider programme view
Digital Capacity
Training

31. Conclusion

32. “Tell me, and I will forget
Show me, and I may remember
Involve me, and I will understand”
(Confucius around 450 BC - read in Henkel et al. 2015)

33. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
LYIT; Tuesday October 31st, 2017
www.teamshp.ie

35. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
31st October 2017; Letterkenny Institute of Technology
www.teamshp.ie

36. Outline
• TEAM Project Steering Committee
• Project concept to successful application
• Practicals in the IoTs
• Project Phases
• Introduction of digital technologies
• Current Status
mindsharing.info

37. School of Health
& Science
Edel Healy
Moira Maguire
Ronan Bree
Don Faller
Nuala Harding
Anne Mulvihill
David Dowling
Gina Noonan
Dina Brazil
Yvonne Kavanagh
Jeremy Bird
Akinlolu Akande
David Doyle

38. MALT Research
Project Concept
Research Application Developed
HoS National Forum
Establishment of project team,
external advisor & steering committee
Stakeholder involvement
Student Partner & Employer
engagement
Ethical Approval x4
Student-Staff Workshop
Lit. Review & Student Survey
Thematic Analysis
Staff recruitment
Pilot implementation
Ethical approval (x4) & Pilot
evaluation

39. Practicals in the IoTs
• Typically 4 x 3-hour sessions per
week for approximately 10
weeks of term
• Used to be 1 assessment per
practical…...240
• 18 students per practical session
(over 18 = 1 demonstrator)
• Lecturer designs the practical
session and develops the associated
practical manual
• Lecturer submits requirements lists
48-hours in advance to technical
team
• Lecturer moderates session, sets
assessment and corrects assessment

40. Overassessment can
represent a block to
deep learning
High student and
staff workload
Format of session?
“powerful learning
environment”
Can Technology
assist students and
staff?

41. Importance of practical element of
Health & Science programmes
PROJECT OVERVIEW
KEY QUESTION 1: How can practical classes be re-aligned to
reduce over assessment and develop essential knowledge, skills
and competence?
KEY QUESTION 2: How can we use digital technologies to
promote assessment for learning in practical settings?

42. Project Stakeholders
TEAM
Project
Group
Partner lecturing teams
Senior Management
Team (Heads of
School)
Student Partners
Industry/Employer
Partners
Library and IT
Learning and
Teaching Units

43. Project Approach: Phase 1
PHASE 1
Baseline Review
LITERATURE REVIEW
International Best Practice
STAKEHOLDER INPUT
Student Needs
STAKEHOLDER INPUT
Employer Needs
STAKEHOLDER INPUT
Staff expertise and feedback

44. Project Approach: Phase 2
PHASE 2
Piloting & Evaluation
of technology-
enhanced assessment
practices’ across all
partner institutions via
academic peer network
DIGITAL
TECHNOLOGIES
ASSESSMENT
APPROACHES
ASSESSMENT of
IMPACT

45. Project Approach : Phase 3
PHASE 3
Dissemination &
Resource Sharing
PROJECT WEBSITE
Assessment Resource
Sharing
DISSEMINATION
Master Classes/Workshop
Publications
COMMUNITY OF
PRACTICE

46. Numerous DkIT TEAM teams!
Steering
Committee
( 4 x partners)
Student
Partners
Technology
/ Discipline
Champions
K. Dunne
R. Bree
S. Loughran
A. Reynolds
J. Treacy
Academic
lead
R.Bree
Academic
Piloting
Team
Student
Groups
CELT;
M.Maguire
HoS; E.Healy

47. Project
Workshop
Student Partners
Keynote Speaker:
Dr. Michael Seery
(University of Edinburgh)
‘Learning in the 21st Century Laboratory’
Presentations on
Literature findings &
practical assessments in
action from frontline
academics
Networking &
Breakout Group
Work Session:
Staff & Students
Project Workshop: 13th June 2016

48. Project Workshop
• 50 lecturers
• 17 students
• Four Partner IOTs
TEAM STORIES FROM THE FRONTLINE
1. AIT “Is the pen mightier than the mouse”
2. DKIT “Using customised videos to develop veterinary nursing
skills”.
3. IT Sligo “Online Grading with Rubrics on Moodle?”
4. IT Carlow “Assessing Practical Skills: Can Student Videos Help?”
5. AIT “Enriching the Chemical Experimental Experience – The
Potential of Developing Pre and Post – Lab Exercises”
6. DKIT "Adopting the Electronic Laboratory Notebook for
Reporting in Science".
7. IT Sligo “Using Moodle Quizzes for Formative and Summative
Assessment”
8. IT Carlow “Physical Physics"

49. Literature
Review
Student
Partners
Employers
Academic
Staff
Priority themes identified for pilot studies
Pre-practical
Videos/Quizzes
ELNs &
ePortfolios
Digital
Feedback
Rubrics
Student
Survey
(n=651)

50. 14 Pilots
9 Pilots
20 Pilots
16 Pilots
59 Pilots
-
50
academic
staff
-
1,481
unique
students

51. 24
19
4
12
Videos/QuizzesELNs/ePortfolios
Digital
Feedback
Rubrics

52. Dundalk IT Athlone IT IT Carlow IT Sligo
B.Sc Pharmaceutical Science
B.Sc Athletic and Rehabilitation
Therapy
Higher Cert in Pharm Tech Studies BSc in Health Science & Physiology
B.Sc Agriculture B.Sc. Applied Bioscience B.Sc Analytical Science B.Sc. Biomedical Science
B.Sc Veterinary Nursing
Higher Cert in Science
(Bioscience/Chemistry),
B.Sc (Honours) Biosciences with
Biopharmaceuticals.
B.Sc in Human Nutrition and
Human Health
B.Sc (Hons) Environmental
Bioscience
B.Sc Biotechnology
B.Sc Sport and Exercise Science, BSc Health Science & Physiology
B.Sc (Hons) in General Nursing
Higher Cert in Bus and Equine
studies
B.Sc TV/Media Production B.Sc Medical Biotechnology,
B.Sc (Hons) in Intellectual Disability
Nursing
B.Sc. Pharmaceutical Science
Higher Cert Physiology and Health
Science
B.Sc Pharmaceutical Science
B.Sc Environmental Bioscience, B.Sc (Ord) in Forensic Toxicology, B.Sc Bioscience,
B.Sc. Health Promotion Public
Health
B.Sc Applied Bioscience Higher cert in Dental Studies B.Sc Sport Rehabilitation Higher Cert Science
Higher Certificate in Agriculture
B.Sc (Hons) in Bioveterinary
Science, B.Sc Strength and Conditioning
B.Sc (Hons) in Midwifery
Higher Certificate Science, BSc
Toxicology B.Sc Analytical Science
B.Sc in Health Science & Nutrition Higher Cert in Pharm Tech
Studies
B.Sc in Veterinary Nursing
B.Sc in Bioscience.
Higher Certificate Science, BSc
Toxicology
B.Sc (Hons) in General Nursing,
B.Sc (Hons) in Psychiatric Nursing
45
Programmes

53. Large piloting team recruited
DkIT AIT ITC ITS
Ronan Bree Anne Mulvihill Dina Brazil
Declan Shelly
Karen Dunne Anne Marie O'Brien Gary Cahill
Tom Patton
Sinead Loughran Anne Friel Carloalberto Petti Maire McCallion
Valerie McCarthy Therese Montgomery Rosemary OHara Mary Heneghan
Bridget Kelly Noreen Morris Niamh Spratt Eimear Donlon
Tony Lennon Carmel Kealey Kieran Germaine Padraig McGourty
Suzanne Linnane Marese Gilhooly Ann-Marie Enright Orla Warren
Joe Treacy Brian Murphy Yvonne Kavanagh
Ann Reynolds Sinead Devery Paula Fitzpatrick
Jean Carragher Ann O'Malley
Myles Hackett Eileen Lane
Brid Delahunt; Mary Booth
Margot Mc Nelis; Siobhán Kavanagh
Tonya Philips
Denise Kennedy
Ann Marie Fitzpatrick
Barbara Grouden
Olivia Corcoran
Lisa Kerr

54. Needs Analysis
Pilot Evaluations
Student surveys Staff Focus Groups
Pilot implementations
Data analysis
Pilot design
Student Focus Groups
Dissemination & Resource Sharing

55. Capacity Building Cross
disciplinary through
identification of priority
themes
Identifying and sharing
best practice
Peer network very well
established
Active student & staff
engagement: Ongoing
recruitment
Project Website:
Dissemination of project
outputs
Presentation at national
and international
conferences
National Impact

56. Project Website: www.teamshp.ie

57. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
Thursday April 6th, 2017
www.teamshp.ie

59. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
TEAM Project Workshop at Letterkenny Institute of Technology
31st October, 2017
www.teamshp.ie

60. Digital Technologies
Pre-practical
Videos/Quizzes
ELNs &
ePortfolios
Digital
Feedback
Rubrics

61. Videos & Quizzes
• Preparation for practicals
• Video assessment
• Cognitive load
• New subject areas for some
• Benefit of video
Video
Quiz
Fill the gaps
Commence

62. Pre-Practical
Preparation
Video
Assessment
(Ronan Bree, Karen Dunne, Dina Brazil, Yvonne Kavanagh, Ann Reynolds, Joe Treacy)

63. Equipment Purchased

64. Video slide

65. Online Quizzes
• VLEs
• Apps

66. Socrative

70. Electronic Lab Notebooks (ELNs)
• Note-taking & Report
generation
• Observations, experimental
results and data =
traditionally paper format
LabArchives
OneNote
• Sharing of data
• Data searching
• Back up possibilities
• User defined templates
• Security
• Metadata
• Employability
(Sinead Loughran, Anne Mulvihill, Ronan Bree)

71. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

72. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

73. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

74. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

75. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

76. Microsoft OneNote
• Part of Microsoft package
• Free for students
• Apps and stand alone
• Auto save
• Digital notebook facility
• Enroll your registered students
• See their work in progress
• Provide formative feedback
• Submission deadline
• Provide feedback (multiple options)

81. Digital Feedback
• In order for assessment to be even
considered effective, the role of
feedback must not be
underestimated.
• Often not engaged with or acted on
• Must be effective
• Delivery important

82. Turnitin

83. Rubrics
• Can evaluate & provide feedback on student work.
• Permits standardisation of an assessment process across
different lecturers in one module, or different modules.
• Can represent complete teaching resources, versus solely
being scoring tools, as they could become reflection
objects during meetings between lecturers and students.
• Student involvement in design
David Doyle, Akinlolu Akande, Dina Brazil

84. Skills Assessment

85. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

86. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

87. Sinead Loughran sinead.loughran@dkit.ie @STLoughran

88. Digital Technologies
Pre-practical
Videos/Quizzes
ELNs &
ePortfolios
Digital
Feedback
Rubrics

89. Scalability
• Demonstrators
• Technical Groups
• Training
• Assessment literacy

90. Further contact
• DkIT, Head of School of Health & Science:
Dr. Edel Healy Edel.Healy@dkit.ie
• AIT, Dean of Faculty of Science & Health:
Dr. Don Faller dfaller@ait.ie
• IT Sligo, Head of School of Science:
Dr. Jeremy Bird bird.jerry@itsligo.ie
• IT Carlow, Head of School of Science:
Dr. David Dowling david.dowling@itcarlow.ie
• Dr. Ronan Bree
• ronan.bree@dkit.ie
• Twitter: @Breebio
• General enquiries
• teamshp@dkit.ie

91. Acknowledgements
• National Forum for Enhancement of Teaching and Learning in Higher
Education
• Project Steering committee, CELT teams, External advisor, Academic piloting
team, student partners and employer contributions.
• …..and thank you for your time this morning

92. Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
TEAM Project Workshop at Letterkenny Institute of Technology
31st October, 2017
www.teamshp.ie

93. Workshop & Feedback
Discussion
www.teamshp.ie
Dr. Ronan Bree ; @Breebio
Dept. of Applied Sciences, Dundalk IT
TEAM Project Workshop at Letterkenny Institute of Technology
31st October, 2017

94. Format
• Break in to groups (by discipline if necessary)
• 4 questions; 5 minutes discussion per topic
• After 20mins, discuss as group to identify any themes

95. Q1
•What do you want your students to be able
to do as a result of their
practical/clinical/laboratory experience?

96. Q2
•How well do you think that current
assessment practices help them to achieve
this?

97. Q3
•Reflecting on today’s session, what are the
top 3 things that have made an impact
with you, and why?

98. Q4
•How might you use this information to
enhance your practical assessment?