Educational Research

From Newcastle. For the world.
Educational Research
Dr Sara Marsham
sara.marsham@ncl.ac.uk
@sara_marine

ncl.ac.uk
Using online marking to improve
feedback and engage students in the
assessment process

From Newcastle. For the world. 3
Engage Students in Assessment Process
Project Aims
1. To engage students in the entire marking process from the
setting of marking criteria through the receipt and feed-forward
application of feedback
2. To write/design effective marking criteria specific to individual
assessments
3. To engage students in the process of using marking criteria in
preparation for an assignment
4. To provide feedback on coursework that links directly to marking
criteria
5. To use Feedback Studio to develop libraries of feedback
comments that can function much like dialogue with students
Implicit questions in our original
proposal:
1. What do students already
know about marking
criteria?
2. Can typed (even repeated!)
comments work like a
dialogue? Will students
recognise this?
3. Can we involve students in
writing marking criteria?
Graham, A.I., Harner, C. & Marsham S. (In prep). Can assessment-specific marking criteria and electronic comment libraries increase student
engagement with assessment and feedback? Assessment and Evaluation in Higher Education.

Engage Students in Assessment Process: Aims 1-3
Understand students’
prior knowledge
Write new
marking criteria
(based on student
knowledge)
Engage
students
with
marking
criteria

Understand Students’ Prior Knowledge
Stage 1 Microbiology Focus Group
How can marking criteria be used to make expectations clear?
I have read a research paper published in a
peer-reviewed journal
1. Yes
2. I’ve read some but
found them
difficult to
understand
3. No
4. I’m not sure what
you mean by a
peer-reviewed
journal
Write your report “in the format of a scientific
paper” - do you know what this means?
1. Yes
2. No
3. To some extent

Write New Marking Criteria
Stage 1
Microbiology
Marking
Criteria

Engage Students with Marking Criteria
Engagement Sessions
Objective #1 - to help students understand the wording in the marking
criteria
Objective #2 - to encourage students to start differentiating between the
descriptions of different grade boundaries and spotting what will help
them to achieve high marks
Objective #3 - to engage students in the practice of peer marking (marking
existing student work against the set of criteria)

Stage 1 Microbiology Engagement Session
1. 0-39%
2. 40-49%
3. 50-59%
4. 60-69%
5. 70-100%
Into what grade boundary would results
example 1 fall?
Which title scored the highest?
1. Example 1
2. Example 2
3. Example 3

Evaluate Engagement Sessions

Mark Against a
Rubric Add
assignment-
specific,
module-
specific,
School or
Faculty-wide
marking
criteria
Mark each piece
of work
according to the
rubric; use
qualitatively or
quantitatively

Turn Criteria into
Comments
S/T
A
R
1 2 3 4 5 6

Create Specific Comment Library
• Each comment linked to one of the criterion with letter and number
For each component, comment on:
• How student meets criterion
• What student could have done to achieve next grade boundary
R 4
R 5

Evaluate Project Aims

NUBS Rubric Workshop
Student Feedback
“This [the rubric] was the most useful aspect of the electronic
feedback as this helped me to gauge which areas of the assignment I
was lacking and therefore where I would need to focus my
improvement for future work.
It also helped me to understand why I had received the mark I had in
relation to the marking criteria for each section and thus why my
overall grade was within a certain grade boundary.”
“I felt like it was easier for the marker to provide positive comments
and this is also important feedback - it is good to know when a
specific section is very good in order to use this style/technique in
another piece of work.
I think the automated comments and marking rubric make the
marker more fair as it ensures they connect each section of work to
the relevant criteria section.”
• Increased perceptions of
fairness and transparency
Overall, the students voted that the
electronic comments were more
positive, more fair, more thorough,
more helpful, easier to understand
and specific (compared to other
feedback).

Impact
o Rolled out in Schools in 2013-2014 and now used extensively in BMSC
o Awarded funding from HEA to host workshop in Newcastle in 2013
o Careers Service adopted for Career Development Module in 2013-2014
o Over 400 students across the University
o Reduction in student complaints as students recognise why they are getting the mark
awarded
o University-wide pilot in 2014-2015
o Sixteen iPads with Turnitin app available to markers interested in trialling electronic marking
o Twenty-two participants signed up to pilot (12 attended training). Seven additional
participants joined over course of academic year

Impact
o SAgE Faculty Innovator of the Year Award in 2014
o Scheme extended across University in 2015-2016 - shifted focus towards its use on
PCs
o Introduced in a number of programmes, and support for academic units to use this
approach is now mainstreamed
o An example of a Faculty Enhancement Project for the successful Institutional QAA
Review and detailed in the Institutional TEF Gold Award
o Recognised by PVC Education and awarded further funding in 2016 for dissemination
o Embedded within current Education Strategy

ncl.ac.uk
Perceptions and attitudes of
undergraduate students to reading

Perceptions and attitudes of undergraduate students to reading
Context
o Reading is vital to student learning and attainment
o Students who read more than their peers achieve higher marks in assessments
o Compliance with reading and relative frequency of reading amongst students
is low
o Possible explanations include lack of motivation, confidence, clarity of benefits;
low levels of reading comprehension, and competing demands for time
Project Aims
1. Examine how long students spent reading academic material
2. Explore students’ reading practices
3. Examine some barriers to reading; lack of time and lack of confidence
St Clair-Thompson H, Graham A, Marsham S. (2018). Exploring the Reading Practices of Undergraduate Students. Education Inquiry, 9(3), 284-298.

Examine how long students spent reading academic material
Study 1
o UG students completed a questionnaire - number hours in previous seven days spent
reading any academic material, textbooks, journal articles, doing guided or assigned
reading, and doing independent reading
o Students spent, on average, 14.05 hrs per week on academic reading, with more time
spent reading journal articles than other reading activities (Table 1)
o Students in later years spent more time reading journal articles (ANOVA, P < 0.01) with
significant differences between Stage 1 and Stage 2, and Stage 1 and Stage 3
Overall Stage 1 Stage 2 Stage 3
All academic reading 14.05 ± 12.61 13.09 ± 13.69 13.71 ± 10.96 16.79 ± 12.53
Textbooks 2.22 ± 4.07 2.59 ± 4.06 2.14 ± 4.65 1.50 ± 2.86
Journal articles 4.68 ± 6.22 2.92 ± 4.02 5.30 ± 5.57 7.67 ± 9.42
Guided/ assigned reading 3.61 ± 5.74 2.73 ± 4.10 4.06 ± 7.11 4.93 ± 6.21
Independent reading 2.21 ± 5.08 1.55 ± 2.66 2.51 ± 5.20 3.21 ± 8.13
Table 1. Mean (± S.D.) length of time in hours spent reading academic materials

Explore students’ reading practices
Study 2
o Student focus groups were held to explore
factors that may be related to students’
reading practices
o Students perceived several benefits of
reading; acquiring background knowledge,
using it to help develop writing skills, helping
visualise words and concepts, and gaining
higher marks in assessments
Students understood what is
expected - whether it be how
much reading they are
supposed to do or which
sources are essential to read
Students use a range of
strategies when reading
Students read
more if it was
required to
understand a
lecture, before
attending a
seminar, or if
they were to be
quizzed on
material
Students
recognise
progression
from St 1 to St
3 - reading
more
textbooks at
St 1 & primary
literature at St
3

Examine some barriers to reading; lack of time and lack of confidence
Study 3
Two key themes emerged from the focus groups:
o Students’ perceived lack of time
o Lack of confidence with reading
o Students completed a closed-response diary about how they had spent
their time for each 60 minute period during the previous 24 hour working
day
o Completed structured and open-ended questions based on their own
confidence with reading difference materials

Students’ perceived lack of time
o Students in later years spent more time on academic activities
o Time spent attending lectures, seminars or labs: Stage 1 > Stage 3 (ANOVA, P <
0.05)
o Time spent conducting independent work: Stage 3 > Stage 2 & Stage 1 (ANOVA, P
< 0.01)
o Time spent preparing for or completing assessments: Stage 2 > Stage 1 (ANOVA, P
< 0.05)
Attending lectures, seminars or labs 1.90 ± 2.02 2.28 ± 2.01 1.76 ± 2.20 0.91 ± 1.08
Conducting independent academic work 2.25 ± 2.35 1.90 ± 2.02 2.05 ± 2.12 3.96 ± 3.18
Preparing for or completing an assignment or exam 2.13 ± 2.43 1.73 ± 1.94 2.77 ± 3.02 1.96 ± 2.10
Total academic activities 6.47 ± 2.80 6.13 ± 2.93 6.73 ± 2.42 7.00 ± 3.15
Leisure activities or socialising 5.22 ± 3.48 5.42 ± 3.74 5.29 ± 3.32 4.39 ± 2.90
Relaxing or sleeping 9.69 ± 2.63 9.57 ± 2.64 10.11 ± 2.61 9.09 ± 2.64
Table 2. Mean (± S.D.) length of time in hours spent on academic and other activities within 24 hours

Lack of confidence with reading
o Students were more confident in reading textbooks (86%), thank journal
articles (79%) with confidence increasing in later years (RM ANOVA, P <
0.01)
o Confidence reading textbooks increased with Stage: Stage 3 > Stage 2 & Stage 1
(ANOVA, P < 0.05)
o Confidence reading journal articles increased with Stage: Stage 3 > Stage 1
(ANOVA, P < 0.05)
Confidence with reading textbooks 2.02 ± 0.65 2.09 ± 0.60 2.07 ± 0.71 1.65 ± 0.57
Confidence with reading journal articles 2.18 ± 0.76 2.27 ± 0.83 2.20 ± 0.70 1.83 ± 0.58
Table 3. Mean (± S.D.) ratings of confidence with reading textbooks and journal articles

Reflections
o Students spend less time reading than the University expectation, with
little support for either lack of time or lack of confidence as an
explanation
o Further explore students reading practices and perceptions of reading in
order to develop ways of encouraging them to read and enhance their
practice
Perceptions and attitudes of undergraduate students to reading

ncl.ac.uk
Links between teaching and research
at Newcastle University

Defining research-led and research-informed teaching
Links between research and teaching
Research-informed
Curriculum emphasises
learning focused on students
writing and discussing
papers or essays
Research-based
Curriculum emphasises
students undertaking inquiry-
based learning or low key
research
Research-led
Curriculum is structured
around teaching subject
content
Research-oriented
Curriculum emphasises teaching
processes of knowledge
construction in the subject
STUDENT-FOCUSED
STUDENTS AS PARTICIPANTS
EMPHASIS ON
RESEARCH CONTENT
EMPHASIS ON
RESEARCH PROCESSES
AND PROBLEMS
TEACHER-FOCUSED
STUDENTS AS AUDIENCE From Healey (2005), after Griffiths (2004)
Marsham S. & Huggins, C. (2016). Links between research and teaching at Newcastle University. University Learning, Teaching and Student
Experience Committee Report.

Data Collection from Schools
NSS QB5.4: My learning has benefited from modules that are informed by
current research
o Identified schools scoring above institutional score
o Met with DELTs/HoS to determine activities
o How do schools/faculties define these links?
o What do schools do to create links?
o Does linking research and teaching add value?
o How do we translate impact of our research?
Year Institution(s) % Agree
2011/12 Newcastle University 84
2011/12 All HEIs 81
2011/12 Top Quartile of sector 81
2011/12 All Institutions in England 81
2012/13 All HEIs 82
2013/14 All HEIs 82
2014/15 All HEIs 82
With thanks to:
Debbie Bevitt, Gwyneth Doherty-Sneddon, Zan Gunn, David Kennedy, Charlotte Patterson, Patrick Rosenkranz, John Sayer, Liz
Stockdale, Tim Townshend, Tony Young

Strong focus on links
between teaching and
research at Open Days
Programmes driven by research
excellence from the bottom up,
rather than research feeding into
the teaching from the top down
Academics embed own research into
generic slides and textbook material -
use their own research stories
Optional research-informed
modules at later stages
Final year students attend
national research-led
conference
School conference led
by Level 6 students
Research lectures / series /
symposia aimed at students
Student
research
projects

Research links in Marine Ecology
Research-informed
Stage 2 Marine Research
and Employability Skills
Research-based
Stage 3 Research Projects
Research-led
Stage 1 Marine Biology
Research-oriented
Stage 2 Marine Ecology
EMPHASIS ON
RESEARCH CONTENT
EMPHASIS ON
RESEARCH PROCESSES
AND PROBLEMS

Data Collection from Students
o Recruited a Student Intern to support research
o Six focus groups (two per Faculty)
o Online survey
o How do our students perceive these links?
o Does linking research and teaching add value?
o Are we translating impact of our research?
o Does it influence the students we recruit?
o Do students recognise our research reputation?
o Do we create an environment for students to see/be
involved in research?
With thanks to:
Charlotte Huggins, Suzanne Cholerton, Richard Harrison, Claire Irving, Sue Gill, and all participating students

Overall Survey Responses
o Received 383 student responses to the online survey
o Responses were largely positive
o Nearly 50% of students felt staff valued research and teaching equally (Fig. 1)
o Almost 70% agreed that staff balanced the demands of research and teaching well (Fig. 2)
8%
19%
49%
19%
5%
0% 10% 20% 30% 40% 50% 60%
Research much more than teaching
Research slightly more than teaching
Reseach & teaching equally
Teaching slightly more than research
Teaching much more than research
9%
25%
67%
0%
10%
20%
30%
40%
50%
60%
70%
Strongly
Disagree/Disagree
Neither Agree nor
Disagree
Strongly
Agree/AgreeFigure 1. Student responses to the statement “Staff in my department generally value…”
Figure 2. Student responses to the statement "University staff generally balance research
and teaching demands well"

0%
10%
20%
30%
40%
50%
60%
70%
80%
Strongly
Disagree/Disagree
Neither Agree nor
Disagree
Strongly Agree/Agree
32
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Strongly
Disagree/Disagree
Neither Agree nor
Disagree
Strongly Agree/AgreeFigure 3. Student responses to the statement “Research
in my department informs the teaching on my course”
Figure 4. Student responses to the statement “Research in my
department has a positive impact on teaching quality”
Figure 5. Student responses to the statement “Research in my
department influenced my decision to come to Newcastle
University"
0%
10%
20%
30%
40%
50%
60%
70%
80%
Strongly
Disagree/Disagree
Neither Agree nor
Disagree
Strongly Agree/Agree

Figure 6. Student responses to the statement "It is
important to me that my teachers are also researchers"
33
Differences between Levels of Study
0%
10%
20%
30%
40%
50%
60%
70%
80%
Undergraduate Postgraduate
Strongly disagree/disagree Neither agree not disagree
Strongly agree/agree
0%
10%
20%
30%
40%
50%
60%
70%
UG1 UG2 UG3
Strongly disagree/disagree Neither agree nor disagree
ANOVA, P < 0.01
0%
10%
20%
30%
40%
50%
60%
70%
UG1 UG2 UG3
Strongly disagree/disagree Neither agree nor disagree
Figure 7. Student responses to the statement “My department provides
enough opportunities for students to conduct their own research”
Figure 8. Student responses to the statement “I feel
knowledgeable about research in my department"
ANOVA, P < 0.05

Seven Key Themes
Research-teacher gap
o Disparities between the priorities of research and teaching among staff
o Disparity of skills between lecturers who were more invested in research
“More as you go”
o Research became more central to learning and teaching as students progressed through
UG studies, and particularly when moving onto PG
“…many lecturers do not appear to be passionate or interested in the teaching of students - but simply do so as a requirement to be
able to conduct their own research”
“I have felt that lecturers / seminar leaders have under prepared for my contact hours because of their own research”
“Can be difficult to arrange time to meet teaching staff with strong research commitments”
“You can usually tell which lecturers have lots of lecturing experience from the researchers who leave the lab to do 1 or 2 lectures a
year. It's not that they're not good or interesting, but sometimes they're delivery isn't as good as the regular lecturers”

Seven Key Themes
Benefits of researcher-teachers
o Students largely value being taught by researchers
Perception of research-knowledge links
o Some students did not seem to understand that research was the way knowledge is
constructed
o See research studies and course content, such as material in text books, as disparate
“Research is of paramount importance. As students we spend vast sums of money on our
education and as such we want to be assured that we are learning from world leaders”
“Its great to know that your lecturers are actively taking part in research of their own. It
increases your confidence in the content they are teaching in the lectures”
“I don't always feel it [research] benefits the individual student. Rather more cynically it merely seems to benefit the University
reputation”
“There is difficulty in maintaining a student's concentration when a lecturer mentions their own research as the information is not
necessarily required for examinations, however if the research directly impacts the content of the module, more attention would be
paid to it”
Benefit
Enthusiasm
Clarity
Staff as
resource
Authority
Cutting
edge

Seven Key Themes
Diversity of staff experience
o Important that there is diversity among the staff of their research interests and experiences
Lecturer as an individual
o Teaching skills and the ability to balance teaching and research demands depended on the
individual lecturer - related to personality
Student motivation
o Key to their understanding of teaching-research links and their engagement in research
“There are some exceptional lecturers who manage to teach us extremely well and are very dedicated, although it is clear that
they do struggle under some of the requirements… I think it is important that lecturers should have the time and resources to both
carry out research and teach effectively, rather than being forced to prioritise one over the other”
“I feel some lecturers don't talk about their own research enough which is sad because all students are curious about what research
projects your lecturer has contributed to or is working on at the moment”
“I would suggest occasional tours of labs and other research facilities to not only tell but show students what research is currently
being conducted and explain how they may help or provide them with inspiration for the future”
“I feel that opportunities for undergraduate students to partake in research is not readily available”

Conclusions
o Similar approaches to both embedding research activities into programme delivery and
communicating these to students
o Supplemented by individual approaches
o Academic staff recognise the importance of informing our students of our research - committed to
delivering research-led/research-based teaching
o Positive reaction to linking teaching and research
o Students value being taught by active researchers
o Students recognised tensions between the two activities, but reflected that on the whole individual
academics dealt with these well when delivering programmes
o Students did not necessarily base their decision to come to Newcastle on our research activities, but
once they were studying here, many felt they would have benefited from being more involved in
research and having more opportunities to understand what research was occurring within their
schools

Impact
o Focus of ULTSEC Away Day September 2016
o Strategic theme for 2017 Innovation Fund and Learning and Teaching Conference
o Requirement of Strategic Approval phase of new programme development
o External Examiners required to comment in their report
o Embedded within current Education Strategy
o Second phase undertaken in 2019 with three Student Interns
o University Experience Day October 2019
o Replicated by colleague at Royal Holloway - further collaboration planned

Newcastle Educators
Edubites 2019-2020
From Newcastle. For Educational Research.

Edubites Series 2019-2020
educators@ncl.ac.uk 41
From Newcastle. For Educational Research
We see this as a fantastic opportunity to
engage in some wide-ranging discussions
around educational research at Newcastle
https://ncleducators.wordpress.com/
Guest blogs welcome
@NewcastleEduca1
Search Newcastle Educators

Thank you
Dr Sara Marsham
sara.marsham@ncl.ac.uk
@sara_marine

Educational Research

More Related Content

What's hot

Similar to Educational Research

More from Sara Marsham

Recently uploaded

Educational Research