Ways to ensure “buy in” from the academics in the transition to digitised assessments Marieke Guy (Head of Digital Assessment) & Claudia Cox (Digital Assessment Advisor) Uniwise partner meeting 2nd November 2023

1. UNIWISE PARTNER MEETING
2ND NOVEMBER 2023
Ways to ensure “buy in” from the academics
in the transition to digitised assessments
Marieke Guy (Head of Digital Assessment) &
Claudia Cox (Digital Assessment Advisor)

2. Today’s session
Segment Duration
Introductory presentation: What UCL has done as an institution to engage academics 10 mins
Mentimeter poll 1 – how well do you feel your institute engages and encourages “buy in” from
academics?
2.5 mins
Presenting the 16 engagement strategies 15 mins
Breakout room activity: groups to discuss what sub-theme areas have the strongest/ weakest
representation at their respective institutes.
20 mins
Feedback from breakout rooms – scribes to summarise findings 15 mins
Mentimeter poll 2 – relative to everything we’ve covered today, how well do you feel your
institutes engages and encourages “buy in” from academics?
2.5 mins
Change management training slides 10 mins
Finish Total: 75
mins

3. University College
London (UCL)
• 11 faculties, 60+ departments
• 43,800 students, 14,300 employees, 440 UG,
675 PG programmes
• 53% international students, 150+ nationalities
• C520,000 student to assessment instances
• Variety of assessments but still majority exams
and coursework
• Procured Wiseflow in 2021

4. Wiseflow and UCL timeline
June/July 2020
• Procurement
process for
Digital
Assessment
platform begins
August-
October 2020
• Request for
Information (RFI)
and Invitation to
tender (ITT)
• Vendor selected
January -
February 2021
• Proof of concept
run
• Resource centre
(wiki) established
• Regulations
written
Summer 2021
• All central Exams
run on Wiseflow
• Staff/student
survey
September
2021
• Digital
Assessment
Team established
• Integrations work
Summer 2022
• All central exams
run on Wiseflow
& some
departmental
exams
• Further
integrations
BUSINESS AS
USUAL
• Agile product team Including
Business Analyst and Change
manager
• Engaged early - Initial pilots and
feedback survey
• AssessmentUCL User group
(academic and admin
representatives)

5. Digital Assessment Team
• Digital Assessment Advisors (DAAs) & Learning Technologist (LT)
• Understand assessment needs and challenges in faculties
• Support for design, creation and delivery of departmental assessments
• Training and support resources
• Listen to concerns and maintain enhancements list
Marieke Guy
Head of Digital
Assessment
Lene-Marie Kjems
Institute of Education &
Laws
Isobel Bowditch
Medical Sciences, Brain
Sciences, Engineering
Claudia Cox
Population Health, Life
Sciences, MAPs
Amanda Seller
AHS, SHS, Bartlett
Nadia Hussain
Learning Technologist

6. Training package
• 100+ training sessions Hands-on training for users –
Managers, Assessors (markers), Reviewers
(moderators) and Authors – over 100
• Maintenance of wiki ‘Resource centre’ (includes
guidance, Blueprints, video guides, step-by-step
guides)
• Creation of supporting visualisations
• Creation of practice assessments on the sandbox site
to support staff training
Support
• Investigate and resolve support queries from departments
• Drop in sessions
• Support model for staff and students
• Report and resolve technical Wiseflow issues with
UNIwise
• Feed into enhancement roadmap of platform
Wiseflow training & support
https://wiki.ucl.ac.uk/x/DhqmCQ

7. • 100+ training sessions delivered (covering
3 main roles – Authoring, Managing &
Marking)
• 600+ staff trained.
• Includes Bespoke training delivered for
CLIE, Laws, Philosophy, IRDR, Biochemistry
• Additional training to meet needs: MCQ &
Rubric masterclass
• Weekly drop-in sessions in 2022, restarting
in 2023 for main exam period.
• 2 x F2F Manager training sessions in 2023
• Lots of ad-hoc training!
30
17
10
19
2022
Bespoke Marker Author Manager
6
9
4
8
2023
Bespoke Marker Author Manager
Bespoke Marker Author Manager Total
2022 30 17 10 19 76
2023 6 9 4 8 27
Training stats

8. Engaging our academics
• Advisory email address & support
processes
• Teams sites for faculties
• Landscape reports (analysis of every
faculty practices)
• Case studies e.g. IRDR’s shift to
AssessmentUCL & Negative marking
• Further engagement with the sector
using the HEI User group
• Next steps – focus on pedagogic
approaches and highlighting benefits
Academic led opportunities

9. Mentimeter 1

10. Understanding academics adoption of learning
technologies
• Literature review Understanding
academics' adoption of learning
technologies: A systematic review by Qian
Liu, Susan Geerthuis and Rebecca Grainger
• Identification of four themes and 16 sub
themes:
1. The Learning Technology (relating to the
nature of a technology)
2. Academic Staff (relating to the individuals who
may utilise a learning technology)
3. Context (cultural and situational sub-themes as
instrumental in adoption)
4. Influencing Adoption (strategies and actions
effected by stakeholders in an effort to
influence adoption or diffusion).

11. Sub-themes Strategies to facilitate academics' adoption
1. Relative advantage
Provide hard, persuasive and visible evidence
Render the advantages of the new technology explicit, obvious and experienced
2. Ease of initial
adoption
Provide immediate expert support
Provide a safe place to experiment
Don't release anything that will frustrate: watch out for bugs
Make the initial experience very positive
3. Availability
Provide reliable access that is integrated with existing IT infrastructures
Provide technically easy access with minimum steps
4. Typologies of
adopters
Design training and support that caters for diversity
Recognise diversity and allow for levels and rates of adoption
5. Attitudes to change
Make learning safe
Recognise staff may become overwhelmed, anxious and defensive
Address the emotions that are associated with change and adoption
Gather concerns and address them
Assess, recognise and accommodate the disruptions to the status quo
6. Control
Academic-led opportunities are likely to lead to adoption
A take-it-or-leave approach to training and technology use does not work
Give academics as much choice and control as possible.

12. 7. Pedagogical beliefs
and practice
Replicate the previous practice in the new
Align the technology with existing thinking and practice: use familiar language
Recognise that teaching is performed in multiple ways
8. Capabilities
Make sure that learning is achievable
Recognise the limited background knowledge and the cognitive load
Include learning design tips and sample cases
Do not assume digital literacy
9. Bureaucracy
Have open and honest conversations that explain and clarify
Bring academic departments on board – aim for ownership not just buy-in
Develop systems for listening and responding to grassroot concerns
Actively disseminate knowledge across the institution
10. Policies and
purpose
Incentivise local leaders and role models and accommodate workload implications
Develop policies that allow for flexibility in implementation and which devolve decision making down -
Determine minimal acceptable levels of adoption and make non-performance uncomfortable
Be clear on the purpose and express it in terms of staff and student benefit
Recognise different interests and drivers behind the learning technology
11. Prioritisation of
research
Recognise the important role of professional staff and teaching assistants
Encourage and use research into large scale adoption initiatives
Establish sustainable mechanisms that recognise and reward teaching excellence
Ensure expectations of time required are accurate, communicated and afforded
Provide role-model adopters who are successful researchers

13. 12. Culture and
discipline
Promote cross-disciplinary conversations that trigger the rethinking of pedagogy
Design discipline-based initial training and support that addresses micro-cultures
Use local champions to explain the change and influence their peers
13. Strategic intent
Monitor progress and evaluate impact
Allow flexibility and contextualisation of the strategy at disciplinary level
Align learning technologies and their adoption with the strategy
Collaborate with the disciplines to formulate institutional strategies on learning technology
14. Participation and
collaboration
Collect and respond to the voice of professional staff
Collect and respond to student voice and feedback
Do not involve a few – use networks to involve as many academics as possible
Involve a diverse range of academics throughout beginning with the initial decision-making stage
15. Facilitative
leadership
Obtain buy-in and collaborate with academic heads and departmental champions
Senior leadership shows visible commitment to learning technologies
16. Academic
Development
Provide for long term progressive development and so support staff as they move from rudimentary adoption to
enriched pedagogies
Minimise technical jargon
Provide a rich support regime so staff can efficiently learn what they want to learn when they want to learn it and
with minimum risk
Ensure that academic development regimes are evidence-based and informed by constructivist, social and affect-
based models of learning

14. Breakout rooms –
20 minutes
In your groups discuss what sub-theme areas have
the strongest/ weakest representation at your
respective institutes. Scribe to feedback
Sub-themes as shown below:
• Relative advantage
• Ease of initial adoption
• Availability
• Typologies of adopters
• Attitudes to change
• Control
• Pedagogical beliefs and practice
• Capabilities
• Bureaucracy
• Policies and purpose
• Prioritisation of research
• Culture and discipline
• Strategic intent
• Participation and
collaboration
• Facilitative leadership
• Academic Development

15. Mentimeter 2

16. Sponsorship
Leadership
Engagement Sustainability
Skills
• Awareness
• Strategic vision
• Strategy
• Build coalition
• Shared values
• Style
• Army of volunteers
• Desire
• Structure
• Create a sense of
urgency
• Short term wins
• Staff
• Knowledge
• Ability
• Removing
barriers
• Systems
• Skills
• Institute change
• Reinforce
• Sustain
acceleration
Commonalities between Change management
models

17. A collaborative tool to gain a deeper
insight into stakeholders/end users
and help you come up with strategies
to engage with your end users.
How do they think and feel?
Can represent a group of users.
Example – buying a TV
Empathy map

18. Keep satisfied
(H,L)
Manage Closely
(H,H)
Monitor
(L,L)
Keep informed
(L,H)
High influence /
High interest
Manage
Closely
Fully engage this group;
make efforts to satisfy
them
High influence /
Low interest
Keep
Satisfied
Engage to keep satisfied,
but moderate the
engagement (new
messages, new
deliverables)
Low influence /
High interest
Keep
Informed
Adequately inform this
group; engage as SMEs /
discuss any issues, gaps
Low influence / Low
interest Monitor
Remain in contact,
monitor, but limit
communication.
Engagement mapping

19. A fictional character
created based on
research and used to
understand needs,
experiences behaviours
and goals.
Again, helpful to come
up with strategies.
UCL has some example
personas
Personas

20. Highlight Benefits:
Clearly communicate
the advantages, such
as reduced grading
time, integrations, and
enhanced data
analytics.
Address Concerns:
Be open to addressing
concerns such as
technical issues,
academic integrity and
accessibility. Establish
a feedback mechanism
for academics.
Share Success
Stories: Share
success stories and
best practices from
early adopters among
the academic staff to
inspire others.
Create and maintain a
good communication &
engagement plan:
Tailor to the impacted
stakeholder groups
ensuring the message
includes the WIIFM -
“What’s in it for me”
Engage Academics
Early: Involve
academics in the
decision-making
process from the
beginning. Seek their
input and feedback on
the transition plan.
Pilot Programs:
Implement small-
scale pilot programs
to demonstrate the
effectiveness of
platform. Use the
results to showcase
the benefits.
Training and Support:
Offer comprehensive
training and ongoing
support for academics
to help them become
comfortable with the
new assessment
methods and tools.
Involve Champions:
Identify and involve
digital assessment
champions among the
academic community
who can advocate for
the transition.
Top tips

21. UNIWISE PARTNER MEETING
2ND NOVEMBER 2023
Thanks!
Marieke Guy (Head of Digital Assessment) m.guy@ucl.ac.uk
Claudia Cox (Digital Assessment Advisor) c.cox@ucl.ac.uk