Ways to ensure “buy in” from the academics in the transition to digitised assessments
Marieke Guy (Head of Digital Assessment) & Claudia Cox (Digital Assessment Advisor)
Uniwise partner meeting
2nd November 2023
The presentation gives an overview of how the HOTEL (Holistic Approach to Technology Enhanced Learning) project intends to develop and test an Innovation Support Model which will help research projects in TEL/e-learning reach sustainability / commercialisation.
Developing a strategy for flexible learning programmes at NorthTecVasi Doncheva
Case Study presented at Future Learning and the Digital Conference June 2013 Auckland, New Zealand
Developing a strategy for flexible learning programmes in your organisation:
• How to develop and implement an institute wide strategy
• How to lead change initiatives and embed them into institutional practice
• Ways to effectively use technologies to engage and inspire learners and create accessible learning opportunities
• Tips on how to effectively integrate technology strategically into teaching and learning
Faculty Learning Communities: A Model for Faculty DevelopmentMatt Lewis
Dr. Nancy Pawlyshyn, Dr. Braddlee, and Dr. Laurette Olson co-authored this presentation. On Feb. 16, 2011 Dr. Olson and I presented this to the ELI Educause event in Washington DC.
The presentation gives an overview of how the HOTEL (Holistic Approach to Technology Enhanced Learning) project intends to develop and test an Innovation Support Model which will help research projects in TEL/e-learning reach sustainability / commercialisation.
Developing a strategy for flexible learning programmes at NorthTecVasi Doncheva
Case Study presented at Future Learning and the Digital Conference June 2013 Auckland, New Zealand
Developing a strategy for flexible learning programmes in your organisation:
• How to develop and implement an institute wide strategy
• How to lead change initiatives and embed them into institutional practice
• Ways to effectively use technologies to engage and inspire learners and create accessible learning opportunities
• Tips on how to effectively integrate technology strategically into teaching and learning
Faculty Learning Communities: A Model for Faculty DevelopmentMatt Lewis
Dr. Nancy Pawlyshyn, Dr. Braddlee, and Dr. Laurette Olson co-authored this presentation. On Feb. 16, 2011 Dr. Olson and I presented this to the ELI Educause event in Washington DC.
Presented at Sloan-C Blended, Milwaukee, WI, July 8th, 2013
With the increase in the diffusion of blended and online programming across higher educational institutions, stakeholders are looking for ways to ensure the quality of the student experience. Quality of blended programs can be ensured through faculty and instructional development and training, faculty and instructor evidence of competence and recognition for excellence, constructive evaluation and feedback on blended and online course design and delivery, and community-building opportunities among instructors and staff. Blended learning is becoming a prominent mode of programming and delivery in education. It is swiftly emerging and transforming higher education to better meet the needs of our students providing them with more effective learning experiences. This movement is leading to a renovation in the way courses are taught and programs support their students. Instructional and faculty development provides the core foundation to institutional programming in providing a framework for implementing blended and online learning pedagogy in the classroom. This student-centered, active learning pedagogy has the potential to alter the traditional classroom by enhancing course effectiveness through increased interactivity leading to superior student outcomes.
A recent study reported that "Respondents ... anticipated that the number of students taking online courses will grow by 22.8% and that those taking blended courses will grow even more over the next 2 years" (Picciano, Seamen, Shea, & Swan, 2012, p. 128). As the demand for blended learning opportunities increases, so does the need for development of instructors to teach and design blended courses and mechanisms to ensure the quality of courses and programs. The University of Wisconsin - Milwaukee (UWM) has been providing instructional development and blended learning opportunities to students for over a decade. Since 2001, UWM has developed 8 blended degree programs. In the fall of 2012, UWM offered approximately 100 blended courses and enrolled 7,655 students (26%) in at least one blended course. UWM continues to see growth, as the nation does, and continues to provide opportunities for students to best meet their needs.
Latest Trends in Standards and Benchmarking for Technology Enhanced Learning:...Charles Darwin University
As TEQSA ramps-up its interest in how higher education institutions are mediating a level of quality in online courses, it is timely for our institutions to understand what options they have at their disposal to help meet, what will eventually become, obligatory reporting. This will predictably involve being able to provide evidence of quality processes across three levels; the unit/course level, the course/program level and at an institutional level. To help an institution establish, or initiate practices to assist them in meeting these requirements, there are a number of existing and emerging tools that are now being used and trialled within the sector. There are also a number of professional bodies who are really keen to get involved in this space, mainly as a way of supporting their members and member institutions.
This presentation will present a systematic review of the literature and a sector scan of the current tools and methodologies being used by a range of institutions and professional bodies. It will also provide some insights into what tools are emerging in this space. To support, or augment these tools and methodologies, there are also a range of support, or scaffolding activities institutions can, or have been be involved in, as they seek to support their internal quality improvement practices.
Examples of current practices will be drawn on to illustrate the main points in this presentation along with a brief report on the most recent ACODE Inter-institutional Benchmarking Summit, held in June 2018 at Griffith University. At this event 25 Australasian Institutions undertook a rigorous benchmarking activity using some or all of the eight benchmarks focusing on technology enhanced learning and teaching.
In conclusion this presentation will challenge institutions to take seriously their mandate to provide their students with learning environments that meet the highest possible quality, particularly now in a higher education setting that will come under increased scrutiny by regulatory bodies. More importantly, it will reflect on what the potential implications are for institutions in moderating their learning management and associated systems.
Designing Exemplary Online Courses in BlackboardJason Rhode
The Blackboard Exemplary Course Program began in 2000 with the goal of identifying and disseminating best practices for designing engaging online courses. Using an established rubric for online course quality, faculty and course designers can evaluate how well their course conforms to proven online teaching best practices for Course Design, Interaction and Collaboration, Assessment, and Learner Support. During this online session offered 12/17/13, we explored suggested best practices included in the Blackboard Exemplary Course Program Rubric for designing engaging online courses. Practical tips for building a course in Blackboard that meets the established quality benchmarks and links to sample award-winning course tours were provided. We also covered the steps and associated deadlines for faculty interested in submitting their course for consideration as a Blackboard Exemplary Course. This workshop was geared toward an audience already familiar with the basic online teaching tools available in Blackboard.
Discusses 8 factors that contribute to the quality of e-learning and how to measure each: PLATFORM = selection/adoption criteria, SUPPORT = technological infrastructure, DESIGN = Standards from the QM Higher Education Rubric, DELIVERY = process (formative) and outcome (summative) evaluation, CONTENT = panel of expert stakeholders, FORMAT = sophistication scale, STUDENT = self-review, and FACULTY = peer-review
A preliminary discussion on the specifics of setting up a quality assurance process for assets, content and metadata in a learning repository. Please don't hesitate to contact me in case you have any relevant input.
The blandness is its formulaic style’: insights to help understand the impact...Marieke Guy
The blandness is its formulaic style’: insights to help understand the impact of AI on assessments
ChangeMakers AI Lunch & Learn sessions
Wednesday 1st November, 1-2pm
More Related Content
Similar to Ways to ensure “buy in” from the academics in the transition to digitised assessments
Presented at Sloan-C Blended, Milwaukee, WI, July 8th, 2013
With the increase in the diffusion of blended and online programming across higher educational institutions, stakeholders are looking for ways to ensure the quality of the student experience. Quality of blended programs can be ensured through faculty and instructional development and training, faculty and instructor evidence of competence and recognition for excellence, constructive evaluation and feedback on blended and online course design and delivery, and community-building opportunities among instructors and staff. Blended learning is becoming a prominent mode of programming and delivery in education. It is swiftly emerging and transforming higher education to better meet the needs of our students providing them with more effective learning experiences. This movement is leading to a renovation in the way courses are taught and programs support their students. Instructional and faculty development provides the core foundation to institutional programming in providing a framework for implementing blended and online learning pedagogy in the classroom. This student-centered, active learning pedagogy has the potential to alter the traditional classroom by enhancing course effectiveness through increased interactivity leading to superior student outcomes.
A recent study reported that "Respondents ... anticipated that the number of students taking online courses will grow by 22.8% and that those taking blended courses will grow even more over the next 2 years" (Picciano, Seamen, Shea, & Swan, 2012, p. 128). As the demand for blended learning opportunities increases, so does the need for development of instructors to teach and design blended courses and mechanisms to ensure the quality of courses and programs. The University of Wisconsin - Milwaukee (UWM) has been providing instructional development and blended learning opportunities to students for over a decade. Since 2001, UWM has developed 8 blended degree programs. In the fall of 2012, UWM offered approximately 100 blended courses and enrolled 7,655 students (26%) in at least one blended course. UWM continues to see growth, as the nation does, and continues to provide opportunities for students to best meet their needs.
Latest Trends in Standards and Benchmarking for Technology Enhanced Learning:...Charles Darwin University
As TEQSA ramps-up its interest in how higher education institutions are mediating a level of quality in online courses, it is timely for our institutions to understand what options they have at their disposal to help meet, what will eventually become, obligatory reporting. This will predictably involve being able to provide evidence of quality processes across three levels; the unit/course level, the course/program level and at an institutional level. To help an institution establish, or initiate practices to assist them in meeting these requirements, there are a number of existing and emerging tools that are now being used and trialled within the sector. There are also a number of professional bodies who are really keen to get involved in this space, mainly as a way of supporting their members and member institutions.
This presentation will present a systematic review of the literature and a sector scan of the current tools and methodologies being used by a range of institutions and professional bodies. It will also provide some insights into what tools are emerging in this space. To support, or augment these tools and methodologies, there are also a range of support, or scaffolding activities institutions can, or have been be involved in, as they seek to support their internal quality improvement practices.
Examples of current practices will be drawn on to illustrate the main points in this presentation along with a brief report on the most recent ACODE Inter-institutional Benchmarking Summit, held in June 2018 at Griffith University. At this event 25 Australasian Institutions undertook a rigorous benchmarking activity using some or all of the eight benchmarks focusing on technology enhanced learning and teaching.
In conclusion this presentation will challenge institutions to take seriously their mandate to provide their students with learning environments that meet the highest possible quality, particularly now in a higher education setting that will come under increased scrutiny by regulatory bodies. More importantly, it will reflect on what the potential implications are for institutions in moderating their learning management and associated systems.
Designing Exemplary Online Courses in BlackboardJason Rhode
The Blackboard Exemplary Course Program began in 2000 with the goal of identifying and disseminating best practices for designing engaging online courses. Using an established rubric for online course quality, faculty and course designers can evaluate how well their course conforms to proven online teaching best practices for Course Design, Interaction and Collaboration, Assessment, and Learner Support. During this online session offered 12/17/13, we explored suggested best practices included in the Blackboard Exemplary Course Program Rubric for designing engaging online courses. Practical tips for building a course in Blackboard that meets the established quality benchmarks and links to sample award-winning course tours were provided. We also covered the steps and associated deadlines for faculty interested in submitting their course for consideration as a Blackboard Exemplary Course. This workshop was geared toward an audience already familiar with the basic online teaching tools available in Blackboard.
Discusses 8 factors that contribute to the quality of e-learning and how to measure each: PLATFORM = selection/adoption criteria, SUPPORT = technological infrastructure, DESIGN = Standards from the QM Higher Education Rubric, DELIVERY = process (formative) and outcome (summative) evaluation, CONTENT = panel of expert stakeholders, FORMAT = sophistication scale, STUDENT = self-review, and FACULTY = peer-review
A preliminary discussion on the specifics of setting up a quality assurance process for assets, content and metadata in a learning repository. Please don't hesitate to contact me in case you have any relevant input.
The blandness is its formulaic style’: insights to help understand the impact...Marieke Guy
The blandness is its formulaic style’: insights to help understand the impact of AI on assessments
ChangeMakers AI Lunch & Learn sessions
Wednesday 1st November, 1-2pm
Redesigning assessments for a world with artificial intelligenceMarieke Guy
Redesigning assessments for a world with artificial intelligence presentation By Marieke Guy, Head of Digital Assessment, UCL
QAA Annual Conference, The Future of Quality: What’s Next?
Wednesday 13 September 2023
MCQs_ The joys of making your mind up.pdfMarieke Guy
Explore the benefits and challenges of using MCQs in both formative and summative assessment, and get practical guidance on designing good MCQs in AssessmentUCL.
4 March, 10.30am-11.30am. Online event.
Multiple choice questions have often had a bad rap in education, sometimes seen as assessing only lower level skills such as factual recall. However, with good question design this assessment approach can allow for testing of more complex cognitive processes. Add in the increasing sophistication of options offered by digital assessment platforms, which allow automatic grading and statistical analysis, and you can begin to significantly streamline your marking processes.
This workshop will explore the benefits and challenges of using MCQs in both formative and summative assessment and provide practical guidance on:
Constructing good MCQs
The range of MCQs available on digital platforms, focussing on AssessmentUCL.
There will be time for discussion and questions.
After attending this session, you will be able to:
Create worthwhile MCQs that test a range of learning outcomes.
Understand the range of MCQs available on digital platforms and how they can be used, focussing on AssessmentUCL.
Who should attend this session
All those engaged in teaching, assessment and the support of learning (academics, administrators, professional service colleagues).
Rubrics_ removing the glitch in the assessment matrix (1).pdfMarieke Guy
Rubrics bring together criteria, grades and feedback into a single scoring matrix. This session will explore how to design a good rubric and the benefits and potential challenges of using rubrics in assessments.
Would you like to increase reliability and consistency in marking, ensure alignment with intended learning outcomes and provide an efficient feedback mechanism for students? If so, this session on rubrics is for you.
Rubrics are a useful way of bringing together criteria, grades and feedback into a single scoring matrix to help streamline marking, provide transparency and support learners to understand how their performance will be judged.
This workshop will focus on the benefits and potential challenges of using rubrics in assessment within your subject area and provide practical guidance on:
How to design a good rubric
Creating and marking with rubrics in Assessment UCL
There will be opportunities for discussion and questions.
After attending this session, you will be able to:
Understand the benefits and potential challenges of using rubrics in assessment
Design an appropriate rubric for your assessments
Understand how to create and mark with rubrics in Assessment UCL
Who should attend this session
All those engaged in teaching, assessment and the support of learning (academics, administrators, professional service colleagues).
This is a presentation by Dada Robert in a Your Skill Boost masterclass organised by the Excellence Foundation for South Sudan (EFSS) on Saturday, the 25th and Sunday, the 26th of May 2024.
He discussed the concept of quality improvement, emphasizing its applicability to various aspects of life, including personal, project, and program improvements. He defined quality as doing the right thing at the right time in the right way to achieve the best possible results and discussed the concept of the "gap" between what we know and what we do, and how this gap represents the areas we need to improve. He explained the scientific approach to quality improvement, which involves systematic performance analysis, testing and learning, and implementing change ideas. He also highlighted the importance of client focus and a team approach to quality improvement.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
What is the purpose of the Sabbath Law in the Torah. It is interesting to compare how the context of the law shifts from Exodus to Deuteronomy. Who gets to rest, and why?
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
The Indian economy is classified into different sectors to simplify the analysis and understanding of economic activities. For Class 10, it's essential to grasp the sectors of the Indian economy, understand their characteristics, and recognize their importance. This guide will provide detailed notes on the Sectors of the Indian Economy Class 10, using specific long-tail keywords to enhance comprehension.
For more information, visit-www.vavaclasses.com
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
2024.06.01 Introducing a competency framework for languag learning materials ...
Ways to ensure “buy in” from the academics in the transition to digitised assessments
1. UNIWISE PARTNER MEETING
2ND NOVEMBER 2023
Ways to ensure “buy in” from the academics
in the transition to digitised assessments
Marieke Guy (Head of Digital Assessment) &
Claudia Cox (Digital Assessment Advisor)
2. Today’s session
Segment Duration
Introductory presentation: What UCL has done as an institution to engage academics 10 mins
Mentimeter poll 1 – how well do you feel your institute engages and encourages “buy in” from
academics?
2.5 mins
Presenting the 16 engagement strategies 15 mins
Breakout room activity: groups to discuss what sub-theme areas have the strongest/ weakest
representation at their respective institutes.
20 mins
Feedback from breakout rooms – scribes to summarise findings 15 mins
Mentimeter poll 2 – relative to everything we’ve covered today, how well do you feel your
institutes engages and encourages “buy in” from academics?
2.5 mins
Change management training slides 10 mins
Finish Total: 75
mins
3. University College
London (UCL)
• 11 faculties, 60+ departments
• 43,800 students, 14,300 employees, 440 UG,
675 PG programmes
• 53% international students, 150+ nationalities
• C520,000 student to assessment instances
• Variety of assessments but still majority exams
and coursework
• Procured Wiseflow in 2021
4. Wiseflow and UCL timeline
June/July 2020
• Procurement
process for
Digital
Assessment
platform begins
August-
October 2020
• Request for
Information (RFI)
and Invitation to
tender (ITT)
• Vendor selected
January -
February 2021
• Proof of concept
run
• Resource centre
(wiki) established
• Regulations
written
Summer 2021
• All central Exams
run on Wiseflow
• Staff/student
survey
September
2021
• Digital
Assessment
Team established
• Integrations work
Summer 2022
• All central exams
run on Wiseflow
& some
departmental
exams
• Further
integrations
BUSINESS AS
USUAL
• Agile product team Including
Business Analyst and Change
manager
• Engaged early - Initial pilots and
feedback survey
• AssessmentUCL User group
(academic and admin
representatives)
5. Digital Assessment Team
• Digital Assessment Advisors (DAAs) & Learning Technologist (LT)
• Understand assessment needs and challenges in faculties
• Support for design, creation and delivery of departmental assessments
• Training and support resources
• Listen to concerns and maintain enhancements list
Marieke Guy
Head of Digital
Assessment
Lene-Marie Kjems
Institute of Education &
Laws
Isobel Bowditch
Medical Sciences, Brain
Sciences, Engineering
Claudia Cox
Population Health, Life
Sciences, MAPs
Amanda Seller
AHS, SHS, Bartlett
Nadia Hussain
Learning Technologist
6. Training package
• 100+ training sessions Hands-on training for users –
Managers, Assessors (markers), Reviewers
(moderators) and Authors – over 100
• Maintenance of wiki ‘Resource centre’ (includes
guidance, Blueprints, video guides, step-by-step
guides)
• Creation of supporting visualisations
• Creation of practice assessments on the sandbox site
to support staff training
Support
• Investigate and resolve support queries from departments
• Drop in sessions
• Support model for staff and students
• Report and resolve technical Wiseflow issues with
UNIwise
• Feed into enhancement roadmap of platform
Wiseflow training & support
https://wiki.ucl.ac.uk/x/DhqmCQ
7. • 100+ training sessions delivered (covering
3 main roles – Authoring, Managing &
Marking)
• 600+ staff trained.
• Includes Bespoke training delivered for
CLIE, Laws, Philosophy, IRDR, Biochemistry
• Additional training to meet needs: MCQ &
Rubric masterclass
• Weekly drop-in sessions in 2022, restarting
in 2023 for main exam period.
• 2 x F2F Manager training sessions in 2023
• Lots of ad-hoc training!
30
17
10
19
2022
Bespoke Marker Author Manager
6
9
4
8
2023
Bespoke Marker Author Manager
Bespoke Marker Author Manager Total
2022 30 17 10 19 76
2023 6 9 4 8 27
Training stats
8. Engaging our academics
• Advisory email address & support
processes
• Teams sites for faculties
• Landscape reports (analysis of every
faculty practices)
• Case studies e.g. IRDR’s shift to
AssessmentUCL & Negative marking
• Further engagement with the sector
using the HEI User group
• Next steps – focus on pedagogic
approaches and highlighting benefits
Academic led opportunities
10. Understanding academics adoption of learning
technologies
• Literature review Understanding
academics' adoption of learning
technologies: A systematic review by Qian
Liu, Susan Geerthuis and Rebecca Grainger
• Identification of four themes and 16 sub
themes:
1. The Learning Technology (relating to the
nature of a technology)
2. Academic Staff (relating to the individuals who
may utilise a learning technology)
3. Context (cultural and situational sub-themes as
instrumental in adoption)
4. Influencing Adoption (strategies and actions
effected by stakeholders in an effort to
influence adoption or diffusion).
11. Sub-themes Strategies to facilitate academics' adoption
1. Relative advantage
Provide hard, persuasive and visible evidence
Render the advantages of the new technology explicit, obvious and experienced
2. Ease of initial
adoption
Provide immediate expert support
Provide a safe place to experiment
Don't release anything that will frustrate: watch out for bugs
Make the initial experience very positive
3. Availability
Provide reliable access that is integrated with existing IT infrastructures
Provide technically easy access with minimum steps
4. Typologies of
adopters
Design training and support that caters for diversity
Recognise diversity and allow for levels and rates of adoption
5. Attitudes to change
Make learning safe
Recognise staff may become overwhelmed, anxious and defensive
Address the emotions that are associated with change and adoption
Gather concerns and address them
Assess, recognise and accommodate the disruptions to the status quo
6. Control
Academic-led opportunities are likely to lead to adoption
A take-it-or-leave approach to training and technology use does not work
Give academics as much choice and control as possible.
12. 7. Pedagogical beliefs
and practice
Replicate the previous practice in the new
Align the technology with existing thinking and practice: use familiar language
Recognise that teaching is performed in multiple ways
8. Capabilities
Make sure that learning is achievable
Recognise the limited background knowledge and the cognitive load
Include learning design tips and sample cases
Do not assume digital literacy
9. Bureaucracy
Have open and honest conversations that explain and clarify
Bring academic departments on board – aim for ownership not just buy-in
Develop systems for listening and responding to grassroot concerns
Actively disseminate knowledge across the institution
10. Policies and
purpose
Incentivise local leaders and role models and accommodate workload implications
Develop policies that allow for flexibility in implementation and which devolve decision making down -
Determine minimal acceptable levels of adoption and make non-performance uncomfortable
Be clear on the purpose and express it in terms of staff and student benefit
Recognise different interests and drivers behind the learning technology
11. Prioritisation of
research
Recognise the important role of professional staff and teaching assistants
Encourage and use research into large scale adoption initiatives
Establish sustainable mechanisms that recognise and reward teaching excellence
Ensure expectations of time required are accurate, communicated and afforded
Provide role-model adopters who are successful researchers
13. 12. Culture and
discipline
Promote cross-disciplinary conversations that trigger the rethinking of pedagogy
Design discipline-based initial training and support that addresses micro-cultures
Use local champions to explain the change and influence their peers
13. Strategic intent
Monitor progress and evaluate impact
Allow flexibility and contextualisation of the strategy at disciplinary level
Align learning technologies and their adoption with the strategy
Collaborate with the disciplines to formulate institutional strategies on learning technology
14. Participation and
collaboration
Collect and respond to the voice of professional staff
Collect and respond to student voice and feedback
Do not involve a few – use networks to involve as many academics as possible
Involve a diverse range of academics throughout beginning with the initial decision-making stage
15. Facilitative
leadership
Obtain buy-in and collaborate with academic heads and departmental champions
Senior leadership shows visible commitment to learning technologies
16. Academic
Development
Provide for long term progressive development and so support staff as they move from rudimentary adoption to
enriched pedagogies
Minimise technical jargon
Provide a rich support regime so staff can efficiently learn what they want to learn when they want to learn it and
with minimum risk
Ensure that academic development regimes are evidence-based and informed by constructivist, social and affect-
based models of learning
14. Breakout rooms –
20 minutes
In your groups discuss what sub-theme areas have
the strongest/ weakest representation at your
respective institutes. Scribe to feedback
Sub-themes as shown below:
• Relative advantage
• Ease of initial adoption
• Availability
• Typologies of adopters
• Attitudes to change
• Control
• Pedagogical beliefs and practice
• Capabilities
• Bureaucracy
• Policies and purpose
• Prioritisation of research
• Culture and discipline
• Strategic intent
• Participation and
collaboration
• Facilitative leadership
• Academic Development
16. Sponsorship
Leadership
Engagement Sustainability
Skills
• Awareness
• Strategic vision
• Strategy
• Build coalition
• Shared values
• Style
• Army of volunteers
• Desire
• Structure
• Create a sense of
urgency
• Short term wins
• Staff
• Knowledge
• Ability
• Removing
barriers
• Systems
• Skills
• Institute change
• Reinforce
• Sustain
acceleration
Commonalities between Change management
models
17. A collaborative tool to gain a deeper
insight into stakeholders/end users
and help you come up with strategies
to engage with your end users.
How do they think and feel?
Can represent a group of users.
Example – buying a TV
Empathy map
18. Keep satisfied
(H,L)
Manage Closely
(H,H)
Monitor
(L,L)
Keep informed
(L,H)
High influence /
High interest
Manage
Closely
Fully engage this group;
make efforts to satisfy
them
High influence /
Low interest
Keep
Satisfied
Engage to keep satisfied,
but moderate the
engagement (new
messages, new
deliverables)
Low influence /
High interest
Keep
Informed
Adequately inform this
group; engage as SMEs /
discuss any issues, gaps
Low influence / Low
interest Monitor
Remain in contact,
monitor, but limit
communication.
Engagement mapping
19. A fictional character
created based on
research and used to
understand needs,
experiences behaviours
and goals.
Again, helpful to come
up with strategies.
UCL has some example
personas
Personas
20. Highlight Benefits:
Clearly communicate
the advantages, such
as reduced grading
time, integrations, and
enhanced data
analytics.
Address Concerns:
Be open to addressing
concerns such as
technical issues,
academic integrity and
accessibility. Establish
a feedback mechanism
for academics.
Share Success
Stories: Share
success stories and
best practices from
early adopters among
the academic staff to
inspire others.
Create and maintain a
good communication &
engagement plan:
Tailor to the impacted
stakeholder groups
ensuring the message
includes the WIIFM -
“What’s in it for me”
Engage Academics
Early: Involve
academics in the
decision-making
process from the
beginning. Seek their
input and feedback on
the transition plan.
Pilot Programs:
Implement small-
scale pilot programs
to demonstrate the
effectiveness of
platform. Use the
results to showcase
the benefits.
Training and Support:
Offer comprehensive
training and ongoing
support for academics
to help them become
comfortable with the
new assessment
methods and tools.
Involve Champions:
Identify and involve
digital assessment
champions among the
academic community
who can advocate for
the transition.
Top tips
21. UNIWISE PARTNER MEETING
2ND NOVEMBER 2023
Thanks!
Marieke Guy (Head of Digital Assessment) m.guy@ucl.ac.uk
Claudia Cox (Digital Assessment Advisor) c.cox@ucl.ac.uk
Editor's Notes
All work is part of the product team – agile and Digital Assessment team
In case people are unaware UCL is a large and broad institution with over 11 faculties, covering areas from brain science and maths and physical sciences, to social and historical sciences and the built environment. Also countless departments, some of which are research only.
There are almost 45 thousand students, a large number of whom are postgraduate.
We are very much an international institution with over 150 nationalities represented.
All of this means that we are very sensitive to scale and large cohorts and have a lot of assessment instances. We are also still exam and course work heavy, though keen to diversify more in the future.
In the picture you can see Jeremy Bentham, the philosopher who devised the doctrine of utilitarianism, arguing that the ‘greatest happiness of the greatest number is the only right and proper end of government’.
He supported the idea of equal opportunity in education and his ideas contributed to the foundation of University College London in 1826. Bentham left his body to medical science and requested that his body be preserved and gifted to UCL.Today Bentham sits in UCL’s South Cloisters dressed in his own clothes and sitting in his chair
In the following case study Dr Alexander Steele, teaching fellow in Earth Sciences, talks about how they have been using FlowMulti (the online exam version of an assessment) to deliver Multiple Choice Questions (MCQs) with negative marking.
In the following case study Rebecca Spencer, Natural Sciences Programme manager, talks about how they have been using AssessmentUCL (Wiseflow) for the Natural Sciences literature project module. AssessmentUCL has proved to be a useful administrative tool in managing markers on the assessment.
In the following video case study (35 minutes 41 seconds) members of the team (Professor Joanna Faure Walker, Dr Saman Ghaffarian, Jose Delgado and John Spittles) talk about the IRDR departmental decision to move to AssessmentUCL/Wiseflow.
131 articles on academics adoption of learning technologies are systematically reviewed.
•Adoption is complex, influenced by technology, academics, context and strategies.
•Multiple implications for practice and policy were identified
•Future research should use longitudinal, multimethod research and robust measurement.
•Future research should avoid the assumptions that adoption is desirable or binary.
What Does ADKAR Stand For?
Awareness of the need for change
Desire to participate and support the change
Knowledge of how to change
Ability to implement desired skills and behaviors
Reinforcement to sustain the change
The commonalities between the change models we looked at, can be grouped into 4 categories:
Sponsorship/Leadership
Engagement
Skills
Sustainability
underneath each category are the components of the various change models we have looked at.
Says – quotes from users in research or from previous data
Thinks – Similar to says, focused on what someone is or may be thinking. Ask people what is on their mind – thinking about what they may be hesitant to share how they are feeling may help
Feels – addresses emotional state – what worries or excites users and why
Does – what actions to they take?
There may be similar info in each section and don’t worry too much. It’s more about getting another view point
Planning to develop some of these – not used them much
Planning to develop some of these – not used them much