Ecosystem Interactions Class Discussion Presentation in Blue Green Lined Styl...
Klaus Dieter Rossade online assessment
1. Online
Assessment
Presenting the work of the EADTU Special Interest Group
Klaus-Dieter Rossade
Associate Dean Curriculum
The Open University
United Kingdom
This Photo by Unknown Author is licensed under CC BY-SA
2. This Photo by Unknown Author is licensed under CC BY-SA
Impossible Missions Force (IMF)
“Your mission, should you
choose to accept it, is …”
“This message will self-
destruct in 5 seconds”
German release: ‘Kobra, übernehmen Sie’
3.
4. Our mission, should we choose to accept it, is …
This Photo by Unknown Author is licensed under
5. Online Assessment SIG Members
• George Ubachs, EADTU, Chair
• Klaus-Dieter Rossade, OUUK
• Piet Henderikx, EADTU
• Carlton Wood, OUUK
• Michal Ben Shaul, OU Israel
• Rasa Urbas, OU Ljubjana
• Marko Papic, OU Ljubjana
• Mary Elizabeth Morocho
Quezada, CALED
• Diogo Casanova, Aberta
• Nadja Gmelch, OUC
• Ana Guerrero, OUC
• Damien Carron, FernUni/UniDistance
• Henrietta
Carbonel, FernUni/UniDistance
• Kirsi Saarinen, TAMK
• Quinten Verdonck, Thomas More
University College (Flemish gov)
• Soetkin Lootens, Artevelde University
College (Flemish gov)
• Efthimios Zervas, HOU
• Dimitris Kalles, HOU
• Miguel Santamaria, UNED
• Christina Sánchez Romero, UNED
• GeorgeÁngeles Sánchez-Elvira
Paniagua
• Jan Beseda, Czech Republic
• Pessi Lyyra, JYU
• Andreas Kempka,
FernUniversität Hagen
• Stefan Stürmer, FernUniversität Hagen
• Jose Janssen, OUNL
13+ nations
Belgium
Czech Republic
Finland
Germany
Greece
Israel
Latin America and
Caribbean region
Netherlands
Portugal
Slovenia
Spain
Switzerland
United Kingdom
(2 experts per institution)
6. Our Mission
Share expertise on institutional strategies and experiences on online
assessment.
• Inventory of assessment practices and problems faced due to Covid-19
• New developments in online assessment incl. results and recommendations of
TESLA project
• Research and innovation within and outside the EADTU on online assessment
• Institutional strategies for online assessment including examples from practice
• Supportive or non-supportive national policies and frameworks
7. Expected outputs
Report about
challenge and context
of institutions related
to online assessment
Inventory of examples
of good practice
Inventory of existing
institutional strategies
Inventory of related
projects and results
Website
Permanent SIG on
online assessment
within EADTU
Community Building Publications (joint)
8. Ways of working
• Sharing experiences in Teams online meetings
• Collating challenges and ideas on interactive
whiteboards (padlet)
• Bottom-up approach to scope the field
• Outcome open
10. Theme
Assessment
design
challenges
Pandemic required changes to
Assessment design and delivery at
speed and at scale
First wave of the 2020 Corona-19
pandemic led to lockdowns right at
the time of final summative module
exams or assignments
Meet needs of students while
maintaining quality standards
(verification, proctoring)
11. Theme
Assessment
Design
emerging
ideas
number and timing of assessment points
stakes – formative vs. summative, weighting of assignments
flexibility – allow assessment modes appropriate to discipline
and pedagogy and ensure operational stability and safety
evidence-based assessment design and transition to digital
assessment
take academic staff along the journey
navigate quality standards, student needs and institutional
contexts
12. Theme
Trust & Ethics
challenges
Trust is a multi-layered concept, including trust
in
• technology (e.g., functions reliably and as
expected; usability)
• the deployment of technology (e.g.
transparency, teacher competence)
• the organisation deploying e-assessment
(e.g. based on reputation)
• personal data processing (e.g. as agreed and
intended).
• reliability/fairness of the assessment process
(identity and authorship verification)
The challenge lies in building trust for example
by providing trustworthy solutions, on all
these levels.
13. Theme
Trust & Ethics
emerging
ideas
• Technology: protocols for cyber security;
share practices between institutions
• Deployment: guidelines that can be
‘translated’ to specific course assessment
design; careful guidance for students at all
stages to reduce stress and support success
• Personal data: comply with GDPR at all levels
(software, data sharing agreements);
consider choice for students
• Reliability: make identification part of the
proctoring process; develop clear protocols
explaining when and how technology might
indicate suspicion of fraud and how this will
be followed up with the students
14. Theme
Operational
process,
technology,
support
challenges
• Pandemic requires solutions at scale
and speed – how do we operationalise
this work?
• How to balance top-down processes
(e.g. executive level decisions) with
bottom-up requirements (pedagogy,
consistency, staff readiness)?
• How do we ensure cybersecurity?
• How do we choose the right
technology?
• How can we contribute to national
and European solutions
15. Theme
Operational
Process,
technology,
support
emerging
ideas
• Organise integrated task force for
communication and alignment of
stakeholders
• Evaluate pedagogical challenges with
online exams and develop guidelines
for future
• Support teachers, course teams
though guidance, resourcing, presence
of support services, support networks
• Ensure cybersecurity within
institutional IT framework, preliminary
condition for trustworthy
examinations
• Set pedagogy and good assessment
design before technology solutions
16. Theme protocols, checklists, frameworks, vademecum
challenges
• Organising online assessment so
students don’t need to attend
physical exams
• Negotiate questions of trust, GDPR,
ethics assessment design,
operational processes, technology
and support
• Guidance to ensure consistent and
quality assured approaches across
the institution, adherence to legal
and privacy regulations
Noun: vade mecum
/ˌvɑːdi ˈmeɪkəm
a handbook or guide
that is kept constantly at
hand for consultation
(Oxford Languages)
17. Theme protocols, checklists, frameworks, vademecum
emerging ideas
• Central role for teaching and learning
services – negotiated through
stakeholder dialogues
• Dedicated wide policy change group;
task force for examination change
work
• Support programmes in place (eg.
instructional videos and web)
• Connect with national policies and
frameworks – bottom up processes
should optimise national regulations
• Professionalised comms to
communicate changes in a
stakeholder targeted way
‘Academics should know
what is possible and what
isn't, making things less
confusing for everyone’
18. Theme
Assessment
Futures
challenges
• Define priorities for the future of
assessment within institutional and
national context
• Role of existing and emerging
technologies (TESLA, End-to-end
solution providers)
• Operational challenge to run
traditional examinations alongside
new digital formats
• Flexibilty and choice that meets
operational requirements and
assessment quality standards
• Alternatives to online exams may be
legally enforceable (right of access)
19. Theme
Assessment
Futures
emerging ideas
• Online assessment is here to stay – What
frameworks, policies and processes will
institutions need?
• Blended learning is new normal – What
are the implications within institutional
and national contexts?
• Explore high level knowledge application
over knowledge recall
• Explore benefits of assessment analytics
to support assessment quality and
standards
• Explore technologies to innovate
assessment (AI/VR)
Consider student experience at all stages of assessment design and
delivery (meaningful assessment, purpose of testing, mental health)
20. Five Principles for Future Assessment
Authentic – from acquiring knowledge to acquiring transferable skills assessed
in a more realistic way
Accessible – design assessment with accessibility-first principle, delivered in
multiple ways depending on needs of learner
Appropriately automated – mix of automated and human marking and
feedback that benefits learners
Continuous – Data and analytics to assess effectiveness and impact of
continuous assessment
Secure – authoring detection and biometic authentification for identity and
remote proctoring
JISC, Feb 2020, The Future of Assessment, https://repository.jisc.ac.uk/7733/1/the-future-of-assessment-report.pdf, p23.
21. Beyond
Enhancement:
Assessment in
2030
Relevant, Adaptable, Trustworthy
1. Meeting needs of students and
employers
2. Student centered and
personalized
3. Anytime and anywhere
4. Efficient and manageable
5. Academic integrity
6. Data use and ownership
7. Fairness
• JISC, 2020, Assessment rebooted, https://repository.jisc.ac.uk/7854/1/assessment-rebooted-report.pdf, pp 21-25
Before I close, let me share with you some exiting news.
In June 2021, we had another addition to our wider family.
Meet Johanna, two weeks old in this picture.
She is held by her grandfather Alan, my brother in law
Her hand holds the finger of her great-grandmother Rennie,
Rennie is 98 years old.
Born in 1923, Rennie never went to university
But she sat exams at school, round about the mid to late 40s.
I asked her:
“Did you sit exams at school
Yes, I did.
Maths? I asked her
Not so much
English?
Oh yes
What, in the class room, sitting at your desk, writing by hand?
Oh yes. “
It occurred to me then that Rennie’s experience of exams is not a whole lot different from the exams our kids take at school or when they go to university.
Not very different for her grandfather Alan who just did some exams for a PG certificate in Global Health, who 40 years ago trained as a Doctor.
1940s ,,, 1970 … 2020s … all these assessment milestones from the people around me and so little change?
That does not seem right.
There is a mission here for us:
We are the experts and can deliver the mission even if it seems impossible sometimes
Our impossible mission is:
Let’s make sure that when Johanna sits her exams at school and then at university,
That the assessment will be different, not the same, or almost the same, as what her great grandmother had to do almost 100 years earlier.
Whatever it takes, let’s not just keep doing what we have done for decades
But find ways to assess students that are appropriate for the time, almost the middle of the 21 century
Let’s not dismiss the disruption from Covid 19 and go back to where we were before
That is our mission,
should we choose to accept it.
This message will self-destruct in
Help me count….
