This document presents a qualitative study on using augmented reality (AR) and virtual reality (VR) to enhance inquiry skills for responsible research and innovation (RRI) in educational settings. It finds that immersive technologies improve student engagement and understanding, while also outlining key benefits and barriers related to their adoption. Recommendations for both educators and technology developers are provided to improve immersive learning experiences and contextual assessments.

1. Project Number: 688520 – TESLA – H2020-ICT-2015/H2020-ICT-2015 Agreement Number: 688520
Formative assessment of inquiry skills for
Responsible Research and Innovation
using 3D Virtual Reality Glasses
and Face Recognition
Alexandra Okada
ale.okada@open.ac.uk
Okada, A.; Rocha, K.; Fuchter, S.; Zucchi, S. ; Wortley, D.

2. 2
Literature
Responsible Research and Innovation
To embed a technological innovation in society,
researchers and innovators must interact with
societal users to discuss the (ethical) acceptability,
sustainability and desirability of the innovation
process and products
(EC, 2012; Von Schomberg 2011)

3. 3
INDEX
• Research Context
• Literature
• Methodology
• Findings
• Conclusion

4. Research Context

5. 5
Context
Can immersive technologies be used to foster
advanced digital skills and scientific literacy
across Economy and Society? To what extent ?
How will technology-enhanced assessment support digital
transformation?
Short-term learning activities?
Medium and Long-term courses?
On-job professional training?
350,000 vacancies in Europe in:
• Artificial intelligence
• Data analytics
• Cybersecurity
• …
What is immersive learning?

6. Literature
How can assessment be changed towards the Generation 4?
Immersive
experiences
AR VR
AI
Formative assessment is vital for students to develop competences:
knowledge, skills and attitude for lifelong learning
What is embedded assessment?
Authentic and
Authenticated
assessment
Robotics

7. Literature

8. Literature
Inquiry Skills for
Responsible Research and Innovation (RRI)
This qualitative study provides:
A model to design and implement AR inquiry game |which engaged
students to
• Care about global issues
• Know science concepts
• Do actions through an open schooling network
The key drivers for teachers’ adoption of AR inquiry games:
1. Making learning meaningful and engaging
2. Students-centered approaches with topical resources
3. Easy-to-use technologies
4. Pedagogical strategies including formative assessment
to identify students’ achievements and difficulties
What are teachers’ views about immersive tools
to foster and assess inquiry skills for RRI?

9. Literature
Intelligent Computing Conference 2019
Fig1. HoloSENAI Motor – example created to illustrate the AR experience
https://www.youtube.com/watch?v=nnebPmlqmw0
This mixed-method study provides:
Students views about using HoloLens to explore and analyze
data about an electric motor.
Immersive Analytics’ benefits for students :
• High level of motivation, curiosity and understanding
• Increased knowledge in terms of features, properties and
functionalities of the motor,
• Better acquisition of information and data analysis skills
after using Microsoft HoloLens and HoloSENAI motor App.
Key barriers highlighted by educational technologies were:
• the high cost of the HoloLens equipment and
• the technical development of new applications.
What are the benefits & barriers of immersive
HoloLens for learners to develop skills for RRI?

10. How can educational technology designers promote
immersive learning & assessment-in-context with virtual reality ?
This qualitative study provides:
• Three types of learners’ interactions features highlighted by
social, virtual and cognitive presence.
• immersive experience with FR authentication assessment.
• Examples about peer-learning and assessment-in-context
enhanced by immersive presence.
• Recommendations about authenticated immersive learning for
teaching staff and technology developers
This study

11. Methodology

12. Research questions:
How can virtual reality be used to foster inquiry skills
for Responsible Research and Innovation (RRI)?
1.How do peers interact with each other and with the 3DVRG?
2.Do these interactions enhance peer-learning and assessment-in-
context of inquiry skills for RRI?
3.What are the key recommendations for teaching staff and technology
teams interested in authentic experiences with virtual reality with
authenticated assessment?

13. This exploratory study focuses on:
• the app “virtual classroom” created in the unit 3D by the authors
• with 360 degree photos and video clips of the famous “Bletchley Park”
the TeSLA e-authentication instruments for voice and face recognition
(funded by EC)
Resources

14. Community of Inquiry (Garrison et al., 2000, 2001 ) extended by authors
Productive conversations are likely to lead to epistemic engagement
VIRTUAL
Students experience
emotional engagement
(Nafukho, 2014), and
Sull (2014)
connectedness
belonging
empathy
Students construct
knowledge together
(Garrison, et al., 2001;
Nagel & Kotzé, 2010)
sensemaking
critical thinking
understanding
Students’ interactions with the
virtual teaching environment and their peers
Yang et al. (2010)
Feedback
Assessment
SupportRaise issues
LEARNING

15. Participants:
• Pilot Study: 30 participants
• Courses: Computing and Educational Technology
• Four pairs of students and two facilitators (10 people) were invited from
three universities :
• Open University - UK
• Estácio de Sá University
• UNISUL Virtual University

16. Procedures:
1. Watch a short video
2. Fill out the participation consent
3. Read the orientations
4. Answer the pre-questionnaire and enrolment
5. Use 3DRVG to explore the Bletchley museum in pairs
6. Complete the assessment task in Moodle with Face Recognition
7. Fill out the post-questionnaire
Methodology
peer-interaction visual-interaction space-exploration formative-assessment
Individual
(with peers)
Individual
(alone)

17. Findings

18. Virtual learning/teaching presence
• Participants shared feedback about their experience.
(if you look down or spin around too much you might feel a bit dizzy)
• They mentioned technical issues.
(It is a “still” photo... if it was more dynamic... )
• They explored and shared technical solutions
(I found a thing which approximates and decreases at least on these glasses.
I can pull forward or back places…)
• Most of students were very engaged with the topic.
They completed the assessment successfully
(Where did you find this information? I would like to read and check more details).
Findings: peers interaction
360 App & 3DVRG

19. Social presence
• Students shared the feeling of being or willing to be together
(Let's go,… Wait,…Wow, We are in,…).
• The social interactions activated participants’ curiosity. They shared
various questions (Is him Turing?, Is it actually him?)
•
They also talked about issues, needs and expectations (Can we walk
forward? …Where are you?....)
Findings: peers interaction
Peer-Learning

20. Cognitive presence
• The interaction in pairs activated learners’ previous knowledge
(at the university, we learn, Turing machine is the origin of everything, right)
It is very known in computing area but not in other fields… why not
• visual immersion was useful to understand the size and
components of the first computers
(The most interesting thing to see is the size right? ! From the machine, which was
needed to process all this) right? …See…electronic components were very
large Yes, with lots of switches…
• Exploring objects and sharing questions created opportunities to
reflect about the social-cultural historical context.
(lets see…What objects did Alan Turing have in his office? …
Findings: peers interaction
What would it happen in the world without Turing’s machine
peer-assessment

21. Findings: issues about
e-authentication
• Commitment to try and solve problems
Initially I had problems with the face recognition tool because the system was not working in
Chrome browser. After changing it, I did not have any technical problem.
• Trust - confidence
I found the whole process longer than traditional assessment without e- authentication,
however, I do believe that the face recognition will increase trust of participants with online
assessment.
• Quality of e-assessment
To ensure quality of the e-assessment activities, I think that various tools could be integrated
with option for students to select in case of technical problems.
e-authentication

22. Kolb (1984) Experiential Learning Experience as a source of learning and development
Extended by authors from 2014 skillsyouneed.com
connectedness
belonging
empathy
sensemaking
critical thinking
understanding
Feedback
Assessment
SupportRaise issues
Findings: inquiry skills for RRI
with immersive learning
QualityCommitment
Trust
• Writing an
article
• Create a
project
• Develop
an app
Authorship
Authenticity
Immersive
do think
acquireextend

23. Recommendations
for teaching activities with 3DVRG :
• Suggest a set of goals for students to expand social, virtual and cognitive
presence.
• Clarify how they can find each other during the virtual tour.
• Provide different pathways with some meeting points (e.g. coffee area).
• Encourage students to connect their virtual experience with current facts or past
events and link previous knowledge with visual information.

24. Recommendations
for VR App developers:
• Enable learners to read information and expand cognitive interactions
• Provide options for students to explore the space ( zoom in and zoom out ) and
enrich their virtual interactions within the environment
• Include a mini-map to get an overview of the place, for planning the navigation and
activating virtual, social, and cognitive interaction

25. Conclusion

26. Discussion:
1. Learners’ “immersion” occurred through the visual interaction in the virtual
environment and also via the verbal interaction with their real peer
2. Peer-to-peer interaction was supported by social, virtual and cognitive presence
3. The immersive learning with 3DVRG was enhanced also through assessment-in-
context.
4. E-Assessment supported by authentication requires commitment and might
increase trust and quality assurance

27. Final remarks:
• This study highlights the value of virtual, social and cognitive interactions
mediated by VR artefacts, peer-interaction and assessment-in-context.
• It offers to educators- instructors, tutors, course-designers and learners, the
possibility of using Virtual Reality with pedagogical activities in pairs including
contextualized assessment with authentication.
• It also reveals some recommendations for technologists interested in creating
apps about real remote places with 360° pictures and videos and for educators
to create activities including assessment-in-context with e-authentication.

28. Alexandra Okada
The Open University, OU UK
ale.okada@open.ac.uk
QUESTIONS?

29. 29
CONSORTIUM PARTNERS

30. 30
THANK YOU FOR YOUR ATTENTION