The document discusses the potential of serious games and immersive technologies like virtual reality (VR) and augmented reality (AR) in enhancing the understanding of cultural heritage. It highlights the challenges in effectively using these tools, such as the lack of user engagement, preservation issues, and interaction design problems. The conclusion calls for better collaboration between the gaming and cultural heritage sectors to create more meaningful and reusable digital experiences.

1. Games as Serious Visualisation Tools For Digital
Humanities, Cultural Heritage and Immersive Literacy
Emeritus Professor Erik Champion, Curtin, UWA, ANU, Australia
University of Graz, Austria – 6.10.2021 @nzerik
Today I
am here

2. Vizà(VR AR MR)=XR for DH
• social and cultural issues
• emerging themes in serious games & immersive literacy
Heaven’s Vault, Inkle Studios: Sci-Fi culture
“Cyberspace is a giant step forward
towards museumization of the world:
where anything remotely different from
Western culture will exist only in digital
form.” (Ziauddin Sardar 1995)

3. 1. Virtual Heritage (& MR)
VH: the attempt to convey not just the appearance but also the meaning and
significance of cultural artifacts and the associated social agency that designed and
used them through the use of interactive and immersive digital media

4. https://alisonjamesart.com/2017/03/12/what-is-a-virtuality /
Milgram Takemura Utsumi & Kishino (1994)
Giovanni Vincenti
MR: “particular subclass of VR related technologies that involve the merging of real and virtual worlds.”

5. AR: “Layered” AR:“Embedded”
Do people care?

6. How to explain VR/XR
VR, AR, MR and XR are typically screen-based,
how can cultural heritage best use XR?
• Move to consumer-component based system.
• OS frameworks improve access.
• Smartphone=stereoscopic viewer, sensor, PC.
• The device won’t matter.
• Software is in the browser.
WebXR or OpenXR works in browser

7. VIRTUAL HERITAGE IS..
• Combination of virtual reality and cultural
heritage (but also the social agency and
cultural significance).
• Promises best of both, fails often.
• Why?
• Few significant interaction design studies
• Lack of preservation
• Forgets end-users, doesn’t provide reuse

8. SS Xantho Steam Engine, Mafkereseb Bekele

9. Mafkereseb Bekele

10. Duyfken, Mafkereseb Bekele

11. Bekele, M. K. (2019). Walkable Mixed Reality Map as interaction interface for Virtual Heritage.
Digital Applications in Archaeology and Cultural Heritage, e00127.
Bekele, M. K., & Champion, E. (2019). A Comparison of Immersive Realities and Interaction
Methods: Cultural Learning in Virtual Heritage. Frontiers in Robotics and AI, 6, 91.
Walkable Maps & Movable 3D
SS Xantho Steam Engine

13. 2. Presence
cultural presence: a visitor’s overall subjective impression when visiting a virtual
environment that people with a different cultural perspective occupy or have occupied as a
place. Such a definition suggests that cultural presence is not just a feeling of being there
but also a sense of being in a foreign time or not-so-well understood place.

14. Even better than the real thing..U2
https://youtu.be/Vo4hCg91Xhk

15. CONCEPTUAL BLINDNESS?
Why virtual reality cannot match the real thing (?)
“… the emotions you feel when you have a virtual experience are not as
valuable. When you actually see Niagara Falls, especially if you get up
close, you feel awe and even fear in the face of an overpowering force
of nature… Computer simulations, however good, contain only what
photography, laser technology and pre-existing expertise put into
them… Real experiences connect us to the deeds of past people and
place us in contexts where history was made… VR will never be a
substitute for encounters with the real thing.”
- Janna Thompson Professor of Philosophy, La Trobe University.
https://theconversation.com/why-virtual-reality-cannot-match-the-real-thing-92035

16. Virtual Places Highly Static..MR?
• Single viewer, not to dwell or move between, few meaningful
between spaces
• Game achievements may change a place, but don’t redefine it.
• VR presence in VR is the sense of being there? Then in a virtual
museum what is being and what is there?
• More user-designed infill, creative tools, a way of recording, sharing,
and commemorating both their designs and their user experiences .
• Place revealed through people etc in terms of what they do, how they
are organized, how they affect others, not just external appearance.
• Should afford, include, and preserve collective learning.
16

17. Evaluation Content (intro, 3 Archaeol., 3
gamic)
Objective: Compare to
Task Performance 6 objects to find Understanding
Cultural
Understanding
6 questions x 3 virtual
environments (learn by
observing, conversing with
bots, activity)
Which VE most challenging to
explore, find or change?
Preference,
demographics, task
performance
Presence Survey
(rank 1 to 7)
Most: interesting; interactive;
related to ‘Mayan’; inhabited;
in the presence of Mayan
culture
Demographics, to task
performance; find
personal preference in
answers, rank 7 VEs
Environmental
recall
Shadows; real people; height
of locals vs tourists; number
of people
Demographics and to
task performance and
to understanding
Subjective
Experience (of time
passing)
In which VE did time pass
quickest? How fast did they
update the screen?
Subjective preference
and to demographics
PhD: U. of Melbourne, Lonely Planet 2001-4

18. Aspects of Cultural Significance Culturally Significant Place Examples in Games
Dynamic & often contested:
cultural significance is variable;
places have range of values over
time for different individuals or
groups - places reflect diversity.
Embodies & represents a range of
encounters and differences,
cultural values, and priorities.
Reflects the richness and diversity
of the community or communities
simulated.
Games host trigger points NOT
symbolically defined places with
amorphous boundaries. Culturally
rich bounded worlds in detailed
games with convincing AI?
Evocative: connects a community
to landscape & past experiences
(as historical records).
Connects modes of inhabitation,
identity, & place via situated past
experiences.
Few ways to store, preserve, &
access memory for players or Non-
Playing Characters (NPCs).
Shared fragility: places are fragile
& must be conserved for present &
future generations in line with the
principle of inter-generational
equity.
Conveys fragility but also care & the
means to care for the cultural
prestige and shared meaning of the
digitally simulated environment.
Few examples of cultural fragility in
games as focus typically on avatar
health.
http://neveralonegame.com/
https://www.polygon.com/features/2020/2/25/21150973/indigenous-representation-in-video-games

19. 3. Interaction Design Issues
Preservation, formats, features, user skills, audience

20. 1950s-62 Sensorama
2018 Selfie+HMD+Pano-movie
1965-8 Sword of Damocles
1952 Cinerama (1939 Vitarama) 1992 CAVE
Why are these impressive, under-used, and/or disappear?

21. The Vanishing Virtual
Addison, Thwaites, Stone
Beyond Time and Space … GONE!
http://www.geek.com/news/expore-the-virtual-forbidden-city-
courtesy-of-ibm-593731/
https://twitter.com/plevy/status/433058523836985344/photo/1
“Museums of Tomorrow”
http://www.beyondspaceandtime.org/

23. 140+3D FORMATS!!

24. Publications don’t
link to 3D assets

26. Susannah Emery-Sims Machinima

27. 3D content is only
PART of problem

28. Biofeedback controls characters, music, graphics, gameplay (A. Dekker, UQ,
2007)
• Phillips, P., Hartup, M., and Champion, E. (2009). “A survey of 10 free massive multiplayer online games that may help augment social
interaction and positive mental health.” The ANZ Association of Psychiatry, Psychology And Law (Inc.) Conference, Fremantle,
BIOFEEDBACK CINEMATICS AND XRAY VISION

29. Shown at VSMM2012 conference, Milan, Italy 2012.
Chinese Taoism Touch Screen by Li (Neil) Wang and Erik Champion.

31. Active Classrooms
Non Playing Characters informed by gesture posture etc.
NPCs

32. Digital Literacy>Immersive literacy
• American Library Association: “’Digital literacy is the ability to use information
and communication technologies to find, understand, evaluate, create, and
communicate digital information, an ability that requires both cognitive and
technical skills’ (p2, 2013).”
• VERSUS Association of College & Research Libraries (ACRL): “’Information
literacy is the set of integrated abilities encompassing the reflective discovery of
information, the understanding of how information is produced and valued, and
the use of information in creating new knowledge and participating ethically in
communities of learning’.”
• Becker: DL encourages technical AND cognitive skills in a digitally literate person
• discernment and judgement;
• understanding (of relationships between learning, privacy and stewardship);
• ability to socially connect;
• and civic participation.
• ME; DL implies a more interactive and participative skill but does not clearly
show importance of non-textual literacy skills in design + participation.

33. ALIA
• Include suitable content in an appropriate, accessible space.
• Feature engaging personalisable content that can be examined in
stages or detail.
• Affords opportunities for reflection, and, [adding media literacy]
affords guides and learners the ability to construct meaning, think
creatively and solve problems.
• Provides multiple learning modes, media and material and, [adding
digital dexterity]: critical engagement with media and lifelong
learning.
• Allows collaboration.

34. AR Tools
Criteria ARCore ARKit A-Frame Unity Unreal
Appropriate to
space, context-
appropriate, and
yet also allows
life-long learning.
Android-
centric, free
Apple-centric,
Yes
Yes, web-
based
Requires
powerful
computer
Requires
powerful
computer
Allows engaging
personalisable
content
Yes Yes Yes Difficult Difficult
Allows
collaboration.
Yes Yes Yes Yes Yes
https://immersive-literacy-digital-liter.gitbook.io/

35. Recogito
https://recogito.pelagios.org/
Tutorials and asset examples:
Unity, MeshLab, 3D modelling (Blender or
Photogrammetry), and Recogito.
Recogito + a national gazetteer dataset to extract
latitude and longitude data for historical place names
http://www.tlcmap.org/guides/recogito/
Aim: interactive 2D map in Unity that could be used as
an interface to change the other media elements.

36. Skills needed
• Basic understanding of Unity required.
• 3D model behind viewer. 3 walls in front:
a picture-map+clickable locations, a video
(clip) wall, + image wall.

37. VR/XR content, Rebecca Kerr

38. 4. GLAM versus AR VR MR
Galleries libraries archives museums
Expensive and need to train staff
Software/hardware obsolescence, visitors don’t download phone apps, internet
Human Guides are superior

39. http://www2017.com.au/ webGLAM Easter 2017, Google Cultural Lab?
Digital Heritage Interfaces 2017 speakers from Oceania/SE Asia

40. Agathe Limouzy Toulouse student intern project (“collaborative Learning” one character shared by 2

41. Beata Dawson, PhD project, Pauline Joseph co-supervisor

42. Rusaila Bazlamit,
PhD project, (& Andrew Hutchison) vsmm2017

43. Workshops
Playtest Playtest Playtest

44. http://caaconference.org/program/sessions/#title20
https://video.twimg.com/ext_tw_video/842014243193016320/pu/vid/1280x720/LpXIOXvAM3o7V5iv.mp4
CAA Atlanta 2017

45. Digital Cities, Digital Humanities Summer School: Game Prototypes, Politecnico, Italy 2018; 2019.

46. • Student AR work, Museum of Cinema, Turin Italy

47. Dhdownunder 2019 Sydney
• If people enjoy playing the prototype they’ll enjoy the game

48. Games: goals, challenges, risks, strategies, rewards
goals
challenges strategies
rewards
risks

49. Design Diagram / history-heritage

50. Creating a Simple Game
1. What is the goal? Why try to achieve it?
2. Why is it an engaging challenge?
3. Does it involve competition/mastery,
chance, imitation, controlling vertigo/rush
of movement/flight?
4. What is the feedback system, affordances +
constraints, rewards and punishments?
5. How level up/advance via mechanics?
6. How does it offer different strategies,
options?
7. What is learnt during or after the
experience?

51. 5 Further afield
Machinima, Escape Rooms, Workshops
Socrates and Kassandra discuss game mods-Ubisoft
https://experience.ubisoft.com/event/en-
gb/birmingham/news-detail.aspx?news_id=353950

52. Facebook Announces $50M Research & Partnership
Fund to “build the metaverse responsibly”
• Economic opportunity: how we can give
people more choice, encourage
competition and maintain a thriving
digital economy
• Privacy: how we can minimize the
amount of data that’s used, build
technology to enable privacy-protective
data uses and give people transparency
and control over their data
• Safety and integrity: how we can keep
people safe online and give them tools to
take action or get help if they see
something they’re not comfortable with
• Equity and inclusion: how we can make
sure these technologies are designed
inclusively and in a way that’s accessible

53. With Dr Juan Hiriart + Rapa Nui children
https://padlet.com
https://milanote.com/

54. Conclusion
• Discuss, ideate, and prototyping immersive AR-MR
• Better pathways for open-source software
• New and reusable Cultural Heritage
• Cultural tourism and DH education
• Closer ties with the game industry
• Academic incentives for scholarly and shared paradata
• A new cultural heritage based on digital technology
• Reusable, scalable, reconfigurable, providence-rich, linkable (L.o.D),
• Greater emphasis on sharable, contestable, multimodal experiences
• Understanding of culturally significant presence