Erik Champion, Curtin University PISA 9 SEPTEMBER 2014 heritage visualisation and serious game design • major concepts and issues in the field • learning from game design • problems that arise when entertainment, heritage, history and education collide

1. Heritage Visualisation:
Lessons From The Fun Side
Erik Champion, Curtin University
PISA 9 SEPTEMBER 2014

2. abstract
• heritage visualisation and serious game design
• major concepts and issues in the field
• learning from game design
• problems that arise when entertainment, heritage,
history and education collide

3. Why Virtual Heritage?
• Background in architecture, (art
history) and philosophy
• PhD with Lonely Planet in VEs for
travel and tourism
• Taught interaction design and
game design at UQ, UNSW,
Massey
• Project Manager at DIGHUMLAB
(Denmark)
• Professor of Cultural Visualisation,
Curtin University, Perth, Western
Australia

4. Cultural Visualisation?
Feature Science Art Culture
Reusable data Yes No Seldom
Standard tools Yes Seldom Thematic & communal
Clear research question Yes Seldom Depends
Null hypothesis Yes No Not often
Extensible Mostly Seldom Important
Falsifiable Yes Seldom Difficult
Stored Surely Unlikely Vital

5. Visualisation is creative
Problem: A farmer was not allowed to build a barn to shelter his horses.
Solution: He was allowed to build furniture.

6. Virtual Heritage is..
• “[It is]…the use of computer-based interactive
technologies to record, preserve, or recreate
artifacts, sites and actors of historic, artistic,
religious, and cultural significance and to deliver
the results openly to a global audience in such a
way as to provide formative educational
experiences through electronic manipulations of
time and space.”
• Stone & Ojika, 2000

7. What Is Left Out?
• Beliefs, rituals, other cultural behaviours and activities?
• Traces the level of certainty, and authenticity of
reproduction and reveals process? I.e. scholastic rigor?
• Sensitive to the needs of audience & shareholder?
Virtual heritage is the attempt to convey not just the
appearance but also the meaning and significance of
cultural artefacts and the associated social agency
that designed and used them, through the use of
interactive and immersive digital media.

8. Visualisation & Simulation
• “to form a mental image of something incapable of being
viewed or not at that moment visible… (Collins
Dictionary)...a tool or method for interpreting image data
fed into a computer and for generating images from
complex multidimensional data sets” (McCormick et al.
1987).
• “The purpose of a simulation is to study the characteristics
of a real-life or fictional system by manipulating variables
that cannot be controlled in a real system…While a model
aims to be true to the system it represents, a simulation can
use a model to explore states that would not be possible in
the original system.” (Beat Schwendimann, 2010).

9. For sites too fragile inaccessible damaged scattered or
dangerous

10. VH is not VR
• “Virtual reality is the use of computers and human-computer
interfaces to create the effect of a three-dimensional
world containing interactive objects
with a strong sense of three-dimensional
presence.”
• The importance of using HMDs or CAVEs, for VR
apparently requires “a head-tracked, usually a
stereoscopic, display that presents the virtual world
from the user’s current head position, including the
visual cues required..”

11. New media
the act of reshaping the user experience of exploring realms or worlds
through the innovative use of digital media. !

12. Wild Reindeer Exhibit Gagarin Interactive Iceland

13. INSTANTAR.ORG
Fraunhofer IDG CHESS project-Markerless tracking

14. KINECT-BOLOGNA
CINECA APA REUSABLE GAME

15. http://www.cineca.it/en

16. Definitions
NEW MEDIA: the act of reshaping the user experience
through the innovative use of digital media.
VIRTUAL HERITAGE: convey the appearance, meaning,
significance and social agency that designed and used
cultural artefacts and sites, (through the use of interactive
and immersive digital media).
NEW HERITAGE: re-examine the user experience that
digital media can provide for the understanding and
experiencing of tangible and intangible cultural heritage
!
Erik Champion, in Y. E. Kalay, T. Kvan, & J. Affleck, New Heritage: new
media and cultural heritage. New York: Routledge, 2008.

17. Edu-retainment
• Game based learning, serious
games, playful learning,
game-based learning…
• “Anyone who thinks there is a
difference between education
and entertainment doesn’t
know the first thing about
either.”
—Marshall McLuhan,
Communications Theorist
Sims: Virtual Montecello

18. Four Hypotheses
• Social learning is inter-active but Culture is also
materially embedded or embodied.
• To teach and disseminate immersive Digital History and
Virtual Heritage, interaction and the learning that results
from that interaction is crucial (see Mosaker, 2001).
• To improve interaction, examine games and why they are
so successful; academic literature suggests games are
best examples of interactive digital engagement
(references in Champion, 2008 et al.).
• Game-based interaction has to be modified for DH/VH.

19. 4 Issues With Culture
1. Definition (relation to place and inhabitation)?
2. How is culture transmitted?
3. Transmit local situated cultural knowledge to
“others”?
4. VH: how to transmit via digital & augmented
media?

20. Central Point
• Games are great learning environments
• Except for Cultural Significance, history and heritage
• Conclusion: problems and solutions
• Technology=barrier but not issue: learning is the problem.
• Which historical principles are used, learnt and applied?
• Inhabitants’ points of view (heritage) is missing
• Scholarly cycle incomplete, community cycle inextensible

21. Five Features of Place
1. A place can have a distinct theme, atmosphere, and contextually
related artefacts.*
2. Some places have the capacity to overawe.
3. Place has the power to evoke memories and associations.
4. Place has the capability to act as either stage or framework in
which communal and individual activity can ‘take place’.
5. Place has the ability to transmit the cultural intentions of individual
participants and social ‘bodies’.
* Place is a process not a product, and can consist of multiple interpretations, conflicts, and a unique
combination of borrowed histories. (Doreen Massey).

22. Why 3D?
• To evoke +communicate historical situations or heritage values find
deeper understandings not simply memorize facts (Bloom, 1956).
• Place is a cultural setting, it gives cultural interaction a time and a
location, Crang (1998, p.103), “Spaces become places as they
become ‘time-thickened’”
• Places do not just organise space, they orient, identify, and animate
the bodies, minds, and feelings of both inhabitants and visitors.
• Cultural presence: a feeling in a virtual environment that people
with a different cultural perspective occupy or have occupied that
virtual environment as a ‘place’.

23. Placeness Is More Than 3D

24. secondlife parody
youtube (French)

27. Croquet
http://www.moddb.com/mods/sourceforts

29. Drawing interfaces
http://www.drawastickman.com/episode1

30. Crayon Physics
http://steamcommunity.com/app/26900
https://www.youtube.com/watch?v=L9YR_5SpLCg

34. Blender 3D-Jim Pachal

35. Why Recreate
Place-ness?
• Fix locations in the memory (hippocampus).
• Reveal design based on scale and senses.
• Reveal limitations or principles of historical 2D
images.
• Provide a heightened sense of difficulty,
occasion, ritual, social proxemics (social
hierarchies).
• Afford a sense of place: peripherality, centre
directionality.
• Cultural landscapes affected by topography and
climate and proximity to resources.
• Parts of language affected by geolocation of
cultures.
iSphere copyright Paul Bourke

36. Monkey Brain, Human Brain
Caption: The flow of object information in a monkey brain (left) and a human brain.
Credit: Sabine Kastner, Princeton University
!

37. Culture is a feedback loop
• A visitor perceives space as place, and inhabits (modifies
a place), place 'perpetuates culture’, and thus influences
the inhabitants in turn.
• We might say that social behaviour is behaviour between
two or more people.
• Cultural behaviour is a subset of social behaviour, where
behaviour is governed by or understood in terms of a
cultural setting.
• As culture almost inevitably involves transactions, there
must be objects of shared transactional value.

38. Serious Games in CH
• Gaius Day, Augmented Reality on site
• Roma Nova Petridis 2012
• Canada: A People’s History Game OR building Detroit
• On site historical (Schrier 2006)reliving the revolution
• Eduventure I: (Ferdinand 2005) Rhine valley history, played in real Marksburg castle, on
tablet, webcam ARToolkitPlus
• Virtuso Arts history (Wagner 2007), sort a collection of artworks or monuments
• ThIATRO art history (Froschauer et al, VSMM 2012)
• Escaladieu (IRIT 2010) Abbey in Pyranees, 3D AR
• Studierstube ES game handheld AR platform
• Strategy eg Battle of Waterloo (BBC)

39. Video Games Pros & Cons
Factors Weaknesses Strengths
Interaction
Agency destroys historic causality. Simplistic
interaction, may be difficult for older audiences.
Helps teach interaction design.
Engagement Educational games: worst of both worlds? Well-known & popular.
Learning How to promote heritage & knowledge transfer.
Learn by trial and error. Leveling
allow for skills learnt
Technical issues Often contains many bugs. Often platform specific.
Speed, lighting, avatar design,
peripheries, networking
Support
Support by the actual company can be slow, and they
may avoid listing intended future features.
Community support (internet
forums).
Game
development
Non proprietary formats, changing game engine code
requires high coding skills
Education discounts available, some
games are easily “modded”.
Access/ cost
Expensive software development kits and commercial
licenses. Expensive.
Take them home, personalize modify
and share them.
Institutional value Not taken seriously. Employability for students.

40. PALADIN-QUMULUS
INTERACTIVE WEATHER

41. cryengine4

42. unreal engine-UDK

43. Kinect 1/2: voice + skeleton
Skyrim has motion tracking and voice commands

44. Type to enter text
Type to enter text

45. LBP1+2
• http://www.mediamolecule.com/blog/article/
kareems_talk_from_learning_without_frontiers_2011/

46. Games For History
1. Play and and answer questions
2. Play and classroom discuss and debate authenticity
3. Role-play with games, puppets, or narrators
4. Mod cities, empires events based on theories
5. Film events etc. using Machinima tools
6. Combine images or panoramas with other media
7. Design past artefacts, events, rituals or customs
8. Create VEs using games and game mods or using VR

47. Playing History
Plague – Slave trade - Vikings
Challenge: ..the belief that it is exciting
to learn about history.
Integrates learning and playing in a way
that engages pupils and gives them a
concrete feel for the historical time and
setting
Solution:
Platform: Mac/PC, single player, browser
Technology: 3D Unity game engine
Playtime: Per game 60 minutes
Target group: 9-14 years old

48. 2. Discuss and debate
• Watch the movie,
‘Gladiator’ ..Identify an item of
material culture (building,
object, ‘thing’) that is
important to the plot and
structure of the movie, and..
• http://proteus.brown.edu/
romanarchaeology08/4986
NB http://www.playthepast.org/

49. 3. Role-play
http://publicVR.org
OR video at http://vimeo.com/25901467

50. Assassins Creed
flickr Niranjan cc shanewarne_60000

52. 4. RTS-Mod
cities empires
Kurt Squire:
“We are interested in: the
processes by which players
develop an interest in history,
what historical understandings
develop, and if participation
has consequences for
activities such as school.”

53. 5. Film Events (Machinima)
http://www.sourcefilmmaker.com/
http://moviesandbox.com
http://www.thesims.com/de-de

54. Filmic 3D narratives
APA reusable game: CINECA

55. 6. Combine images, panos
http://www.petermorse.com.au/vrar/vr/
Iphone: http://www.youtube.com/watch?v=s9sBtuCuju0
Technical description http://paulbourke.net/dome/UnityiDome/
Other pano examples http://paulbourke.net/transient/Beacon/beacontour.html

56. 7. Design past artifacts,
events, rituals or customs
Unreal Tournament / Xibalba-Palenque

58. 8a. CAVE via game engine
http://cryve.id.tue.nl

59. 8b. iVEs from VR systems
http://www.ntnu.no/ub/omubit/bibliotekene/gunnerus-1/mubil

61. 8c.other mods

62. 2001-2004 PhD thesis
• Place versus Cyberspace: What creates a sensation of place (as a
cultural site) in a virtual environment in contradistinction to a sensation
of a virtual environment as a collection of objects and spaces?
• Cultural Presence versus Social Presence and Presence: Which
factors help immerse people spatially and thematically into a cultural
learning experience?
• Realism versus Interpretation: Does an attempt to perfect fidelity to
sources and to realism improve or hinder the cultural learning
experience?
• Education versus Entertainment: Does an attempt to make the
experience engaging improve or hinder the cultural learning
experience?

65. Surround projection 2005
•

66. 2005Unreal-music_Xavier Quijas
Yxayotl-www.yxayotl.com.mp4

67. Film Students
Into VEs
•Recreation
of
Maltese
temples,
the
students
created
ghost
narrators
by
videoing
themselves
recreating
imagined
rituals
and
inserting
these
translucent
videos
into
the
game
level.
This
was
done
in
less
than
12
weeks
part-­‐time
by
3
students
(undergraduate)
in
2005.
CAVI
(lower
left)
at
Aarhus
can
do
this
and
project
videos
onto
3D
statues
and
monuments.

69. Warping

71. 3D and CMS
Unity inside Moodle

72. Lazy Susan Panorama
Hybrid 2D - 3D Navigation

73. Our$work$in$sensory$input/output$

74. meditation game
Andrew Dekker 2005

75. biofeedback zombies
Andrew Dekker

77. Lessons badly learnt
• Very difficult to recreate original action scenes
and moments of discovery as game devices.
• Chinese players, familiar with a distorted
version of the original, not aware their cultural
knowledge was not accurate, did not
appreciate being told this.
• Recreating linear narrative via game design is
torturous.
• OR: simulate the procedural knowledge of
rituals and symbol-making via thematically-akin
interaction..
• ref: Game Mods: Design Theory and Criticism

80. Games aren’t Challenging?
• A rule-based formal system with a variable and quantifiable
outcome, where different outcomes are assigned different values,
the player exerts effort in order to influence the outcome, the
player feels attached to the outcome, and the consequences of
the activity are optional and negotiable. (Juul 2003, para 15).
• A system in which players engage in an artificial conflict,
defined by rules, that results in a quantifiable outcome. (Salen
and Zimmerman, 2004).
• A challenge that offers up the possibility of temporary or
permanent tactical resolution without harmful outcomes to the real
world situation of the participant (Champion, 2006).

81. Prescriptive or Procedural
learning?
Gamer: Reach objectives as quickly and vividly
as possible.
Activity
Tourist: Enjoy highlights safely and conveniently.
Viz: Weekend in
Capri
Traveller: complete tasks via local affordances. Activity: Myst
Archaeologist: Discover past via examining
material remains, geographical changes,
epigraphy etc.
Viz of process:
ArcDig, detective
series?
Anthropologist: Understand the beliefs roles and
relationships of inhabitants in context.
geographical changes, epigraphy etc.
Hermeneutic: Myst,
Sims? Oblivion?

82. Problem: Interaction /History
• Ritual knowledge: Match artefacts with events to progress through time
• Memetic Cause &effect (Guess results or memes to progress history)
• Extrapolate from clues in NPC dialogue
• Role-play minor characters, “History” not affected
• Counterfactual histories (create many possible worlds)
• Augment virtual world with historical or current media
• Sentiment analysis (observe the emotional impact of events on NPCs)
• Separate lies from truth to progress
• Mimic NPCs (as a kind of reverse Cultural Turing Test)

83. Problem: Avatars
• Realistic depiction
• Social behaviour
• Interface issues
http://www.interactivestory.net/
Eric Fassbender: Macquarie Lighthouse

84. Problem: Inhabitants’ PoV
• Can users learn via interaction the meanings and values
of others, do we need to interact as the original
inhabitants did?
• How can we find out how they interacted?
• Can the limited and constraining nature of current
technology help interaction become more meaningful,
educational and enjoyable (Handron & Jacobson, 2010)?
• How do we even know when meaningful learning is
reached?

85. Problem: Rituals
• Attention/focus
• Social judgement
• Territoriality
• Social Proxemics
• Being “in the flow”
• Physical delineation (profane vs sacred)
• Event-based or regular
http://www.youtube.com/watch?v=MqMXIRwQniA
Image: http://www.virtualtripping.com/google-earths-rome-reborn/ 2008

86. Problem: Sensory immersion

87. Affective Process

88. Kinect 1/2: voice + skeleton
Skyrim has motion tracking and voice commands

89. Problem: Integrate Text+Model
http://gap.alexandriaarchive.org/gapvis/index.html#index

90. Problem: Violence
• No realistic humans
• No social judgement
• No time to think
• Gun based genres are
commonplace
• Weaponry skill can be easily
levelled up
• Typical single player
• Demographics

91. Alternatives To Violence
• Reflexivity: A reflective space,
where players relax & consider the
consequences of their actions
• Performativity: Players asked to
perform or orate and present their
experience of the VE in class.
• RPG Virtue Ethics: Characters
change in relation to development
of virtue ethics.
• Consequentialism: Consequences
of player actions affect their future
gameplay. through the game.
• Creative Uses For Weapons.
• NPC distaste and disparagement:
they discourage violence.
• Biofeedback: Performance based
on calmness.
• Expressive and embodied modes
of interaction.
• Non-violent competition.
• P l a y e r s b e c o m e m o r a l l y
accountable for their actions.
• Ritual or mythical use of weapons.

92. Evaluating VES - People
• Task performance (quantitative or
qualitative)
• Likert or statistical evaluation
• Extrapolated understanding
• Personal ‘sense’ of cultural
presence
• What do they choose next (exit
strategies)
• ‘Teach the teacher’ et al methods
• Excitement recorded from
biofeedback

93. http://www.onzeonze.com.br/blog360/toursaofrancisco/index.html

94. UNESCO
1. Create list and host online available 3D heritage
models, concentrate on Australia and the Pacific
region.
2. Collate and archive VH resources via HuNI &
AARNET.
3. Show community groups how to develop and modify
their own online virtual heritage models and sites.
4. Provide training material on 3D capture and design
that can be expanded by others. Test via workshops
at Curtin & partners.
5. Recommend long-term archive guidelines, ways of
linking 3D models to scholarly publications &
resources.
6. Advice & modify suitable FOSS-Free and Open
software and creative copyright for 3D heritage.

95. The HIVE@CURTIN

96. Tiled Display

97. Jeffrey Jacobson
talking at Curtin

99. Jeffrey Jacobson
Curved Screen, HIVE, Curtin

105. The cost of Stereo VR

106. Linked Open Data And Motion
Capture

107. Conclusion
• Games as Virtual Environments may connect more people,
more thematically without competing with book learning.
• Background research needed for public vs. scholar needs.
• Game conventions ‘work’ but meaningful learning elusive.
• We lack interactive and immersive digital history projects
that are meaningful and engaging learning experiences.
• Mixed reality in history and heritage has many advantages
but few working exemplars.