presentation at the Blander conference in Amsterdam (25/10/15, http://www.blender.org/conference/ ) about interaction with 3D through natural interaction and tangible interfaces

1. Forbidden
not to grab
or touch
Daniel Pletinckx
Dries Nollet
Carlotta Capurro
Visual Dimension bvba
Belgium
V-MUST is funded by the European Commission under the Community's Seventh
Framework Programme, contract no. GA 270404.

2. Visual Dimension bvba
• Active in digital heritage and architecture
• Created in 2003 (Ename, Belgium)
• Active on European scale as innovation company
• Active in commercial & European projects

3. Visual Dimension bvba
• Digital Heritage projects
• 3D virtual reconstruction & digital restoration
• Interaction with 3D digital heritage (TimeGate)
• Interactive storytelling for digital heritage (TimeFrame, TimeScope)
• European projects
• EPOCH, Etruscanning
• V-MusT
• CARARE, 3D-ICONS
• Project development

4. Ir. Daniel Pletinckx
• Civil engineer ICT & Communication Technology
• Active in digital heritage since 1997
• Awards
• Flemish Monument Award 1998
• VGI ICT Innovation Award 2004
• Tartessos Award 2009 for virtual
archaeology (Sevilla)

5. 3D Digital and Virtual museums
• What does 3D offers to digital & virtual museums?
– Digitise existing buildings and objects
– Recreate and visualise the past (monuments, sites, landscapes)
– Digitally restore museum objects
– Explore virtual spaces
– See evolution in time until today
• Monuments & landscapes are essential part of
digital & virtual museums
• Provide context of museum objects
• Provide a visual reference framework for 2 people out of 3

6. Interaction with 3D Digital Heritage
• Creation of 3D digital heritage
– All 3D modeling in Blender (legacy data from ArchiCAD)
– 3D digitisation in AgiSoft (5 %)
– 3D sculpting (Blender & Z-Brush)
– Landscape reconstruction in Vue or Unity (Terrain Composer plugin)
• Natural interaction
– Kinect camera for Windows, Leap Motion sensor
– Interface to Unity3D
• Interactive objects
– 3D printing (print ready 3D models in Blender)
– Interface to integrated hardware in Unity3D
– Interactivity in Unity3D

7. Ename, Belgium
Archaeological site of Ename, Belgium

8. Ename, Belgium
Archaeological site and village of Ename, Belgium

9. Ename 1065
4D visualisation of evolution of the Ename abbey site (1065)

10. Ename 1085
4D visualisation of evolution of the Ename abbey site (1085)

11. Ename 1150
4D visualisation of evolution of the Ename abbey site (1150)

12. Ename 1500
4D visualisation of evolution of the Ename abbey site (1500)

13. Ename 1595
4D visualisation of evolution of the Ename abbey site (1595)

14. Ename 1665
4D visualisation of evolution of the Ename abbey site (1665)

15. Ename 1730
4D visualisation of evolution of the Ename abbey site (1730)

16. Ename 1020
Resistivity measurements at the archaeological site of Ename

17. Ename 1020
Resistivity results of the archaeological site of Ename
E
BEA
ARN
STRAAT
UC
ENAM
L
EP
EIN
St.- SALVAT
R
TRAA
O
S
T
SCHELDE
N
0
AKKER
BRAAK
AKKER
WEILAND
WEILAND
BRAAK
RIETVELD
BRAAK
10.000
12.299
12.058
9
12.16
12.2
12
10.850
9.420
.59
89
9815.
9.648
10.
9
13
9.14
8
365
9.
9.193
10.2
20
9.874
10.027
9.978
9.495
9.251
9.030
14
9.0
10.057
07
10.7
964
9.
7
9.92
9.869
9.06
8
9.755
9.696
9.629
56
9.
5
9.494
9428.
9367.
.2899
.1749
9065.
9.216
9.414
9.568
9.672
754
9.
9.87
7
9.818
9.850
9
83
.8
9
9.21
9.974
10.0
05
10.111
01.170
1
192
0.
7
10.18
10
4
.12
10.134
10.138
10.128
10.131
1013
6
.
9.907
9942.
11.792
1
94
1.7
90
11.8
1196
5
.
833
11.
11
6
.81
11.887
9
13.18
24.363
12.584
10.00
0
9.520
fenopa
l
pavilioe
n
kijkente
r
a
v
s
enopal
w
k
t
F
-
a
erfee
s
Dorpel
elektrici
teitcab
nei
pse
lunt
o
t
p
p-udna
ar
v
o
e
de
GIN
-t
k
este
vpsie
-ijkvn
r
e
vp-site-k
et
mepu
nt
et
P
H
Hm
P
© Archaeological Prospection
Services of Southampton
Ename, Belgium
0 100m
N

18. Ename 1020
Magnetometer results of the archaeological site of Ename
E
BEA
ARN
STRAAT
UC
ENAM
L
EP
EIN
St.- SALVAT
R
TRAA
O
S
T
SCHELDE
N
0
AKKER
BRAAK
AKKER
WEILAND
WEILAND
BRAAK
RIETVELD
BRAAK
10.000
12.299
12.058
9
12.16
12.2
12
10.850
9.420
.59
89
9815.
9.648
10.
9
13
9.14
8
365
9.
9.193
10.2
20
9.874
10.027
9.978
9.495
9.251
9.030
14
9.0
10.057
07
10.7
964
9.
7
9.92
9.869
9.06
8
9.755
9.696
9.629
56
9.
5
9.494
9428.
9367.
.2899
.1749
9065.
9.216
9.414
9.568
9.672
754
9.
9.87
7
9.818
9.850
9
83
.8
9
9.21
9.974
10.0
05
10.111
01.170
1
192
0.
7
10.18
10
4
.12
10.134
10.138
10.128
10.131
1013
6
.
9.907
9942.
11.792
1
94
1.7
90
11.8
1196
5
.
833
11.
11
6
.81
11.887
9
13.18
24.363
12.584
10.00
0
9.520
fenopa
l
pavilioe
n
kijkente
r
a
v
s
enopal
w
k
t
F
-
a
erfee
s
Dorpel
elektrici
teitcab
nei
pse
lunt
o
t
p
p-udna
ar
v
o
e
de
GIN
-t
k
este
vpsie
-ijkvn
r
e
vp-site-k
et
mepu
nt
et
P
H
Hm
P
© Archaeological Prospection
Services of Southampton
Ename, Belgium
0 100m
N

19. Ename 1020
Reconstruction of Ename in 1020, before the abbey phase

20. Ename 1020
Reconstruction of Ename in 1020, trade settlement

21. Ename 1020
Reconstruction of Ename in 1020, harbour and castrum

22. Ename 1020
Reconstruction of Ename in 1020, harbour and Saint Salvator church

23. Ename 1020
Reconstruction of Ename in 1020, market place and Saint Salvator church

24. Workflow of landscape simulation
Conceptual model of reconstructed buildings and terrain features

25. Workflow of landscape simulation
Top view map in Photoshop

26. Workflow of landscape simulation
Separation of terrain types in layers in Photoshop

27. Workflow of landscape simulation
Creation of height maps through painting in Photoshop

28. Workflow of landscape simulation
Scaled height maps in Photoshop (16 bit TIFF)

29. Workflow of landscape simulation
Terrain types and applied height map (Vue)

30. Workflow of landscape simulation
Terrain with procedural textures and height maps but without vegetation (Vue)

31. Workflow
Terrain with water and vegetation after population (Vue)

32. Workflow
Terrain with vegetation, buildings and (animated) objects (Collada)

33. 4D visualisation: 1020-1730
Procedural landscape simulation in Vue (1730)

34. Real-time 3D historical landscapes
• Documented historical landscapes (images)
• Vegetation maps (terrain types)
• Height maps
• Re-generation as real-time 3D vegetation
• Terrain Composer (Unity3D plugin)
• Highly optimised terrain and vegetation (LOD)
• From the same vegetation and height maps

35. Real-time 3D historical landscapes
Landscape simulation by TerrainComposer of Ename abbey and village

36. Real-time 3D visualisation: 1290
Optimised 3D landscape simulation (1290) in Unity3D

37. Real-time 3D visualisation: 1290
Optimised 3D landscape simulation (1290) in Unity3D

38. Real-time 3D visualisation: 1290
Optimised 3D landscape simulation (1290) in Unity3D

39. The abbey interior in 1288
Educational game – conceptual model – reconstructed interior

40. The abbey interior in 1288
Educational game – chapter room remains

41. The abbey interior in 1288
Ename archaeological site – chapter room

42. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – chapter room

43. The abbey interior in 1288
Ename abbey in 1288 – real time implementation – chapter room

44. The abbey interior in 1288
Ename archaeological site – refectory and lavatory

45. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – refectory

46. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – refectory

47. The abbey interior in 1288
Ename abbey in 1288 – real time implementation – refectory

48. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – scriptorium

49. The abbey interior in 1288
Ename abbey in 1288 – real time implementation – scriptorium

50. The abbey interior in 1288
Ename abbey in 1288 – archaeological site– scriptorium

51. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – scriptorium

52. The abbey interior in 1288
Ename abbey in 1288 – conceptual model – scriptorium

53. The abbey interior in 1288
Ename abbey in 1288 – real time implementation – library

54. The abbey interior in 1290
Ename abbey in 1288 – real time implementation – cloister

55. The abbey interior in 1290
Ename abbey in 1288 – real time implementation – cloister

56. The abbey interior in 1290
Ename abbey in 1288 – real time implementation – cloister

57. The abbey interior in 1290
Ename abbey in 1288 – real time implementation – guest quarter dining room

58. The abbey interior in 1288
Educational game – shape design – guest quarter table ware

59. The abbey interior in 1288
Educational game – museum obejcts – guest quarter table ware

60. The abbey interior in 1288
Educational game – iconography – guest quarter table ware

61. The abbey interior in 1288
Educational game – recontextualisation – scriptorium and library objects

62. The abbey interior in 1288
Educational game – recontextualisation – scriptorium and library objectsDEMO

63. Natural Interaction
• Exploration through POIs (touchscreen)
• Panorama based (TimeScope)
• ObjectVR based (TimeLine)
• Tangible Interfaces
• Virtex (museum objects)
• Virtex Light (monuments)
• Natural interaction – hands (Leap)
• Spatial Augmented Reality
• Natural interaction – body (Kinect)

64. Virtex
Interactive 3D print of Augustus Prima Porta statue

65. Virtex
Electronics inside the interactive 3D print

66. Virtex
Interactive 3D print of Augustus Prima Porta statue at APM

67. Virtex
Interactive 3D print of Augustus Prima Porta statue at APM

68. Virtex
Interactive 3D print of Augustus Prima Porta statue in Sarajevo

69. Virtex Light
Ara Pacis 3D model

70. Virtex Light
Ara Pacis interactive 3D print

71. Virtex Light
Ara Pacis interactive 3D print

72. Virtex Light
Ara Pacis interactive 3D print

73. Virtex Light
Interactive Ara Pacis at Allard Pierson museum

74. Virtex Light
Interactive Ara Pacis at Allard Pierson museum

75. NI Spatial Augmented Reality
Interactive Egyptian stele (Allard Pierson Museum)

76. NI Spatial Augmented Reality
Interactive oscillum (Allard Pierson Museum)

77. NI Spatial Augmented Reality
Interpretation of fragmentary wall paintings (Mercati Traianei)

78. NI Spatial Augmented Reality
Greco-Roman Museum - Submerged Antiquities in Bibliotheca Alexandrina

79. NI Spatial Augmented Reality
Caesarion in Bibliotheca Alexandrina

80. NI Spatial Augmented Reality
Caesarion – digital restoration – the parents

81. NI Spatial Augmented Reality
Caesarion – digital restoration

82. NI Spatial Augmented Reality
Caesarion – digital restoration – hypothesis for laurel wreath

83. NI Spatial Augmented Reality
Caesarion – digital restoration

84. NI Spatial Augmented Reality
Caesarion – natural interaction to show digital restoration

85. Natural interaction: Etruscanning
Museo Gregoriano Etrusco, Vatican Museums

86. Natural interaction: Etruscanning
Regolini-Galassi tomb, excavated in 1836, objects in Vatican Museums

87. Natural interaction: Etruscanning
Interactive virtual Regolini-Galassi tomb (CNR-ITABC & Visual Dimension)

88. Natural interaction: Etruscanning
Interactive virtual Regolini-Galassi tomb (CNR-ITABC & Visual Dimension)

89. Natural interaction: Etruscanning
Interactive virtual Regolini-Galassi tomb (CNR-ITABC & Visual Dimension)

90. Natural interaction: Etruscanning
Etruscanning setup at the National Museum of Antiquities, Leiden, Netherlands

91. Natural interaction: Etruscanning
Etruscanning setup at the Allard Pierson Museum, Amsterdam

92. Natural interaction: Etruscanning
Etruscanning setup at the Vatican Museums, Rome

93. Natural interaction: Vrouw Maria
Shipwreck Vrouw Maria in Maritime Museum of Finland (2012)

94. Natural interaction: Imago Bononiae
Imago Bononiae in Digital Heritage 2013 conference

95. Natural interaction: Keys2Rome
AdMotum setup in Keys2Rome exhibition in the Town Hall, Sarajevo

96. Educational game @ Ename
Ename Heritage Centre and archaeological site

97. Natural interaction: TimeGate
Ename crosier – Ename archaeological museums

98. Natural interaction: TimeGate
Natural interaction – gesture editor

99. Natural interaction: TimeGate
Gesture editor – selection of significant body parts

100. Natural interaction: TimeGate
Gesture editor – gesture recording

101. Natural interaction: TimeGate
Ename abbey in 1290: interactive quest – part 1

102. Natural interaction: TimeGate
Ename abbey in 1290: interactive quest – part 2

103. Natural interaction: TimeGate
Ename abbey in 1290: interactive quest – part 3

104. Natural interaction: TimeGate
Ename abbey in 1290: interactive quest – part 4

105. 3D game & walk-through
Ename Heritage Centre

106. Video
https://www.youtube.com/watch?v=rb41jKX9Cac

107. Many thanks!
heritage@visualdimension.be
http://heritage.visualdimension.be/