This document describes an interactive augmented reality installation created by students at Namibia University of Science and Technology to showcase Namibian cultural diversity. The installation uses a camera and projector to track physical objects marked with distinct cultural patterns, and plays corresponding visuals and sounds when each object is detected. Users can interact by moving the objects, with the goal of enhancing the cultural experience through augmented reality without constraints of screens or input devices. The installation was developed using techniques including computer vision, audio playback, and interactive displays to blend real and virtual environments.

1. An Audio-Visual Multi-User Interactive Object Tracking
Installation for Presenting Namibian Cultural Diversity
Richard N. Plaatjes
Namibia University of Science and
Technology
1501 Gladiola St. Windhoek, Namibia
+264 81 486 9292
richardplaatjes@gmail.com
Baptista João
Namibia University of Science and
Technology
1002 Virgo St. Windhoek, Namibia
+264 81 877 5446
baptista.jaj@gmail.com
Steve Basimike
Namibia University of Science and
Technology
71 Potgeiter St. Windhoek, Namibia
+264 81 237 9999
sbasimike@gmail.com
Trevor Kabajani
Namibia University of Science and
Technology
174 Socrates St. Windhoek, Namibia
+264 81 693 0261
trevkabajani@gmail.com
Malan G. de Wet
Namibia University of Science and
Technology
43 Anna St. Windhoek, Namibia
+264 81 281 0755
mgdewet@gmail.com
Adilson Clemente
Namibia University of Science and
Technology
Windhoek, Namibia
+264 81 843 4702
adilsonclemente15@gmail.com
ABSTRACT
The paper describes an interactive multi-user application which
has different trackable objects that produce a variety of cultural
sounds and visualisations that represent Namibian cultural
diversity. The setup enables the users to interact with the
application by moving the objects on the canvas; the application
detects and recognises objects by means of a camera with the aid
of the defined detection and recognition algorithm within the
application. Several different patterns with a Namibian cultural
theme have been designed and implanted on the objects to be
tracked. These distinctly featured objects represent different
corresponding cultural sounds and visuals to be generated.
Categories and Subject Descriptors
• Human-centered computing ~ Mixed / augmented reality
• Human-centered computing ~ Sound-based input / output
• Human-centered computing ~ Graphics input devices • Human-
centered computing ~ Auditory feedback
General Terms
Algorithms, Documentation, Design, Experimentation, Human
Factors, Languages.
Keywords
Artificial, Augmented Reality, Visualisation, Cultural Diversity.
1. INTRODUCTION
Namibia is a fascinating nation. Its history is engraved in its cities,
towns, foods, artefacts, architecture and diverse residing tribes. As
a matter of first importance Namibia is an African nation, a
southern African nation particularly and home to the Khoisan
people who lived a hunter-gatherer lifestyle in the desert districts
of southern Africa. Today numerous groups still carry on with the
same traditions and visitors can visit San cultural centers and
living historical centers.
Alongside the Khoisan there are numerous different cultures
including Herero, Rehoboth Basters, Damara, San (Bushmen),
Nama, Coloureds, Whites, Kavango, Caprivians Topnaars,
Tswanas, Himba and the Owambo, and every one of these ethnic
groups are proud of their culture, way of life and their country.
Technological advancements have made consumption, sharing,
and creation of media for the average person fairly simple. At the
heart of this paper is the concept of ”combining culture and
augmented reality” which is casually defined as an environment
resulting from the interplay between the real world and the virtual
world. By combining real and virtual environments, many
approaches have emerged since 1968 when the first head-mounted
display (HMD) was developed by Sutherland [9].
2. RELATED WORKS
There is a considerable increase in the number of implementations
that demonstrate the use of interactive systems and augmented
reality. The work by Bergig, Soreq, Hagbi, Pevzner, Levi, Blau,
Smelansky and El-Sana implemented a game based on the
different combination of Rubik’s cube being detected to change
events in the game [1]. Zhou, Cheok, Pan and Li implemented
Storytelling with the use of foldable cubes with markers being
recognised to trigger visuals and audio sensory effects about a
story [10]. The same concept was also used by Juan, Canu and
Gmenez [5].
According to Roush, the combination of real-world and virtual
reality creates a “second life, or something like it” and is
something that has been long predicted [7]. This includes work
such as the Economic Weather Map project which is an approach
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
Conference’10, Month 1–2, 2010, City, State, Country.
Copyright 2010 ACM 1-58113-000-0/00/0010 …$15.00.

2. that has mostly included real-world data which can be referred to
as “reality mining" [2].
There have also been several other efforts that bring real-life and
virtual reality together, such as the Human Pacman [3],
Augmented-Reality Quake [6], and DynaDOOM [8]. Although
such work remains exclusively within the realm of video games,
they have incorporated some audio and video streaming concepts
as well. More augmented reality projects are being used to involve
users in the collaborative planning of real urban areas to create
interactive systems, such as the expensive projects that have been
used for the military although these type of systems are heavily
reliant on geographic information systems (GIS) than the usual
sensor networks embedded in an environment.
3. INSTALLATION
3.1 Installation Setup
The installation is composed of several devices such as a camera
for tracking the objects, projector for screening the visual effects,
speakers for sound
effects and the
computer for
running the
implemented
algorithms (see
Figure 1).
Additionally
several patterns
(markers) have
been produced
and implanted on
the objects to be
tracked. These distinctly featured objects represent different
corresponding events to be generated (see Figure 2).
Based on the tracked object, a distinct signal should be made
available for the generation of visual and audio effects through the
defined algorithms.
3.2 Technical Implementation
The full functionality of the installation is underpinned by the
development of three modules; namely the tracking module which
is developed based on the C++ programming language for fast
computation. The ArUco library proved to be quite useful for the
detection of several distinct objects simultaneously (Figure 3).
RapidJson was used to write the tracked data to file in a
lightweight storage format known as JSON; the visualisation
module based on the combination of technologies and libraries
such as HTML5, CSS3, JavaScript, JQuery, JCanvas and AJAX
to generate the visual effects and ultimately trigger the response of
the audio module; the audio module is based on the Media
Control Interface
(MCI) from the
.NET framework
with sounds
produced with
Fruity Loop
studios
pertaining to
prerecorded
cultural sounds
each
multithreaded to
allow
overlapping. The
complete and
final solution was
furthermore
highly dependent on the requirements of the key devices and
equipment such as the projector for screening, camera for
tracking, speakers for sound output and the physical featured
objects to be tracked.
3.3 Cultural Elements of the Installation
Various cultural elements are integrated with the installation to
ensure cultural user experience when users interact with the
application. All the objects tracked by the tracking module are
covered with distinct fabrics/images (Figure 4) each representing
a different Namibian cultural diversity. Interesting facts are placed
below each object being tracked, to give the
end user more knowledge about that specific
culture when they turn it upside down. The
visualisation module displays distinct cultural
images which are linked directly to the
objects being tracked. All the images have a
direct relation with the facts and cultural
group and are also placed below each distinct
object. The audio module generates various
traditional music pieces and effects that are
directly related to the corresponding distinct
object. As the object is placed on the canvas,
the relevant traditional music piece/effect and
image for that specific culture group is played by the sound
system. Each audible traditional clip is produced with various
cultural music instruments and effects from the different cultural
groups making them each unique to the group’s cultural diversity.
3.4 User Interaction
Augmented reality is aimed at augmenting the physical world with
computer-generated digital information. This has enabled this
installation to enhance the cultural entertainment experience by
allowing users to physically move or place objects within a
designated area to explore different cultural based sounds and
visuals. A visible line of light is seen moving across the
designated area and the cultural sounds and visuals are produced
when this line moves over the placed object. The users are not
constrained to any computer screen or input device for interaction
but rather interact by physically moving the objects.
Figure 1. Physical Layout
Figure 2. Object
Being Tracked
Figure 3. Detected Objects and
Intersecting Line
Figure 4. Himba
Fabrics [11]

3. 4. BACKGROUND
The installation presented in this paper is a prototype developed
by a group of Computer Science students from the Namibia
University of Science and Technology (NUST). It allows objects
to be tracked close to real time using a camera (webcam) and a
projector to show visual and sound effects. By leveraging the
capabilities of the existing technologies, the students were tasked
to develop an application that provided a cultural experience to
Namibian users. The installation enabled users to actively interact
within an augmented reality setting by means of interactive
systems.
5. CONCLUSION
The presented setup provided in this paper entailed tracking of
marked objects and the generation of distinct cultural sounds and
visual effects, all based on the interaction of the users placing or
moving the objects around. This is one of the experiments within
the broader space of emerging technologies where the team had
time constraints and encountered several technical challenges.
This experiment was also aimed at promoting Namibian cultural
diversity.
6. ACKNOWLEDGMENT
Our thanks goes to Prof. H. Winschiers-Theophilus for aiding in
the conceptualisation of this endeavour as well as a special thanks
to Namibian University of Science and Technology and its staff
for the continuous support and encouragement throughout the
development and realisation of the experiment.
7. ADDITIONAL AUTHORS
Anton N. T. Lungameni (Namibia University of Science and
Technology, 2565 David Bezuidenhout St. Windhoek, Namibia,
+264818935632, ndelitunga.l@gmail.com), Auryl Greyeno
Rusberg (Namibia University of Science and Technology, 1770
Stonechat St. Windhoek, Namibia, +264813791908,
greyeno@gmail.com), Conrad Nampolo (Namibia University of
Science and Technology, 7400 Dam St. Windhoek, Namibia,
+264814569696, conrad.nampolo@gmail.com), Isak Amunkete
(Namibia University of Science and Technology, 1603 Tobol St.
Windhoek, Namibia, +264814287088, iamunkete94@gmail.com)
), Josua Jonas (Namibia University of Science and Technology,
+264811500206, josua.oletu@gmail.com), Rodney Santambwa
(Namibia University of Science and Technology, 20 Pullman St.
Windhoek, Namibia, +264814569696, ssrodoo@gmail.com)
8. REFERENCES
[1] Bergig, O., Soreq, E., Hagbi, N., Pevzner K., Levi, N., Blau
S., Smelansky, Y. and El-Sana J. Out of the Cube:
Augmented Rubik’s Cube. Retrieved March 20, 2016, from
Namibia University of Science and Technology:
https://www.researchgate.net/publication/220061083_Out_
of_the_Cube_Augmented_Rubik's_Cube.
[2] Boone, G. Reality Mining: Browsing Reality with Sensor
Networks. Sensors Magazine, 2004, 21(9).
[3] Cheok, A. D., et al. Human Pacman: A Mobile
Entertainment System with Ubiquitous Computing and
Tangible Interaction over a Wide Outdoor Area. In Fifth
Int'l Symposium on Human Computer Interaction with
Mobile Devices and Services (Mobile HCI), 2003, 209-223.
[4] Jensen, R., Tasoluk, C., Marshall, H. and Green, G.
Integrating Intelligent Structured Training with a Virtual
Dismounted Environment, 2007.
[5] Juan, C., Canu, R., and Gmenez, M. Augmented Reality
interactive Storytelling Systems Using Tangible Cubes for
Edutainment. Retrieved May 15, 2016, from Namibia
University of Science and Technology:
http://www.ieconference.org/ie2004/proceedings/019%20z
hou.pdf.
[6] Piekarski, W., Thomas, B. AR Quake: The outdoor
augmented reality gaming system, Communications of the
ACM, 2002, v.45 n.1.
[7] Roush, W. Second Earth, 2007
[8] Sukthankar, G. The DynaDOOM Visualization Agent: A
Handheld Interface for Live Action Gaming. in Workshop
on Ubiquitous Agents on Embedded, Wearable, and Mobile
Devices, 2002.
[9] Sutherland, I. A head-mounted three dimensional display,
Proceedings of the December 9-11, 1968, fall joint
computer conference, part I, December 09-11, 1968, San
Francisco, California
[10] Zhou, Z., Cheok, A. D., Pan, J., Li Y. Magic Story Cube: an
Interactive Tangible Interface for Storytelling. Retrieved
June 5, 2016 from Namibia:
https://static.aminer.org/pdf/PDF/000/005/007/magic_story
_cube_an_interactive_tangible_interface_for_storytelling.p
df
[11] Novak, J. 2008. Himba: Young Girl