November 2007 Confidential proposal for Mobile muse
COMMUNITY GENERATED FLOOR-SCREEN INTERFACES
Bozena Kaminska, Aleksandra Kaminska, Brent Carmichael, William New
kaminska@sfu.ca
1. BACKGROUND
For the majority of those living in developed urban centers, “digital” has become an aspect of day-to-
day life. In many respects the novelty has worn off: from work to play we are part of a digital techno-
culture, our cities are powered by sophisticated digital mechanisms and our media is provided through
a variety of digital sources and networks. However, the development and integration of digital
technology into our urban landscape and artistic practices is still in its infancy as scientists, engineers,
architects, designers and artists are all working to create meaningful and productive technologies.
While we are in effect living in digital environments, new technological innovations are constantly
remodelling our cities, our relationships and our cultural products. As “digital” becomes not only
ubiquitous but also increasingly innovative, there is great opportunity to explore the ways in which
technology is changing the architecture of our urban landscape and the production and consumption of
art.
2. PROJECT OVERVIEW
We propose to combine our work involving RF transceivers, self-powering body networks and sensors
assembled into self-organizing wireless networks with existing capabilities of the MUSE3 Platform (bi-
directional SMS/MMS + web application + public screen displays) to create a new fundamental platform
capability: instant geolocation and graphical mapping of individual participants within and around
a crowd. The aggregated map of all individual members of the crowd is continuously displayed on large
public screens or private mobile phone screens for anyone to see. On this map, an individual can instantly
visualize his/her own location relative to the assembled crowd and relative to visible features within the
environment. The community itself thus generates immediate knowledge of both the ever-moving location
of crowd assembly and disaggregation, as well as the immediate location of any individual within and
around that crowd. The overall effect is a ‘smart crowd’ with symmetrical intelligence: the individual sees
the crowd’s spatial distribution and his/her position relative to that crowd; the crowd can see the exact
location of any participating individual on the public map display, the individual can identify and
continuously locate another individual participant within the crowd (through pre-existing networks such as
Facebook, LinkedIn, etc.), and the crowd can see itself as a collective living self-aware fluid super-
organism comprised of a cellular network of identifiable traceable individuals.
3. 2010 SPECIFIC SCALABLE IMPLEMENTATIONS AND BEYOND
Several possible scenarios for 2010 exist depending on the amount of implementation infrastructure and
the venue. We propose 2 scenarios each of differing complexity, interactivity and engagement. Then
beyond 2010 we describe numerous related applications for the MUSE3 technology.
Scenario 1 – One World
This is a mapping project where the position/presence/condition/identity of participants is first sensed and
then transformed into information/data which can then be manipulated into an artistic/creative display and
ultimately projected on a public display 'map' (Figure describing Scenario 1 in Appendix A).
Proposed for the first pilot implementation, the first scenario will connect a large screen with a sensory
floor. The floor will be a representation of a world map on which people will be able to stand and affect
the display on the screen. Users will identify themselves through a place of origin on a world map and
thus become an individual in the crowd. It allows any individual to be participatory in contributing to the
1

November 2007 Confidential proposal for Mobile muse
map’s content displayed on a screen.
There will be two ways of interacting as an individual within the crowd:
1) The first is more passive, generic and basic: when a person stops in a certain country space on the
map – ideally the person’s place of origin – a small white light will be displayed on the screen in
that country (design or artistic representation to be determined). If more people stand in the same
country the light can become bigger and/or brighter. With a more sophisticated pressure-sensitive
floor users might have to jump on the spot to indicate where they are from (analogous to physically
sticking a pin in that location) and these points could be sustained for the day, the duration of the
event, festival, etc.. Also, such activities could be used to allow the crowd to “vote with their feet”
and perhaps show their support of a particular country by cheering during televised competitions
or perhaps by more actively competing in virtual competitions/games where the more people who
support a particular team (or again with a more sophisticated floor, the faster they run on the spot)
the better their team does. For these setups a person will not need to have any token/sensor or RF
tag since they will already be participants through their presence.
2) The second way of interactive participation is more individualistic. A person can obtain a special
token (similar to an RFID tag, no battery required) permitting the participant to represent him-,
herself on the screen in a more personalized way. The token can generate a light of a special colour
to differentiate from the basic white or possibly display an avatar or some other symbol
representing the individual. The tokens can collect and include data such as dates of stay in the
city, voting preferences, avatar design, movement or geographical position information,
physiological values (e.g. heart rate) or simple background information users would be
comfortable sharing (hometown, name, etc.), or other interactions controlled via a communication
interface with their token or cellular phone. The screen, either figuratively through a world map, or
through a more abstract light and colour show, will show the geographic distribution of the
population/crowd on the floor at any given time and allow participants to create a personal
response to the screen through their physical positioning on the map and their differentiation from
the crowd through their token generated information. The token can have a time-programmed
association, for example a person’s time of stay in a city (or venue) so that a light (i.e. an
individual) can be a part of the screen for a period of time even when they are no longer on the
sensory floor. Tokens can be also used as identifiers of a person in multiple locations and the
overall setup could be richly applied to individual venues and dynamic trajectories can be added
depending on the specific location as described above.
Within this scenario, more visual effect such as the trajectory from the entrance to the floor-screen space
of the person's country of origin may be displayed. The walking path would show up as an equivalent
trajectory on the screen map; for example from Vancouver to France. So, at the moment of obtaining a
token/ticket a person becomes a white light in Vancouver which then moves across the map to the country
of origin at which time it changes colour and takes on person’s heartbeat rhythm, for example. Much of
this would depend on the overall artistic design of the venue, but overall the idea of individual movement
together with community observation on the screen is very attractive.
Scenario 2 – Who am I? Where am I? Where am I going?
Our miniature devices (very inexpensive and easy to install) can be placed on the floor in order to guide
visitors to places of interest such as their seat in the performance hall or sporting venue (Figure describing
Scenario 2 in Appendix B). For example, this system could be activated by linking a visitor’s token to
their ticket information either at the time of purchase or at public self-serve linking kiosks (whereby the
user would flash their token over an RFID reader and then scan the barcode on her ticket in order to link
the information). Then as they enter the venue or any guided public area floor markers would either
2

November 2007 Confidential proposal for Mobile muse
automatically, or upon request (perhaps by squeezing the token), illuminate and lead them to their
destination.
Building on this technology, the screen in the performance/game hall could display a seating plan of the
venue and populate it with the colours or national flag or other personal graphical representations such as
avatars or favourite sports or hobbies of the individuals present in the crowd. In this way, unacquainted
but neighbouring parties might be prompted to interact with each other in far more meaningful ways than
currently take place. Such audience ice-breakers certainly promote the spirit of the 2010 Olympics as well
as the user generated content phenomenon. Not only will individuals be able to broadcast where they’re
at, whom they’re with and who they are, but also who they’ve met!
Furthermore, if each audience member is given the ability to communicate information such as text
messages, or changes and movements of their avatar many more interactive possibilities will exist
depending on design. We can envision real-time, real-life displays of personal interests and networks
commonly found in virtual arenas such as Second Life, Facebook and MySpace. What’s more, data
from these social networking environments could also be linked to each person’s token/avatar.
Information you consent to display online could be forwarded to the big screen in a summarized and
controlled fashion (e.g. your favourite hobby as mentioned above) and information regarding who among
your contacts is in attendance at the event could be communicated to you.
Overall we want to enable individuals to stand out in a crowd while appreciating the diversity of the other
individuals around them. Attendance at this event will be like no other.
Communication:
A number of spaces or locations could be mapped and the screen content shared through the wireless
infrastructure developed by the MUSE3 project. The data from each location could also be stored and used
in different applications by multiple organizations and consent for storage and transmission of information
could be administered by the individual users themselves. Scaling up from these simple venue-specific
models supports numerous business models that would support the overall infrastructure of such
environments.
The communication infrastructure developed by the MUSE3 project will be used and integrated for the
exchange of screen content between sites and personal portable devices. Wireless networks such as cell
phones and internet communication can be used to access/display any public space included in the system
of ‘smart’ interfaces. The internet, messaging, and displays are part of the proposed project
communication modality. Tokens/sensors provided to the public will enable personal connections between
the users and the global system.
Interest:
There are several levels of participation:
1) Passive access to the information – “read only” interactions
2) Access and feedback relative to the data – “read and comment” interactions
3) Interactive access, content generation and sharing which is the main MUSE mission “Community
Generated Media” – full “read/write” interactions
Impact, Scalability and Applications Beyond 2010:
There are multiple parallel interactive and participatory angles. The artistic, entertainment and social
aspects can be combined with crowd analysis, statistics, targeted advertisement, identification and
security.
3

November 2007 Confidential proposal for Mobile muse
The model proposed for locating, identifying, and measuring/monitoring participants is inherently
distributed across large public areas. Wireless methods are easily scalable across residential, work and
recreational spaces. Wireless transceivers (cell phone and otherwise) communicate to nearby hubs that
surround the geographical space (“cell”) in which the transceiver is located. Such hubs (and thereby cells
from which communications can be received) can be distributed within practical limits across the entire
community, regionally, and indeed nationally (c.f. commercial cellular telephone networks). The overall
extent of coverage is thereby a business, marketing and economic decision, not a technology decision.
Organizations interested in the information provided might be emergency personnel charged with
determining the position of participants/visitors, or possibly entertainers or venders interested in the
preferences and individual characteristics such as place of origin that comprise the crowd before them.
The country tourist map example can be replaced by other maps (e.g. store locations in a shopping centre
or district) or positional representations (e.g. course location and speed of marathon runners or cross-
country racers –many of whom already wear RFID tags). Managers/curators of museums might want to
observe and monitor over time which art pieces attract the largest crowds and what these crowds “look”
like and what they think of the exhibit. Shopping centers and department stores may want to gather
similar information on shoppers; tourism offices might want to track the distribution of their visitors.
Interactive paths to specific places and access and interaction with online and displayed content further
options for implementations.
Other examples include the following:
• Security initiatives (active and proactive interventions, crowd control, tracking, etc.)
• Social networking (career-based, entertainment, dating, etc.)
• Sport activity participation (spectators, virtual play, interactive games, etc.)
• Children’s parties
• Neighbourhood activities (BBQ’s, garage sales, etc.)
• Entertainment and performances (crowd interaction)
• Targeted advertisement
• Data collection
The data continuously available as to position, movement, paths taken, identity of participants, their
interests and characteristics combined or compared with historical data provide a broad unprecedented
palette for artistic expression. We propose to collaborate with an artistic team to transform data into an
engaging visual, which could be adjusted accordingly under varying conditions. Public art in which the
individual participant can observe and modify their own contribution is a new genre which promises to be
of intense interest from the individual cooperative groups and the collective crowd overall. The wisdom
and behaviour of crowds quickly becomes apparent at an unprecedented speed of recognition, whether
expressed artistically or statistically, visually or musically, in the participant’s choice of language and
form.
4. TECHNOLOGY
The “smart” floor sensitive to position and activity uses innovative inexpensive green technology
energized by ultra-low power scavenged directly from the human participant. The technology has proven
useful in the European automotive industry. The novel concept relies on the human body acting as a
conductive path to a handheld monitoring/identity device from the ambient electric fields within a
building that generate micro currents through that body. One can continuously detect a person’s position,
moving, dancing, wheeling, and so on. Installation is simple and inexpensive. The challenge is to design
4

November 2007 Confidential proposal for Mobile muse
an application with a graphically attractive screen display plus a communication scheme appropriate for
the particular venue.
Technology can be as simple as a distributed mat of transducers embedded throughout the floor/carpet to
detect when someone is standing/sitting or walking/wheeling on certain areas of the carpet. More complex
will be an arrangement where every participant is carrying some form of monitoring tag (within their
admission ticket, a pocket token, a wristband, etc). More involved still, is an arrangement where each
participant has a wireless transceiver, perhaps an ordinary mobile phone. This transceiver periodically, or
upon command, or with an event occurrence, transmits a short burst of self-identifying information by
SMS or other simple protocol.
5. COLLABORATIONS
The self-organizing wireless mesh network technology as well as the thin-film microelectronic
transceivers and sensors are working today in the CiBER laboratories of Simon Fraser University
[www.ciber.ca]. CiBER Team has recently presented a live invitational demonstration of advanced
wireless sensors during the world exposition of innovative technologies sponsored by WIRED NextFest in
Los Angeles.
Our collaboration with Toronto media group (multi-university program) includes PhD student Aleksandra
Kaminska and her professors. Aleksandra brings together the interests she pursued in her two Master
degree projects. This first, supervised by Dr. Radha Hegde, compared the contemporary art scenes in
Hong Kong and Beijing as they related to their respective cities, to each other, their country and the world.
The second, with Dr. Rodney Benson, was a study of Marc Augé’s non-places and the global citizen. She
will also draw on her experience as a member of the Visible Cities project and the LOT media arts
collective and urban research group, both under the wing of Dr. Janine Marchessault. LOT develops ways
of using artistic practice as urban methodology. Dr. Marchessault is also a founding member of the Future
Cinema Lab, the first dedicated facility in Canada devoted to the design of new forms of storytelling,
development of prototypes for urban research, and creation of innovative projects within networked and
hybrid media environments [www.futurecinema.ca].
Adigy Canada is a valued corporate contributor to our MUSE collaboration, led by Dr. William New,
a physician/engineer widely recognized for his work in continuous measurement of human activity and
physiology. Dr. New has founded several successful companies involving technology that
unobtrusively monitors medical conditions involving the heart, breathing, and brain activity. His most
recent work focuses on wireless connectivity that allows unrestrained observation of ambulatory
subjects. Adigy will provide the conduit to product development and commercialization of innovations
demonstrated in our MUSE participation.
Much of the project’s visual appeal will depend and rely on collaborating with a creative artistic team
to transform the data collected from the tokens and the floor into an engaging screen display.
6. BUDGET
Detailed budgets for both scenarios described above are listed in Appendix A. The project can take
different granularity and implementation levels. The simplest one is a proof of concept with one simple
screen and one space. This would require the engineering time, artist/designer involvement, overall
material supply, and project coordination. The integration with MUSE3 infrastructure should be
included. The approximate estimation for a one year project is in a range of $300,000 to $500,000
depending on specific choices.
5

November 2007 Confidential proposal for Mobile muse
Appendix A – Scenario 1
Name: Annie
Heart Rate: 74bpm
Current Location: 49.15N | 123.10W
Hometown: Melbourne
Connects to
the Internet
DB
No Token, position
sensed by floor map
Base Station
Base Station
Nearby Friends:Jorge
User Wearing Active Token
User Wearing Passive Token
6

November 2007 Confidential proposal for Mobile muse
Appendix B – Scenario 2
Coloured Dots Represent Tokens
Screen
TXT MSG:
Display Flag
Avatar
Token Reader/Locators Distributed
throughout the Venue Connect with Hubs
and Internet
Ticket-Token
Linking Kiosk
Hub Connects to Internet
Seating Area
On Cell Phone:
Friends Nearby:
Jorge and
Alice
7
Directions to Sec 11, Row J, Seats 14 & 15

November 2007 Confidential proposal for Mobile muse
Appendix C – Detailed Budget
Scenario 1 - World Map Interactive Display
1st Prototype design (for March 2009) would include a Sensitive Floor Display communicating the presence of
individuals by physical location and tracking of worn Tokens/Tags to the Interactive Screen Display Software
(we assume screens will already exist in the public space and are not funded here). Token IDs will be linked to
data manipulated and controlled by users via the internet. More sophisticated Tokens/Tags will record other
movement and physiological data and will wirelessly transmit to stationary base-stations if implementation of
this more involved design is funded.
Salary Salary $/Wk # of Weeks Cost
Project Manager $1,344 $69,888
52
H/W Design/Test (Tokens, Base Station, Floor) $1,160 $60,320
52
H/W Design/Test (Tokens, Base Station) $1,080 $56,160
52
S/W Design/Test (Firmware, Interface, Web Apps) $1,080 $56,160
52
Artistic/Industrial Designer (Interface, Web, Form Design) $1,080 $56,160
52
Salary Subtotal $298,688
Equipment, Materials & Labour Cost per Unit # of Units Cost
Passive Token $200 25 $5,000
Active Token $700 10 $7,000
Base Station (for active and passive tokens) $1,000 5 $5,000
Sensitive Floor $20,000 1 $20,000
Miscellaneous Dev. Materials (Paper, Toner, Food) $2,500 1 $2,500
S/W Development Environment and Tools $1,200 1 $1,200
Web Server and Hosting $2,500 1 $2,500
Equipment
Subtotal $43,200
Scenario 1 Total $341,888
Scenario 2 - Venue Interactive Display and Tracking
1st Prototype design (for March 2009) would include a self-serve Ticket-Token Linking Kiosks, Token
Readers/Locators (that double as floor lights) throughout a venue space and the to the Interactive Screen
Display Software (we assume screens will already exist in the public space and are not funded here). Token
IDs will be linked to data manipulated and controlled by users via the internet.
Salary Salary $/Wk # of Weeks Cost
Project Manager $1,344 $69,888
52
H/W Design/Test (Tokens, Readers, Hubs, Kiosk) $1,160 $60,320
52
H/W Design/Test (Tokens, Readers) $1,080 $56,160
52
S/W Design/Test (Firmware, Interface, Web Apps) $1,080 $56,160
52
Artistic/Industrial Designer (Interface, Web, Form Design) $1,080 52 $56,160
Salary Subtotal $298,688
Equipment, Materials & Labour Cost per Unit # of Units Cost
Passive Token $200 25 $5,000
Token Readers/Locators (floor lights) $500 50 $25,000
Token Readers/Locators Communication Hubs $1,000 10 $10,000
Ticket-Token Linking Kiosk $10,000 1 $10,000
Miscellaneous Dev. Materials (Paper, Toner, Food) $2,500 1 $2,500
S/W Development Environment and Tools $1,200 1 $1,200
Web Server and Hosting $2,500 1 $2,500
Equipment
Subtotal $56,200
Scenario 2 Total $354,888
8