SlideShare a Scribd company logo
1 of 20
Download to read offline

Between Mind & Environment
(Digest Version)
Wei Xu
32
ABSTRACT
ABSTRACT
The development of technology has fundamentally
changed our daily lives. Without it, we would not have
FaceTime to connect with loved ones when we first wake
up, Google Maps to check the traffic over breakfast, or
Pokemon Go to play on the way to work. As the quantity of
information increases, we shift our behavior to adapt to this
new digital environment.
As Diane Ackerman writes in her book A Natural History of
The Senses, “There is no way in which to understand the
world without first detecting it through the radar-net of our
sensors.”1
Each sense collects a different kind of informa-
tion, and the collaboration of multiple senses gathers and
stores information about the external world. Our brain then
processes this information and gives it order. Our senses
connect our inside selves with the outside world, and are
lenses which shape our understanding of our environment.
With the help of technology, we now receive more data
than before, but the sensory input is limited by the interface
we use, and often is just visual and auditory. By using more
senses than just sight and sound, can we get benefits from
data technology to explore neglected information? Can this
kind of information enhance feelings? Will we gain a new
understanding of ourselves?
Through my projects I wish to create multi-sensory
experiences to help people notice ignored information
and question the lenses through which information is
54
gathered. These projects use algorithms that I have defined
to transform information without physical form, such as feel-
ings, the social atmosphere of a room, and the relationship
between individuals and a group. During the transformation
processes, I will observe how data is translated through
different senses, and how data enhanced by algorithms
influences people’s communication, behavior, emotions,
reactions and feedback. I hope that through these experi-
ences, people can have more opportunities to explore
themselves and their surroundings.
Photo of my prototypes at DMI the 15th Anniversary Night
ABSTRACT
76
Ideas mapping process for thesis in the third semester
CASESTUDIES
98
CASE STUDIES
1110
BIG MOUTH
One of my projects was to build a useless
machine, the Big Mouth Scent Machine. It
was a dynamic interactive installation, shaped
like a mouth. When participants wrote about
their feelings using a hashtag “#” on Twitter,
such as “#happy,” the data were sent to my
computer and triggered the scent machine.
The machine generated a smell which repre-
sented that emotion. In this way, participants
could enjoy their friends’ emotions in a physi-
cal space following the smell out of the “big
mouth”. The shape of mouth was a metaphor
to show that the smell was related to a digital
stream of talk.
This project was created to reflect my under-
standing of communication. In my opinion,
communication is not only the way people
express their feelings, but also a way of ex-
changing emotions, opinions and ideas.
Big Mouth
CASESTUDIES
1312
Background
In my Design Symposium class in the 3rd semester, we dis-
cussed how technology has changed communication, and
asked questions such as: what new social phenomena are
created by technology? What is the role of the communica-
tion designer in our generation? These questions are not
only related to design, but also to the designer’s attitude
about society, communication, technology, beauty, and
identification. We approached these questions by discuss-
ing six topics: technology, networks, algorithms, selfies, the
Internet of things, and data visualization.
For designers, experimentation is always the best way to
help us understand our roles and think about how dynamic
media influences our lives. A practical project can push a
designer to observe his or her environment more closely,
learn more from participants’ reactions, and inspire more
thinking about what the role of a designer is.
In order to help us to think about these questions, we were
asked to build a machine without any practical function.
Useless machines were first created by Bruno Munari, the
famous Italian visual communication designer, who con-
tributed to the fundamentals of modernism, futurism, and
concrete art. He created his useless machines in the 1930s2
.
They were a series of works made with light materials such
as wood and paper, connecting with each other. Munari put
his useless machines in a vacant space, in order to observe
how the visual experience of the materials dynamically
changed with the influence of the environment and time3
.
The useless machines created beautiful shapes and
shadows in those spaces, but they didn’t have a function. In
this way, the useless machines provided an opportunity for
Munari to observe and describe the dynamic changes in a
space and develop his theory of the time-space continuum,
which he describes as “how to make a work of art that could
interact with the environment and change accordingly.”4
Munari’s work was at the vanguard of manipulating kinetic
movement to influence the experience of a space5
. Rather
than a project, the core purpose of a useless machine
is a new perspective, which allows people to create art
and design without the limitation of traditional physical
materials. The “useless machine” raises the question of
the meaning of dynamic media design, and how can these
machines become media for communication?
In order to help us to establish our view on current commu-
nication through making a useless machine, we were given
restrictions. First, we had to consider the ritual view of com-
munication, which is the understanding of communication
from our unique point of view determined by our environ-
ment6
. Second, the machine had to connect to a network
via Application Program Interface (API), have an output, and
have an observable change affected by an algorithm. In
this way, we reflected our understanding of communication
today in a dynamic way, which in turn helped to form a
personal definition of dynamic media in communication.
Useless Machine, Bruno Munari
CASESTUDIES
1514
Thought and Concept
When we talk about communication, it is impossible not to
look at technology. Ten years ago, I couldn’t imagine the
ways in which technology would change our fundamental
human interactions. We didn’t have Facebook, Twitter
or Instagram. I remember when I was a child, my family
needed to go to the telecom company to pick up a call.
Even five years ago, if I lost my phone, it meant I had lost
most of my connections in this world. Now I can easily find
my kindergarten classmates on social media. Technology
helps us keep in touch with each other.
Zeynep Tufekci discusses technology in his article, Social
Media’s Small, Positive Role in Human Relationships7
. He
states: “Social media is an enhancing human connectivity as
people can converse in ways that were once not possible.”8
I would go further to say that the Internet has decreased
the limitations of time and space, so that people now have
more opportunities to exchange information. Now my
friends don’t need to worry about “losing” me. Everyone
can locate me via an e-mail address. This convenience
of exchanging information brings people a quantity of
information, but does it enhance the quality of communica-
tion? When we text our friends, we can see only the written
words, without the help of other contexts such as facial
expressions, voices or gestures. Compared with in person
communication, the short message lacks sensory stimula-
tion, and transfers very limited emotional information
among people.
Several centuries ago, people used multi-sensory means to
express themselves and create a community. A traditional
East Asian Ko-do ceremony uses scent to enable the
communication of abstract information in an unexpected
physical form. During the ceremony, the leader prepares
jars with different scents made by burning. The participants
pass the jars, and take turns smelling incense. Participants
voice observations about the incense, and play games to
guess what is being burned. In the past, this was a common
means of social communication. Without speakers, monitors,
or projectors, people invented a multi-sensory way to
exchange thoughts and feelings. Some scents had a very
nice smell and always brought people pleasure. Friends
were invited to scent ceremonies to experience and share
the pleasure of appealing scents. In this way, communities
were created through non-verbal communication.
Ko-do is a slow, physical form of communication that
reflects an East Asian attitude about communication. In
Japan, people use the phrase “Read Air”. In China, saying
less is considered common sense. People tend to believe
that words are limited to express the quantity of informa-
tion beneath the surface. Instead of words, people create
poetic ways to build a bridge between people with the
help of different sensory experiences. The meaning of the
communication is not to tell people something precisely,
but to create a connection with others.
This slow communication connects people more inti-
mately than e-mail, Facebook and Twitter. I believe scent
has strong potential even today to enhance the process
of communication.
Asian Ko-do ceremony
CASESTUDIES
1716
Research
In his news article, Smell you later: Researchers find humans
can communicate through scent, Mark Prigg introduces a
theory from Dr. Gun Semin that a smell can trigger the same
emotion in multiple people9
. As such, two people may be
able to communicate their emotions directly through smell.
Smell can likewise trigger forgotten memories10
. The
memories related to smell usually are buried deeply and
a memory that can be stored for a long time. A smell can
easily bring up a deep memory. For example, the smell of
carrot soup evokes for me my lunch time in kindergarten,
and the smell of an orange perfume brings me an image of
my grandmother.
Scent can even alert us to danger. For instance, natural gas
has the added strong and unpleasant scent of rotten eggs
so that we can’t miss a hazardous leak. Our minds store
both smells we like and those we don’t, to remind us of the
beauty and danger in our lives.
As Diane Ackerman points out in A Natural History of the
Senses, we have trouble capturing a smell with precise
language. “Instead we tend to describe how they make us
feel.”11
According to Ackerman, “When we use words such
as smoky, sulfurous, floral, fruity, sweet, we are describing
smells in terms of other things (smoke, sulfur, flowers, fruit,
sugar).”11
The scent information is translated to our brain
without help of a verbal description. The interoperation of
smell is a good bridge to connect multisensory memories of
information decoded from other senses.
Design
For my useless machine, I started by identifying the
input information. I chose a Twitter feed as the database,
because Twitter represents a new way of communication
for this generation. It only allows a reductive 140 characters
per message. Literally, when you share your ideas through
Twitter, everyone can see them.
Sherry Turkle criticizes Twitter as a place where people’s
words can be automatically heard by anonymous listeners12
.
Twitter gives people a platform for knowing what happens
among their friends, but it is not a platform for nuanced
give and take. Face to face communication allows for the
layered information given by facial expressions, context,
and ambiance.
I hoped that my useless machine would allow me to send a
sensory message that I thought would attract more atten-
tion from people than the message loading on our phones
or laptops. When the information is sent with multisensory
elements, the experience may help users have a different
perspective, deepen their online communications, and
even question the severely limited sensory input forced by
Twitter’s format.
I separated the task into several steps: finding usable
information inherent in Twitter, creating a prototype of
the scent machine, and connecting the database with the
physical machine.
Twitter API Arduino gets infromation Big Mouth is triggered
CASESTUDIES
1918
First, I defined my database. Tweets are marked by
hashtags to label the type of content. These metadata
labels can be used to group all related tweets for readers,
and allow developers to get real-time information. Using
the program Processing, I designed my scent machine to
connect to Twitter’s server via an API, and pull all messages
with the hashtags I defined, such as #feelinghappy. Once
people sent their messages with #feelinghappy, my ma-
chine would generate a scent which represented “happy”.
When I designated more words to described emotions, I
could create a corresponding scent.
The second step was to build a prototype of a physical
scent machine. I thought of using different perfumes to
represent different emotions. For example, a sweet cream
perfume could represent happiness. A bitter one could
represent sadness. In my prototype, I decided to use a
jasmine flower scent perfume as the example which could
represent “happy,” and a rotten egg to represent “sad.”
Emotions can hardly be separated from feelings. In order
to tell a story of emotions I used those scents which I
experienced with different emotions to trigger similar
feelings for my participants. I set up an early prototype
(not yet connected to the live feed) in an open space, but
it was hard to recognize each of the scents, as they diffuse
and evaporate quickly. So I redesigned the scent generating
process. In order to keep scents from escaping before they
were recognized, every update would trigger a little bit of
one scent generated from a blocked bottle. My participants
were supposed to sniff the smell by putting their noses
close to the bottle. In this way, I could differentiate the
different scents and the feelings they evoked according
to the Twitter feed information. I tried different physical
methods to squeeze the scent from the bottle. In the end,
an ultrasonic vibration helped me overcome the problem.
It worked like a humidifier, by sucking the liquid perfume
from the bottle and misting it into air. With the control of a
switch, I could decide which smell I wanted to disperse.
Next, I connected my laptop to the prototype. When
Processing identified a tweet with the appropriate hashtag,
it triggered the ultrasonic vibration under the appropriate
perfume bottle, thus replacing the physical switches.
It took me a long time to think about the proper form for
this project. Psychologist J. J. Gibson invented a word,
affordance, to refer to “the actionable properties between
the world and an actor (a person or animal).”13
Based on
this, I wanted to use my interface to show people what to
do as well as to express my thinking about communication.
After some thought I decided to use an open mouth shape
which represented Twitter, the platform where you can
say anything. The mouth was a symbol of a vivid, nuanced
physical experience, unlike Twitter’s blunt, digital broadcast.
Test of a mist module
The making process
Processing sends Twitter
feeds data to Arduino
CASESTUDIES
2120
Testing
When I tested my machine with my audience, they were
surprised by a gap they perceived between the romantic
idea and the creepy interface. They were attracted by the
strangeness of the form, and they thought it was ridiculous
to approach the “big mouth” with their noses. There was
a lot of laughing as users experienced the mood of each
tweet. Although they couldn’t figure out the correct mean-
ing of every tweet, the scent influenced their feedback.
Most users didn’t care about the contents of the tweet
at all, but they were excited to guess what the author’s
emotion was. Through this wordless communication, people
seemed to care more about the lives of the Twitter users.
The scent caused people to slow down and consider each
tweet more thoughtfully. This project positively influenced
the relationship of the friends in a nonverbal way. As one
of my classmates Yanjun said, she expected to find that
her friend was feeling happy, and that their feelings could
trigger a nice smell when she used the machine.
Image of the Big Mouth
CASESTUDIES
2322
Conclusion
This project made use of the sense of smell to translate data
into an unexpected sensory experience, bringing a virtual
community back to a physical space. From the written
language that defines Twitter, I was able to create a more
abstract, slow experience of shared emotion. This vivid
experience brought more engagement than users’ verbal
expressions. The sample of users I tested understood that
the flower smell correlated to happiness, and this created
a shared understanding among the users. This experiment
supported my hypothesis that with the help of additional
and unexpected senses, designers can affect the quality
of digital communication. There is a potential to design
interfaces using our five senses. Stimulation of more senses
depicts a more vivid picture. For a further version of this
project, I want to see people’s reactions if I involve more
emotions in this project. I am also interested in testing
features of other senses for next projects.
Image of Big Mouth
CASESTUDIES
2524
Big Mouth
Chocolate Lift
The Birds on The Wires
Lighting Tutu
My work is centered on stimulating our five senses. In this
process, I used real-time information shaped by algorithms
as input, which generate sensory stimulations as output. In
some of the projects, I worked with an individual; in others,
with multiple people or communities. I compared my users’
feelings and reactions before and after the sensory experi-
ences. I used a chart to record participants’ feedback and
the changes I observed in each project.
Smell TouchTasteListenSee
METHODOLOGY
2726
Big Mouth Chocolate Lift The Birds on The Wires Lighting Tutu
Big Mouth Chocolate Lift The Birds on The Wires Lighting Tutu
Twitter feed
Twitter
information
Happiness/
Sadness
Love/
Belief
Belonging/
Powerful
Encouragement/
Passion
Empathy Friendship/
togetherness
Sense of
community
Connection
InternetEnvironments
Existing Context
Input
Participants
Multi sensory experiences
(Changed) Perception
Feelings and emotions from participants
Two people in a
physical space
Crowd in an
open space
Concert hall
Chocolate/
Memories
Pitch of
sounds
Light
patterns
Bio-data Group size Motion
Before the multi sensory experiences
After the multi sensory experiences
How my multi sensory experiences work
PROJECTS SUMMARY Projects
CASESTUDIES
2928
Psychoanalyst D.W. Winnicott writes that “in childhood we
are watching the human being only gradually becoming
able to distinguish between the subjective and objective.”14
He analyzes the way infants learn about “me” and “not
me.” When babies develop a sense of proprioception,
knowing the parts of the body and how they work together,
they can begin to distinguish their inside feelings from
the outside environment. As we grow up, our responses
become automatic, so we become so familiar with this
body system that we are unable to distinguish how our
senses work to connect us to our environment. We have
lost the memory of how we learned about the relationships
between our senses and our surroundings.
Furthermore, no two people experience the environment
in the same way. As Mark Solms and Oliver Turnbull state
in The Brain and the Inner World,”emotion is … internally
directed, only you can feel your emotions.”15
All the recog-
nition of our environment is based on subjective feelings.
VALUE
WHO AM I
The shadow experinment in the You Are Here project
NOTES
3130
1. Ackerman xv
2. Antonello 317
3. Antonello 328
4. Robinson “Bruno Munari, The Man and His ‘Useless Machines’.”
5. Antonello 328
6. Carey 43
7. Zeynep 2012
8. Zeynep 28
9. Prigg 2012
10. Stafford “Why Can Smells Unlock Forgotten Memories?”
11. Ackerman 7
12. Turkle 2012
13. Gibson 1977
14. Winnicott 207
15. Solms 106
16. Kim ”Internet Encyclopedia of Philosophy: Wang Yangming”
17. Blakeslee 142
18. Eagleman “Can We Create New Senses for Humans?”
NOTES
WORKSCITED
3332
Book
Ackerman, Diane. A Natural History of the Senses. New York:
Random House, 1990. Print.
Antonello, Pierpaolo. “Beyond Futurism: Bruno Munari’s Useless
Machines.” Pp. 315-36 in Günter Berghaus, Futurism and the
Technological Imagination, 2009. Web
Bollas, Christopher, Allan Sekula, and Sally Stein. Being a Character:
Psychoanalysis and Self Experience. London: Routledge, 1993. Print.
Blakeslee, Sandra, and Matthew Blakeslee. The Body Has a Mind
of Its Own: How Body Maps in Your Brain Help You Do (almost)
Everything Better. New York: Random House, 2007. Print.
Braun, Björn Michael. “Methodological Advancements of Cross-
Cultural User-Centered Product Development.” Google Books. N.p.,
19 June 2009.
Bush, Vannevar. “As We May Think.” Interactions 3.2 (1996): 35-46
Carey, James W., and Stuart G. Adam. Communication as Culture
Revised Edition: Essays on Media and Society. New York: Routledge,
1992. Print.
Carey, James W. “Technology and Ideology: The Case of the
Telegraph.” Prospects 8 (1983): 303-25.
Flusser, Vilém (2013-03-28). The Shape of Things: A Philosophy of
Design. Reaktion Books. Kindle Edition.
Freud, Sigmund. Civilization and Its Discontents. New York: W.W.
Norton, 2010. Print.
Gibson, James J. “The Theory of Affordances.” Pp. 67-82 in
Perceiving, Acting, and Knowing, edited by Robert Shaw and John
Bransford. Hillsdale, NJ: Erlbaum, 1977.
Gibson, James J. (1979). “The Ecological Approach to Visual
Perception.” Boston: Houghton Mifflin.
Hara, Ken’ya. Designing Design. Baden, Switzerland: Lars MuÌller,
2007. Print.
Harlow, Harry, Robert Dodsworth, and Margaret Harlow. Total Social
Isolation in Monkeys. Proceedings of the National Academy of
Sciences 12124, National Library of Medicine (1965): 90-97.
Hinde, Robert A. Non-verbal Communication. Cambridge, UK:
Cambridge University, 1972. Print.
Ishii, Hiroshi. “Tangible Bits.” Proceedings of the 8th International
Conference on Intelligent User Interfaces - IUI ‘03 (2003): n. pag. Web.
Katz, Joel. Designing Information: Human Factors and Common
Sense in Information Design. Print.
Munro, Alice. Runaway: Stories. New York: Knopf, 2004. Print.
Norman, Donald A. Emotional Design: Why We Love (or Hate)
Everyday Things. New York: Basic, 2004. Print.
Nørretranders, Tor. The User Illusion: Cutting Consciousness down
to Size. New York: Viking, 1998. Print.
Rand, Ayn. The Romantic Manifesto: A Philosophy of Literature.
New York, NY: World Publishing Co., 1969. Print.
Reiners, Torsten, and Lincoln C. Wood. Gamification in Education
and Business. Cham: Springer, 2015. Print.
Ong, Walter J. Orality and Literacy: The Technologizing of the Word.
London, UK: Methuen, 1982. Print.
Seibt, Johanna. Process Theories: Crossdisciplinary Studies in
Dynamic Categories. Dordrecht: Kluwer Acad. Publ., 2003. Print.
Shedden, David. “Today in Media History: First Colonial Newspaper
Published in 1690.” Poynter. Poynter, 25 Sept. 2014. Web.
Solms, Mark, and Oliver Turnbull. The Brain and the Inner World:
An Introduction to the Neuroscience of Subjective Experience. New
York: Other, 2002. Print.
Turkle, Sherry. Alone Together: Why We Expect More from
Technology and Less from Each Other. New York, NY: Basic, 2011.
WORKS CITED
3534
WORKSCITED
Torres, Ana. Sensing Tangible Experiences. 2015. Print.
Winnicott, D.W. (1975), Primitive Emotional Development and
Aggression in Relation to Emotional Development. Through Pediatrics
to Psycho-analysis. New York: International Universities Press.
Web Resource
Bates, Claire. “Two Hearts Really DO Beat as One If You’re in Love:
Scientists Find Couples’ Vital Signs Mimic Each Other.” Mail Online.
Associated Newspapers, 12 Feb. 2013. Web.
Cage, John. ”Empty Words.” John Cage :: Official Website. Web.
http://www.johncage.org/empty.html
Cannes Moments: Perfume’s Amazing Digital Light Show at Cannes
Lions. Perfume. Youtube. 20 Nov. 2013. Web. https://www.youtube.
com/watch?v=GOUcCrjiOu0.
Cheney-Lippold, John. “A New Algorithmic Identity; Soft Biopolitics
and the Modulation of Control.” Theory, Culture & Society, 2011,
28(6):164-181. Web.
“Coldplay’s Mylo Xyloto Tour Launched Xylobands LED Wristbands
Worldwide.” Coldplay’s Mylo Xyloto Tour Launched Xylobands
LED Wristbands Worldwide. Company Newsroom https://
tlccreativespecialeffects.newswire.com/news/coldplays-mylo-xyloto-tour-
launched-xylobands-led-wristbands-worldwide-15717342
Deutsch, Diana. “Musical Language.” Radiolab. 24 Sept. 2007. Web.
http://www.radiolab.org/story/91512-musical-language/
Eagleman, David. “Can We Create New Senses for Humans?”
TED Talk. Mar. 2015. Web. https://www.ted.com/talks/
david_eagleman_can_we_create_new_senses_for_humans
“Hatsune Miku EXPO 2016 North America.” Web.http://mikuexpo.com/
na2016/
“Hillary Clinton Email Archive.” WikiLeaks. WikiLeaks, 16 Mar. 2016.
Historycomestolife. “First Television Broadcast NBC/RCA July 7,
1936 Part 1 of 2.” YouTube. YouTube, 28 Jan. 2014.
Hong, Dennis. “Making a Car for Blind Drivers.” TED
Talk. Mar. 2011. Web. https://www.ted.com/talks/
dennis_hong_making_a_car_for_blind_drivers
Hutchings, Emma. “Coldplay Wristbands Turn Concert Audience
Into Giant LED Display - PSFK.” PSFK.14 June 2012. http://www.
psfk.com/2012/06/coldplay-audience-led-display.html
Kim, Youngmin. “Internet Encyclopedia of Philosophy: Wang
Yangming.” Internet Encyclopedia of Philosophy. Internet
Encyclopedia of Philosophy and Its Authors. ISSN 2161-0002, Web.
http://www.iep.utm.edu/wangyang/
McCandless, David. “The Beauty of Data Visualization.”
TED Talk. July 2010. Web. http://www.ted.com/talks/
david_mccandless_the_beauty_of_data_visualization
Lee, Jinsop. “Design for All 5 Senses.”TED Talk.
Feb. 2013. Web.https://www.ted.com/talks/
jinsop_lee_design_for_all_5_senses?language=en.
Leithinge, Daniel. “InFORM.” Tangible Media Group. MIT Media
Lab, 2013. Web. http://tangible.media.mit.edu/project/inform
Prigg, Mark. “Smell You Later: Researchers Find Humans Can
Communicate Through Scent.”Daily Mail (London), November
6, 2012. http://www.dailymail.co.uk/sciencetech/article-2228619/
Smell-later-Researchers-humans-communicate-smell-scared-
disgusted.html
Reiners, Torsten, and Lincoln Wood. “Gamification in Education and
Business” Springer International Publishing, n.d. Web.
Roethke, Theodore. “The Waking by Theodore Roethke | Poetry
Foundation.” Poetry Foundation. Poetry Foundation. https://www.
poetryfoundation.org/poems-and-poets/poems/detail/43333.
3736
Robinson, Mark. “Bruno Munari, The Man and His ‘Useless
Machines’.” OEN, 22 Aug. 2012. Web. http://the189.com/design/
bruno-munari-the-man-and-his-useless-machines
Saul McLeod “Visual Perception Theory.” Visual Perception. N.p.,
01 Jan. 1970. Web. http://www.simplypsychology.org/perception-
theories.html
Spencer Kelly. Can an ‘electronic Lollipop’ Simulate Taste? Web.
http://www.bbc.com/news/technology-26487218.
Stafford, Tom. “Why Can Smells Unlock Forgotten Memories?” BBC.
13 Mar. 2012.
Taylor, Jill Bolte. “My Stroke of Insight.” TED Talk. Feb. 2008. Web.
http://www.ted.com/talks/jill_bolte_taylor_s_powerful_stroke_of_insight
TeamLab, ”Drawing on the Water Surface Created by the Dance of
Koi and People - Infinity”.2016. Web. http://www.team-lab.net/w/
koi_and_people/
Tufekci, Zeynep. “Social Media’s Small, Positive Role in Human
Relationships.” The Atlantic, 25 Apr. 2012. Web.
Twilley, Nicola. “Accounting for Taste.” The New Yorker. 26 Oct.
2015. Web.
“What Is Augmented Reality (AR).” IGI Global: International Publisher
of Progressive Academic Research Books and Journals. Web. http://
www.igi-global.com/dictionary/augmented-reality-ar/1859
WORKSCITED
Thesis_Digest

More Related Content

What's hot

New pace approaches for digital storytelling: a practical case
New pace approaches for digital storytelling: a practical caseNew pace approaches for digital storytelling: a practical case
New pace approaches for digital storytelling: a practical caseHumberto Neves
 
IMBA Application - Lorenzo sforza - Question J
IMBA Application - Lorenzo sforza - Question JIMBA Application - Lorenzo sforza - Question J
IMBA Application - Lorenzo sforza - Question JLorenzo Sforza
 
Using the present to create the future - the Web in South Africa
Using the present to  create the future - the Web in South AfricaUsing the present to  create the future - the Web in South Africa
Using the present to create the future - the Web in South AfricaDerek Keats
 
Things issue1 communication
Things issue1 communicationThings issue1 communication
Things issue1 communicationswaipnew
 
Digital Anthropology
Digital AnthropologyDigital Anthropology
Digital AnthropologyFifi Stahlman
 
Winkwaves Designing Social Media
Winkwaves Designing Social MediaWinkwaves Designing Social Media
Winkwaves Designing Social MediaRene Jansen
 
Library Bus Project - Thesis Work in progress
Library Bus Project - Thesis Work in progressLibrary Bus Project - Thesis Work in progress
Library Bus Project - Thesis Work in progressDipti Sonawane
 
Process book
Process bookProcess book
Process bookLeonLiu57
 
Personal media
Personal mediaPersonal media
Personal mediaharrison88
 
Ubiquitous Commons workshop at transmediale 2015, Capture All
Ubiquitous Commons workshop at transmediale 2015, Capture AllUbiquitous Commons workshop at transmediale 2015, Capture All
Ubiquitous Commons workshop at transmediale 2015, Capture AllSalvatore Iaconesi
 
Lucio Margulis Juego de la Prevención de Riesgos de Trabajo
Lucio Margulis Juego de la Prevención de Riesgos de TrabajoLucio Margulis Juego de la Prevención de Riesgos de Trabajo
Lucio Margulis Juego de la Prevención de Riesgos de TrabajoLucio Margulis
 
Libraries with Social Media XFactor
Libraries with Social Media XFactor Libraries with Social Media XFactor
Libraries with Social Media XFactor Judy O'Connell
 
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic Document
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic DocumentParamount Studios Anti-Piracy Co-Op Initiative -- Strategic Document
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic DocumentGunther Sonnenfeld
 
As we may feel - PhD presentation
As we may feel - PhD presentationAs we may feel - PhD presentation
As we may feel - PhD presentationMartin Christensen
 
The Internet As Directed Anticipation
The Internet As Directed AnticipationThe Internet As Directed Anticipation
The Internet As Directed AnticipationZotronix
 
Open Collaborative Making a Digital Perspective
Open Collaborative Making a Digital PerspectiveOpen Collaborative Making a Digital Perspective
Open Collaborative Making a Digital PerspectiveIrini Mirena Papadimitriou
 
2013 khci_virtual space syntax analysis on pervasive social computing
2013 khci_virtual space syntax analysis on pervasive social computing2013 khci_virtual space syntax analysis on pervasive social computing
2013 khci_virtual space syntax analysis on pervasive social computing4dspace
 

What's hot (20)

New pace approaches for digital storytelling: a practical case
New pace approaches for digital storytelling: a practical caseNew pace approaches for digital storytelling: a practical case
New pace approaches for digital storytelling: a practical case
 
be.app
be.appbe.app
be.app
 
IMBA Application - Lorenzo sforza - Question J
IMBA Application - Lorenzo sforza - Question JIMBA Application - Lorenzo sforza - Question J
IMBA Application - Lorenzo sforza - Question J
 
Using the present to create the future - the Web in South Africa
Using the present to  create the future - the Web in South AfricaUsing the present to  create the future - the Web in South Africa
Using the present to create the future - the Web in South Africa
 
Things issue1 communication
Things issue1 communicationThings issue1 communication
Things issue1 communication
 
Digital Anthropology
Digital AnthropologyDigital Anthropology
Digital Anthropology
 
Winkwaves Designing Social Media
Winkwaves Designing Social MediaWinkwaves Designing Social Media
Winkwaves Designing Social Media
 
be.app edit
be.app editbe.app edit
be.app edit
 
Library Bus Project - Thesis Work in progress
Library Bus Project - Thesis Work in progressLibrary Bus Project - Thesis Work in progress
Library Bus Project - Thesis Work in progress
 
Process book
Process bookProcess book
Process book
 
Personal media
Personal mediaPersonal media
Personal media
 
Ubiquitous Commons workshop at transmediale 2015, Capture All
Ubiquitous Commons workshop at transmediale 2015, Capture AllUbiquitous Commons workshop at transmediale 2015, Capture All
Ubiquitous Commons workshop at transmediale 2015, Capture All
 
Lucio Margulis Juego de la Prevención de Riesgos de Trabajo
Lucio Margulis Juego de la Prevención de Riesgos de TrabajoLucio Margulis Juego de la Prevención de Riesgos de Trabajo
Lucio Margulis Juego de la Prevención de Riesgos de Trabajo
 
Libraries with Social Media XFactor
Libraries with Social Media XFactor Libraries with Social Media XFactor
Libraries with Social Media XFactor
 
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic Document
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic DocumentParamount Studios Anti-Piracy Co-Op Initiative -- Strategic Document
Paramount Studios Anti-Piracy Co-Op Initiative -- Strategic Document
 
As we may feel - PhD presentation
As we may feel - PhD presentationAs we may feel - PhD presentation
As we may feel - PhD presentation
 
The Internet As Directed Anticipation
The Internet As Directed AnticipationThe Internet As Directed Anticipation
The Internet As Directed Anticipation
 
Open Collaborative Making a Digital Perspective
Open Collaborative Making a Digital PerspectiveOpen Collaborative Making a Digital Perspective
Open Collaborative Making a Digital Perspective
 
2013 khci_virtual space syntax analysis on pervasive social computing
2013 khci_virtual space syntax analysis on pervasive social computing2013 khci_virtual space syntax analysis on pervasive social computing
2013 khci_virtual space syntax analysis on pervasive social computing
 
IDYL2S book
IDYL2S bookIDYL2S book
IDYL2S book
 

Viewers also liked

Synthèse des travaux SEPL 2014-2015
Synthèse des travaux SEPL 2014-2015Synthèse des travaux SEPL 2014-2015
Synthèse des travaux SEPL 2014-2015Franck THOUNY
 
Synthèse des travaux SEPL 2013-2014
Synthèse des travaux SEPL 2013-2014Synthèse des travaux SEPL 2013-2014
Synthèse des travaux SEPL 2013-2014Franck THOUNY
 
The Logo Design Process
The Logo Design ProcessThe Logo Design Process
The Logo Design ProcessZillionDesigns
 
청석의 좋은 글 모음 제 74호 2016년 12월
청석의 좋은 글 모음 제 74호 2016년 12월청석의 좋은 글 모음 제 74호 2016년 12월
청석의 좋은 글 모음 제 74호 2016년 12월올포피엠
 
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsae
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsaeLeyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsae
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsaeeCityclic
 
Halvar Flake: Why Johnny can’t tell if he is compromised
Halvar Flake: Why Johnny can’t tell if he is compromisedHalvar Flake: Why Johnny can’t tell if he is compromised
Halvar Flake: Why Johnny can’t tell if he is compromisedArea41
 
A World Class Strategy to Sustainable SHEQ
A World Class Strategy to Sustainable SHEQA World Class Strategy to Sustainable SHEQ
A World Class Strategy to Sustainable SHEQPECB
 
10分で出来る!?プラグインライブコーディング
10分で出来る!?プラグインライブコーディング10分で出来る!?プラグインライブコーディング
10分で出来る!?プラグインライブコーディングHiroki Kondo
 
Redhat Virualization Technology: A Detailed Manual.
Redhat Virualization Technology: A Detailed Manual.Redhat Virualization Technology: A Detailed Manual.
Redhat Virualization Technology: A Detailed Manual.Ankur Verma
 
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017MICEboard
 

Viewers also liked (15)

Filières inclusives : Analyser les modèles d'affaire
Filières inclusives : Analyser les modèles d'affaireFilières inclusives : Analyser les modèles d'affaire
Filières inclusives : Analyser les modèles d'affaire
 
Minorty scheme booklet
Minorty scheme bookletMinorty scheme booklet
Minorty scheme booklet
 
Filières inclusives : concept et définitions
Filières inclusives : concept et définitionsFilières inclusives : concept et définitions
Filières inclusives : concept et définitions
 
Synthèse des travaux SEPL 2014-2015
Synthèse des travaux SEPL 2014-2015Synthèse des travaux SEPL 2014-2015
Synthèse des travaux SEPL 2014-2015
 
Synthèse des travaux SEPL 2013-2014
Synthèse des travaux SEPL 2013-2014Synthèse des travaux SEPL 2013-2014
Synthèse des travaux SEPL 2013-2014
 
Chromium ppt
Chromium pptChromium ppt
Chromium ppt
 
The Logo Design Process
The Logo Design ProcessThe Logo Design Process
The Logo Design Process
 
청석의 좋은 글 모음 제 74호 2016년 12월
청석의 좋은 글 모음 제 74호 2016년 12월청석의 좋은 글 모음 제 74호 2016년 12월
청석의 좋은 글 모음 제 74호 2016년 12월
 
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsae
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsaeLeyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsae
Leyes 39 y 40 de 2015, del procedimiento administrativo. novedades. obsae
 
Artificial Intelligence Preparing for the Future of AI
Artificial Intelligence Preparing for the Future of AIArtificial Intelligence Preparing for the Future of AI
Artificial Intelligence Preparing for the Future of AI
 
Halvar Flake: Why Johnny can’t tell if he is compromised
Halvar Flake: Why Johnny can’t tell if he is compromisedHalvar Flake: Why Johnny can’t tell if he is compromised
Halvar Flake: Why Johnny can’t tell if he is compromised
 
A World Class Strategy to Sustainable SHEQ
A World Class Strategy to Sustainable SHEQA World Class Strategy to Sustainable SHEQ
A World Class Strategy to Sustainable SHEQ
 
10分で出来る!?プラグインライブコーディング
10分で出来る!?プラグインライブコーディング10分で出来る!?プラグインライブコーディング
10分で出来る!?プラグインライブコーディング
 
Redhat Virualization Technology: A Detailed Manual.
Redhat Virualization Technology: A Detailed Manual.Redhat Virualization Technology: A Detailed Manual.
Redhat Virualization Technology: A Detailed Manual.
 
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017
The Circle Convention Centre Hyatt Zürich - MICE Presentation 2017
 

Similar to Thesis_Digest

Thesis Documentation
Thesis DocumentationThesis Documentation
Thesis DocumentationAnran Zhou
 
Optimizing interconnectivity inhabiting virtual cities of common practice
Optimizing interconnectivity  inhabiting virtual cities of common practiceOptimizing interconnectivity  inhabiting virtual cities of common practice
Optimizing interconnectivity inhabiting virtual cities of common practiceJonathan Buffa
 
Describe some effects that cybertechnology has had so far for our se.pdf
Describe some effects that cybertechnology has had so far for our se.pdfDescribe some effects that cybertechnology has had so far for our se.pdf
Describe some effects that cybertechnology has had so far for our se.pdfcalderoncasto9163
 
The Evolution Of Human Communication
The Evolution Of Human CommunicationThe Evolution Of Human Communication
The Evolution Of Human CommunicationSharon Roberts
 
Baym, nancy k. (2015). personal connections in the digital age
Baym, nancy k. (2015). personal connections in the digital ageBaym, nancy k. (2015). personal connections in the digital age
Baym, nancy k. (2015). personal connections in the digital ageRAJU852744
 
Top10 Chinese Social Trends (2020) Report -Monzayin Consulting
Top10 Chinese Social Trends (2020) Report  -Monzayin ConsultingTop10 Chinese Social Trends (2020) Report  -Monzayin Consulting
Top10 Chinese Social Trends (2020) Report -Monzayin Consultingjeqh0228
 
Pablo Rivilla - How do you imagine social interaction within 10 years, takin...
 Pablo Rivilla - How do you imagine social interaction within 10 years, takin... Pablo Rivilla - How do you imagine social interaction within 10 years, takin...
Pablo Rivilla - How do you imagine social interaction within 10 years, takin...pablorivilla
 
sxsw-interactive-festival-2013)
sxsw-interactive-festival-2013)sxsw-interactive-festival-2013)
sxsw-interactive-festival-2013)Kristin Milburn
 
Learning with Technology - week 2 lecture-social media
Learning with Technology - week 2 lecture-social mediaLearning with Technology - week 2 lecture-social media
Learning with Technology - week 2 lecture-social mediaCharles Darwin University
 
Mini Lecture for UoL Alumni: Social Media: Uses and Abuses
Mini Lecture for UoL Alumni: Social Media: Uses and AbusesMini Lecture for UoL Alumni: Social Media: Uses and Abuses
Mini Lecture for UoL Alumni: Social Media: Uses and AbusesJennifer Jones
 
The Institution as Broadcaster
The Institution as BroadcasterThe Institution as Broadcaster
The Institution as BroadcasterRChimera
 
Pedagogy and School Libraries
Pedagogy and School LibrariesPedagogy and School Libraries
Pedagogy and School LibrariesJudy O'Connell
 
Care Review Roadshow July 2019
Care Review Roadshow July 2019Care Review Roadshow July 2019
Care Review Roadshow July 2019John Morrison
 
Internet of Affect (08.06.2016)
Internet of Affect (08.06.2016)Internet of Affect (08.06.2016)
Internet of Affect (08.06.2016)Sumandro C
 
A WEARABLE HAPTIC GAME CONTROLLER
A WEARABLE HAPTIC GAME CONTROLLERA WEARABLE HAPTIC GAME CONTROLLER
A WEARABLE HAPTIC GAME CONTROLLERijgttjournal
 
Media role in communication
Media role in communicationMedia role in communication
Media role in communicationSalman Jailani
 
Essays on New Information Environment
Essays on New Information EnvironmentEssays on New Information Environment
Essays on New Information EnvironmentParsa Shirazi
 

Similar to Thesis_Digest (20)

J. Social Interaction
J. Social InteractionJ. Social Interaction
J. Social Interaction
 
Turning Content into Conversation
Turning Content into ConversationTurning Content into Conversation
Turning Content into Conversation
 
Thesis Documentation
Thesis DocumentationThesis Documentation
Thesis Documentation
 
Optimizing interconnectivity inhabiting virtual cities of common practice
Optimizing interconnectivity  inhabiting virtual cities of common practiceOptimizing interconnectivity  inhabiting virtual cities of common practice
Optimizing interconnectivity inhabiting virtual cities of common practice
 
Describe some effects that cybertechnology has had so far for our se.pdf
Describe some effects that cybertechnology has had so far for our se.pdfDescribe some effects that cybertechnology has had so far for our se.pdf
Describe some effects that cybertechnology has had so far for our se.pdf
 
The Evolution Of Human Communication
The Evolution Of Human CommunicationThe Evolution Of Human Communication
The Evolution Of Human Communication
 
Baym, nancy k. (2015). personal connections in the digital age
Baym, nancy k. (2015). personal connections in the digital ageBaym, nancy k. (2015). personal connections in the digital age
Baym, nancy k. (2015). personal connections in the digital age
 
Top10 Chinese Social Trends (2020) Report -Monzayin Consulting
Top10 Chinese Social Trends (2020) Report  -Monzayin ConsultingTop10 Chinese Social Trends (2020) Report  -Monzayin Consulting
Top10 Chinese Social Trends (2020) Report -Monzayin Consulting
 
Pablo Rivilla - How do you imagine social interaction within 10 years, takin...
 Pablo Rivilla - How do you imagine social interaction within 10 years, takin... Pablo Rivilla - How do you imagine social interaction within 10 years, takin...
Pablo Rivilla - How do you imagine social interaction within 10 years, takin...
 
sxsw-interactive-festival-2013)
sxsw-interactive-festival-2013)sxsw-interactive-festival-2013)
sxsw-interactive-festival-2013)
 
Learning with Technology - week 2 lecture-social media
Learning with Technology - week 2 lecture-social mediaLearning with Technology - week 2 lecture-social media
Learning with Technology - week 2 lecture-social media
 
Mini Lecture for UoL Alumni: Social Media: Uses and Abuses
Mini Lecture for UoL Alumni: Social Media: Uses and AbusesMini Lecture for UoL Alumni: Social Media: Uses and Abuses
Mini Lecture for UoL Alumni: Social Media: Uses and Abuses
 
The Institution as Broadcaster
The Institution as BroadcasterThe Institution as Broadcaster
The Institution as Broadcaster
 
Pedagogy and School Libraries
Pedagogy and School LibrariesPedagogy and School Libraries
Pedagogy and School Libraries
 
Care Review Roadshow July 2019
Care Review Roadshow July 2019Care Review Roadshow July 2019
Care Review Roadshow July 2019
 
Internet of Affect (08.06.2016)
Internet of Affect (08.06.2016)Internet of Affect (08.06.2016)
Internet of Affect (08.06.2016)
 
A WEARABLE HAPTIC GAME CONTROLLER
A WEARABLE HAPTIC GAME CONTROLLERA WEARABLE HAPTIC GAME CONTROLLER
A WEARABLE HAPTIC GAME CONTROLLER
 
Routine Breaker
Routine BreakerRoutine Breaker
Routine Breaker
 
Media role in communication
Media role in communicationMedia role in communication
Media role in communication
 
Essays on New Information Environment
Essays on New Information EnvironmentEssays on New Information Environment
Essays on New Information Environment
 

Thesis_Digest

  • 1.  Between Mind & Environment (Digest Version) Wei Xu
  • 2. 32 ABSTRACT ABSTRACT The development of technology has fundamentally changed our daily lives. Without it, we would not have FaceTime to connect with loved ones when we first wake up, Google Maps to check the traffic over breakfast, or Pokemon Go to play on the way to work. As the quantity of information increases, we shift our behavior to adapt to this new digital environment. As Diane Ackerman writes in her book A Natural History of The Senses, “There is no way in which to understand the world without first detecting it through the radar-net of our sensors.”1 Each sense collects a different kind of informa- tion, and the collaboration of multiple senses gathers and stores information about the external world. Our brain then processes this information and gives it order. Our senses connect our inside selves with the outside world, and are lenses which shape our understanding of our environment. With the help of technology, we now receive more data than before, but the sensory input is limited by the interface we use, and often is just visual and auditory. By using more senses than just sight and sound, can we get benefits from data technology to explore neglected information? Can this kind of information enhance feelings? Will we gain a new understanding of ourselves? Through my projects I wish to create multi-sensory experiences to help people notice ignored information and question the lenses through which information is
  • 3. 54 gathered. These projects use algorithms that I have defined to transform information without physical form, such as feel- ings, the social atmosphere of a room, and the relationship between individuals and a group. During the transformation processes, I will observe how data is translated through different senses, and how data enhanced by algorithms influences people’s communication, behavior, emotions, reactions and feedback. I hope that through these experi- ences, people can have more opportunities to explore themselves and their surroundings. Photo of my prototypes at DMI the 15th Anniversary Night
  • 4. ABSTRACT 76 Ideas mapping process for thesis in the third semester
  • 6. 1110 BIG MOUTH One of my projects was to build a useless machine, the Big Mouth Scent Machine. It was a dynamic interactive installation, shaped like a mouth. When participants wrote about their feelings using a hashtag “#” on Twitter, such as “#happy,” the data were sent to my computer and triggered the scent machine. The machine generated a smell which repre- sented that emotion. In this way, participants could enjoy their friends’ emotions in a physi- cal space following the smell out of the “big mouth”. The shape of mouth was a metaphor to show that the smell was related to a digital stream of talk. This project was created to reflect my under- standing of communication. In my opinion, communication is not only the way people express their feelings, but also a way of ex- changing emotions, opinions and ideas. Big Mouth
  • 7. CASESTUDIES 1312 Background In my Design Symposium class in the 3rd semester, we dis- cussed how technology has changed communication, and asked questions such as: what new social phenomena are created by technology? What is the role of the communica- tion designer in our generation? These questions are not only related to design, but also to the designer’s attitude about society, communication, technology, beauty, and identification. We approached these questions by discuss- ing six topics: technology, networks, algorithms, selfies, the Internet of things, and data visualization. For designers, experimentation is always the best way to help us understand our roles and think about how dynamic media influences our lives. A practical project can push a designer to observe his or her environment more closely, learn more from participants’ reactions, and inspire more thinking about what the role of a designer is. In order to help us to think about these questions, we were asked to build a machine without any practical function. Useless machines were first created by Bruno Munari, the famous Italian visual communication designer, who con- tributed to the fundamentals of modernism, futurism, and concrete art. He created his useless machines in the 1930s2 . They were a series of works made with light materials such as wood and paper, connecting with each other. Munari put his useless machines in a vacant space, in order to observe how the visual experience of the materials dynamically changed with the influence of the environment and time3 . The useless machines created beautiful shapes and shadows in those spaces, but they didn’t have a function. In this way, the useless machines provided an opportunity for Munari to observe and describe the dynamic changes in a space and develop his theory of the time-space continuum, which he describes as “how to make a work of art that could interact with the environment and change accordingly.”4 Munari’s work was at the vanguard of manipulating kinetic movement to influence the experience of a space5 . Rather than a project, the core purpose of a useless machine is a new perspective, which allows people to create art and design without the limitation of traditional physical materials. The “useless machine” raises the question of the meaning of dynamic media design, and how can these machines become media for communication? In order to help us to establish our view on current commu- nication through making a useless machine, we were given restrictions. First, we had to consider the ritual view of com- munication, which is the understanding of communication from our unique point of view determined by our environ- ment6 . Second, the machine had to connect to a network via Application Program Interface (API), have an output, and have an observable change affected by an algorithm. In this way, we reflected our understanding of communication today in a dynamic way, which in turn helped to form a personal definition of dynamic media in communication. Useless Machine, Bruno Munari
  • 8. CASESTUDIES 1514 Thought and Concept When we talk about communication, it is impossible not to look at technology. Ten years ago, I couldn’t imagine the ways in which technology would change our fundamental human interactions. We didn’t have Facebook, Twitter or Instagram. I remember when I was a child, my family needed to go to the telecom company to pick up a call. Even five years ago, if I lost my phone, it meant I had lost most of my connections in this world. Now I can easily find my kindergarten classmates on social media. Technology helps us keep in touch with each other. Zeynep Tufekci discusses technology in his article, Social Media’s Small, Positive Role in Human Relationships7 . He states: “Social media is an enhancing human connectivity as people can converse in ways that were once not possible.”8 I would go further to say that the Internet has decreased the limitations of time and space, so that people now have more opportunities to exchange information. Now my friends don’t need to worry about “losing” me. Everyone can locate me via an e-mail address. This convenience of exchanging information brings people a quantity of information, but does it enhance the quality of communica- tion? When we text our friends, we can see only the written words, without the help of other contexts such as facial expressions, voices or gestures. Compared with in person communication, the short message lacks sensory stimula- tion, and transfers very limited emotional information among people. Several centuries ago, people used multi-sensory means to express themselves and create a community. A traditional East Asian Ko-do ceremony uses scent to enable the communication of abstract information in an unexpected physical form. During the ceremony, the leader prepares jars with different scents made by burning. The participants pass the jars, and take turns smelling incense. Participants voice observations about the incense, and play games to guess what is being burned. In the past, this was a common means of social communication. Without speakers, monitors, or projectors, people invented a multi-sensory way to exchange thoughts and feelings. Some scents had a very nice smell and always brought people pleasure. Friends were invited to scent ceremonies to experience and share the pleasure of appealing scents. In this way, communities were created through non-verbal communication. Ko-do is a slow, physical form of communication that reflects an East Asian attitude about communication. In Japan, people use the phrase “Read Air”. In China, saying less is considered common sense. People tend to believe that words are limited to express the quantity of informa- tion beneath the surface. Instead of words, people create poetic ways to build a bridge between people with the help of different sensory experiences. The meaning of the communication is not to tell people something precisely, but to create a connection with others. This slow communication connects people more inti- mately than e-mail, Facebook and Twitter. I believe scent has strong potential even today to enhance the process of communication. Asian Ko-do ceremony
  • 9. CASESTUDIES 1716 Research In his news article, Smell you later: Researchers find humans can communicate through scent, Mark Prigg introduces a theory from Dr. Gun Semin that a smell can trigger the same emotion in multiple people9 . As such, two people may be able to communicate their emotions directly through smell. Smell can likewise trigger forgotten memories10 . The memories related to smell usually are buried deeply and a memory that can be stored for a long time. A smell can easily bring up a deep memory. For example, the smell of carrot soup evokes for me my lunch time in kindergarten, and the smell of an orange perfume brings me an image of my grandmother. Scent can even alert us to danger. For instance, natural gas has the added strong and unpleasant scent of rotten eggs so that we can’t miss a hazardous leak. Our minds store both smells we like and those we don’t, to remind us of the beauty and danger in our lives. As Diane Ackerman points out in A Natural History of the Senses, we have trouble capturing a smell with precise language. “Instead we tend to describe how they make us feel.”11 According to Ackerman, “When we use words such as smoky, sulfurous, floral, fruity, sweet, we are describing smells in terms of other things (smoke, sulfur, flowers, fruit, sugar).”11 The scent information is translated to our brain without help of a verbal description. The interoperation of smell is a good bridge to connect multisensory memories of information decoded from other senses. Design For my useless machine, I started by identifying the input information. I chose a Twitter feed as the database, because Twitter represents a new way of communication for this generation. It only allows a reductive 140 characters per message. Literally, when you share your ideas through Twitter, everyone can see them. Sherry Turkle criticizes Twitter as a place where people’s words can be automatically heard by anonymous listeners12 . Twitter gives people a platform for knowing what happens among their friends, but it is not a platform for nuanced give and take. Face to face communication allows for the layered information given by facial expressions, context, and ambiance. I hoped that my useless machine would allow me to send a sensory message that I thought would attract more atten- tion from people than the message loading on our phones or laptops. When the information is sent with multisensory elements, the experience may help users have a different perspective, deepen their online communications, and even question the severely limited sensory input forced by Twitter’s format. I separated the task into several steps: finding usable information inherent in Twitter, creating a prototype of the scent machine, and connecting the database with the physical machine. Twitter API Arduino gets infromation Big Mouth is triggered
  • 10. CASESTUDIES 1918 First, I defined my database. Tweets are marked by hashtags to label the type of content. These metadata labels can be used to group all related tweets for readers, and allow developers to get real-time information. Using the program Processing, I designed my scent machine to connect to Twitter’s server via an API, and pull all messages with the hashtags I defined, such as #feelinghappy. Once people sent their messages with #feelinghappy, my ma- chine would generate a scent which represented “happy”. When I designated more words to described emotions, I could create a corresponding scent. The second step was to build a prototype of a physical scent machine. I thought of using different perfumes to represent different emotions. For example, a sweet cream perfume could represent happiness. A bitter one could represent sadness. In my prototype, I decided to use a jasmine flower scent perfume as the example which could represent “happy,” and a rotten egg to represent “sad.” Emotions can hardly be separated from feelings. In order to tell a story of emotions I used those scents which I experienced with different emotions to trigger similar feelings for my participants. I set up an early prototype (not yet connected to the live feed) in an open space, but it was hard to recognize each of the scents, as they diffuse and evaporate quickly. So I redesigned the scent generating process. In order to keep scents from escaping before they were recognized, every update would trigger a little bit of one scent generated from a blocked bottle. My participants were supposed to sniff the smell by putting their noses close to the bottle. In this way, I could differentiate the different scents and the feelings they evoked according to the Twitter feed information. I tried different physical methods to squeeze the scent from the bottle. In the end, an ultrasonic vibration helped me overcome the problem. It worked like a humidifier, by sucking the liquid perfume from the bottle and misting it into air. With the control of a switch, I could decide which smell I wanted to disperse. Next, I connected my laptop to the prototype. When Processing identified a tweet with the appropriate hashtag, it triggered the ultrasonic vibration under the appropriate perfume bottle, thus replacing the physical switches. It took me a long time to think about the proper form for this project. Psychologist J. J. Gibson invented a word, affordance, to refer to “the actionable properties between the world and an actor (a person or animal).”13 Based on this, I wanted to use my interface to show people what to do as well as to express my thinking about communication. After some thought I decided to use an open mouth shape which represented Twitter, the platform where you can say anything. The mouth was a symbol of a vivid, nuanced physical experience, unlike Twitter’s blunt, digital broadcast. Test of a mist module The making process Processing sends Twitter feeds data to Arduino
  • 11. CASESTUDIES 2120 Testing When I tested my machine with my audience, they were surprised by a gap they perceived between the romantic idea and the creepy interface. They were attracted by the strangeness of the form, and they thought it was ridiculous to approach the “big mouth” with their noses. There was a lot of laughing as users experienced the mood of each tweet. Although they couldn’t figure out the correct mean- ing of every tweet, the scent influenced their feedback. Most users didn’t care about the contents of the tweet at all, but they were excited to guess what the author’s emotion was. Through this wordless communication, people seemed to care more about the lives of the Twitter users. The scent caused people to slow down and consider each tweet more thoughtfully. This project positively influenced the relationship of the friends in a nonverbal way. As one of my classmates Yanjun said, she expected to find that her friend was feeling happy, and that their feelings could trigger a nice smell when she used the machine. Image of the Big Mouth
  • 12. CASESTUDIES 2322 Conclusion This project made use of the sense of smell to translate data into an unexpected sensory experience, bringing a virtual community back to a physical space. From the written language that defines Twitter, I was able to create a more abstract, slow experience of shared emotion. This vivid experience brought more engagement than users’ verbal expressions. The sample of users I tested understood that the flower smell correlated to happiness, and this created a shared understanding among the users. This experiment supported my hypothesis that with the help of additional and unexpected senses, designers can affect the quality of digital communication. There is a potential to design interfaces using our five senses. Stimulation of more senses depicts a more vivid picture. For a further version of this project, I want to see people’s reactions if I involve more emotions in this project. I am also interested in testing features of other senses for next projects. Image of Big Mouth
  • 13. CASESTUDIES 2524 Big Mouth Chocolate Lift The Birds on The Wires Lighting Tutu My work is centered on stimulating our five senses. In this process, I used real-time information shaped by algorithms as input, which generate sensory stimulations as output. In some of the projects, I worked with an individual; in others, with multiple people or communities. I compared my users’ feelings and reactions before and after the sensory experi- ences. I used a chart to record participants’ feedback and the changes I observed in each project. Smell TouchTasteListenSee METHODOLOGY
  • 14. 2726 Big Mouth Chocolate Lift The Birds on The Wires Lighting Tutu Big Mouth Chocolate Lift The Birds on The Wires Lighting Tutu Twitter feed Twitter information Happiness/ Sadness Love/ Belief Belonging/ Powerful Encouragement/ Passion Empathy Friendship/ togetherness Sense of community Connection InternetEnvironments Existing Context Input Participants Multi sensory experiences (Changed) Perception Feelings and emotions from participants Two people in a physical space Crowd in an open space Concert hall Chocolate/ Memories Pitch of sounds Light patterns Bio-data Group size Motion Before the multi sensory experiences After the multi sensory experiences How my multi sensory experiences work PROJECTS SUMMARY Projects
  • 15. CASESTUDIES 2928 Psychoanalyst D.W. Winnicott writes that “in childhood we are watching the human being only gradually becoming able to distinguish between the subjective and objective.”14 He analyzes the way infants learn about “me” and “not me.” When babies develop a sense of proprioception, knowing the parts of the body and how they work together, they can begin to distinguish their inside feelings from the outside environment. As we grow up, our responses become automatic, so we become so familiar with this body system that we are unable to distinguish how our senses work to connect us to our environment. We have lost the memory of how we learned about the relationships between our senses and our surroundings. Furthermore, no two people experience the environment in the same way. As Mark Solms and Oliver Turnbull state in The Brain and the Inner World,”emotion is … internally directed, only you can feel your emotions.”15 All the recog- nition of our environment is based on subjective feelings. VALUE WHO AM I The shadow experinment in the You Are Here project
  • 16. NOTES 3130 1. Ackerman xv 2. Antonello 317 3. Antonello 328 4. Robinson “Bruno Munari, The Man and His ‘Useless Machines’.” 5. Antonello 328 6. Carey 43 7. Zeynep 2012 8. Zeynep 28 9. Prigg 2012 10. Stafford “Why Can Smells Unlock Forgotten Memories?” 11. Ackerman 7 12. Turkle 2012 13. Gibson 1977 14. Winnicott 207 15. Solms 106 16. Kim ”Internet Encyclopedia of Philosophy: Wang Yangming” 17. Blakeslee 142 18. Eagleman “Can We Create New Senses for Humans?” NOTES
  • 17. WORKSCITED 3332 Book Ackerman, Diane. A Natural History of the Senses. New York: Random House, 1990. Print. Antonello, Pierpaolo. “Beyond Futurism: Bruno Munari’s Useless Machines.” Pp. 315-36 in Günter Berghaus, Futurism and the Technological Imagination, 2009. Web Bollas, Christopher, Allan Sekula, and Sally Stein. Being a Character: Psychoanalysis and Self Experience. London: Routledge, 1993. Print. Blakeslee, Sandra, and Matthew Blakeslee. The Body Has a Mind of Its Own: How Body Maps in Your Brain Help You Do (almost) Everything Better. New York: Random House, 2007. Print. Braun, Björn Michael. “Methodological Advancements of Cross- Cultural User-Centered Product Development.” Google Books. N.p., 19 June 2009. Bush, Vannevar. “As We May Think.” Interactions 3.2 (1996): 35-46 Carey, James W., and Stuart G. Adam. Communication as Culture Revised Edition: Essays on Media and Society. New York: Routledge, 1992. Print. Carey, James W. “Technology and Ideology: The Case of the Telegraph.” Prospects 8 (1983): 303-25. Flusser, Vilém (2013-03-28). The Shape of Things: A Philosophy of Design. Reaktion Books. Kindle Edition. Freud, Sigmund. Civilization and Its Discontents. New York: W.W. Norton, 2010. Print. Gibson, James J. “The Theory of Affordances.” Pp. 67-82 in Perceiving, Acting, and Knowing, edited by Robert Shaw and John Bransford. Hillsdale, NJ: Erlbaum, 1977. Gibson, James J. (1979). “The Ecological Approach to Visual Perception.” Boston: Houghton Mifflin. Hara, Ken’ya. Designing Design. Baden, Switzerland: Lars MuÌller, 2007. Print. Harlow, Harry, Robert Dodsworth, and Margaret Harlow. Total Social Isolation in Monkeys. Proceedings of the National Academy of Sciences 12124, National Library of Medicine (1965): 90-97. Hinde, Robert A. Non-verbal Communication. Cambridge, UK: Cambridge University, 1972. Print. Ishii, Hiroshi. “Tangible Bits.” Proceedings of the 8th International Conference on Intelligent User Interfaces - IUI ‘03 (2003): n. pag. Web. Katz, Joel. Designing Information: Human Factors and Common Sense in Information Design. Print. Munro, Alice. Runaway: Stories. New York: Knopf, 2004. Print. Norman, Donald A. Emotional Design: Why We Love (or Hate) Everyday Things. New York: Basic, 2004. Print. Nørretranders, Tor. The User Illusion: Cutting Consciousness down to Size. New York: Viking, 1998. Print. Rand, Ayn. The Romantic Manifesto: A Philosophy of Literature. New York, NY: World Publishing Co., 1969. Print. Reiners, Torsten, and Lincoln C. Wood. Gamification in Education and Business. Cham: Springer, 2015. Print. Ong, Walter J. Orality and Literacy: The Technologizing of the Word. London, UK: Methuen, 1982. Print. Seibt, Johanna. Process Theories: Crossdisciplinary Studies in Dynamic Categories. Dordrecht: Kluwer Acad. Publ., 2003. Print. Shedden, David. “Today in Media History: First Colonial Newspaper Published in 1690.” Poynter. Poynter, 25 Sept. 2014. Web. Solms, Mark, and Oliver Turnbull. The Brain and the Inner World: An Introduction to the Neuroscience of Subjective Experience. New York: Other, 2002. Print. Turkle, Sherry. Alone Together: Why We Expect More from Technology and Less from Each Other. New York, NY: Basic, 2011. WORKS CITED
  • 18. 3534 WORKSCITED Torres, Ana. Sensing Tangible Experiences. 2015. Print. Winnicott, D.W. (1975), Primitive Emotional Development and Aggression in Relation to Emotional Development. Through Pediatrics to Psycho-analysis. New York: International Universities Press. Web Resource Bates, Claire. “Two Hearts Really DO Beat as One If You’re in Love: Scientists Find Couples’ Vital Signs Mimic Each Other.” Mail Online. Associated Newspapers, 12 Feb. 2013. Web. Cage, John. ”Empty Words.” John Cage :: Official Website. Web. http://www.johncage.org/empty.html Cannes Moments: Perfume’s Amazing Digital Light Show at Cannes Lions. Perfume. Youtube. 20 Nov. 2013. Web. https://www.youtube. com/watch?v=GOUcCrjiOu0. Cheney-Lippold, John. “A New Algorithmic Identity; Soft Biopolitics and the Modulation of Control.” Theory, Culture & Society, 2011, 28(6):164-181. Web. “Coldplay’s Mylo Xyloto Tour Launched Xylobands LED Wristbands Worldwide.” Coldplay’s Mylo Xyloto Tour Launched Xylobands LED Wristbands Worldwide. Company Newsroom https:// tlccreativespecialeffects.newswire.com/news/coldplays-mylo-xyloto-tour- launched-xylobands-led-wristbands-worldwide-15717342 Deutsch, Diana. “Musical Language.” Radiolab. 24 Sept. 2007. Web. http://www.radiolab.org/story/91512-musical-language/ Eagleman, David. “Can We Create New Senses for Humans?” TED Talk. Mar. 2015. Web. https://www.ted.com/talks/ david_eagleman_can_we_create_new_senses_for_humans “Hatsune Miku EXPO 2016 North America.” Web.http://mikuexpo.com/ na2016/ “Hillary Clinton Email Archive.” WikiLeaks. WikiLeaks, 16 Mar. 2016. Historycomestolife. “First Television Broadcast NBC/RCA July 7, 1936 Part 1 of 2.” YouTube. YouTube, 28 Jan. 2014. Hong, Dennis. “Making a Car for Blind Drivers.” TED Talk. Mar. 2011. Web. https://www.ted.com/talks/ dennis_hong_making_a_car_for_blind_drivers Hutchings, Emma. “Coldplay Wristbands Turn Concert Audience Into Giant LED Display - PSFK.” PSFK.14 June 2012. http://www. psfk.com/2012/06/coldplay-audience-led-display.html Kim, Youngmin. “Internet Encyclopedia of Philosophy: Wang Yangming.” Internet Encyclopedia of Philosophy. Internet Encyclopedia of Philosophy and Its Authors. ISSN 2161-0002, Web. http://www.iep.utm.edu/wangyang/ McCandless, David. “The Beauty of Data Visualization.” TED Talk. July 2010. Web. http://www.ted.com/talks/ david_mccandless_the_beauty_of_data_visualization Lee, Jinsop. “Design for All 5 Senses.”TED Talk. Feb. 2013. Web.https://www.ted.com/talks/ jinsop_lee_design_for_all_5_senses?language=en. Leithinge, Daniel. “InFORM.” Tangible Media Group. MIT Media Lab, 2013. Web. http://tangible.media.mit.edu/project/inform Prigg, Mark. “Smell You Later: Researchers Find Humans Can Communicate Through Scent.”Daily Mail (London), November 6, 2012. http://www.dailymail.co.uk/sciencetech/article-2228619/ Smell-later-Researchers-humans-communicate-smell-scared- disgusted.html Reiners, Torsten, and Lincoln Wood. “Gamification in Education and Business” Springer International Publishing, n.d. Web. Roethke, Theodore. “The Waking by Theodore Roethke | Poetry Foundation.” Poetry Foundation. Poetry Foundation. https://www. poetryfoundation.org/poems-and-poets/poems/detail/43333.
  • 19. 3736 Robinson, Mark. “Bruno Munari, The Man and His ‘Useless Machines’.” OEN, 22 Aug. 2012. Web. http://the189.com/design/ bruno-munari-the-man-and-his-useless-machines Saul McLeod “Visual Perception Theory.” Visual Perception. N.p., 01 Jan. 1970. Web. http://www.simplypsychology.org/perception- theories.html Spencer Kelly. Can an ‘electronic Lollipop’ Simulate Taste? Web. http://www.bbc.com/news/technology-26487218. Stafford, Tom. “Why Can Smells Unlock Forgotten Memories?” BBC. 13 Mar. 2012. Taylor, Jill Bolte. “My Stroke of Insight.” TED Talk. Feb. 2008. Web. http://www.ted.com/talks/jill_bolte_taylor_s_powerful_stroke_of_insight TeamLab, ”Drawing on the Water Surface Created by the Dance of Koi and People - Infinity”.2016. Web. http://www.team-lab.net/w/ koi_and_people/ Tufekci, Zeynep. “Social Media’s Small, Positive Role in Human Relationships.” The Atlantic, 25 Apr. 2012. Web. Twilley, Nicola. “Accounting for Taste.” The New Yorker. 26 Oct. 2015. Web. “What Is Augmented Reality (AR).” IGI Global: International Publisher of Progressive Academic Research Books and Journals. Web. http:// www.igi-global.com/dictionary/augmented-reality-ar/1859 WORKSCITED