Originally run at University of Tartu for Undergraduates and up. Audience: anyone with an interest in the meaning and philosophy behind our interaction with the technological world around us.

1. SEMIOTICS OF DIGITAL
INTERACTIONS
Kyle Davidson – kyle.Davidson@ut.ee

2. "A sign, or representamen, is something which stands to somebody
for something in some respect or capacity. It addresses somebody,
that is, creates in the mind of that person an equivalent sign, or
perhaps a more developed sign. That sign which it creates I call
the interpretant of the first sign. The sign stands for something,
its object. It stands for that object, not in all respects, but in
reference to a sort of idea, which I have sometimes called
the ground of the representamen." (CP 2.228, 1897)

3. APPLE

4. object

6. embodiment

7. The x – various meanings

9. APPLE
Person to person
Machine to machine
Machine to person

12. • MICROSOFT BOB: “A software development platform generates a user
interface that adopts a real-world metaphor. In particular, the user
interface appears to a user as containing rooms of a home”
• BumpTop; “3D desktop user interface inspired by real desks using physics,
multi-touch gestures to drive towards a more expressive, human vision for
computing”
• PreSonus Studio One: “An efficient, single-screen interface [that] houses
an unlimited number of tracks, intuitive editing tools, and advanced virtual
instruments. […] Studio One doesn’t dictate how you work or what you
work on
• Microsoft HoloLens; “the first self-contained, holographic computer,
enabling you to engage with your digital content and interact with
holograms in the world around you”
• IBM Watson IoT and Thomas Jefferson Hospital; “Via a speaker-based
interface, the envisioned smart hospital room can answer questions and
execute requests that are very specific to the context of the user”

15. JURI LOTMAN

16. JURI LOTMAN
• Hierarchical structuralist semiotic approach
• Saw oral language as primordial with written language, art, play and so as
superstructures.
• This has been criticised by Sebeok who highlights gesture in animal
communication. Also Derrida.
• Dynamic
• Tension vs explosion
• The semiosphere is culture – culture is the semiosphere

17. JURI LOTMAN
• Consciousness is impossible without communication. In this sense it can be said that
dialogue precedes language and generates the language. The idea of semiosphere
is based exactly on this: the ensemble of semiotic formations precedes (not
heuristically, but functionally) a single isolated language and is a precondition for its
existence. Without semiosphere a language not only does not work, but does not
even exist. (Lotman 1984: 16)

18. JURI LOTMAN
• Since all the levels of the semiosphere — ranging from a human individual or an
individual text to global semiotic unities — are all like semiospheres inserted into
each other, then each and one of them is both a participant in the dialogue (a part
of the semiosphere) as well as the space of the dialogue (an entire semiosphere).
(Lotman 1984: 22)

19. HIERARCHY OF LANGUAGE

21. HIERARCHY OF LANGUAGE
•An object language is a language which is the
"object" of study in various fields
•Metalanguage is language or symbols used when
language itself is being discussed or examined
• Wikipedia

22. FEEDBACK SYSTEMS
•Explosion of understanding – discrete.
•Tension of understanding – continuous.

23. SEMIOSPHERES
•Social groupings of semiotic understanding
•Core is more rigid in its meaning
•Edges are permeable and allow for translation
•Oral language is the base semiosphere
•Culture – art – are superstructures
•Physical, metaphorical, digital and so on

24. UMWELT
• Semiospheres are interacting umwelten (Kull 1998)
• Defined as the conceptual/semiotic universe of an organism based on the signs, meanings,
and relationships one perceives
• Umwelt theory differs from the black box theory of perception in that the comprehensible
world delimits the external world via a feedback process, rather than input and output being
calculated by the inner, unknowable, consciousness.
• Umwelt theory can be split further into the separate, exterior Umgebung, and the
unmediated Innenwelt of the organism’s inner world.
• Kull writes; “Organisms are themselves creating signs, which become the constituent parts of
the semiosphere. This is not an adaptation to environment, but the creation of a new
environment” (Kull 1998).
• Magnus and Kull (2012) also state that for Lotman, a semiosphere is a precondition of
culture.

25. CYBERNETICS
• The father of umwelt theory (Jacob von Uexküll) is also perhaps the first
cyberneticist.
• Cybernetics is the mechanism of feedback between action and intent – such as
processing the visual data of a ball and catching it (Emmeche 2001). The topics of
cybernetics, umwelt, and semiotics have been compressed and attempted to be
applied to artificial intelligence. While it is obvious that a basic mechanical system
doesn’t have a “self”, the idealised AI as a self-governing automaton has presented
researchers with the question of whether a robot can have an umwelt (Emmeche
2001).

26. MIKHAIL BAKHTIN

27. BAKHTIN
• any element of an utterance that forwards a thought and is being foregrounded, or
even a full utterance is translated by us into corresponding context that is different
and active. Any understanding is dialogic. Understanding is contrasted to utterance
as a speaker’s words are contrasted to those of another speaker in a dialogue.
Understanding is looking for a counterword to the word of a speaker. Only
understanding of a foreign word seeks for “a similar” word in the native language.
(Bakhtin 2000: 436)

28. BAHKTIN AND CARNIVALESQUE
The term “carnival” comes up when reading about avatars and VR – it is worth just considering it
from a semiotic point of view, as a narrative that mocks authority. Consider this with meme too.
Mikhail Bakhtin's four categories of the carnivalesque sense of the world:
• Familiar and free interaction between people: carnival often brought the unlikeliest of people
together and encouraged the interaction and free expression of themselves in unity.
• Eccentric behaviour: unacceptable behaviour is welcomed and accepted in carnival, and one's
natural behaviour can be revealed without the consequences.
• Carnivalistic mésalliances: familiar and free format of carnival allows everything that may normally
be separated to reunite — Heaven and Hell, the young and the old, etc.
• Profanation: in carnival, the strict rules of piety and respect for official notions of the 'sacred' are
stripped of their power— blasphemy, obscenity, debasings, 'bringings down to earth', celebration
rather than condemnation of the earthly and body-based.
• https://en.wikipedia.org/wiki/Carnivalesque

29. APPLYING LOTMAN TO DIGITAL
INTERACTIONS

30. SEMIOSPHERES OF SOCIAL MEDIA
•Mass media communication
•Fake news and political semiosphere
•International context
•Age, gender, location etc
•Communication online via jargon and esoteric
language

31. SEMIOSPHERES OF VR/AVATARS
• The communication passes between two virtual entities,
controlling their avatars. Thus, the message is directed
to the avatar, and received from the avatar, mediated,
and then reaches the user.
• Consider the use of gesture, oral language, and flexible
identity and communication.
• Different planes of reality crossing over a single
semiosphere of communication. What happens to the
message?

32. SUMMARY OF SEMIOTICS OF
DIGITAL INTERACTIONS
• Semiotics is about modelling the transmission of information.
• Semiospheres demonstrate a model of translation between medias and
cultures. Umwelt represent the individual interpretation.
• The “digital interaction” is anything that passes information via – or to – a
digital (0/1) interface.
• How and where one interacts with these interfaces affects the interaction,
both individually and socially.
• How culture and language is altered by the interface is key. Translation
between media, cultures, and languages.
• NB – the avatar as another interface in the communication model mediates
the message.

33. NEXT WEEK
• “Culture and semiotics”
• We will look at Barthes metadisciplinary approach to semiotics and culture.
• We will also look at Eco’s concept of the sign.
• READ: Mythologies – Roland Barthes (chapter on the Dominici trial) and the Dick
Hebdige chapter from last week.
• READ: http://www.angelfire.com/md2/timewarp/eco.html,
http://www.signosemio.com/hjelmslev/semiotic-hierarchy.asp,
http://www.signosemio.com/eco/modes-of-sign-production.asp,
http://www.signosemio.com/eco/semiotic-process-and-classification-of-signs.asp