Second Life, Second Body Nonverbal Communication in Multiuser, 3D Virtual Environments Smiljana Antonijevi ć Virtual Knowledge Studio for the Humanities and Social Sciences BEST, Nis July 22, 2008

Overview Why Second Life ? Why nonverbal communication? Computer-mediated nonverbal communication orientation of correspondence orientation of autonomy Second Life, Second Body digital nonverbal act as: communicative device; socio-cultural phenomenon; new media object six-month long observational study structural and functional analyses digital nonverbal act: an epistemic tool juxtaposed with users’ agency; computer-mediated nonverbal behavior: a field of tension between the User discourse and the Designer discourse.

Why Second Life ?

Why nonverbal communication?

Computer-mediated nonverbal communication

orientation of correspondence

orientation of autonomy

Second Life, Second Body

digital nonverbal act as: communicative device; socio-cultural phenomenon; new media object

six-month long observational study

structural and functional analyses

digital nonverbal act: an epistemic tool juxtaposed with users’ agency; computer-mediated nonverbal behavior: a field of tension between the User discourse and the Designer discourse.

Why Second Life? Internet-based, multiuser virtual environment Launched in 2003; 13 million accounts (March, 2008) Capitol Hill, BBC, Harvard, IBM, NASA, Toyota, The Second Louvre Museum, Barack Obama … The future of online communication

Internet-based, multiuser virtual environment

Launched in 2003; 13 million accounts (March, 2008)

Capitol Hill, BBC, Harvard, IBM, NASA, Toyota, The Second Louvre Museum, Barack Obama …

The future of online communication

Why Nonverbal Communication? “… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3). Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) … The first communicative ability a person develops. Two thirds of behavior in dyadic interaction.

“… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3).

Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) …

The first communicative ability a person develops.

Two thirds of behavior in dyadic interaction.

Gesture

Facial Expressions

Facial Expressions

Artifacts

Dress

Interpersonal Distance

Why Nonverbal Communication? “… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3). Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) … The first communicative ability a person develops. Two thirds of behavior in dyadic interaction.

“… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3).

Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) …

The first communicative ability a person develops.

Two thirds of behavior in dyadic interaction.

Why Nonverbal Communication? “… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3). Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) … The first communicative ability a person develops. Two thirds of behavior in dyadic interaction.

“… all of the ways in which communication is effected between persons when in each other’s presence, by means other than words.” (Kendon, 1981:3).

Communicative use of: body ( kinescis ), space ( proxemics ) , artifacts ( objectics ) , touch ( haptics ) , voice ( vocalics ) …

The first communicative ability a person develops.

Two thirds of behavior in dyadic interaction.

Digital Nonverbal Communication Wiener (1948), parallels between electrical communication systems and human nervous system, perception, and motion; cybernetics: communicative rather than expressive aspects of nonverbal behavior. Computer Sciences; Internet Studies; Social Sciences; Communication Studies. The orientation of correspondence aims to identify the corresponding patterns in the perception and use of physical and digital nonverbal cues. The orientation of autonomy aspires to recognize the distinctiveness of digital nonverbal behavior.

Wiener (1948), parallels between electrical communication systems and human nervous system, perception, and motion; cybernetics: communicative rather than expressive aspects of nonverbal behavior.

Computer Sciences; Internet Studies; Social Sciences; Communication Studies.

The orientation of correspondence

aims to identify the corresponding patterns in the perception and use of physical and digital nonverbal cues.

The orientation of autonomy

aspires to recognize the distinctiveness of digital nonverbal behavior.

The Orientation of Correspondence Kiesler et al . (1984); cues-filtered-out approach Bates (1994); a believable agent – a believable illusion of life. Guye-Vuilleme et al. (1999); Virtual Life Network -- mimetic system design, tries to represent social cues from the physical world, as literally as possible, in the digital domain. Fabri et al. (2002) ; nonverbal abilities of human-like agents the essence of their capacity for behavioral resemblance. Allbeck and Balder (2002) ; PARSYS program – generates realistic movement through dynamic simulation. Yee et al. (2007); people behave according to the same social rules in both physical and virtual worlds; it is possible to study social interaction in virtual environments and generalize about social interaction in the “real world”.

Kiesler et al . (1984); cues-filtered-out approach

Bates (1994); a believable agent – a believable illusion of life.

Guye-Vuilleme et al. (1999); Virtual Life Network -- mimetic system design, tries to represent social cues from the physical world, as literally as possible, in the digital domain.

Fabri et al. (2002) ; nonverbal abilities of human-like agents the essence of their capacity for behavioral resemblance.

Allbeck and Balder (2002) ; PARSYS program – generates realistic movement through dynamic simulation.

Yee et al. (2007); people behave according to the same social rules in both physical and virtual worlds; it is possible to study social interaction in virtual environments and generalize about social interaction in the “real world”.

The Orientation of Autonomy Haraway (1991); the fusion between the human body and technological devices . Emoticons ( Rivera et al. 1996; Walther et al. 2001; Riva 2002 ). Erickson and Kellog (2000); abstract design approach; digital representation of nonverbal cues that are not closely tied to their physical analogs. Talamo and Ligorio (2000); the construction of identity in “Euroland”. Walther et al. (2005); social information processing theory. Munster (2006); to think through what the experience of having a body means in information cultur e, our understanding of the body must be reconsidered.

Haraway (1991); the fusion between the human body and technological devices .

Emoticons ( Rivera et al. 1996; Walther et al. 2001; Riva 2002 ).

Erickson and Kellog (2000); abstract design approach; digital representation of nonverbal cues that are not closely tied to their physical analogs.

Talamo and Ligorio (2000); the construction of identity in “Euroland”.

Walther et al. (2005); social information processing theory.

Munster (2006); to think through what the experience of having a body means in information cultur e, our understanding of the body must be reconsidered.

Communicative Device Digital Nonverbal Act Cultural Phenomena New Media Object

Communicative Device

Digital Nonverbal Act

Cultural Phenomena New Media Object

Digital Nonverbal Act as a Communicative Device Origin : the way in which a certain nonverbal act has become part of a person’s nonverbal repertoire. Coding : the principle of correspondence between the act and its meaning. Usage : circumstances under which a nonverbal act occurs. Functions : communicative functions of nonverbal cues in human interaction.

Origin : the way in which a certain nonverbal act has become part of a person’s nonverbal repertoire.

Coding : the principle of correspondence between the act and its meaning.

Usage : circumstances under which a nonverbal act occurs.

Functions : communicative functions of nonverbal cues in human interaction.

Digital Nonverbal Act as a Cultural Phenomenon Marcel Mauss , Techniques of the Body Not: natural; culturally universal; individual. Learned through “prestigious imitation”; shaped and legitimized by the social authority. Mary Douglas, The Physical and the Social Body Consonance among the semiotic systems; the use of the body synchronized with other means of expression. The forms of bodily control correspond to the forms of societal control; both types of control stem from the same cultural premises. Pierre Bourdieu, Habitus and Body Hexis Body hexis the basis of habitus; functions beyond the oral and/or the written discourse. Theory of practice; a scientific model of practice can never account for all the instances that can and do happen in practice; a practical activity never actually has the form represented in a scientific model.

Marcel Mauss , Techniques of the Body

Not: natural; culturally universal; individual.

Learned through “prestigious imitation”; shaped and legitimized by the social authority.

Mary Douglas, The Physical and the Social Body

Consonance among the semiotic systems; the use of the body synchronized with other means of expression.

The forms of bodily control correspond to the forms of societal control; both types of control stem from the same cultural premises.

Pierre Bourdieu, Habitus and Body Hexis

Body hexis the basis of habitus; functions beyond the oral and/or the written discourse.

Theory of practice; a scientific model of practice can never account for all the instances that can and do happen in practice; a practical activity never actually has the form represented in a scientific model.

Digital Nonverbal Act as a New Media Object Lev Manovich (2001); the principles of New Media objects numerical representation modularity automation variability transcoding Clarisse Sieckenius de Souza (2005); Semiotic Engineering human-computer interaction as a process of continuous meaning-exchange between the new media object designers and the new media objects users new media objects as intellectual and metacommunicative artifacts the design structure of new media objects determines both the available and the appropriate forms of users’ interaction

Lev Manovich (2001); the principles of New Media objects

numerical representation

modularity

automation

variability

transcoding

Clarisse Sieckenius de Souza (2005); Semiotic Engineering

human-computer interaction as a process of continuous meaning-exchange between the new media object designers and the new media objects users

new media objects as intellectual and metacommunicative artifacts

the design structure of new media objects determines both the available and the appropriate forms of users’ interaction

Methodology 6 months long ethnographic study (October 2006-March 2007) 108 Second Life locations selected in a non-structured manner Short behavioral episodes 843 episodes of naturally occurring user interaction Blaze Media Pro software Analysis focused on proxemic and kinesic cues (interpersonal distance, body orientation, posture, gesture)

6 months long ethnographic study (October 2006-March 2007)

108 Second Life locations selected in a non-structured manner

Short behavioral episodes

843 episodes of naturally occurring user interaction

Blaze Media Pro software

Analysis focused on proxemic and kinesic cues (interpersonal distance, body orientation, posture, gesture)

Structural Analysis Classification of nonverbal cues in Second Life User-Defined Cues Predefined Cues Blended Cues Missing Cues

Classification of nonverbal cues in Second Life

User-Defined Cues

Predefined Cues

Blended Cues

Missing Cues

User-Defined Cues The user deliberately performs and individually encodes a nonverbal act. Proxemic cues (interpersonal distance, body orientation). Important role: communicating interactional intent; structuring interaction; sending relational messages. Closely related with co-occurring textual discourse. Not significantly correlated with the users’ physical appearance (human or other) and gender, or with the communicative context.

The user deliberately performs and individually encodes a nonverbal act.

Proxemic cues (interpersonal distance, body orientation).

Important role: communicating interactional intent; structuring interaction; sending relational messages.

Closely related with co-occurring textual discourse.

Not significantly correlated with the users’ physical appearance (human or other) and gender, or with the communicative context.

Signaling interactional intent

Signaling interactional intent

Signaling interactional intent Can I steal a dance?

Signaling interactional intent

Signaling interactional intent

Signaling interactional intent

Signaling interactional intent Hi Lucy

Signaling interactional intent

Signaling interactional intent

Signaling interactional intent Hi

User-defined cues: sending relational messages

User-defined cues: sending relational messages

User-defined cues: sending relational messages

User-defined cues: sending relational messages

User-defined cues: sending relational messages

Predefined Cues System generated and encoded nonverbal acts. Kinesic cues (hand movement, gaze, posture). Indicate communicative activity; mimic interactional synchrony. Unrelated with co-occurring textual discourse. Not related with the users’ physical appearance (human or other) and gender, or with the communicative context.

System generated and encoded nonverbal acts.

Kinesic cues (hand movement, gaze, posture).

Indicate communicative activity; mimic interactional synchrony.

Unrelated with co-occurring textual discourse.

Not related with the users’ physical appearance (human or other) and gender, or with the communicative context.

Predefined cues: indicating communicative activity

Predefined cues: indicating communicative activity

Predefined cues: indicating communicative activity

Predefined cues: mimicking interactional synchrony

Predefined cues: mimicking interactional synchrony

Blended Cues The user deliberately performs a nonverbal act but does not encode it. Kinesic and proxemic cues. Simulate various communicative functions. Highly context and gender dependent. Often in collision with co-occurring textual discourse.

The user deliberately performs a nonverbal act but does not encode it.

Kinesic and proxemic cues.

Simulate various communicative functions.

Highly context and gender dependent.

Often in collision with co-occurring textual discourse.

Hi, V

Hi

How are you?

Please May I?

Digital nonverbal act: an epistemic tool juxtaposed with user agency. Double objectification of digital nonverbal behavior; a nonverbal act becomes a new media object . Digital nonverbal behavior: field of tension between the User discourse and the Design discourse. User discourse: the virtual body augments the user’s physical body and acts in a way called for by a particular interactional situation. Design discourse: the virtual body is acted upon in a way called for by the system epistemology and pragmatics. T he design discourse is detached from actual practice of user interaction and linked to abstract, precoded practice built into the system . U ser discourse realized through users’ interaction with current capacities and limitations of the system ; confluence between technology and the user.

Digital nonverbal act: an epistemic tool juxtaposed with user agency.

Double objectification of digital nonverbal behavior; a nonverbal act becomes a new media object .

Digital nonverbal behavior: field of tension between the User discourse and the Design discourse.

User discourse: the virtual body augments the user’s physical body and acts in a way called for by a particular interactional situation. Design discourse: the virtual body is acted upon in a way called for by the system epistemology and pragmatics.

T he design discourse is detached from actual practice of user interaction and linked to abstract, precoded practice built into the system .

U ser discourse realized through users’ interaction with current capacities and limitations of the system ; confluence between technology and the user.

Questions? Smiljana Antonijevic [email_address]

Questions?

Smiljana Antonijevic

[email_address]