Visualizing Voice audio as social catalyst Karrie Karahalios UIUC 2009

audio as social catalyst

What attracts people most is other people. - William H. Whyte

Social Catalysts Triangulation : This is the process by which some external stimulus provides a linkage between people and prompts strangers to talk to each other as if they were not.” -William H. Whyte

Triangulation : This is the process by which some external stimulus provides a linkage between people and prompts strangers to talk to each other as if they were not.”

-William H. Whyte

Goal: the design of augmented social spaces for people Challenge: 1. explore which social cues to transmit between mediated spaces 2. sense cues (gaze, agreement, etc.) 3. visualize social patterns, feedback 4. incorporate cues and media space into catalyst for interaction

Goal: the design of augmented social spaces for people Challenge: 1. explore which social cues to transmit between mediated spaces 2. sense cues (gaze, agreement, etc.) 3. visualize social patterns, feedback 4. incorporate cues and media space into catalyst for interaction cue variety feedback message personalization simultaneously being reminded of a need to talk to someone having a communication channel turn-taking, repair, stylized openings Beyond Being There Hollan and Stornetta

cue variety

feedback

message personalization

simultaneously being reminded of a need to talk to someone

having a communication channel

turn-taking, repair, stylized openings

Beyond Being There Hollan and Stornetta

Goal: the design of augmented social spaces for people Challenge: 1. explore which social cues to transmit between mediated spaces 2. sense cues (gaze, agreement, etc.) 3. visualize social patterns, feedback 4. incorporate cues and media space into catalyst for interaction

Telemurals Karrie Karahalios, 2002-3

Hole in Space, 1980 Kit Galloway and Sherrie Rabinowitz

Microsoft, 2001 Virtual Kitchen

When we abstract an image through cartooning, we’re not so much eliminating details as we are focusing on specific details. By stripping down an image to its essential “meaning”, an artist can amplify that meaning in a way that realistic art can’t. Scott McCloud, Understanding Comics

Chit Chat Club Karrie Karahalios & Kelly Dobson Spring 2001

Karrie Karahalios & Fernanda Viegas Spring 1999 Visiphone

Siggraph EmergingTechnologies, 1999

Interaction and Wearability

Conversation Clock

each color represents different person length of bar represents volume dots represent moments of silence

each color represents different person

length of bar represents volume

dots represent moments of silence

turn-taking interruptions time-span moments of silence conversational dominance rhythm and flow of conversation

turn-taking

interruptions

time-span

moments of silence

conversational dominance

rhythm and flow of conversation

Turn Taking Interruption

Turn Taking

Interruption

Conversational Dominance Silence

Conversational Dominance

Silence

Agreement Aural Back-channels

Agreement

Aural Back-channels

Mimicry Rhythm and Flow Time Spans

Mimicry

Rhythm and Flow

Time Spans

Everyone sees the same thing Reveal Patterns that are not apparent otherwise. Social Mirror

Everyone sees the same thing

Reveal Patterns that are not apparent otherwise.

Self Reflection Third person viewpoint Realtime Feedback Social Mirror

Self Reflection

Third person viewpoint

Realtime Feedback

Persistent History Cues occur over time Structured and Interpretable at a glance Social Mirror

Persistent History

Cues occur over time

Structured and Interpretable at a glance

Context of Conversation Evocative artifact for participants Varied interpretations Social Mirror

Context of Conversation

Evocative artifact for participants

Varied interpretations

Learned in Pilot speakers try to ignore the visualization listeners contemplate

speakers try to ignore the visualization

listeners contemplate

Study Methodology food movies class project academic papers music Familiar Groups and Topics

food

movies

class project

academic papers

music

Measurements turns length of speech leads glances gestures surveys interviews

turns

length of speech

leads

glances

gestures

surveys

interviews

Qualitative Feedback “ It was easy to judge who is driving conversation.” “ I was trying to look at the circle to see whether we were balanced” “ I realized that I could monitor my speech patterns by watching the colors. It was interesting to train myself not to say ‘umm’ as much or pause.”

“ It was easy to judge who is driving conversation.”

“ I was trying to look at the circle to see whether we were balanced”

“ I realized that I could monitor my speech patterns by watching the colors. It was interesting to train myself not to say ‘umm’ as much or pause.”

Qualitative Feedback “ I noticed when you’re the one talking, you want to stop. But if you’re mid-topic you couldn’t stop, because you had to finish your topic. But as soon as you finished your topic you’d shut up.” “ It became all red; should green or yellow speak next?” -Yellow

“ I noticed when you’re the one talking, you want to stop. But if you’re mid-topic you couldn’t stop, because you had to finish your topic. But as soon as you finished your topic you’d shut up.”

“ It became all red; should green or yellow speak next?” -Yellow

Summary Participants found the visualization to be revealing of their interaction People glance, not focus, on visualization Above and Below average participants react differently to the same visualization Participants showed extensive interest in themselves

Participants found the visualization to be revealing of their interaction

People glance, not focus, on visualization

Above and Below average participants react differently to the same visualization

Participants showed extensive interest in themselves

Interpret participant roles in interaction

Ongoing Work As a public display, what does it convey to others How can participants augment the mirror with their knowledge of context? Develop into an archival tool Language discourse Encouraging turn-taking of children with autistic spectrum disorder (ASD)

As a public display, what does it convey to others

How can participants augment the mirror with their knowledge of context?

Develop into an archival tool

Language discourse

Encouraging turn-taking of children with autistic spectrum disorder (ASD)

Group Awareness “ I could get a visual grasp of argument/conversation successes (i.e. winning others over).” “ [I would] check if others were agreeing with the point presented, not necessarily by me.”

“ I could get a visual grasp of argument/conversation successes (i.e. winning others over).”

“ [I would] check if others were agreeing with the point presented, not necessarily by me.”

Change in participation Feeling Represented

Voting Results Opened a back-channel for all Social Mirror Karma Balanced Conversations Awareness of Self and Others Voter Vocabulary

Opened a back-channel for all

Social Mirror Karma

Balanced Conversations

Awareness of Self and Others

Voter Vocabulary

Topic Table

Claude Fischer. America Calling: A Social History of the Telephone to 1940.