Capturing Different Media1. Sound2. Video3. Photo4. Observation (Field Notes)
Media ModesSound Speech, music, laughter, background noiseVideo Lighting Ambient lighting in homes, daylight Physical presence Posture, facial and bodily gestures, eye contact, pointing Production elements On screen effects, framing of shots, cutting between scenes, etc. Physical objects Placing objects in field of view, gesturing with objects,Photography Similar to video but without the movement/time aspectsObservation Spoken language, physical behaviours, social(field notes) dynamics, printed text/signs, etc.
Multimodal ActivityObserve and record examples of how technologiesinfluence the way that people relate to one anotherUse one of the following media:o Sound Recordingo Observation (Field Notes)l Still Photosl Video Recording
Things to look out for? Reciprocity in communication coordination of turn taking in conversation ‘gift’ exchange Ritualised behaviour openings and closings Defining the ‘meaning’ of physical space What behaviours are allowed/expected/norms Reinforcement of these norms Divided Consciousness/Absent Presence Dealing with immediate and distant social contact simultaneously
Feedback Session What modes have been captured? What different stories do they tell? Were there any issues with collecting/using particular media?
Developing Multimodal Themes Watch the videos/listen to recordings/view images/ Identify themes ◦ Observe and identify significant moments that highlight multimodal interaction as an aspect of communication – note the time ◦ Refer to existing literature on multimodal aspects of communication to check what you see and hear – for example Goffman (1959)3. Watch the videos3 Describe the multimodal aspects of the videost Roughly transcribe the video data to clarify significant moments in communication and identify commonalities across videosd Watch the videose Final transcription in a suitable form to explain themes and subthemes ◦ Include standard annotation conventions
Multimodal transcription of video data(using my own coding scheme)
Multimodal transcription – from conversational analysis: (n) pause noted in seconds = joining of words : an extension of the preceding syllable; ; rising or falling intonation for subsequent utterance underlined text indicates spoken with particular vocal energy - a sudden cut off to an utterance; ? a rising tone . a falling tone , a shifting continuous tone bridging utterances.
ReferencesClark, H. H. (1992). Arenas of language use. Chicago: The University of Chicago Press.Clark, H. H., & Schaefer, E. F. (1989). Contributing to discourse. Cognitive Science, 13, 259–294Kendon, A., Harris, R. M., & Ritchie Key, M. (1975). Organization of behavior in faceto-face interaction. Paris: Mouton Publishers.Kendon, A. (1990). Conducting interaction: Patterns of behavior in focused encounters.Cambridge: Cambridge University Press.Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press.Ten Have, P. (1999). Doing conversation analysis. A practical guide. London: Sage Publications.