Notes on: Visual Music, Audio-Vision, Synaesthetic Cinema, Audiovisual Composition"Nuno N. Correia, firstname.lastname@example.org
Visual Music"• When deﬁning visual music, Ox and Keefer (2008)distinguish between four “differently formed visualstructures”:"– A visualization of music, which translates sound into visuals,“with the original syntax being emulated in the new visualrendition”. According to Ox and Keefer, this can also bedeﬁned as intermedia."– A time-based visual composition, which is similar to thestructure of a kind or style of music – “as if it were an auralpiece”. It can have sound, or exist silent."– A direct translation of image to sound – “literally, what you seeis also what you hear”. Some of Norman McLaren’s works(where scratchings on ﬁlm produce simultaneously image andsound) ﬁt in this category."– A static visual composition, “as in Klee”."
Audiovisual Contract"• Sound does not correspond "naturally" to an image."!• Audiovisual contract:"– "a kind of symbolic contract that the audio-viewerenters into, agreeing to think of sound and imageas forming a single entity" (Chion 1994, p.216).""
Added Value"• Added value: "– "the expressive and informative value with whicha sound enriches a given image "– so as to create the deﬁnitive impression, in theimmediate or remembered experience one hasof it, "– that this information or expression naturallycomes from what is seen, "– and is already contained in the imageitself" (Chion 1994, p.5)."
Added Value"• Added value works reciprocally: "– on the one hand, "sound shows us the image differentlythan what the image shows alone"; "– on the other hand, image "makes us hear sounddifferently than if the sound were ringing out in thedark" (Chion 1994, p.21)."• Added value is the most important of the relationsbetween sound and image (Chion 1994, p.5). "
Synchresis"• Based on the idea of synchronization (and alsosynthesis), Chion created the notion of synchresis:"– "the forging of an immediate and necessaryrelationship between something one sees andsomething one hears at the same time" (Chion1994, p.224)."
Synchresis"• According to Chion, synchresis allows for numerouscombinations of possible sounds with possibleimages: "for a shot of a hammer, any one of ahundred sounds will do" (Chion 1994, p.63)."• But random associations may not generatesynchresis: "play a stream of random audio andvisual events, and you will ﬁnd that certain ones willcome together through synchresis and othercombinations will not" (Chion 1994, p.63)"
Senses as Channels"• Chion states that there is no "sensory given" that isisolated from the start: "the senses are channels,highways more than territories or domains”."• He clariﬁes this, stating that "when Kineticsensations organized into art are transmittedthrough a single sensory channel", they can conveyall the other senses via that one channel. (Chion1994, p.137). "• He exempliﬁes with the inherent visuality of concretemusic, and the implied sound behind silent movies. "
Expanded Cinema"• Expanded cinema has been expanding for a longtime. "• Since it left the underground and became a popularavant-garde form in the late 1950s the new cinemaprimarily has been an exercise in technique, "– the gradual development of a truly cinematic languagewith which to expand further mans communicative powersand thus his awareness. "• If expanded cinema has had anything to say, themessage has been the medium."• Youngblood 1970, p. 75"
Expanded Cinema"• Slavko Vorkapich: "– "Most of the ﬁlms made so far are examples not ofcreative use of motion-picture devices and techniques, butof their use as recording instruments only.” "– “There are extremely few motion pictures that may becited as instances of creative use of the medium, "– and from these only fragments and short passages maybe compared to the best achievements in the other arts”."• Youngblood 1970, p. 75"
Synaesthetic Cinema"• The new cinema has emerged as the only aestheticlanguage to match the environment in which we live."• Emerging with it is a major paradigm: a conceptionof the nature of cinema so encompassing andpersuasive that it promises to dominate all image-making in much the same way as the theory ofgeneral relativity dominates all physics today. I call itsynaesthetic cinema."• Youngblood 1970, p. 76"
Synaesthetic Cinema"• The new artist and the new scientist recognize thatchaos is order on another level, and they set aboutto ﬁnd the rules of structuring by which nature hasachieved it. "• Thats why the scientist has abandoned absolutesand the ﬁlmmaker has abandoned montage."• Youngblood 1970, p. 76"
Synaesthetic Cinema"• Synaesthetic cinema is the only aesthetic languagesuited to the post-industrial, post-literate, man-madeenvironment with its multi- dimensional simulsensorynetwork of information sources. "• Its the only aesthetic tool that even approaches thereality continuum of conscious existence in thenonuniform, nonlinear, nonconnected electronicatmosphere of the Paleocybernetic Age."• Youngblood 1970, p. 77"
Audiovisual Composition"• Conventionally, audiovisual composition may betaken to mean the process of putting sound andmusic to visual material in order to create asoundtrack to an existing ﬁlm or animation. "• The study of this type of practice is often referred toas sound on ﬁlm, audio for video or sound design. "• The theoretical debates which surround this practiceare informed in particular by Michel Chions Audio-Vision:Sound on Screen."• Grierson 2005, p. 9"
Audiovisual Composition"• However, Chion also introduces concepts whichpoint to a more complex type of audiovisual practice.With this in mind, it is illuminating that Chion takesexception to the idea of the soundtrack. Accordingto Chion, there is no such thing as a soundtrack."• Grierson 2005, p. 9"• Chions justiﬁcation for this view is bound up with hisnotion of added value. This is, in essence, the ideathat through the combination of audio and visualelements, a third audiovisual element is generated."• Grierson 2005, p. 10"
Audiovisual Composition"• Therefore, the audiovisual work is more than acombination of its component parts. "• As such, the practice of audiovisual composition isnot simply the production of audio with video. "• It is the process of composing audiovisual workswhich exploit added value."• Grierson 2005, p. 10""
Audiovisual Composition"• So in audiovisual composition and analysis, addedvalue shifts the focus from separate audio and visualcomponents, to the relationship between audio andvisual components. "• This is particularly the case where a work has beendesigned from the outset with great attention todetail regarding the composition of both the sonicand visual material and their effect together, "• and even more so if the work attempts to exploitideas about the formal relationships between sonicand visual material."• Grierson 2005, p. 10"
Bibliography"• Chion, M., 1994. Audio-Vision: Sound on Screen, NewYork: Columbia University Press."• Grierson, M., 2005. Audiovisual Composition. Kent:University of Kent. Available at: http://www.strangeloop.co.uk/Dr.%20M.Grierson%20-%20Audiovisual%20Composition%20Thesis.pdf."• Ox, J. & Keefer, C., 2008. On Curating Recent DigitalAbstract Visual Music. In Abstract Visual Music. TheNew York Digital Salon. Available at: http://www.centerforvisualmusic.org/Ox_Keefer_VM.htm."• Youngblood, G., 1970. Expanded Cinema, New York: P.Dutton & Co. Available at: http://www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html."!