AVOL: Towards an Integrated Audiovisual ExpressionISEAIstanbul, 14.9.2011Nuno N. CorreiaAalto University, Media Lab Helsinki
Introduction• AVOL (AudioVisual Online) is an interactive audiovisual project for the Web,• by Portuguese new media artists Video Jack (Nuno N. Correia and André Carrilho).• It aims to allow for an integrated musical and visual expression, in a way that is playful to use and engaging to experience.• It was released in December 2007, and further developed until 2010.• Link: http://www.videojackstudios.com/avol/
Introduction• AVOL is one of the 4 projects chosen by the Portuguese Ministry of Culture for their net art portal.• It follows upon previous work by Video Jack, such as Heat Seeker (2006).• In previous Video Jack projects, the sound element was manipulated separately from the visual element.• In AVOL, Video Jack aimed to integrate the two elements – audio and image – under the same application and the same interface.
Introduction – Presentations• In 2008, AVOL was presented as installation at the following festivals: – Cartes Flux, Espoo; – Re-New, Copenhagen; – Create, London; – Live Herring, Jyväskylä.• And as performance (also in 2008) at these festivals: – Abertura Festival, Lisbon; – Electro-Mechanica Festival, St. Petersburg.
Introduction – Recent Developments• In 2010, 3 of the 28 audio loops were updated.• Also in 2010, video captures of AVOL were uploaded to Video Jack video websites (YouTube and Vimeo), and embedded on the Video Jack website.• AVOL music was also composed and embedded.• These additional media elements are meant to provide complementary ways for users to experience the project, and also to understand its possibilities.• http://www.videojackstudios.com/projects/avol/
Introduction – Contextualization• AVOL can be contextualized with other works that have explored integrated audiovisual expression.• In the early 20th century, Oskar Fischinger and Walther Ruttman created visual music films in Germany – a combination of tinted animation with live music (Moritz 1997).• John Whitney is “widely considered ‘the father of computer graphics’” for his explorations of computer- generated manipulation of visuals through mathematical functions (Paul 2003, p. 15).
Introduction – Contextualization• Golan Levin is one of the artists that have explored audiovisual creative expression, in works such as Audiovisual Environment Suite (AVES) (1998-2000), – “an interactive software that allows for the creation and manipulation of simultaneous visuals and sound in real time” (Paul 2003, p. 133).• In 2006, Nintendo released Electroplankton, by artist Toshio Iwai, – a series of aquatic worlds “filled with different species of plankton that can produce sound and light when you interact with them” (Davis 2006).
Design – IAVOs• Visuals and music, together with the graphical user interface (GUI) that controls them, are grouped in 7 IAVOs (Interactive AudioVisual Objects).• Each IAVO contains 4 audiovisual options (“sets”), allowing users to create diversified combinations of music and visuals.• The visuals consist of abstract concentric vector animations that are audio-reactive.• The reactivity to each sound is based on the scaling of the respective animation proportionally to the sound amplitude.
Design – Sound• The sound loops were grouped into coherent entities as much as possible, similarly to band members on a stage.• Four of the loops are rhythmical (bass drum, snare drum, hats and clicks) and the remaining three melodic (keyboard, guitar, strings).• All loops have same tempo (120 bpm) and the same duration (16 seconds).• They were composed with the purpose of being coherent and harmonious independently of the active combination of sounds.
Design – Graphics• Each of the 4 “sets” of animations has its color palette and type of shape.• There was an interpretative differentiation according to the nature of the corresponding sound.• Despite these nuances, the animations were designed to be harmoniously interchangeable.• The use of vector graphics ensures that the animations are scalable and react fluidly to sound.• The animations, although abstract, suggest natural shapes.
Design – Interaction• The four white petal-shaped buttons trigger each of the four loops associated with the IAVO.• When users activate one of the loops, new elements appear on the IAVO’s interface: – three “traffic light” buttons (stop, auto and solo), – and a draggable “ring” encompassing the “petal” buttons, – Incorporating minus and plus volume buttons.• The “petal” aesthetics of IAVO buttons is meant to be harmonious with the animations, which also resemble flowers.
Design – Comparisons• The animations in AVOL resemble John Whitney’s floral compositions: – “all colors move into the ring simultaneously from all sides, forming circles within circles all scintillating smoothly in a floral configuration” (Youngblood 1970, p. 220).• There is also some resemblance between AVOL’s flower-like objects and the plankton in Electroplankton, even more apparent when collisions occur.
Evaluation – Initial Conclusions• The author considers that AVOL was successful in terms of playfulness and engagement, due to: – the IAVO approach – entities composed of UI elements controlling sound and audio-reactive animations; – the harmony, coherence and mutual agreement of the audiovisual content.
Evaluation – Initial Conclusions• However, the author detects several shortcomings in AVOL: – a limited amount of sounds and animations to interact with; – few audio manipulation options.• These shortcomings limit the versatility and expressiveness of AVOL.• A new project was released in 2010 to address these issues: AV Clash (http://www.avclash.com).
Evaluation – AV Clash• AV Clash connects to Freesound.org, retrieving nearly 240 sounds from the online database.• It contains a larger number of animations (96).• Audio manipulation capabilities include audio effects (echo and filter) and sound trimming.• The sound character is substantially different.• But there are several similarities between both, mainly in terms of visuals and interaction design• The number of IAVOs in AV Clash was reduced to 4.
Evaluation – Questionnaire• In the first half of 2011, an online questionnaire was setup to evaluate AV Clash.• The questionnaire contained closed and open-ended questions, and was answered by 22 anonymous respondents.• It included questions comparing AV Clash with AVOL.
Evaluation Did you spend more time interacting with AV Clash or with AVOL? 1 5 AV Clash AVOL Approximately the same time 16
Evaluation• Why did you spend more time interacting with AV Clash than AVOL? [16 users; multiple choice question] – 73% : larger amount of manipulation options; – 40% : larger amount of content; – 47% : other.
Evaluation Did you find the possibility of accessing more sounds and visuals in AV Clash appealing, compared with AVOL? 9 8 7 6 5 4 3 2 1 0 1-Not 2 3 4 5-Very appealing at appealing all
Evaluation Comparing to AVOL, are the additional audio manipulation options in AV Clash interesting? 9 8 7 6 5 4 3 2 1 0 1-Not 2 3 4 5-Very interesting at interesting all
Evaluation Comparing the different sound and music approaches of AV Clash and AVOL, which one contributes to a more enjoyable experience? 7 6 5 4 3 2 1 0 1-AVOL, much 2 3 4 5-AV Clash, more much more
Evaluation• Why did you prefer the sound and music approach of AVOL? [7 users; open-ended question] – synchronization of loops; – curation of sounds; – inclusion of percussive elements.
Evaluation Which is easier / more intuitive to use, AV Clash or AVOL? 6 AV Clash 8 AVOL They are on the same level regarding intuitiveness 8
Evaluation• Why is AVOL easier to use? [8 users; open-ended question] – 6 users mention the simpler interface and fewer options.
Evaluation Which project gives you a higher feeling of creativity? AV Clash 2 AVOL 5 They both gave 13 me the same feeling of creativity 2 I didnt get any feeling of creativity from either
Evaluation• Why does AV Clash give you a higher feeling of creativity? [13 users; open-ended question] – 7 mention more manipulation options, – 1 mentions more variety in sound.
Evaluation• Why does AVOL give you a higher feeling of creativity? [2 users; open-ended question] – One user mentions that the sounds "fit together nicely", and that switching between them created interesting results.
Evaluation• Why did you get the same / no feeling of creativity from either? [7 users; open-ended question] – One user considers that both projects shape the sound and visuals too much to allow for his/her own creativity, – and another mentions that the projects are "too structured".
Conclusions• The results show that users value the additional content and functionalities of AV Clash.• These additions contribute to a greater engagement than its predecessor.• However, extra manipulation options and added content come at a cost in terms of usability.• Moreover, AVOL comes close to AV Clash in terms of sonic enjoyment.
Conclusions• Therefore, even though the majority of respondents favor AV Clash,• there seems to be a significant number of users who prefer a simpler project such as AVOL,• with fewer options and content,• but with curated and more harmonious sounds.• A possible future path could concentrate more on playfulness and intuitiveness, targeting these users.
Conclusions• Some of the users are dissatisfied with the creative and expressive potential of both projects, – considering that they are "too structured" and shape the end result too much.• Further developments in this line of projects could address these limitations, – expanding the manipulation options, content, and customization possibilities.
Conclusions• Personal views on the future of related projects:• In future developments, emphasis should be given to recording and sharing capabilities. – Net art audiovisual projects should take advantage of their online presence to facilitate the recording and distribution of user-generated content, namely by leveraging social media.• Multi-touch mobile devices are attractive for future interactive audiovisual projects, – since they allow for a more direct and flexible manipulation of visuals than a pointing device interface.
References• Davis, R., 2006. Electroplankton Review. GameSpot, January. http://www.gamespot.com/ds/puzzle/electroplankton/ review.html, referenced 22/08/2011.• Moritz, W., 1997. The Dream of Color Music and the Machines that Made it Possible. In Animation World Magazine, Apr. 1997. http://www.awn.com/mag/issue2.1/ articles/moritz2.1.html, referenced 22/08/2011.• Paul, C., 2003. Digital Art. Thames & Hudson, London.• Youngblood, G., 1970. Expanded Cinema. P. Dutton & Co., New York. http://www.vasulka.org/Kitchen/ PDF_ExpandedCinema/ExpandedCinema.html, referenced 22/08/2011.