AV CLASH – Online Audiovisual ProjectA Case Study of Evaluation in New Media Art
What is AV Clash?• AV Clash (http://www.avclash.com) is a Web-based artistic project by Video Jack – Nuno N. Correia and André Carrilho, with Gokce Taskan• that allows the creation of audiovisual compositions, consisting of combinations of sound and animation loops.• The sounds are retrieved from Freesound.org.• Visualization and UI are integrated into Interactive AudioVisual Objects (IAVOs).• AV Clash was released in November 2010.
IAVO – “Front”
IAVO – “Back”
The Questionnaire• Soon after the project was released, an evaluation of the project was planned, based on an online questionnaire.• An online questionnaire seemed to be an adequate format for evaluation, given that the project is Web- based, and the audience of the project is global.
The Questionnaire• The questionnaire was designed in order to assess if AV Clash fulfills its objectives, as defined in the research question it addresses:• how to create a tool for integrated audiovisual creativity, with customizable content, that is flexible, intuitive, playful to use and engaging to observe?
The Questionnaire• To address the issues of creativity, playfulness and experience, and also the overall idea of engagement,• recent texts regarding experience-focused approaches to human-computer interaction (HCI) were important references.• The notion of aesthetic interaction also influenced the design of the questionnaire.
The Questionnaire• Different authors have identified gaps between HCI methodologies and the methodologies used by new media artists.• These authors suggest that – on the one hand, new media artists have ignored user studies; – on the other hand, HCI methodologies are often task-focused, and not experience-focused, which artists would find more useful.
The Questionnaire• The questionnaire was composed of 81 questions, mostly close-ended, but also open-ended ones (13).• The close-ended questions included mostly five-point Likert scales.• The questionnaire was answered by 22 anonymous respondents.• For this paper, sections of the questionnaire related to the comparison of AV Clash with previous Video Jack projects, and future developments, are excluded.
Integration of Media, Amount of Content and Flexibility of Manipulation• The integration of sound and visuals seems to be successful, with the vast majority of the respondents considering the visuals well suited to the sound.• A large amount of sounds and visuals may have remained undiscovered for users who only interacted briefly with the system.• More could be done to increase the amount of content, but it might be equally important to focus on mechanisms that facilitate the access to that content.
Integration of Media, Amount of Content and Flexibility of Manipulation• Users are split regarding the importance of customizing sounds and visuals,• which may reflect that there are more advanced users who desire a larger diversity and deeper intervention on the content side,• and those who are satisfied with exploring and manipulating a limited amount of existing materials.
Usability and Interactivity• Usability should be substantially improved, particularly in order to captivate casual users.• Half of the users consider the interface to be unclear.• The answers to open-ended questions also highlighted this aspect—usability issues appeared even in answers to unrelated questions.
Usability and Interactivity• The approach of integrating the interface elements with the animations seems positive for the vast majority of test users.• An equal percentage of participants also prefer the interactive approach of AV Clash toward music and visuals than linear sound or video.• The appreciation for this interactivity was also recurrent in the answers to the open-ended questions.
Creativity, Playfulness and Engagement• Three quarters of the participants feel creative when playing with AV Clash.• The vast majority of the users stated that they had a "fun" experience, with half answering that they had "very much fun".• An equal percentage of users declared they would use the project again.
Creativity, Playfulness and Engagement• When asked to describe the experience with the project in an open-ended question, nine of the respondents used the word "fun", with others using expressions such as "playful", "immersive", "engaging", "surprising" and "meditative".
Creativity, Playfulness and Engagement• The positive results regarding the experience-related questions seem to indicate that:• despite the usability problems identified,• a majority of users were willing to invest time exploring the interface and the functionalities in order to be rewarded in terms of playfulness and engagement.