1. Issues and techniques for collaborative music
making on multi-touch surfaces
Anna Xambó
Visiting Research Student
Centre for Research in Computing
Supervisor: Robin Laney
CRC PhD Student Conference 2010
The Open University
2. Team
• Dr Robin Laney (Open University)
• Chris Dobbyn (Open University)
• Anna Xambó (Universitat Pompeu Fabra)
• Mattia Schirosa (Universitat Pompeu Fabra)
• Prof Dorothy Miell (University of Edinburgh)
• Prof Karen Littleton (Open University)
• Sheep Dalton (Open University)
3. Project
• In collaboration with the MK Art Gallery (2009).
• Simple multi-touch music prototype.
• Pre-composed materials.
Sample loop
1
Active sample loop
Headphones
4
Speakers
2 3
1 Bass line
2 Drum line
3 Keyboard line
4 Percussion line
Interface diagram. Four users playing with the prototype.
4. Research question
• What aspects maximize collaboration for a particular user group?
• How can we evaluate effectively music creation systems that support
collaboration on tabletop surfaces?
• How can we design systematically music creation systems that support
collaboration on tabletop surfaces?
5. Motivation
• Improved collaboration can help in many contexts of music making
(composition, performance, improvisation or musical education).
• Interaction concepts applicable to other creative domains (gaming,
brainstorming).
• Evaluation of NIME using HCI methods is considered a novel and promising
field of research.
7. Musical multi-touch surfaces
• Audiopad [1], 2002, selection and edition of loops.
• Reactable [2, 3], 2004, modular synthesizer.
• Composition on the Table [4], 1998, audio-visual creation.
• Stereotronic Multi-Synth Orchestra [5], 2009, concentric seq.
8. Creative engagement
• Understanding creativity
• Psychological perspective of “flow” or full immersion in an activity
(Csikszentmihalyi [6]); the experience of fun and pleasure (Blythe and
Hassenzahl [7]).
• Creativity in group
• In the collective music composition attunement to others’ contributions
(Bryan-Kinss et al. [8]); “flow” extended to group productivity (Sawyer [9]).
9. Design considerations
• Multi-user instruments ([10], [11], [12])
• Shared vs. local control; complexity vs. simplicity.
• Multi-touch interaction [13])
• Discrete vs. continuous actions; display size; sense of touch; multiplicity.
10. Method
• Iterative process: Design -> Evaluation -> Design
• User study
• Participants: 12 people, 3 groups of 4 users (music skills: G1=16/20, G2=8/20, G3=9/20).
• Video recordings: 3 musical tasks + informal discussion.
• Questionnaire.
• Data analysis
• Open coding (derived from GT [14]).
• Structured coding (derived from qualitative CA [15]).
11. Findings from OC
• Collaboration
• Awareness of others; visual feedback; decision making.
• Musical aesthetics
• Emotiveness; playfulness.
• Learning process
• Easiness vs. difficulty.
• System design
• Responsiveness; individual expressivity.
12. Findings from structured coding
• Codes used
• Codes from [16]: tangible manipulation (consistent physical-digital), spatial
interaction, embodied facilitation (affordances), expressive representation
(legibility).
• Codes from [8]: mutual awareness, shared representations (collective
legibility), mutual modifiability (level of democracy), annotation (opinions).
• Results
• Consistent evidence: some content already discussed in the OC.
• New: multiple access points; shareable controls; conversations.
13. Findings from questionnaire
• Q1. I felt we operated as a team. !" $'**
!# %'")
• Q2. I felt part of a collaborative
process.
!$ #'#&
• Q3. It was difficult to play. !% %'&*
!& $'()
• Q4. I enjoyed the music making task.
" +,-./011 # $ % 2/011 &
Averages for the 5 statements.
• Q5. I concentrated intensely on the
music making.
14. Conclusions
• In general
• A simple (and constrained) prototype can be highly engaging (mainly for novices).
• This evaluation method provide us evidence of creative engagement.
• This approach can help us improve the prototype design (participatory design).
• In particular
• Roles: no dominant figure emerged, one or another took the lead.
• Conversation: The prototype strongly facilitated conversation (group
productivity).
15. Future work
• Better responsiveness
• Responsiveness-emotiveness; audiovisual feedback.
• Adding shared controls
• Support of both shared and individual controls.
• Adding features
• Balance complexity-simplicity (experts and novices).
16. References
[1] J. Patten, B. Recht, and H. Ishii, “Audiopad: a tagbased interface for musical performance,” in NIME ’02: Proceedings of the 2002 conference on
New interfaces for musical expression, (Singapore), pp. 1–6, National University of Singapore, 2002.
[2] S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,” in Proceedings of the International Computer Music Conference (ICMC
2005), (Barcelona, Spain), 2005.
[3] S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable: Exploring the synergy between live music performance and tabletop
tangible interfaces,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp.
139–146, ACM, 2007.
[4] T. Iwai, “Composition on the table,” in International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special
Interest Group on Computer Graphics and Interactive Techniques, ACM, 1999.
[5] http://www.fashionbuddha.com/, 15/3/2010.
[6] M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. Jossey-Bass, 1975.
[7] M. Blythe and M. Hassenzahl, The semantics of fun: differentiating enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic
Publishers, 2004.
[8] N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement” Behaviour and Information Technology, 2009.
[9] K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic Books, 2007.
[10] [2] S. Jordà, “Multi-user instruments: models, examples and promises,” in NIME ’05: Proceedings of the 2005 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 23–26, National University.
[11] R. Fiebrink, D. Morris, and M. R. Morris, “Dynamic mapping of physical controls for tabletop groupware,” in CHI ’09: Proceedings of the 27th
international conference on Human factors in computing systems, (New York, NY, USA), pp. 471–480, ACM, 2009.
[12] T. Blaine and S. Fels, “Contexts of collaborative musical experiences,” in NIME ’03: Proceedings of the 2003 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 129–134, National University of Singapore, 2003.
[13] B. Buxton, Multi-Touch Systems that I Have Known and Loved. Microsoft Research, 2007.
[14] J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human-Computer Interaction. Wiley, 2010.
[15] D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol. 10, pp. 65–77, 1987.
[16] E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A framework on physical space and social interaction,” in CHI ’06: Proceedings
of the SIGCHI conference on Human Factors in computing systems, (New York, NY, USA), pp. 437–446, ACM Press, 2006.
