This document summarizes research into collaborative music making using multi-touch surfaces. It describes a study with 12 participants testing a simple music prototype. The study found that the prototype facilitated conversation and group productivity between participants, even for novices. It also provided evidence that roles were shared between participants rather than one person dominating. Future work could improve responsiveness and add both shared and individual controls to better support experts and novices.

1. Issues and techniques for collaborative music
making on multi-touch surfaces
Anna Xambó
Visiting Research Student
Centre for Research in Computing
Supervisor: Robin Laney
CRC PhD Student Conference 2010
The Open University

2. Team
• Dr Robin Laney (Open University)
• Chris Dobbyn (Open University)
• Anna Xambó (Universitat Pompeu Fabra)
• Mattia Schirosa (Universitat Pompeu Fabra)
• Prof Dorothy Miell (University of Edinburgh)
• Prof Karen Littleton (Open University)
• Sheep Dalton (Open University)

3. Project
• In collaboration with the MK Art Gallery (2009).
• Simple multi-touch music prototype.
• Pre-composed materials.
Sample loop
1
Active sample loop
Headphones
4
Speakers
2 3
1 Bass line
2 Drum line
3 Keyboard line
4 Percussion line
Interface diagram. Four users playing with the prototype.

4. Research question
• What aspects maximize collaboration for a particular user group?
• How can we evaluate effectively music creation systems that support
collaboration on tabletop surfaces?
• How can we design systematically music creation systems that support
collaboration on tabletop surfaces?

5. Motivation
• Improved collaboration can help in many contexts of music making
(composition, performance, improvisation or musical education).
• Interaction concepts applicable to other creative domains (gaming,
brainstorming).
• Evaluation of NIME using HCI methods is considered a novel and promising
field of research.

6. Context
• Musical multi-touch surfaces
• Creative engagement
• Design considerations
• Multi-user instruments
• Multi-touch interaction

7. Musical multi-touch surfaces
• Audiopad [1], 2002, selection and edition of loops.
• Reactable [2, 3], 2004, modular synthesizer.
• Composition on the Table [4], 1998, audio-visual creation.
• Stereotronic Multi-Synth Orchestra [5], 2009, concentric seq.

8. Creative engagement
• Understanding creativity
• Psychological perspective of “flow” or full immersion in an activity
(Csikszentmihalyi [6]); the experience of fun and pleasure (Blythe and
Hassenzahl [7]).
• Creativity in group
• In the collective music composition attunement to others’ contributions
(Bryan-Kinss et al. [8]); “flow” extended to group productivity (Sawyer [9]).

9. Design considerations
• Multi-user instruments ([10], [11], [12])
• Shared vs. local control; complexity vs. simplicity.
• Multi-touch interaction [13])
• Discrete vs. continuous actions; display size; sense of touch; multiplicity.

10. Method
• Iterative process: Design -> Evaluation -> Design
• User study
• Participants: 12 people, 3 groups of 4 users (music skills: G1=16/20, G2=8/20, G3=9/20).
• Video recordings: 3 musical tasks + informal discussion.
• Questionnaire.
• Data analysis
• Open coding (derived from GT [14]).
• Structured coding (derived from qualitative CA [15]).

11. Findings from OC
• Collaboration
• Awareness of others; visual feedback; decision making.
• Musical aesthetics
• Emotiveness; playfulness.
• Learning process
• Easiness vs. difficulty.
• System design
• Responsiveness; individual expressivity.

12. Findings from structured coding
• Codes used
• Codes from [16]: tangible manipulation (consistent physical-digital), spatial
interaction, embodied facilitation (affordances), expressive representation
(legibility).
• Codes from [8]: mutual awareness, shared representations (collective
legibility), mutual modifiability (level of democracy), annotation (opinions).
• Results
• Consistent evidence: some content already discussed in the OC.
• New: multiple access points; shareable controls; conversations.

13. Findings from questionnaire
• Q1. I felt we operated as a team. !" $'**
!# %'")
• Q2. I felt part of a collaborative
process.
!$ #'#&
• Q3. It was difficult to play. !% %'&*
!& $'()
• Q4. I enjoyed the music making task.
" +,-./011 # $ % 2/011 &
Averages for the 5 statements.
• Q5. I concentrated intensely on the
music making.

14. Conclusions
• In general
• A simple (and constrained) prototype can be highly engaging (mainly for novices).
• This evaluation method provide us evidence of creative engagement.
• This approach can help us improve the prototype design (participatory design).
• In particular
• Roles: no dominant figure emerged, one or another took the lead.
• Conversation: The prototype strongly facilitated conversation (group
productivity).

15. Future work
• Better responsiveness
• Responsiveness-emotiveness; audiovisual feedback.
• Adding shared controls
• Support of both shared and individual controls.
• Adding features
• Balance complexity-simplicity (experts and novices).

16. References
[1] J. Patten, B. Recht, and H. Ishii, “Audiopad: a tagbased interface for musical performance,” in NIME ’02: Proceedings of the 2002 conference on
New interfaces for musical expression, (Singapore), pp. 1–6, National University of Singapore, 2002.
[2] S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,” in Proceedings of the International Computer Music Conference (ICMC
2005), (Barcelona, Spain), 2005.
[3] S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable: Exploring the synergy between live music performance and tabletop
tangible interfaces,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp.
139–146, ACM, 2007.
[4] T. Iwai, “Composition on the table,” in International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special
Interest Group on Computer Graphics and Interactive Techniques, ACM, 1999.
[5] http://www.fashionbuddha.com/, 15/3/2010.
[6] M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. Jossey-Bass, 1975.
[7] M. Blythe and M. Hassenzahl, The semantics of fun: differentiating enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic
Publishers, 2004.
[8] N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement” Behaviour and Information Technology, 2009.
[9] K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic Books, 2007.
[10] [2] S. Jordà, “Multi-user instruments: models, examples and promises,” in NIME ’05: Proceedings of the 2005 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 23–26, National University.
[11] R. Fiebrink, D. Morris, and M. R. Morris, “Dynamic mapping of physical controls for tabletop groupware,” in CHI ’09: Proceedings of the 27th
international conference on Human factors in computing systems, (New York, NY, USA), pp. 471–480, ACM, 2009.
[12] T. Blaine and S. Fels, “Contexts of collaborative musical experiences,” in NIME ’03: Proceedings of the 2003 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 129–134, National University of Singapore, 2003.
[13] B. Buxton, Multi-Touch Systems that I Have Known and Loved. Microsoft Research, 2007.
[14] J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human-Computer Interaction. Wiley, 2010.
[15] D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol. 10, pp. 65–77, 1987.
[16] E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A framework on physical space and social interaction,” in CHI ’06: Proceedings
of the SIGCHI conference on Human Factors in computing systems, (New York, NY, USA), pp. 437–446, ACM Press, 2006.