Issues and techniques for collaborative music
making on multi-touch surfaces
Visiting Research Student
Centre for Research in Computing
Supervisor: Robin Laney
CRC PhD Student Conference 2010
The Open University
• Dr Robin Laney (Open University)
• Chris Dobbyn (Open University)
• Anna Xambó (Universitat Pompeu Fabra)
• Mattia Schirosa (Universitat Pompeu Fabra)
• Prof Dorothy Miell (University of Edinburgh)
• Prof Karen Littleton (Open University)
• Sheep Dalton (Open University)
• In collaboration with the MK Art Gallery (2009).
• Simple multi-touch music prototype.
• Pre-composed materials.
Active sample loop
1 Bass line
2 Drum line
3 Keyboard line
4 Percussion line
Interface diagram. Four users playing with the prototype.
• What aspects maximize collaboration for a particular user group?
• How can we evaluate effectively music creation systems that support
collaboration on tabletop surfaces?
• How can we design systematically music creation systems that support
collaboration on tabletop surfaces?
• Improved collaboration can help in many contexts of music making
(composition, performance, improvisation or musical education).
• Interaction concepts applicable to other creative domains (gaming,
• Evaluation of NIME using HCI methods is considered a novel and promising
field of research.
Musical multi-touch surfaces
• Audiopad , 2002, selection and edition of loops.
• Reactable [2, 3], 2004, modular synthesizer.
• Composition on the Table , 1998, audio-visual creation.
• Stereotronic Multi-Synth Orchestra , 2009, concentric seq.
• Understanding creativity
• Psychological perspective of “flow” or full immersion in an activity
(Csikszentmihalyi ); the experience of fun and pleasure (Blythe and
• Creativity in group
• In the collective music composition attunement to others’ contributions
(Bryan-Kinss et al. ); “flow” extended to group productivity (Sawyer ).
• Multi-user instruments (, , )
• Shared vs. local control; complexity vs. simplicity.
• Multi-touch interaction )
• Discrete vs. continuous actions; display size; sense of touch; multiplicity.
• Iterative process: Design -> Evaluation -> Design
• User study
• Participants: 12 people, 3 groups of 4 users (music skills: G1=16/20, G2=8/20, G3=9/20).
• Video recordings: 3 musical tasks + informal discussion.
• Data analysis
• Open coding (derived from GT ).
• Structured coding (derived from qualitative CA ).
Findings from OC
• Awareness of others; visual feedback; decision making.
• Musical aesthetics
• Emotiveness; playfulness.
• Learning process
• Easiness vs. difficulty.
• System design
• Responsiveness; individual expressivity.
Findings from structured coding
• Codes used
• Codes from : tangible manipulation (consistent physical-digital), spatial
interaction, embodied facilitation (affordances), expressive representation
• Codes from : mutual awareness, shared representations (collective
legibility), mutual modifiability (level of democracy), annotation (opinions).
• Consistent evidence: some content already discussed in the OC.
• New: multiple access points; shareable controls; conversations.
Findings from questionnaire
• Q1. I felt we operated as a team. !" $'**
• Q2. I felt part of a collaborative
• Q3. It was difficult to play. !% %'&*
• Q4. I enjoyed the music making task.
" +,-./011 # $ % 2/011 &
Averages for the 5 statements.
• Q5. I concentrated intensely on the
• In general
• A simple (and constrained) prototype can be highly engaging (mainly for novices).
• This evaluation method provide us evidence of creative engagement.
• This approach can help us improve the prototype design (participatory design).
• In particular
• Roles: no dominant figure emerged, one or another took the lead.
• Conversation: The prototype strongly facilitated conversation (group
• Better responsiveness
• Responsiveness-emotiveness; audiovisual feedback.
• Adding shared controls
• Support of both shared and individual controls.
• Adding features
• Balance complexity-simplicity (experts and novices).
 J. Patten, B. Recht, and H. Ishii, “Audiopad: a tagbased interface for musical performance,” in NIME ’02: Proceedings of the 2002 conference on
New interfaces for musical expression, (Singapore), pp. 1–6, National University of Singapore, 2002.
 S. Jordà, M. Kaltenbrunner, G. Geiger, and R. Bencina, “The reacTable*,” in Proceedings of the International Computer Music Conference (ICMC
2005), (Barcelona, Spain), 2005.
 S. Jordà, G. Geiger, M. Alonso, and M. Kaltenbrunner, “The reacTable: Exploring the synergy between live music performance and tabletop
tangible interfaces,” in TEI ’07: Proceedings of the 1st international conference on Tangible and embedded interaction, (New York, NY, USA), pp.
139–146, ACM, 2007.
 T. Iwai, “Composition on the table,” in International Conference on Computer Graphics and Interactive Techniques, SIGGRAPH: ACM Special
Interest Group on Computer Graphics and Interactive Techniques, ACM, 1999.
 http://www.fashionbuddha.com/, 15/3/2010.
 M. Csikszentmihalyi, Beyond Boredom and Anxiety: Experiencing Flow in Work and Play. Jossey-Bass, 1975.
 M. Blythe and M. Hassenzahl, The semantics of fun: differentiating enjoyable experiences, pp. 91–100. Norwell, MA, USA: Kluwer Academic
 N. Bryan-Kinns and F. Hamilton, “Identifying mutual engagement” Behaviour and Information Technology, 2009.
 K. Sawyer, Group Genius: The Creative Power of Collaboration. Basic Books, 2007.
  S. Jordà, “Multi-user instruments: models, examples and promises,” in NIME ’05: Proceedings of the 2005 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 23–26, National University.
 R. Fiebrink, D. Morris, and M. R. Morris, “Dynamic mapping of physical controls for tabletop groupware,” in CHI ’09: Proceedings of the 27th
international conference on Human factors in computing systems, (New York, NY, USA), pp. 471–480, ACM, 2009.
 T. Blaine and S. Fels, “Contexts of collaborative musical experiences,” in NIME ’03: Proceedings of the 2003 conference on New interfaces for
musical expression, (Singapore, Singapore), pp. 129–134, National University of Singapore, 2003.
 B. Buxton, Multi-Touch Systems that I Have Known and Loved. Microsoft Research, 2007.
 J. Lazar, J. Feng, and H. Hochheiser, Research Methods in Human-Computer Interaction. Wiley, 2010.
 D. L. Altheide, “Ethnographic content analysis,” Qualitative Sociology, vol. 10, pp. 65–77, 1987.
 E. Hornecker and J. Buur, “Getting a grip on tangible interaction: A framework on physical space and social interaction,” in CHI ’06: Proceedings
of the SIGCHI conference on Human Factors in computing systems, (New York, NY, USA), pp. 437–446, ACM Press, 2006.