THE INFLUENCE OF PERCEPTUAL ATTACK
              TIMES IN
  NETWORKED MUSIC PERFORMANCE
   Pilot Study conducted at CCRMA, Stanford University (2011)
      Álvaro Barbosa, João Cordeiro
Research Center for Science and Technology of the Arts
           CITAR (UCP–Porto, Portugal)
Background on Network Music Performance




       Latency and Networked Music
      In a vocal conversation it is possible to maintain it even with one-way
      delays of up to 500 ms.
           Holub, J., Kastner, M., Tomiska, O. 2007
      In order to maintain a synchronized and smooth musical interaction
      reduces drastically to the order of tens of milliseconds.
           Schuett, N. 2002
           Chafe C., Gurevich M, 2004
           Lago, N and Kon, F. 2004
           Barbosa, A., Carôt, A. 2005
           Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005
           Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006
           Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009
           Chafe, C., Cáceres, J-P., Gurevich, M., 2010
Background on Network Music Performance




      Latency and Networked Music

      For the Human ear to perceive the order of two simultaneous sounds,
      they should not be displaced in time over 20ms (Hirsh, 1959)



              A difficulty in discerning the order of sounds events
                                       
              hard to maintain a synchronous musical interaction.
Background on Network Music Performance




      Latency and Networked Music
      The ability to perform music synchronously is strongly dependent on:

      •   The music expressive qualities (Dynamics and Articulation)
      •   The music style (rhythm, melody, harmony)
      •   The music perceptual qualities (pitch, texture, timbre)
      •   The music structure/form
      •   The musician’s experience and practicing strategies
      •   Complementary feed-back modalities (visual, tactile)
      •   The listening conditions
Background on Network Music Performance




      Latency and Networked Music
                      Basic Principles
   (1) Ensemble Performance Threshold (EPT)

   (2) Echo Feed-Back (Self Delay)

   (3) Inverse Proportion to Tempo

   (4) Reverb and Complementary Modalities

   (5) Slow Attack Times
Background on Network Music Performance




   (1) Ensemble Performance Threshold (EPT)

      (CCRMA 2004, USC 2005)
Background on Network Music Performance




   (2) Echo Feed-Back (USC 2005)




         Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005
Background on Network Music Performance




   (2) Echo Feed-Back (UCP-Porto 2005)
Background on Network Music Performance




   (3) Inverse Proportion to Tempo (UCP-Porto 2005)
Background on Network Music Performance




   (3) Inverse Proportion to Tempo (CCRMA/Banff 2006)


                                               St. Lawrence
                                              String Quartet
                                            (over 50ms Delay)
Background on Network Music Performance




   (4) Reverb and Complementary Modalities

   Studies on the effect of Reverberation

   (Farner 2009)
   The Influence of Delay and Various Acoustic Environments

   Studies on the effect of Visual Feed-Back

   Pilot experiments at CCRMA since 2006

   Studies on the effect of Haptic Feedback ???
The Influence of Attack Times in NMP




   (5) Slow Attack Times
       Bregman (1990)
       Auditory Scene Analysis




   This experiment clearly reveals that
   it is harder to perceive the order of
   overlapping sound events when
   their rise time (attack) is slower.

   In other words, overlapping sounds
   with slower rise times are better
   perceived as synchronous even
   when their onsets are not
   physically simultaneous.
The Influence of Attack Times in NMP




    Perceptual Attack Time (PAT)

    The fact that two sounds are physically synchronous does not necessarily
    make them sound perceptually synchronous.

    Perceptual Attack Time (PAT) is a subjective measure of the time that is
    perceived by the listener as the moment of rhythmic placement for a
    musical sound event.

    According to Matt Wright
    “for highly percussive sounds, the PAT might be the same as, or just a few
    milliseconds after, the onset time, but for sounds with a slow attack, for
    example, bowed violin, the PAT might be dozens of milliseconds after the
    onset”.
The Influence of Attack Times in NMP




    Pilot Study Hypothesis

    If slow attack times allow a better perception of sound simultaneity this
    might also lead to a better ability to perform synchronous musical
    interaction


    Experimental Setup
The Influence of Attack Times in NMP




    Rhythmic Performance
    80 BPM (to avoid possible overlaying of subsequent notes)
    Slow Tempo => Higher EPT
    Range from 25 ms to 110 ms (with steps of 10 ms)
    28 trials for Slow and Fast attack strokes
The Influence of Attack Times in NMP




    Results #1
    Could each musician maintain a 80 BPM steady rhythm, throughout the
    range of latencies introduced on the feedback loop?
    We performed the analysis of the average BPM progression with the
    “Tempo and Beat Tracker” tool from Queen Mary’s Sonic Visualizer.
The Influence of Attack Times in NMP




    Results #2
    How each musician performs in relation to the other?
    analysis was performed using the software tool MATCH: Music Alignment
    Tool Chest, developed by Simon Dixon (this software analyses the
    alignment of audio files using the OLTW algorithm)
The Influence of Attack Times in NMP




    Conclusions & Future Work

    1. The results from the “Attack Time Pilot Experiment” suggest that for
       higher latencies than the generally established EPT (20-40 ms), a
       better musical performance can be achieved when musical notes are
       played with slow attack times.

    2. Further investigation remains to be done, specifically by expanding the
       experiment to a larger number of subjects and a larger scope of
       musical instruments.

    3. It remains to be determined what is if there is a functional relationship
       between the rise times of Attack in musical notes and the performance
       Tempo, that affects the EPT.
Thank You!!!!
abarbosa@porto.ucp.pt | http://www.abarbosa.org

jcordeiro@porto.ucp.pt | http://www.joaocordeiro.eu

44th AES conference (2011)

  • 1.
    THE INFLUENCE OFPERCEPTUAL ATTACK TIMES IN NETWORKED MUSIC PERFORMANCE Pilot Study conducted at CCRMA, Stanford University (2011) Álvaro Barbosa, João Cordeiro Research Center for Science and Technology of the Arts CITAR (UCP–Porto, Portugal)
  • 2.
    Background on NetworkMusic Performance Latency and Networked Music In a vocal conversation it is possible to maintain it even with one-way delays of up to 500 ms. Holub, J., Kastner, M., Tomiska, O. 2007 In order to maintain a synchronized and smooth musical interaction reduces drastically to the order of tens of milliseconds. Schuett, N. 2002 Chafe C., Gurevich M, 2004 Lago, N and Kon, F. 2004 Barbosa, A., Carôt, A. 2005 Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005 Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006 Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009 Chafe, C., Cáceres, J-P., Gurevich, M., 2010
  • 3.
    Background on NetworkMusic Performance Latency and Networked Music For the Human ear to perceive the order of two simultaneous sounds, they should not be displaced in time over 20ms (Hirsh, 1959) A difficulty in discerning the order of sounds events  hard to maintain a synchronous musical interaction.
  • 4.
    Background on NetworkMusic Performance Latency and Networked Music The ability to perform music synchronously is strongly dependent on: • The music expressive qualities (Dynamics and Articulation) • The music style (rhythm, melody, harmony) • The music perceptual qualities (pitch, texture, timbre) • The music structure/form • The musician’s experience and practicing strategies • Complementary feed-back modalities (visual, tactile) • The listening conditions
  • 5.
    Background on NetworkMusic Performance Latency and Networked Music Basic Principles (1) Ensemble Performance Threshold (EPT) (2) Echo Feed-Back (Self Delay) (3) Inverse Proportion to Tempo (4) Reverb and Complementary Modalities (5) Slow Attack Times
  • 6.
    Background on NetworkMusic Performance (1) Ensemble Performance Threshold (EPT) (CCRMA 2004, USC 2005)
  • 7.
    Background on NetworkMusic Performance (2) Echo Feed-Back (USC 2005) Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005
  • 8.
    Background on NetworkMusic Performance (2) Echo Feed-Back (UCP-Porto 2005)
  • 9.
    Background on NetworkMusic Performance (3) Inverse Proportion to Tempo (UCP-Porto 2005)
  • 10.
    Background on NetworkMusic Performance (3) Inverse Proportion to Tempo (CCRMA/Banff 2006) St. Lawrence String Quartet (over 50ms Delay)
  • 11.
    Background on NetworkMusic Performance (4) Reverb and Complementary Modalities Studies on the effect of Reverberation (Farner 2009) The Influence of Delay and Various Acoustic Environments Studies on the effect of Visual Feed-Back Pilot experiments at CCRMA since 2006 Studies on the effect of Haptic Feedback ???
  • 12.
    The Influence ofAttack Times in NMP (5) Slow Attack Times Bregman (1990) Auditory Scene Analysis This experiment clearly reveals that it is harder to perceive the order of overlapping sound events when their rise time (attack) is slower. In other words, overlapping sounds with slower rise times are better perceived as synchronous even when their onsets are not physically simultaneous.
  • 13.
    The Influence ofAttack Times in NMP Perceptual Attack Time (PAT) The fact that two sounds are physically synchronous does not necessarily make them sound perceptually synchronous. Perceptual Attack Time (PAT) is a subjective measure of the time that is perceived by the listener as the moment of rhythmic placement for a musical sound event. According to Matt Wright “for highly percussive sounds, the PAT might be the same as, or just a few milliseconds after, the onset time, but for sounds with a slow attack, for example, bowed violin, the PAT might be dozens of milliseconds after the onset”.
  • 14.
    The Influence ofAttack Times in NMP Pilot Study Hypothesis If slow attack times allow a better perception of sound simultaneity this might also lead to a better ability to perform synchronous musical interaction Experimental Setup
  • 15.
    The Influence ofAttack Times in NMP Rhythmic Performance 80 BPM (to avoid possible overlaying of subsequent notes) Slow Tempo => Higher EPT Range from 25 ms to 110 ms (with steps of 10 ms) 28 trials for Slow and Fast attack strokes
  • 16.
    The Influence ofAttack Times in NMP Results #1 Could each musician maintain a 80 BPM steady rhythm, throughout the range of latencies introduced on the feedback loop? We performed the analysis of the average BPM progression with the “Tempo and Beat Tracker” tool from Queen Mary’s Sonic Visualizer.
  • 17.
    The Influence ofAttack Times in NMP Results #2 How each musician performs in relation to the other? analysis was performed using the software tool MATCH: Music Alignment Tool Chest, developed by Simon Dixon (this software analyses the alignment of audio files using the OLTW algorithm)
  • 18.
    The Influence ofAttack Times in NMP Conclusions & Future Work 1. The results from the “Attack Time Pilot Experiment” suggest that for higher latencies than the generally established EPT (20-40 ms), a better musical performance can be achieved when musical notes are played with slow attack times. 2. Further investigation remains to be done, specifically by expanding the experiment to a larger number of subjects and a larger scope of musical instruments. 3. It remains to be determined what is if there is a functional relationship between the rise times of Attack in musical notes and the performance Tempo, that affects the EPT.
  • 19.
    Thank You!!!! abarbosa@porto.ucp.pt |http://www.abarbosa.org jcordeiro@porto.ucp.pt | http://www.joaocordeiro.eu

Editor's Notes

  • #13 The Timedelaybetweenclustersis 120 ms