<ul><li>      History of Internet Audio Experiments at McGill  (and elsewhere) </li></ul><ul><li>Jeremy R. Cooperstock </l...
<ul><li>Software available from: http://ultravideo.mcgill.edu </li></ul><ul><li>Research supported by: </li></ul>
Some Early (Pre-)History <ul><li>Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993) </li></ul><ul>...
World's first Internet AC-3 stream Montreal-New York for the AES 107th Convention, Sept. 26, 1999
System Diagram McGill University Montreal, Canada NYU Cantor Hall New York, USA 1.5 Mbps 1.5 Mbps SGI Indy PC Client Cisco...
In action Receiver Transmit Receive,check,  playback Send Audio Queue Sender Receive Audio Queue Read
Recording Studio that Spanned a Continent Montreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000 McGill Redpath...
Microphone Configuration
3 Mbps
27.6 Mbps
Some Modern History <ul><li>Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000) </li></ul><ul><li>mu...
&quot;Frères Jacques&quot; experiment Montreal, RISQ 2000 Conference <ul><li>&quot;low-latency&quot; MPEG-2 videoconferenc...
&quot;Playing Together&quot; experiment New York-Ottawa, Remote Masters class, Dec. 8, 2000 <ul><li>used Litton MPEG-2 cod...
Quality vs. Latency <ul><li>near-equivalent quality of encoded video at greatly reduced bandwidth </li></ul><ul><li>but......
Tolerated Synchronicity 10 0 20 40 30 50 70 60 80ms speed of light: Mtl-LA net latency Mtl-LA networks * This figure is a ...
Latency:   The Interaction-Killer or put another way...
Interactive Success! <ul><li>Chris Chafe: SoundWIRE (August 2001) </li></ul><ul><li>streaming pro audio from remote locati...
The future according to Sympatico?
For  those with DSL at home... yeah, right! (tu n'es pas sérieux!)
So what does it take? <ul><li>much  bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast E...
So what does it take? <ul><li>much  bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast E...
So what does it take? <ul><li>much  bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast E...
Sometimes things work...
Sometimes things work... and sometimes they don't note: look of extreme anguish
Distributed Violin Duet McGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001 <ul><li>connected two Montreal...
&quot;The medium is the message&quot;  Marshall McLuhan <ul><li>Cooperstock's Corollary: </li></ul><ul><li>&quot;The mediu...
Size  does  matter Montreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002 <ul><li>connected McGill to National Rese...
Distributed Jazz Jam Montreal-Stanford (CCRMA),  Research Demo, Oct. 24, 2002 <ul><li>uncompressed audio and video @ < 50m...
Remote Video Interpretation (Sign Language) Montreal-Fredericton (UNB), research trials, 2003 <ul><li>DV (patient, physici...
Vibrosensory Transmission VRQ demo, March 8, 2004
Coming next: DSD Transmission AES demo, October 31, 2004
Milestones Ahead <ul><li>multicast </li></ul><ul><li>traffic shaping </li></ul><ul><li>region-of-interest client selection...
Self-Signal Isolation: Echo-suppression <ul><li>the  big problem in videoconferencing  </li></ul><ul><li>same issue for vi...
<ul><li>achieving high-fidelity distributed interaction, at levels of presence and latency that support the most demanding...
 
<ul><li>http://ultravideo.mcgill.edu </li></ul>
Upcoming SlideShare
Loading in …5
×

History of Internet Audio Experiments at McGill

304 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
304
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

History of Internet Audio Experiments at McGill

  1. 1. <ul><li>      History of Internet Audio Experiments at McGill (and elsewhere) </li></ul><ul><li>Jeremy R. Cooperstock </li></ul><ul><li>Centre for Intelligent Machines </li></ul><ul><li>Centre for Interdisciplinary Research in Music Media and Technology </li></ul><ul><li>McGill University </li></ul>
  2. 2. <ul><li>Software available from: http://ultravideo.mcgill.edu </li></ul><ul><li>Research supported by: </li></ul>
  3. 3. Some Early (Pre-)History <ul><li>Eve Schooler: Distributed Music: A Foray into Net Performance (Sept. 1993) </li></ul><ul><li>synchronized three real-time streams from different hosts; delays in the order of 200 ms </li></ul><ul><li>difficult for performers to be listeners </li></ul><ul><li>Paul Hoffert: CyberSoiree (Feb. 1996) </li></ul><ul><li>ATM-based technology for audio and video streaming of a four-way jazz performance </li></ul><ul><li>delay >0.5s delay but musicians earned to compensate through extensive practice </li></ul><ul><li>Dimitri Konstantas: Distributed Musical Rehearsal Studio (May 1996) </li></ul><ul><li>ATM based distributed rehearsal with conductor at different location from musicians </li></ul><ul><li>80 ms one-way delay for audio-video synch; echo resulted in &quot;extreme confusion&quot; </li></ul><ul><li>Seiji Ozawa: Opening Ceremony Nagano Winter Olympics (1998) </li></ul><ul><li>conduct choruses on 5 continents: singers in Sydney, New York, Beijing, Berlin, False Bay </li></ul><ul><li>time lag adjustor used to eliminate satellite delay </li></ul>
  4. 4. World's first Internet AC-3 stream Montreal-New York for the AES 107th Convention, Sept. 26, 1999
  5. 5. System Diagram McGill University Montreal, Canada NYU Cantor Hall New York, USA 1.5 Mbps 1.5 Mbps SGI Indy PC Client Cisco IP/TV PC Server SGI Indy Network Dolby encoder AC-3 (640 Kbps) (5.1 channels,16 bits @ 48kHz) coded in AES/EBU
  6. 6. In action Receiver Transmit Receive,check, playback Send Audio Queue Sender Receive Audio Queue Read
  7. 7. Recording Studio that Spanned a Continent Montreal-Los Angeles for the AES 109th Convention, Sept. 23, 2000 McGill Redpath Hall, Montreal USC Norris Theatre, LA 12 channels, 96kHz, 24 bits/channel
  8. 8. Microphone Configuration
  9. 9. 3 Mbps
  10. 10. 27.6 Mbps
  11. 11. Some Modern History <ul><li>Internet2 Initiative: World's First Remote Barbershop Quartet (Nov. 2000) </li></ul><ul><li>multi-location quartet; each of the 4 singers in different cities, conductor in 5th </li></ul><ul><li>network delay variances prevented singers from hearing each other or conductor </li></ul><ul><li>Internet2 Initiative: Music Video Recording via Internet2 (Nov. 2000) </li></ul><ul><li>multi-location music video recording session using real-time streaming video </li></ul><ul><li>musicians simultaneously connected via timing tracks to a mixing board </li></ul><ul><li>Chris Chafe: QoS Enabled Audio Teleportation (Nov. 2000) </li></ul><ul><li>CD quality sound (750 kbps) of 2 separated musicians in Dallas streamed to Stanford </li></ul><ul><li>musicians played &quot;together&quot; in same space (Stanford) but delay was severe </li></ul><ul><li>John Wawrzynek: Network Musical Performance (May 2001) </li></ul><ul><li>gestural coding (e.g. MIDI) used to manage data for distributed musical performance </li></ul><ul><li>musicians at Berkeley and CalTech, playing on MIDI keyboards; local feedback only </li></ul>
  12. 12. &quot;Frères Jacques&quot; experiment Montreal, RISQ 2000 Conference <ul><li>&quot;low-latency&quot; MPEG-2 videoconferencing </li></ul>
  13. 13. &quot;Playing Together&quot; experiment New York-Ottawa, Remote Masters class, Dec. 8, 2000 <ul><li>used Litton MPEG-2 codec @ 10 Mbps </li></ul>Pinchas Zuckerman, Columbia U, NYC Wu Ji, Canarie ARDNOC, Ottawa
  14. 14. Quality vs. Latency <ul><li>near-equivalent quality of encoded video at greatly reduced bandwidth </li></ul><ul><li>but... compression takes time </li></ul>
  15. 15. Tolerated Synchronicity 10 0 20 40 30 50 70 60 80ms speed of light: Mtl-LA net latency Mtl-LA networks * This figure is a theoretical “best-case” scenario. Actual performance may vary based on local hardware. The manufacturer makes no warranties, either express or implied, and will not be held liable for damage resulting from use of the product. Certain restrictions may apply. Offer void where prohibited by law. Batteries not included. compress transmit small ensemble large ensemble lip synch musical activity MPEG* acquire
  16. 16. Latency: The Interaction-Killer or put another way...
  17. 17. Interactive Success! <ul><li>Chris Chafe: SoundWIRE (August 2001) </li></ul><ul><li>streaming pro audio from remote locations for musical collaboration </li></ul><ul><li>UDP/IP streaming: good results between Armonk, NY and Stanford </li></ul><ul><li>round trip delay 125ms, musicians able to &quot;catch-up&quot; during pauses </li></ul>
  18. 18. The future according to Sympatico?
  19. 19. For those with DSL at home... yeah, right! (tu n'es pas sérieux!)
  20. 20. So what does it take? <ul><li>much bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast Ether (100Mbps) for audio </li></ul></ul><ul><ul><li>GigE (1000Mbps) for video </li></ul></ul><ul><ul><li>more efficient network topologies – hops add delay </li></ul></ul>
  21. 21. So what does it take? <ul><li>much bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast Ether (100Mbps) for audio </li></ul></ul><ul><ul><li>GigE (1000Mbps) for video </li></ul></ul><ul><ul><li>more efficient network topologies – hops add delay </li></ul></ul><ul><li>more powerful hardware </li></ul><ul><ul><li>CPU speed (fast process switching, especially for mixed media) </li></ul></ul><ul><ul><li>pro-audio interfaces with small buffers </li></ul></ul>
  22. 22. So what does it take? <ul><li>much bigger network pipes (&quot;abusive bandwidth&quot;) </li></ul><ul><ul><li>need Fast Ether (100Mbps) for audio </li></ul></ul><ul><ul><li>GigE (1000Mbps) for video </li></ul></ul><ul><ul><li>more efficient network topologies – hops add delay </li></ul></ul><ul><li>more powerful hardware </li></ul><ul><ul><li>CPU speed (fast process switching, especially for mixed media) </li></ul></ul><ul><ul><li>pro-audio interfaces with small buffers </li></ul></ul><ul><li>more masochistic researchers </li></ul>
  23. 23. Sometimes things work...
  24. 24. Sometimes things work... and sometimes they don't note: look of extreme anguish
  25. 25. Distributed Violin Duet McGill-Université de Montréal for RISQnet Conference, Nov. 21, 2001 <ul><li>connected two Montreal universities via IP over 1 Gbps fiber link </li></ul><ul><li>uncompressed audio and video </li></ul><ul><li>< 30ms latency </li></ul>
  26. 26. &quot;The medium is the message&quot; Marshall McLuhan <ul><li>Cooperstock's Corollary: </li></ul><ul><li>&quot;The medium must be faster than the message.&quot; </li></ul>
  27. 27. Size does matter Montreal-Ottawa (NRC), Distance Teaching Trial, Feb. 20, 2002 <ul><li>connected McGill to National Research Council (Ottawa) with SDI video and multichannel 96kHz/24bit audio </li></ul><ul><li>display on 50&quot; plasma screen (near life-size) </li></ul>
  28. 28. Distributed Jazz Jam Montreal-Stanford (CCRMA), Research Demo, Oct. 24, 2002 <ul><li>uncompressed audio and video @ < 50ms latency </li></ul>
  29. 29. Remote Video Interpretation (Sign Language) Montreal-Fredericton (UNB), research trials, 2003 <ul><li>DV (patient, physician) exchanged with remote sign-language interpreter </li></ul>physician's setup patient's setup
  30. 30. Vibrosensory Transmission VRQ demo, March 8, 2004
  31. 31. Coming next: DSD Transmission AES demo, October 31, 2004
  32. 32. Milestones Ahead <ul><li>multicast </li></ul><ul><li>traffic shaping </li></ul><ul><li>region-of-interest client selection </li></ul><ul><li>multichannel echo-supression/cancellation </li></ul>
  33. 33. Self-Signal Isolation: Echo-suppression <ul><li>the big problem in videoconferencing </li></ul><ul><li>same issue for video and vibrosensory data </li></ul>
  34. 34. <ul><li>achieving high-fidelity distributed interaction, at levels of presence and latency that support the most demanding applications and doing so in spite of sensor and bandwidth limitations </li></ul>
  35. 36. <ul><li>http://ultravideo.mcgill.edu </li></ul>

×