The document discusses spatial conferencing, which brings telepresence capabilities to smartphones using regular headsets. Spatial conferencing uses techniques like spatial audio and head related transfer functions to make it sound like participants are located in the same physical room even when communicating remotely. It achieves this through high quality, low latency 14 kHz+ audio and positioning participants spatially through a multi-channel audio conferencing system. A demonstration showed promising results, with testers feeling like they were sitting in the same room. The document proposes further development and standardization of spatial conferencing technologies.

1. © Fraunhofer IIS 1
Spatial Conferencing
Redmond, 16 Oct 2014
Spatial Conferencing
HP Baumeister

2. © Fraunhofer IIS 2 S p atial Conferencing
Telepresence Rooms:
High End Conferencing

3. © Fraunhofer IIS 3 S p atial Conferencing
General Reaction:
“As if speaking to someone in the same room”

4. © Fraunhofer IIS 4 S p atial Conferencing
General Reaction:
“As if speaking to someone in the same room”
Why?
In part, because the audio is “transparent”, and comes from the appropriate
direction!
“Spatial Audio”

5. © Fraunhofer IIS 5 S p atial Conferencing
“Spatial Audio”
• audio: 3 channels of Fraunhofer AAC-LD @ 48kHz
• video: 3 x H.264 @ 1080p

6. © Fraunhofer IIS 6 S p atial Conferencing
Telepresence for Everyone:
Spatial Conferencing
 Bring the “telepresence experience” to smartphones
 Using regular headsets/earphones

7. © Fraunhofer IIS 7 S p atial Conferencing
Telepresence for Everyone:
Spatial Conferencing
“Billions of smartphone and tablets”

8. © Fraunhofer IIS 8
Spatial Conferencing
Spatial Conferencing: Key Elements
Highest Quality Audio: “Full-HD Voice”
“Spatialization”

9. © Fraunhofer IIS 9
Spatial Conferencing
Spatial Conferencing: “Full-HD Voice”
14 kHz+ audio bandwidth

10. © Fraunhofer IIS 10
Spatial Conferencing
Full-HD Voice Audio Quality
source: http://www.3gpp.org/ftp/tsg_sa/WG4_CODEC/TSGS4_52/Docs/S4-090080.zip (Feasibility study on EVS audio bandwidth, Ericsson 3GPP/SA4 01/2009)
0
1
2
3
4
5
Mean Opinion Score
Fair
Good
Excellent
Bad
Poor
NB
WB
SWB

11. © Fraunhofer IIS 11
Spatial Conferencing
Spatial Conferencing “Full-HD Voice”
14 kHz+ audio bandwidth
Very low delay
Multi-channel capability

12. © Fraunhofer IIS 12
Spatial Conferencing
Spatial Conferencing
“Full-HD Voice”
ELD = Enhanced Low Delay
AAC-ELD is a low latency mode of the widely adopted music codec AAC.
CD-like audio quality for telephone calls
full audio bandwidth up to 20 kHz
low coding delay down to 15 ms, crucial for natural conversations
optimized for a bit-rate range of 24 kbit/s to 64 kbit/s per channel
AAC-ELDv2 enhances bit-rate range down to 24 kbit/s stereo
Most widely adopted codec offering Full-HD Voice today
FaceTime
Native in iOS
Native in Android

13. © Fraunhofer IIS 13
Spatial Conferencing
Spatial Conferencing for Everyone “Spatialization”
Geometric Localization of Participants
With or without Video……

14. © Fraunhofer IIS 14
Spatial Conferencing
Spatial Conferencing “Head Related Transfer Function”

15. © Fraunhofer IIS 15
Spatial Conferencing
Spatial Conferencing “Head Related Transfer Function”
Credit: University of Maryland

16. © Fraunhofer IIS 16
Spatial Conferencing
Spatial Conferencing “Out of Head Experience”
Stereo = “Inside Head”
Spatial Rendering = “outside Head”

17. © Fraunhofer IIS 17
Spatial Conferencing
Audio Conferencing
MCU

18. © Fraunhofer IIS 18
Spatial Conferencing
Audio Conferencing “single channel” – “everyone is in the same location”
MCU

19. © Fraunhofer IIS 19
Spatial Conferencing
Spatial Conferencing “Spatialization” – “participants can be localized as if in the room”
MCU
positioning & out of head
Full-HD Voice (spatialized)

20. © Fraunhofer IIS 20
Spatial Conferencing
Spatial Conferencing MCU

21. © Fraunhofer IIS 21
Spatial Conferencing
3-Point conference via IP
Full Band Audio, 44.1 kHz
Immersive, spatial experience
Spatial Conferencing Demo Setup
3x Nexus 5
-Android 4.4.3.2.1
-VoIP App using native ELD
standard
in-ear headset
Mac-Mini running MCU with binaural processing
64 kbps mono
128 kbps
stereo

22. © Fraunhofer IIS 22 S p atial Conferencing
Developed and evaluated in collaboration with
Spatial Conferencing Trials
PSTN Full-HD Voice
“I found it really amazing, because one really
thought you are sitting in the same room”

23. © Fraunhofer IIS 23
Spatial Conferencing
Member Companies should consider spatial conferencing in future systems
Define uses cases
Interface Specification / Standardisation
Interoperability
Spatial Conferencing IMTC Opportunities

24. © Fraunhofer IIS 24
Spatial Conferencing
See me: HP Baumeister
Call me: 408 573 9903
 Mail Me hp.baumeister@dmt.fraunhofer.org
Thank you! Questions? Live Demo?

25. © Fraunhofer IIS 25
Spatial Conferencing
Spatial Conferencing “Spatialization”
Full-HD Voice (mono)
MCU
positioning & out of head
Full-HD Voice (spatialized)