Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
2017 06 oba aes ebu mise en forme
1. Lidwine Hô
B A
T24 Object-Based Audio for Broadcasters
Tue. May 23| 13:00 - 14:30 | Salon 7 Vienna
Session 4: OBA And Interactivity
From linear to interactive contents : sound and transmedia
Berlin - Geneva 2017
OBJECT-BASED
AUDIO
17 - 18 May 2017
at EBU, Geneva
2. Broadcasters need
to be a part of the
change
Broadcasters need
to be force of
proposal in terms of
contents
World is changing, and is challenging.
People, Users, want more.
Lidwine Hô
3. The Enemy : Karim Ben Khelifa
Engage the audience
- A face-to-face installation where the visitor wears a VR headset and is placed in between combatants of
opposing sides.
- The objective is to question : Could I be you if I was on the other side ?
The paradigm is totally new. The user becomes a part of the content, he is include in.
It’s a way to allow him to feel more engaged
Lidwine Hô
4. Users become part of the contentUsers become part of the content
When your
experimenting this face
to face with these
characters, Your
behavior is a part of the
content. You become
one of them, for the
other participants and
you’re a part of the
content
Lidwine Hô
5. What Do
Broadcasters Need ?
Synchronization
Images / Sounds
Movement / Rendering
Visual / Acoustic
Keys to Interaction with content
What Do
Broadcasters Need ?
What is
Perception ?
Perception is Multi-sensorial
in a Coherent Whole
What is
Perception ?
To See, Hear and Feel
All at once
To Interact with Environment
More than
AnyTimeAnyWhereAnyDevice
What Do Users
Want ?
Be Surprised
Personalized contents
High Quality contents
Be Immersed by Images and
Sounds
What Do Users
Want ?
Believe it’s Real
The Feeling of reality
Lidwine Hô
6. What Do
Broadcasters Need ?
What Do
Broadcasters Need ?
What is
Perception ?
What is
Perception ?
What Do Users
Want ?
What Do Users
Want ?
Perception in real life is to see, hear and feel all in the same time coherently
It’s also about interactivity : if I walk, I want the surrounding sound and
visual environment to change around me
Broadcasters need to bring this feeling of reality to users
They need synchronization of the different keys of human perception .
Lidwine Hô
7. What Do Broadcasters Need ?
Attracting The Audience
More than
AnyTime, AnyWhere, AnyDevice
Attracting The Audience
?
Lidwine Hô
8. Quality of Experience
Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
The Person
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
I’m Running
These are information
those could be useful
to deliver an
appropriate content
to one user
Lidwine Hô
9. Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
The Person
CONTEXT
CUSTOMIZATION
ACCESSIBILTY
I’m Running
Quality of Experience
Lidwine Hô
10. Attracting The Audience
Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
CONTEXT
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
CUSTOMIZATION
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
The Person
ACCESSIBILTY
I’m Running
Interactivity
=
AnySituation
Personalization
=
AnyBody
Lidwine Hô
12. Recording Editing/ Mixing Spatialization Exporting Live Experiences
Ambix
tbe
Object
Ambisonic order6
AnyDevice
=
Interoperability
Former Production Process
5.1
ST
Natif binaural
AnyTime, AnyWhere, AnyDevice, AnyContent
AnyBody, AnySituation
Lidwine Hô
13. Former Production Process
Editing/
Spatialization
Recording
Editing/ Mixing
Export 1 :
Atmosphere : Scene based : B-format
Characters : objects
Music : 5.1
Export 2 :
Atmosphere , characters, music :
Scene based : B-format/ ambix
Export 3 :
Atmosphere , characters, music : 5.1
English langage : Object
Export 4 :
binaural natif
Etc….
AnyDevice
Lidwine Hô
14. Next Production Process in OBA
Editing/
Spatialization
Recording
Editing/ Mixing
ADM for
AnyDevice
including VR
Ambisonic 6th Order
Ambix
5.1 / stereo / 22.2
TbeAudio
Video
Data
realtime
Rendering
One export to Any Device
E
X
P
O
R
T
R
e
n
d
e
r
i
n
g
AnyDevice
realtime
Rendering
realtime
Rendering
Lidwine Hô
16. Data
Data
Data
B A
Data
This is music non diegetic
It has not to move when point of you is
changing
Intensity has to be reduce when dialog
enhancement is on
It’s classified in jazz music and it could
be switch with other music if allowed by
the player
This is an atmosphere that has to move with the
point of view.
It has to be reduce when dialog enhancement is
on
in case of 5.1 output setup, it has to be
distributed equally in the LRLsRs speakers
This sound is a German
voice and should be muted
in case of English voice
selection.
It has to be enhance in case
of noisy environment.
It has to be binauralized in
case of headphone plug
It position is
• Azimuth : 30
• Elevation 10
• Distance 1, 5 meters
Audio
Channel
Audio
Scene
Audio
Object
Mono Voices
Ambisonic
spatialize
Atmosphere
5.1 non
diegetic
Music
What it is?
Lidwine Hô -
17. OBA
Standard
OBA to Auro 3D
Decoder
OBA to Dolby
Decoder
OBA to binaural
Decoder
OBA to 5.1
Decoder
OBA to WFS
Decoder
OBA to HOA
Decoder
OBA to Youtube
Decoder
OBA to Unity
Decoder
B-Format Ambix
Beds and objetcs
2 tracks binauralized
with HRTFs
Channels
LRCLsRsLfe
Wave field synthesis
HOA order N
Object + B-format +
Channels
Reaper to OBA
Encoder
Magix to OBA
Decoder
Pyramix to OBA
decoder
Protools to OBA
Decoder
sound &
position
sound &
position
sound &
position
sound &
position
Data input
Position / interaction
Audio + Data
Lidwine Hô -
Any DAW to Any Device
Etc…
18. Decoder
OBA to 5.1
5.1
Channel
Reaper to OBA
Encoder
sound &
position
Data
Any DAW to Any Device
TransportArchive
Dynamic
Data input:
choice of
the point of
view
Lidwine Hô -
Sound
+
Data input:
Position
Interaction
posibilities
format
Name
Etc…
Example of rendering on 5.1
Relative to
content
Relative to
Users
Data
• 5.1 listening setup
• Dialog enhancement On
• Audio-description
• Object 1: voice English
• Atmosfere B-format
• Musique non diegetic than cannot move
Data
19. OBA to Unity
Decoder
Object + B-format +
Channels
Reaper to OBA
Encoder
Data
Position / interaction
Any DAW to Any Device
TransportArchive
Dynamic
Data input:
user position
Noise
environment
Interaction
Lidwine Hô -
Sound
+
Data input:
Position
Interaction
format
Name
Etc…
Example of rendering 3D Audio on SDK
sound &
position
Relative to
content
Relative to
Users
• HMD 3D sound
• Movement of the head
• Localization
• Interaction with object 1
• Object 1: voice English
• Atmosfere B-format
• Musique non diegetic than cannot move
Data Data
20. OBA to Unity
Decoder
Object + B-format +
Channels
Reaper to OBA
Encoder
sound &
position
Data
Position / interaction
Any DAW to Any Device
TransportArchive
Dynamic
Data input:
user position
Noise
environment
Interaction
head
tracking
hand
tracking
User
preferences
Etc…
OBA
Lidwine Hô -
<audioContent
audioContentID="ACO_1001"
audioContentName="Music">
<audioObjectIDRef>AO_1001</
audioObjectIDRef>
<loudnessMetadata>
<integratedLoudness>-23.0</
integratedLoudness>
<maxTruePeak>-2.3</maxTruePeak>
</loudnessMetadata>
</audioContent>
From
sound designer
From
DAW
• Diegetic Sound
• Voice in Chinese
• Has to be played on
1 Loud speaker at a
time
• Ambiance : as to be
reduce when dialog
enhancement is ON
• It as to move when
point of view is
changing
• Reverberation
Sound
+
Sound
Spatial information
Other Information
Sound formats
Data input:
21. What is the perfect OBA for broadcasters?
An Audio
Object Mix made from
independent objects, scene
based ambiances, channel
based audio and data that allow
to use all audio the better, in any
situation
Audio
+
Information
Lidwine Hô -
22. Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
CONTEXT
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
CUSTOMIZATION
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
The Person
ACCESSIBILTY
I’m Running
Interactivity
=
AnySituation
Personalization
=
AnyBody
Personalization is a key to real life perception feeling
Lidwine Hô
23. Attracting The Audience
Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
CONTEXT
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
CUSTOMIZATION
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
The Person
ACCESSIBILTY
I’m Running
Interactivity
=
AnySituation
Moving my
head =
head
tracking
Lidwine Hô -
25. AnySituation
=
Interactivity
•Dialog enhancement adapted
to noise environment
•Geo-localization
•Head tracking
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
I’m looking at the butcher,
right now and I want to now
more about him
The Person
I’m Running
CONTEXT
OBA
Lidwine Hô
27. Functionalities allow by OBA
ATAWAD
OBA
ObjectBasedAudio
Listening Context
Customization to users preferences
Accessibility for impaired
Interoperability of contents on any device
Immersive Audio with binaural rendering
360° AudioVisual contents
Visually
impaired
Deaf person
Person who don’t
like Pink color
I hate Jazz music
I’ve ever seen
previous opus
Noisy
environment
I’m in Paris
It’s the evening
My baby’s sleeping
Audio
Description
Hearing-
impaired
I’m looking at the butcher,
right now and I want to now
more about him
I’m Running
Langage
Audio-description
Dialog enhancement
Use of geo-localization
OBA is the
key to
interactive,
personalized
and
immersive
contents
Lidwine Hô
28. What do we need to create now ?
Listening Context
Customization to users preferences
Accessibility for impaired
Interoperability of contents on any device
Immersive Audio with binaural rendering
360° AudioVisual contents
Langage
Audio-description
Dialog enhancement
Use of geo-localization
Open Format
understandable
by Any Device
and AnyPlatform
Allowing all this
Functionality
ObjectBasedAudio
for
AnyThing
including VR
Lidwine Hô
