Teleclassing: opzet en technologische uitdagingen - Bart De Schuymer - Televic

Teleclassing
Opzet & TechnologischeuitdagingenBart De Schuymer

Agenda
Project goal
Technological challenges

Project goal
Bologna declaration
29 European countries
reform higher education
bachelor/master system
but also: joined forces between universities
solve student’s mobility problem
increasing interest in teleclassing: synchronous communication between classrooms
4

Project goal
IBBT-Teleclassing: distance education in Flanders
joined forces of VUB and UGent
in cooperation with regional industry
small classrooms
combination of two teleclasses
ex cathedra
teacher-to-student/student-to-teacher interaction
5

Project goal
immersive
cfr. telepresence:
but in classrooms:
6

Project goal
original goals
user’s research
related, specific technological developments
fully functional, realistic setup
7

Project goal
budget shortages due to
economic crisis
change of government/university policy
restriction to lab demonstrator
realistic setup
real lessons with teachers & students
user’s research
8

Project goal
local site
9

Project goal
remote site
10

Agenda
Project goal
Technological challenges

Setup
12

Network
13

Network Requirements to support a Teleclassing environment
Teleclassing – characteristics :
High Quality / High Definition
Real-Time
Interactive

Network Requirements to support a Teleclassing environment
Quality of Service definitions :

Smartboard
16

Interactive whiteboard
Central application for teaching a class
Based on Smartboard technology
One whiteboard per class room
Synchronised

Functionality
Support of multiple types of media
Annotations
Annotations on top of media
“classical” whiteboard function
Collaboration between whiteboards is possible
Teacher keeps overall control
18

Media
Powerpoint
PDF
Images
Movies
3D content
19

Example
20

Audio/Video encoding & streaming
21

Streaming components
Using open standards
Hardware interfacing (Camera’s and microphones)
Capture cards
Encoders
DirectShow based software encoders
Video: H.264/AVC (SD)
Audio: AAC
Streaming protocol: RTP/RTCP
One RTP/RTCP stream per media stream
Decoders
Regular media players
Streaming server
Darwin Streaming Server
22

Capture
Standard pan-tilt-zoom cameras
23

Visualisation
Pixel-canvas
Standard projectors
24

Results lab demonstrator
Encoders streaming to streaming server
Streams available for multiple clients
Round trip delay: approximately 600ms
Capture cards
Inherent delay encoding
Buffering decoders
25

Advanced HD video coding using H.264/AVC FRExt.
Immersive teleclassing experience requires transmission of multiple high resolution video streams between locations
This consumes a significant amount of bandwidth
Efficient compression is needed
Improve compression performance of H.264/AVC coding for HD material
Adaptive quantization
Post-filtering for quantization noise suppression
26

Adaptive quantization
Contrast sensitivity of the human eye:
More sensitive to low spatial frequencies than to high spatial frequencies
More sensitive to luma (intensity) than to chroma (color) information
Coefficients representing high spatial frequencies and color information arerepresented with less accuracy
Accuracy/quantization determined based on model of the contrast sensitivity of the HVS.
Up to 5% bit-rate savings for the same perceptual quality compared to uniform quantization
Luminance
Red-Green chrominance
No. of perceivable levels
Blue-Yellow chrominance
Spatial frequency (cycles/degree)
27

Post-filtering for quantization noise suppression
Filter
Filter
bitstream
calculation
reconstruction
&
&
encoding
post
-
filtering
bitstream
Encoder
Decoder
Lossy compression introduces (quantization) noise into the video frames.
Design a filter to suppress this noise at the encoder and send coefficients to the decoder for post-processing
Studied state-of-the-art filter: up to 12% bit-rate reduction for the same quality can be obtained.
h(t) chosen such that e2 is minimized
Decoded
Original
Filtered
Decoded
28

Audio
29

Audio capture
teacher
standard wireless headset
remote students
microphone array
developed during the project
30

Audio Captation for Teleclassing
Internet
Requirements for audio captation in a classroom setup
Good SNR
Physically robust
Flexible
Immersive
User-friendly
Simple maintenance
31

Single Microphone vs. Microphone Array
90
1
60
120
0.8
0.6
30
150
0.4
0.2
180
0
210
330
240
300
270
Omnidirectional microphone
Microphone array
Flexible
No Noise
suppression
Directive microphone
90
1
60
120
0.75
0.5
30
150
90
1
60
120
0.25
Good Noise Suppression
Fixed speaker position
0.75
0
0.5
30
150
0.25
210
330
180
0
240
300
270
210
330
Flexible (steerable)
Good Noise suppression
240
300
270
32

Application 1: Noise Suppression
original speech
single microphone
microphone array
0.1
0.1
0.1
0.08
0.08
0.08
0.06
0.06
0.06
0.04
0.04
0.04
0.02
0.02
0.02
0
0
0
-0.02
-0.02
-0.02
-0.04
-0.04
-0.04
-0.06
-0.06
-0.06
-0.08
-0.08
-0.08
-0.1
-0.1
-0.1
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
time (s)
time (s)
time (s)
0.1
0.08
0.06
0.04
0.02
0
-0.02
-0.04
-0.06
-0.08
-0.1
0
0.2
0.4
0.6
0.8
1
time (s)
Multi-channel Noise Suppression
Microphone Array Beamforming
e.g.: 8-microphone linear array
Environmental noise: up to 18dB SNR gain
Localised noise: up to 25dB SNR gain
Single-Channel Noise Suppression
Spectral Subtraction
Interference Cancellation
MA+ Spectral Subtraction
33

Application 2: Acoustic Echo Suppression
MA + interference canceller
single microphone
microphone array
0.1
0.1
0.1
0.08
0.08
0.08
0.06
0.06
0.06
0.04
0.04
0.04
0.02
0.02
0.02
0
0
0
-0.02
-0.02
-0.02
-0.04
-0.04
-0.04
-0.06
-0.06
-0.06
-0.08
-0.08
-0.08
-0.1
-0.1
-0.1
0
0.5
1
1.5
2
2.5
0
0.5
1
1.5
2
2.5
time (s)
0
0.5
1
1.5
2
2.5
5
x 10
time (s)
5
x 10
time (s)
5
x 10
Internet
Multi-channel Echo Suppression
Single-channel Echo Suppression
Interference Cancellation
Student
Teacher
34

Application 3: Reverberation Suppression
Multi-Channel Reverberation Suppression

Microphone Arrays: Small to Big
Microphone Arrays come in many different shapes and sizes:
From 2 to 1020 microphones
From 6 cm to 3 m wide
Linear, circular, arc shaped,...
36

Microphone array: realisation
technical requirements
small size
non-intrusive
can be integrated
changeable listening direction
good audio quality
patent application filed
37

lab demonstrator setup
38

later prototypes
39

directivity
40

Teleclassing: opzet en technologische uitdagingen - Bart De Schuymer - Televic

by IBBT on Apr 07, 2010

Accessibility

Upload Details

Usage Rights

1 Embed 16

Statistics

Teleclassing: opzet en technologische uitdagingen - Bart De Schuymer - Televic — Presentation Transcript