VERDIKT conference 2012

1. Improving
the
user
experience
of
mul4media
streaming
services
in
highly
dynamic
environments
Frank Eliassen (frank@ifi.uio.no)
Verdikt conference, 26th April 2012
4/26/2012
ROMUS
project
@
VERDIKT
1

2. Many problems still hampering Internet
live video streaming
• Increased heterogeneity of
networks and terminals
• Variability in resource availability GSM
The
Internet
such as bandwidth, CPU, and /UMTS
joining and leaving of devices
• Challenge: How
to
provide
each
consumer
with
the
best
possible
Car computer
viewing
experience
when
considering
heterogeneity
and
variability,
while
BT/
WLAN
maintaining
efficiency
and
scalability
Home PC
4/26/2012
ROMUS
project
@
VERDIKT
2

3. ROMUS
project:
main
objec4ve
• To
inves4gate
and
provide
solu4ons
for
mul4media
(video)
streaming
services
to
provide
each
consumer
best
possible
experience
in
a
highly
dynamic
environment
– best
possible
quality
(image
quality,
con4nuous
playback)
– least
possible
visual
distor4on
4/26/2012
ROMUS
project
@
VERDIKT
3

4. ROMUS
–
three
main
areas
of
results
• Adapta4on
and
Robustness
in
Live
P2P
Streaming
– Chameleon:
a
novel
adap4ve
P2P
streaming
protocol
targe4ng
live
video
streaming
(video
“broadcasts”)
– S*r:
A
social
network
based
P2P
streaming
solu4on
to
be[er
handle
peer
dynamics
in
live
video
streaming
• Video
quality
assessment
– Randomized
Pair
Comparison
(R/PC),
a
novel
test
method
for
subjec4ve
video
quality
assessment
– A
set
of
guidelines
to
reduce
visual
distor4on
in
scalable
video
streaming
• Mul4core
Processing
to
handle
mul4media
workloads
– Techniques
for
exploi4ng
mul4core
processing
and
graphical
processing
units
on
individual
peer
nodes
to
improve
video
quality
– P2G:
a
framework
for
distributed
processing
on
computer
nodes
in
a
cluster
suppor4ng
mul4media
workloads
with
so_
deadlines.
4/26/2012
ROMUS
project
@
VERDIKT
4

5. ROMUS
team
• Professor
Frank
Eliassen,
University
of
Oslo
(project
leader)
• Professor
Carsten
Griwodz,
Simula
Research
Laboratory
• Professor
Pål
Halvorsen,
Simula
Research
Laboratory
• Dr.
Viktor
S.
Wold
Eide,
University
of
Oslo
(post
doc
for
18
months)
• Dr.
Eli
Gjørven,
University
of
Oslo
(post
doc
for
6
months)
• Anh
Tuan
Nguyen,
University
of
Oslo
(PhD
scholar)
• Pengpeng
Ni,
Simula
Research
Laboratory
(PhD
scholar)
• Håkon
Stensland,
Simula
Research
Laboratory
(PhD
scholar)
4/26/2012
ROMUS
project
@
VERDIKT
5

6. ROMUS
project
Adapta4on
and
Robustness
in
Live
Peer-­‐to-­‐Peer
Streaming
4/26/2012
ROMUS
project
@
VERDIKT
6

7. Mo4va4on
• Limita4ons
of
tradi4onal
live
P2P
streaming
systems
– No
differen4ated
QoS:
users
must
receive
the
same
stream
regardless
of
their
bandwidth
(high
capacity
users
perceive
the
same
low
quality
as
average
users)
– No
con4nuous
playback/black
block
images:
with
the
current
best-­‐effort
Internet
and
the
peer
dynamics,
the
streaming
quality
at
each
peer
is
easily
impaired
(when
the
available
bandwidth
at
a
peer
drops
below
the
streaming
rate,
it
may
suffer
playback
skips)
4/26/2012
ROMUS
project
@
VERDIKT
7

8. Main
hypotheses
• Adaptable
coding
techniques
(such
as
SVC)
can
bring
significant
benefits
in
terms
of
differen4ated
QoS
and
con4nuous
playback
to
live
P2P
streaming
• Network
coding
and
social
networking
can
improve
the
robustness
of
the
P2P
system
with
respect
to
network
fluctua4ons
and
peer
dynamics
• Quality-­‐aware
overlays
can
ensure
high
capacity
peers
will
receive
high
quality
video
4/26/2012
ROMUS
project
@
VERDIKT
8

9. Scalable
Video
Coding
(H.264
AVC/
SVC)
• A
video
coding
technique:
encodes
a
video
into
layers
of
quality
• Standardized
in
July
2007
by
ITU-­‐T
(H.264)
• ~10%
bitrate
overhead
and
an
indis4nguishable
visual
quality
compared
to
H.264
AVC
4/26/2012
ROMUS
project
@
VERDIKT
9

10. Scalable
Video
Coding
(source:
h[p://www.hhi.fraunhofer.de)
“Any” sub-stream can be extracted The three scalability dimensions
4/26/2012
ROMUS
project
@
VERDIKT
10

11. Network
Coding
(Linear
network
coding)
• Instead
of
simply
forwarding
data,
intermediate
nodes
may
recombine
several
input
packets
into
one
or
several
output
packets
• Perfect
collabora4on
– Poten4al
throughput
improvements
– A
high
degree
of
robustness
4/26/2012
ROMUS
project
@
VERDIKT
11

12. Chameleon:
a
pull-­‐based
P2Pstreaming
protocol
Chameleon’s architecture with key components
4/26/2012
ROMUS
project
@
VERDIKT
12

13. Evalua4on:
Baseline
• FABALAM:
Y.
Liu,
W.
Dou,
and
Z.
Liu,
“Layer
Alloca4on
Algorithms
in
Layered
Peer-­‐to-­‐Peer
Streaming,”
in
Proc.
of
IFIP
interna*onal
conference
on
network
and
parallel
compu*ng
(NPC),
Oct.
2004,
pp.
167–174
• Commons
– Pull-­‐based
P2P
streaming
protocol
– Adaptability
• Differences
Chameleon
FABALAM
H.264/SVC
Synthe4c
layered
data
Network
coding
Approxima4on
algorithm
-­‐
A
layer
is
delivered
from
-­‐
A
layer
is
delivered
from
mul4ple
senders
one
sender
4/26/2012
ROMUS
project
@
VERDIKT
13

14. Evalua4on:
on
scalability
Chameleon vs. FABALAM: Skip rates Chameleon vs. FABALAM: Quality satisfaction
Chameleon is scalable and offers much lower skip rates and higher quality satisfaction
4/26/2012
ROMUS
project
@
VERDIKT
14

15. Evalua4on:
on
peer
dynamics
Using weibull(k,2) for generation of
different levels of peer dynamic Skip rates
Chameleon can adapt well to
peer dynamics to maintain
low skip rates and high Quality satisfaction
quality satisfaction
4/26/2012
ROMUS
project
@
VERDIKT
15

16. ROMUS
project
Video
Quality
Assessment
Flicker
effects
in
Adap4ve
Video
Streaming
4/26/2012
ROMUS
project
@
VERDIKT
16

17. Quality
adapta4on
mechanism
in
Chameleon
at
work
What are acceptable limits of quality fluctuations for the user?
Can we provide guidelines for how to adapt to reduce visual distortion?
4/26/2012
ROMUS
project
@
VERDIKT
17

18. Visual perception of dynamically adaptive video (1)!
Understanding and using limits of user perception and perceived quality"
Signal-­‐to-­‐noise
ra4o
(SNR)
scaling
Noise
flicker
Blur
flicker
Resolu4on
scaling
Frame
rate
scaling
Judder
Three main types of visual artifacts"
Media Performance Group

19. Visual Perception of dynamically adaptive video (2)
Two main fluctuation factors"
High
Frequency
Encoding
Layers
Low
Frequency
Frames
over
Time
Amplitude" Frequency"
Field study"
• mobile devices, free seating, resolution 480x320@30fps, no sunlight,
lounge chairs"
Experiment design"
• repeated measures, single-stimulus, randomized block design"
• blocking by flicker type and amplitude level"
• baselines for highest and lowest quality without quality fluctuations"
Media Performance Group

20. Visual Perception of dynamically adaptive video (3)!
Three influential factors"
Amplitude"
Most dominant effect"
Flicker is almost undetectable at
amplitudes < 8QP and almost
always detectable for larger Frequency"
amplitudes" Major effect"
Acceptance thresholds compared
to constant low quality video:"
Content" worse when above 1 Hz,"
often better when below 0.5 Hz"
Minor effect"
"
But: content can influence"
flicker perception;"
low interaction for noise flicker and
stronger for blur flicker"
"
Media Performance Group

21. ROMUS
project
P2G:
Parallel
Processing
Graphs
A
Framework
for
Distributed
Real-­‐Time
Processing
of
Mul*media
Data
4/26/2012
ROMUS
project
@
VERDIKT
21

22. Mo4va4on
• The
poten4al
to
use
mul4core
processing
and
new
heterogeneous
technology
(e.g.,
GPUs)
to
handle
mul4media
workloads
on
individual
peer
nodes
to
meet
new
demands
• Challenge:
difficult
to
program
because
of
heterogeneous
architectures
(e.g.,
data
parallelism
vs.
thread
parallelism)
• Exis4ng
frameworks
have
all
some
short-­‐
comings;
do
not
meet
all
requirements
of
mul4media
data
processing
4/26/2012
ROMUS
project
@
VERDIKT
22

23. P2G
main
features
• The P2G framework allows developers to
express continuous multimedia workloads with
fine grained parallelism in a single language
• The runtime decides itself how a program should
be partitioned, and which execution node should
execute them
• Open source at http://p2gproject.org
4/26/2012
ROMUS
project
@
VERDIKT
23

24. Mo4on
JPEG
Experiment
• Continuous workload, CIF resolution
• DCT consumes most of the encoding time.
4/26/2012
ROMUS
project
@
VERDIKT
24

25. Mo4on
JPEG
Results
4-­‐way
Core
i7
8-­‐way
Opteron
30
22
21
30
28
20
26
18
24
16
22
20
14
Time (s)
Time (s)
18
12
11
15
16
10
14
8
7
7
12
10
8
6
6
10
8
6
5
8
6
4
6
5
5
5
4
2
2
0
0
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
Threads
4/26/2012
ROMUS
project
@
VERDIKT
Threads 25

26. Conclusions
(1)
• The development of a simulation model to simulate and
evaluate novel adaptive and robust live P2P video streaming
solutions is essentially important for long-term research in the
field
• It is essential to find solutions to important limitations of live
P2P streaming technologies. Our basic solutions and findings
could be inherited by other initiatives to build a more practical
protocol taking other network metrics into account.
• Tools and guidelines for how to design real time video
streaming infrastructure adopting SVC techniques is crucial
for being able to provide the best possible user experience
with minimal visual distortion.
4/26/2012
ROMUS
project
@
VERDIKT
26

27. Conclusions
(2)
• Our results indicate that quality adaptation can
outperform constant low quality, but frequency and
amplitude, as well as switching patterns are relevant
• Multicore processor scheduling to handle multimedia
workloads on individual peer nodes will be important in
the future and may improve the multimedia experience of
the user even further.
• Our results shows the feasibility of providing a
programming framework for automatic parallel, real-time
processing of multimedia workloads exploiting
heterogeneous multicore processors.
4/26/2012
ROMUS
project
@
VERDIKT
27

28. Thank
you!
Q&A
4/26/2012
ROMUS
project
@
VERDIKT
28