1. Generic Video Adaptation
Framework Towards Content –
and Context Awareness in
Future Networks
W.Aubry1,2,3, D.Négru2, B.Le Gal1, S.Desfarges2, D.Dallet1
1University of Bordeaux, IMS Lab, France
2University of Bordeaux, LaBRI Lab, France
3Viotech Communications, France

2. Outline
 Context Awareness
 Video Adaptation
 Proposed Framework
 Format Adapter
 Conclusion

3. Video Consumption
 Increasing consumption of Video content
 Youtube, IPTV, …
 Video requires a lot of bandwidth
 HD file : 8-10 MB/s
 Average Website page : 320 kB
 Network congestion happens more and more
often

4. Actual Network
 Agnostic nature of the actual network
 Mail, Music, Video, … = data
10011101001
Content Provider Network End user

5. Content & Context awareness
 Network & Content management considering
 Content (mail, music, video, …)
 Context (network state, terminal, …)
 e.g. reducing video bitrate to fit in the available
bandwidth
 Less packet loss
 Less buffering time (lag)
 Less video quality

6. Quality of Experience
 Quality of Experience (QoE)
 Video quality
 Viewing condition
 Accessibility, lag …

7. Video Adaptation to Context
 Network state
 Bitrate to match bandwidth
 Terminal
 Frame resizing to screen resolution
 Codec change to decoding capabilities
 e.g. frame resizing to reduce energy
consumption at the terminal side

8. Video Adaptation System
modified video stream
video Home End User
source Gateway Terminal
configuration protocol
General
Interface
Interface
Interface
Network
Network
Network
Low Power
purpose
processor
processor
video
FPGA Video Decoder Display
accelerator (ASIC/ASIP) Controller display

9. Video adaptation constraints
 High quality
 QoE depends on video quality
 Real time processing
 lag avoidance
 Generic enough

10. Video adaptation
 Video adaptation researches aim at
 Reducing computational cost
 Maintaining high quality output
1 I. Ahmad et al. “Video Transcoding: An Overview of Various Techniques and Research Issues”
IEEE Transactions on Multimedia, October 2005.
2 Y. Xin et al. “Digital Video Transcoding” Proceedings of the IEEE, Vol. 93, No. 1, January 2005

11. Generic ?
 No
 Proposals optimize a specific process:
 MPEG-2 to h.264
 Frame resizing in h.263
…
 It is too costly to implement each and
every possible adaptation process

12. Generic video adaptation
 Any kind of adaptation
 Frame rate
 Bitrate
 Frame resolution
 Any codec

13. Our proposition
Decoding Pool Adaptation Pool Encoding Pool
MPEG-2 id MPEG-2
H.264 Adapt 2 H.264
... Adapt 1
...
...
Video Format Video Format Video
Decoder Adapter Adaptation Adapter Encoder
Decoding Path Adaptation Path Encoding Path

14. Our proposition
 Unified adaptation format
 Codec is seamless
 codec re-usability
 Possibility to use a 3rd party codec
 Format defined to enable
 already proposed adaptation algorithms
 codec features

15. Standard features
h.264/MPEG-4
Features MPEG-2 h.263
AVC
Macroblock size 16x16 16x16 16x16
MV Resolution ½ ½ ¼
Transform 8X8 DCT 8X8 DCT 4X4, 8X8 Integer
16x16, 16x8,
Vector Block Size 16x16, 16x8 16x16,8x8 8x16, 8x8, 8x4,
4x8, 4x4
Support Intra
No No Yes
prediction

16. Adaptation Format
 Data : Features
h.264/MPEG-4
AVC
 Pixeldomain
Macroblock size 16x16
 4x4 blocks
MV Resolution ¼
Transform 4X4, 8X8 Integer
 Metadata
16x16, 16x8,
 H.264 based Vector Block Size 8x16, 8x8, 8x4,
4x8, 4x4
Support Intra
Yes
prediction

17. Format converters
 Specific to a decoder (or encoder)
implementation
 Main tasks
 Re-order pixel data
 16x16  4x4
 Translate Metadata

18. Metadata Conversion
 from decoder to adaptatation
 From 16x16 block to 4x4 block
 Duplicate metadata
MacroBlock
Divider
Metadata Metadata Metadata Metadata
Macroblock

19. Metadata Conversion
 from adaptation to coder
 From 4x4 block to 16x16 block
 Use 4 to 1 downscaling algorithms
 e.g. : for MPEG-2
 MV : Linear average
 Quantizer scale : min
…

20. Motion Vector block size for
H.264
 h.264 introduces variable vector block size
0
7
10
15
0 3 7 15

21. Motion Vector block size for
H.264
0 0 0
7 7 7
10
15
0 3 7 15 0 3 7 15 0 3 7 15
Decoder Output Adaptation Input Adaptation Output Encoder input

22. Block Recombination
 Block selection
 Motion Comparison
 Combination

23. Recombination Possibilities

24. Block Recombination
 Recombination done 4 times
 Procedure repeated on coarser 4 blocks
More than one vector
Block not considered

25. Conclusion
 A « generic » adaptation framework
 pixel domain
 encoder/decoder re-usability
 adaptation algorithm agnostic
 Unified adaptation format
 h.264 based
 format adapter

26. Future works
 Extend adaptation format to HEVC
 Evaluate computational cost
 Computational time
 Silicon space

27. Thank you for your attention