Background Information & Suggestions for Joint Research Topics IVY Lab & MMLab.

1. Background Information & Suggestions for
Joint Research Topics IVY Lab & MMLab
Wesley De Neve
ICU, Daejeon, South Korea
September 2007

2. I am from...
• Belgium
– located in Europe
– 30,528 square km
– 10.5 million people
– weather has four seasons
– three official languages
• Dutch
– my native language
• French
• German
http://en.wikipedia.org/wiki/Belgium
Belgium

3. I was born...
• on May the 31st, 1980
• in Ghent
– 233,120 inhabitants
Ghent
http://en.wikipedia.org/wiki/Ghent

4. I am living...
• in Aalter
– a city close to Ghent
– 18,841 inhabitants

5. Background MMLab (1/2)
• Multimedia Lab (MMLab)
– research group
– headed by professor Rik Van de Walle
– part of Ghent University – IBBT
• Ghent University
– about 25,000 students
• IBBT (Institute for BroadBand Technology)
– fosters cooperation between different universities and
industrial partners in Belgium
» Alcatel-Lucent, Scientific Atlanta (Cisco), Barco, ...
– people
• 6 staff members
• currently 25 researchers (Ph.D. students and “others”)
• about 15 master students each year

6. Background MMLab (2/2)
• Multimedia Lab (MMLab)
– teaching
• five courses in the field of multimedia
– main research topics
• advanced video applications
– e.g., adaptation of scalable and non-scalable video content
• mobile multimedia applications
– e.g., transparent transfer of multimedia sessions
• standardization in the field of multimedia systems and
applications
– MPEG, JVT, and W3C
http://multimedialab.elis.ugent.be/

7. My Education @ Ghent University
• Master in Computer Science (1998 - 2002)
– Master’s Thesis
• “Practical study and analysis of the QuickTime framework”
– programming on top of the QuickTime API using Visual C++
• Ph.D. in Computer Science Engineering (2002 - 2007)
– Ph.D. Dissertation
• “Description-driven adaptation of media resources”
– format-independent video content adaptation using XML
• other research topics
– assessment of video quality
– GPU-based video processing
• complete list of publications (published)
– http://www.elis.ugent.be/en/node/648?G=*&A=De Neve,+W

8. Video Content Adaptation
• Investigation of format-independent techniques for
the efficient adaptation of (scalable) video content
– using an XML-based approach
• inspired by MPEG-21 BSDL and XFlavor
• novel contribution: design of BFlavor (BSDL + XFlavor)
– results verified for H.264/AVC, SVC, VC-1, and so on
original bitstream
I
BB
P
adapted bitstream MPEG-21 BSDL
I
P
transformed
XML
STX
XMLBFlavor

9. XML-based Content Adaptation:
Principles
I picture
B picture
B picture
P picture
t
Bitstream Syntax Description (BSD) Compressed bitstream
<bitstream
xml:base="bitstream_30hz.mpg">
<header>0 24</header>
<I_picture>24 2637</I_picture>
<B_picture>2661 746</B_picture>
<B_picture>3407 903</B_picture>
<P_picture>4310 1657</P_picture>
</bitstream>
adapted bit stream suited for
playback in a constrained environment

10. Video Quality Assessment
• Analysis of rate-distortion performance of H.264/AVC
– by means of an extensive number of experiments
– compared to state-of-the-art video coding formats
• VC-1
– Windows Media Video 9
• MPEG-4 Visual
– XviD and DivX
• MC-EZBC
– wavelet-based video codec developed by John Woods
– use of different objective quality metrics
• PSNR and JND

11. GPU-based Video Processing
• Use of the GPU
– for color space conversions
• e.g., YCoCg-R to RGB
– for video effects
• e.g., blurring and sharpening
– for H.264/AVC decoding
• using Direct3D and NVIDIA CUDA
– Compute Unified Device Architecture
• focus on motion compensation
– results presented at WIAMIS 2007 and SPIE Optics +
Photonics 2007

12. Other Experiences
• Participated in MPEG standardization
– contributed to improving MPEG-21 Digital Item Adaptation
• in particular, MPEG-21 BSDL
– proposed extensions to BSDL included in an amendment
to MPEG-21 Digital Item Adaptation
• Workpackage management
– for projects at a national (i.e., Belgian) and European level
• Supervised 15 master students
– mainly in the domain of video adaptation, video coding,
and rich media applications
• rich media applications: based on MPEG-4 XMT or Microsoft
Silverlight

13. Remaining Activities at MMLab
• Follow-up of research regarding
– XML-driven video content adaptation
– GPU-based video processing
• design of a novel video codec with following requirements
– GPU-friendly
– lossless
– scalable
– support for high dynamic range imaging
• Activities are supposed to be limited in effort

14. Personal Research Goals
• To work towards a number of co-publications
• To increase (practical) knowledge w.r.t.
– scalable video coding (SVC)
– programming for the GPU using NVIDIA CUDA
– new coding formats such as HD Photo / JPEG XR
– lossless video coding
– high dynamic range imaging (HDRI)

15. Possible Research Topics (1/2)
• Topic 1
– comparison of adaptivity provisions of
• Microsoft HD Photo / JPEG XR (http://blogs.msdn.com/billcrow/)
• JPEG 2000
• intra-only SVC
• (MPEG-4 VTC)
– from an MPEG-21 DIA point of view
• this is, using MPEG-21 BSDL
• how can MPEG-21 DIA be useful for a service such as Flickr?
– adaptation in the compressed domain
– rotation in the compressed domain
– allows for cooperation with MMLab
– needs further thinking to find good research questions...

16. Possible Research Topics (2/2)
• Topic 2
– GPU-based video processing using CUDA
• new computing paradigm
• http://developer.nvidia.com/object/cuda.html
– e.g., in the context of SVC
• motion estimation has to be done at several spatial layers
• computationally expensive
– allows for cooperation with Bart Pieters in MMLab
– my preferred research topic...
– needs further thinking to find good research questions...
• Other proposals or suggestions?
– e.g., multimedia ontologies for semantic adaptation, ...?

17. Some Remarks
• Constraints
– time constraints
• difficult to start a completely new research topic
• probably, the best thing to do is to continue to build on my
previous research experience
– group constraints
• preference for close cooperation with people in the group
– e.g., people working on video adaptation, SVC, GPU, ...?
• Proposed topics
– allow for cooperation between IVY Lab and MMLab, and
hence, allow for co-publications