Presentation of the paper "Adaptive Video and Metadata Display using Multimedia Documents" during International Workshop on Social, Adaptive and Personalized Multimedia Interaction and Access, Florence, Italy, October 2010

1. Adaptive Video and Metadata Display using Multimedia Documents Cyril Concolato ACM MM 2010 / SAPMIA Workshop 29/10/2010

2. Personalized Video Viewing with ROIRelated works Previous works “The big picture on small screens delivering acceptable video quality in mobile TV”, Knoche et al., TOMCCAP 2009 Discusses best zooming factor depending on the content “Adding dynamic visual manipulations to declarative multimedia documents”, Kuikjet al., DocEng 2009 Zooming onto pictures and creating animated camera motions “Animated Picture Presentation Steered by Natural Language”, Reiterer et al., UCMedia 2009 Virtual camera motion driven by ROI and textual description More recent works @ ACM MM 2010 “Crowd-sourced Automatic Zoom and Scroll for Video Retargeting”, Carlier et al. Learning the ROI based on user interaction, and creating a retargeted video based on ROI “Impact of Zooming and Enhancing Region of Interests for Optimizing User Experience on Mobile Sports Video”, Song et al. User study on the usefulness of ROI for improving the user experience “Video Retargeting for Aesthetic Enhancement”, Xiang et al. Automatic ROI detection and video creation page 1

3. Our approach vs. related works Automatic ROI detection (RWTH Aachen) Similar to existing works with specific detection Differentiated H.264|AVC encoding (IBBT-MMLAB) Balanced encoding between background and ROIs Use of a rich media document To display video To let the user select a ROI and zoom or not To show additional metadata with adaptation features page 2 “Annotation based personalized adaptation and presentation of videos for mobile applications”, S. De Bruyne, P. Hosten, C. Concolato, M. Asbach, J. De Cock, M. Unger, J. Le Feuvre and R.Vande Walle, Multimedia Tools and Applications, 2011, DOI: 10.1007/s11042-010-0575-2.

4. Our System Principles Generate rich media documents from video annotations Based on semi-automatic annotations Based on templates Hierarchical Rich Media Documents MPEG-4 BIFS for synchronized & interactive ROI W3C SVG & JavaScript for adaptive metadata layout & interaction page 3

5. Adaptive Rich Media Documents Part of a global problem of media adaptation (e.g. MPEG-21 DIA) Specificities of documents Structured information (e.g. XML) The use of media The spatial organization (2D/3D, …) The temporal aspects (animations, synchronization …) The interactive behavior (events, modifications) Existing methods for document adaptation Alternatives/Switch between document branches Constraints solving problem Interpolation between key scenes (e.g. automatic layout, “artistic resizing”) Scalable documents page 4

6. Example of spatial adaptation of Rich Media Documents page 5

7. Our choices in thiswork Adaptation based on constraints solving Screen size, video size, quantity/type of metadata to display Author directives E.g. priority of text over images, relative positioning of elements, … Compiled into a JavaScript algorithm Included in the rich media document Executed at runtime Results Size and positions of metadata, font size, split of metadata over several pages … page 6

8. Video and Metadata Display Results page 7 Le Feuvre, J., Concolato, C., and Moissinac, J. 2007. GPAC: open source multimedia framework. In Proceedings of the 15th international Conference on Multimedia (Augsburg, Germany, September 25 - 29, 2007). MULTIMEDIA '07. ACM, New York, NY, 1009-1012. DOI= http://doi.acm.org/10.1145/1291233.1291452

9. Demonstrations page 8

10. Conclusions and Future Work Functionnal proof of concept How media annotations can leverage document adaptation How different rich media languages can be mixed How user preferences expressed by interactions can drive the adaptation Many aspects can be improved Add more constraints Pixel density, screen orientation, … Improve algorithm for constraint solving Better use of screen space Work on the User Interface When ROI don’t last long enough to be clicked When many ROIs are present on the screen at the same time When the font size is too small User Studies Future work Authoring of adaptive documents page 9

11. Thank you for your attention!Questions ? Suggestions ? cyril.concolato@telecom-paristech.fr page 10