1. Latest Developments of
SVG 2
Streaming and Gradient
Technologies
Cyril Concolato
Klagenfurt, 14/12/2012

2. Overview
 Presentation of Telecom ParisTech
 Latest developments in SVG 2
 My research work related to SVG
• Streaming
• Advanced Gradients
page 2 Séminaire Adaptation

3. Telecom ParisTech
 Engineering School founded in 1878
• The French word télécommunication was
coined in 1904 by its director É. Estaunié.
 Small size university
• 205 full-time staff
• 1400 students
(undergraduate, master and 330 PhD)
 Organization
• Member of ParisTech
- Group of engineering schools in the Paris area
• Member of the French Telecom Institute
- Group of engineering schools specialized in
Telecom
• Collaborations/Double-degree with 70
partner universities around the world
page 3 Séminaire Adaptation

4. Missions
 Focused on Telecommunications
• Communications and Electronics
• Computer Science and Networking
• Signal and Image Processing
• Economics and Social Sciences
 Education
• Delivers “Ingénieur”, PhD, Masters degrees…
 Research
• 700 publications per year + Patents
• French-, European-, industrial-funded projects
• Standardization (MPEG, W3C)
 Entrepreneurship - Incubator
• 185 sustainable companies started since 1999
page 4 Séminaire Adaptation

5. The Multimedia Group: Research Themes
 Multimedia Content Representation
• Description, Compression
 Delivery of Multimedia Content
• File delivery, Streaming, Broadcasting, Adaptation
 Visualization of Multimedia Content
• Compositing, Rendering, Interactivity, Adaptation
 Multimedia Interactive Applications
• Digital Television and Digital Radio
• Mobile Applications
• Rich Internet Applications
page 5 Séminaire Adaptation

6. The GPAC Open Source Project
 GPAC (http://gpac.sourceforge.net)
• Multimedia player, encoder, and delivery tools
• Distributed Open Source under LGPL License
• ~ 630 000 lines of code, ~ANSI C, Cross-platform
• 5000 downloads per month
• 2nd Best OSS @ ACM Multimedia 2007
 Goals
• Platform for research and academic purposes
• Dissemination of multimedia standards
 Main tools
• Multimedia Packaging and Streaming: MP4Box
• Player: Osmo4 and MP4Client
page 6 Séminaire Adaptation

7. Scalable Vector Graphics
 Royalty-free Web Standard for Vector Graphics
• Including animation, styling, interactivity, …
 Developed publicly by the SVG Working Group
• Officially 43 participants (8-10 active)
• Companies represented
- Mozilla, Opera, Google, Adobe, Microsoft, Apple, Canon,
Motorola, IBM, Telecom ParisTech (since 2004) …
 Supported by all recent web browsers
• Almost interoperable: TTWF
 New SVG version to come: SVG 2
page 7 Séminaire Adaptation

8. What is SVG 2?
 Development process started in 2011
• Open Call for requirements: ~180 input requirements
• Discussions: ~140 accepted
• Working on ~100 for SVG 2
 Roadmap
• FPWD published in 08/2012
• WD: 02/2013
• LCWD: 09/2013
• CR: 12/2014
 Current draft
page 8 Séminaire Adaptation

9. SVG 2 main graphical features
 For map applications
• Markers
• Level of details
• Non scaling objects,
• More control over dashing, …
 More tools for designers, artists
• Gradients,
• Hatching,
• Catmull-Rom Curves,
• New line joins…
 Color Management
• Color spaces
• Color types
page 9 Séminaire Adaptation

10. SVG 2 and the Web Platform
 Massive harmonization
• Lots of joint CSS/SVG work
- filters, transforms, blending and compositing, display order,
animation, …
• Convergence w/ HTML5 Canvas
• HTML5 audio/video
• JavaScript-related improvements: Better DOM APIs,
SVG Templates, Shadow DOM
 Lots of clean-ups and simplifications
• Namespaces, …
page 10 Séminaire Adaptation

11. Streaming and SVG
page 11

12. Representation of animations
 Graphics animations can be represented differently
• SVG uses SMIL (self contained description)
• Flash animations are frame-based with a different model
• Can we represent Flash animations in SVG ?
 My contributions
• Demonstrated that it is possible to represent frame-based
animations using SVG (automatic conversion tool):
- Careful managment of drawing order, timing
example
C. Concolato, J. C. Dufourd and J. C. Moissinac, SMIL Europe 2003
Representing 2D Cartoons using SVG
• This opened the way to new delivery methods for SVG
- Delivering each frame individually (streaming)
page 12 Séminaire Adaptation

13. Fragmentation of SVG frame-based
animations
page 13 Séminaire Adaptation

14. Handling of animations by browsers
 Long running SVG animations are problematic
• File size becomes big
- This doesn’t fit well in the “download-and-play” web model
• DOM tree keeps growing, rendering becomes slow
- There is a need for some form of garbage collection
 My contributions
• Introducing a declarative garbage collection mechanism:
the <discard> element (part of SVG 2)
- Elements that became unnecessary are discarded at a given
time C. Concolato, J. Le Feuvre and J. C. Moissinac, DocEng 2007
Timed-fragmentation of SVG documents to control the playback memory
usage
• Demonstrating the use of streaming technologies
(storage in MP4, delivery using RTP) to control the
memory
- Elements are loaded only when they become necessary
page 14 Séminaire Adaptation

15. Memory Control in SVG animations
page 15 Séminaire Adaptation

16. Rendering of SVG animations
 Complex layering of different standards
CSS/SVG/SMIL
 My contribution
• An algorithm for efficient rendering
- Minimizing the number of redraw operations
- Detecting changes in complex situation
C. Concolato, J. Le Feuvre and J. C. Moissinac,
Design of an Efficient Scalable Vector Graphics Player for Constrained
Devices, IEEE Transactions on Consumer Electronics
page 16 Séminaire Adaptation

17. Synchronization of SVG and video streams
 Applications
• Existing tools: Advene, Popcorn.js and Popcorn Maker
• Synchronized rich subtitles
• Synchronized annotations
 Can we deliver SVG streams in HTML 5 elements?
• What container, protocol, …?
• What level of synchronization can be achieved?
 Current contributions
• http://concolato.wp.mines-telecom.fr/2012/06/18/html-5-video-and-
svg-graphics-synchronization/
• http://concolato.wp.mines-telecom.fr/2012/07/11/carriage-of-
synchronized-graphics-and-text-in-mp4-files/
• Mix SVG and HTML in the same document while locking timelines
• Use SVG content as an HTML 5 track
- Stored in MP4, stored in WebTT or delivered as an SVG file
- Extensible to HTML and CSS as graphics track
page 17 Séminaire Adaptation

18. Possible Future Work
Streaming and Adaptation for SVG
Responsive Images
Level of Details
Tiling and Layering
MPEG-DASH
page 18 Séminaire Adaptation

19. Research on SVG and
Gradients
page 19

20. Context
 Sabbatical stay
• October 2011 - April 2012
• Sydney, Australia
 Canon R&D center in Australia: CiSRA
• 250 staff
• Research themes
- « Ultra Fast Rendering »
- « Image Compression »
- « Video Surveillance »
- « Digital Imaging »
 My role
• Representing Canon in the SVG WG
• Work on SVG Advanced Gradients
page 20 Séminaire Adaptation

21. Canon + SVG =
« Advanced Gradient Requirements »
 Interesting Technologies
• Coons Patches
• Tensor-Product Patches
• Diffusion Curves
page 21 Séminaire Adaptation

22. Coons Patch
Patch
4 cubic Bézier
Bilinear curves
Interpolation
between the colors
on the 4 corners (u,v)
Mesh = N contiguous patches in rows / columns
page 22 Séminaire Adaptation

23. Gradient Mesh (exemples)
Conical, spherical gradients
Pseudo-3D
(Takashi M
orisaki)
page 23 Séminaire Adaptation

24. Coons Patch (pro/cons)
 Pros
• Proven Technology (1967!), widespread (Illustrator, PDF, CorelDraw,
Inkscape…)
• Can represent rich images
• Fast rendering
 Cons
• Hard to animate: Need to modify the mesh
• Tricky to edit: sharp edge / color discontinuity
• Non-smooth rendering
page 24 Séminaire Adaptation

25. Diffusion Curves – What is that?
 Orzan et al., “Diffusion Curves: A Vector Representation
for Smooth-Shaded Images”, SIGGRAPH 2008
• Representing an image using curves diffusing colors on the
left and on the right (sharp or blurry edges)
• Rendered Image
- Solution of PDE with conditions on the curves
(Poisson, Laplace)
- Rendering using the « Finite Element » approach,
discretizing the on hierarchical grids
• Problems
- Global rendering (zoom & borders)
- Slow rendering (spatially variant blur)
- Few control of the color diffusion
- Sensitive to rasterization (small features)
page 25 Séminaire Adaptation

26. Diffusion Curves – Some improvements
 Jeschke et al., “A GPU Laplacian Solver for
Diffusion Curves and Poisson Image Editing”,
SIGGRAPH Asia 2009
• Rendering Acceleration & Quality Improvment
- Improved PDE solving iterative algorithm
- Better initial solution
- Use of GPU
• No improvement in the control
 Bezerra, Eisemann et al., “Diffusion Constraints for
Vector Graphics ”, NPAR 2010
• Adding more control: diffusion strength, diffusion
barrier
page 26 Séminaire Adaptation

27. Diffusion Curves – Another Approach
 Van de Gronde, “A High Quality Solver
for Diffusion Curves”, PhD Thesis, 2010
• Analytic Element Method
– Constraints are decomposed into simple primitives (line
segment)
– Final Image is formed as the sum of simple fonctions
- Better but slower rendering
page 27 Séminaire Adaptation

28. A very active research area…
 Bowers et al., “A Ray Tracing Approach to Diffusion Curves”
ESGR 2011
 Finch et al., “Freeform Vector Graphics with Controlled Thin-Plate
Splines”
SIGGRAPH Asia 2011
 Pang et al., “Fast Rendering of Diffusion Curves with Triangles”
IEEE Computer Graphics and Applications, 2012
 Sun et al., “Diffusion Curve Textures for Resolution Independent
Texture Mapping”
SIGGRAPH 2012
 Boyé et al., “A Vectorial Solver for Free-form Vector Gradient”
Transaction on Graphics 2012
page 28 Séminaire Adaptation

29. My contributions so far …
 Evaluating the quality of the different methods
• Special focus on animations
- Flickering, robustness, …
• Study of “Freeform Vector Graphics”
- http://concolato.wp.mines-
telecom.fr/2012/05/22/experimenting-with-freeform-vector-
graphics/ (video example)
 Proposing improvements & SVG integration
• Paper in progress
page 29 Séminaire Adaptation

30. Diffusion Curves – Summary
 Strong points
• Rich Representation, more flexible than Coons
patches
- Easier Editing & animations
 Weaknesses (Research Challenges)
• Global processing (depend on the image size)
• Needs better rendering
- Faster
- Better quality in animations
- Better stability in zooming
 Inclusion in SVG 2.x?
page 30 Séminaire Adaptation

31. Thank you
Questions ?
Comments ?