Streaming of SVG animations on the Web

7,353 views
7,423 views

Published on

Presentation given during the HTML5 Dev Conf and The Graphical Web, showing how to create and play SVG content in a streaming manner.

Published in: Technology, Art & Photos
1 Comment
2 Likes
Statistics
Notes
  • It's two years after you posted this presentation and I am attempting to use WebVTT to drive SVG animation. But I don't see a lot of other examples of people doing it. My first task is animating sheet music generated using a custom version of MuseScore, an open source sheet music program that already exports SVG. I have modified my version of MuseScore to output linked SVG and VTT files. I am just today starting to write the animation javascript using requestAnimationFrame(). Eventually I will want to stream this stuff, so I appreciate your posting here. If there are any other live examples of linked SVG/VTT streaming animations I would appreciate seeing some links of what has happened with this concept over the past two years. Thanks!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
7,353
On SlideShare
0
From Embeds
0
Number of Embeds
4,492
Actions
Shares
0
Downloads
25
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Streaming of SVG animations on the Web

  1. 1. Streaming SVG Animations on the Web Cyril Concolato HTML5 Dev Conf / The Graphical Web 22-23 Oct 2013 Cyril Concolato
  2. 2. Context and Motivations  Animating using web technologies is getting popular • Lots of content being produced  « SVG 2 » and « Web Animations 1.0 » being drafted • Opportunities to improve the technologies  My goal: Enable treating web animations like video • Why? • What kind of animations? • How? 2 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  3. 3. Web Animation Use Cases  Animations in User Interfaces  Animations for Data Visualization  Timeline-based Animations • • • Graphics animations (cartoons, …) Synchronized audio & graphics (karaoke, ads …) Synchronized video & graphics (annotations, subtitles) Popcorn.js 3 23/10/2013 SVG Wow! Cyril Concolato Cartoon Streaming SVG animations on the Web
  4. 4. Timeline-based Animations Possibilities  Playback control using a clock • Play(0)/Play(T) • Pause/Resume • Speed-up/slow down  Accurate synchronization • Between animations or between animation and media • Dependent clocks and timestamps Similar to video but not allowed in HTML5 <video> 4 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  5. 5. An SVG video-like JavaScript Player Principles  Using HTML 5 <video> syntax but for SVG content Demo 1 1 2 3 <video controls width="640" height="480"> <source src="myAnimatedFile.svg" type="image/svg+xml" > </video> Demo 2 1 2 3 4 <video controls width="640" height="480"> <source src="file.mp4" type="video/mp4" > <track src="annotations.svg" kind="graphics" type="image/svg+xml"> </video> 5 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  6. 6. An SVG video-like JavaScript Player Status  Simple Implementation • • • • GUI based on Video.js Replacing <video> by <svg> at run-time Using XHR to fetch SVG data and insert in the page Uses get/setCurrentTime() and un/pauseAnimations()  Limitations • • • • 6 23/10/2013 Bugs! Same origin restrictions Incremental/progressive loading not working Cannot reproduce HTML network-related events (can play, can play through) Cyril Concolato Streaming SVG animations on the Web
  7. 7. An SVG video-like JavaScript player: Next Step  Network-aware Controlled Playback • Load SVG data just before it’s needed (and let garbage collection do the rest)  How to achieve controlled playback of SVG ? • 7 23/10/2013 Apply streaming concepts Cyril Concolato Streaming SVG animations on the Web
  8. 8. Applying streaming concepts to SVG  Streaming “Controlled continuous delivery and consumption of data units”  What’s needed to stream SVG content? • Fragmentation of SVG documents into streaming units ─ Needs guidelines for SVG authoring • Assign timing to each unit ─ Mapping between byte-offset/time-position of the SVG document ─ Needs a streaming instruction format • Seek information ─ Needs explicit processing behavior  SVG Streaming draft • • http://dev.w3.org/SVG/modules/streaming/ Feedback welcome  Example: Streaming SVG Cartoons 8 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  9. 9. Cartoons on the Web – Today Source: www.cartoonnetwork.com  Largely based on Vector Graphics • • Bezier curves filled with solid color or simple gradients and/or stroke Sometimes using raster images (e.g. background or texture)  Mostly frame-based content • • • Sometimes with additional interpolation Typically synchronized with a sound track Good candidate for streaming  Currently delivered as Flash or video streams • 9 23/10/2013 Plugins, coding artefacts or bandwidth problems … Cyril Concolato Streaming SVG animations on the Web
  10. 10. Cartoons on the Web – Next Steps  Could be better integrated in the web platform • Using of vector graphics primitives for scalability and new primitives for advanced rendering ─ E.g. coons patches, hatches (SVG 2), diffusion curves, … • Using other web technologies ─ Styling, language translation, …  Attempts to convert Flash to HTML5/CSS/SVG/Canvas • Many JS-based approaches ─ ─ ─ ─ • Google Swiffy Mozilla Shumway Adobe CreateJS PixelPlant Some declarative approaches ─ MP4Box (100% SVG) 10 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  11. 11. Frame-based SVG Cartoons using MP4Box <svg viewBox="0 0 768 576" ...> <rect width="100%" height="100%" fill="lightblue"/> <g id="frame_1" display="none"> <animate attributeName="display" to="inline" begin="0" end="0.16666667"/> <defs> <g id="Shape1"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g> <g id="Shape2"><path fill="rgb(111,90,41)" stroke="none" d="M 524.0 -109.0 ..."/>...</g> <g id="Shape3">...</g> </defs> <use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> <use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> <use xlink:href="#Shape3" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> </g> <g id="frame_2" display="none"> <animate attributeName="display" to="inline" begin="0.16666667" end="0.25"/> <defs> <g id="Shape4"><path .../></g> </defs> <use xlink:href="#Shape1" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> <use xlink:href="#Shape4" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> <use xlink:href="#Shape2" transform="translate(0.0 576.0) rotate(0.0) scale(1.0 1.0) "/> </g>... </svg> 11 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  12. 12. Fragmentation of SVG document into streaming units  Identify stream header • • Static throughout the streaming session Required to initialize the parser  Make sure SVG data is properly ordered • • Use only backward references Document order == time order  Identify units • • Assign timestamps No overlap in position / possible overlap in time (like WebVTT cues)  Make self-discardable units • <discard>, JavaScript, scoped stylesheets  Mark self-contained units • 12 23/10/2013 Random Access Point Cyril Concolato Streaming SVG animations on the Web
  13. 13. Streaming Instructions Example <stream timescale="24" file="cuisine.swf.svg"> <header range-end="204"/> <unit time="0" range-end="353" rap="1" /> <unit time="2" range-end="38403" rap="0" /> <unit time="3" range-end="39591" rap="0" /> <unit time="5" range-end="41923" rap="0" /> <unit time="6" range-end="45825" rap="0" /> <unit time="7" range-end="51276" rap="0" /> <unit time="10" range-end="52596" rap="1" /> <unit time="12" range-end="55261" rap="0" /> … </stream> 13 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  14. 14. How to generate Streaming Instructions ?  Post-authoring • Requires byte positions to be final  Hard to determine without author’s instructions • Where does the frame start/end? • What’s the timestamp of the unit? • Where are the random access points?  Possible automatic generation from Flash  Possible with additional elements/attributes 14 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  15. 15. Flash-to-SVG streaming SWF file Converter SVG file Converter & Packager Packager MP4 file Extractor SVG file 15 23/10/2013 Cyril Concolato Streaming Instructions file Streaming SVG animations on the Web
  16. 16. Generating Streaming Instructions for SVG content converted from Flash MP4Box -add flash.swf:fmt=svg file_with_svg_track.mp4 MP4Box -info file_with_svg_track.mp4 MP4Box -six 1 file_with_svg_track.mp4 16 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  17. 17. SVG Streaming Demo SVG + Streaming Instructions Demo 3 • • • • • Web Server (byte-range support) Browser + JS Player Playing an SVG stream Viewing Random Access Points and others Seeking into an SVG unit not yet downloaded Concatenating SVG content (splicing) Live and pseudo-live streaming of SVG content ─ Web-based Vector Graphics TV channel 17 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  18. 18. SVG streams and Media Containers MP4 Flash MP4 SVG MP4Box Video SVG Audio  Why? What for? • Packaging audio/video/graphics together ─ To download only one resource ─ To facilitate synchronization of tracks • Possible use with HTML5 native <video> elements ─ SVG in-band graphical track • Possible use with HTML 5 Media Source extensions ─ Adaptive streaming of SVG  Formats • • 18 23/10/2013 ISO/IEC-14496:12 (royalty-free) defines how to store timed text in MP4 Could be done for other formats (WebM, Ogg, …) Cyril Concolato Streaming SVG animations on the Web
  19. 19. SVG streams and WebVTT  WebVTT allows metadata tracks and raw cues 1 2 3 4 <video controls width="640" height="480"> <source src="file.mp4" type="video/mp4" > <track src="annotations.vtt" kind="metadata"> </video> SVG WebVTT Flash WebVTT SVG MP4Box MP4Box –webvtt-raw 1 file_with_svg.mp4 MP4Box –webvtt-raw 1:embedded file_with_svg.mp4 19 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  20. 20. SVG embedded in WebVTT WEBVTT Metadata track generated by GPAC MP4Box kind: metadata inBandMetadataTrackDispatchType: image/svg+xml metadata-header: <svg viewBox="0 0 384 288" ...> <rect width="100%" height="100%" fill="lightblue"/> 00:00:00.000 --> 00:00:00.083 isRAP:true <g display="none"> <animate ... /> <defs>...</defs> <use .../> </g> 00:00:00.083 --> 00:00:00.025 isRAP:false <g>...</g> 20 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  21. 21. SVG referenced in WebVTT WEBVTT Metadata track generated by GPAC MP4Box kind: metadata inBandMetadataTrackDispatchType: image/svg+xml baseMediaFile: flash7.svg text-header-length: 207 00:00:00.000 --> 00:00:00.083 mediaOffset:207 dataLength:311 isRAP:true 00:00:00.083 --> 00:00:00.166 mediaOffset:518 dataLength:320 isRAP:false 00:00:00.166 --> 00:00:00.250 mediaOffset:838 dataLength:207 isRAP:false ... 21 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  22. 22. Summary & Conclusion  Presented • • Guidelines for authoring streamable SVG content Tools for generating streaming instructions ─ from Flash, will be extended to support SVG natively • JavaScript-based players for streamable SVG content  Outcome • Provided that your SVG content is well structured and annotated, you can stream SVG content ─ In different modes: Controlled Progressive Download, Live, On Demand, Playlists, Adaptive, … ─ Out of band graphics tracks (currently with JS shim) ─ In band tracks in media containers (in browsers in the future)  Future work • • • 22 23/10/2013 Implement adaptive streaming (MP4 based) Implement media synchronization (WebVTT) Work with CSS-based or JS-based animations Cyril Concolato Streaming SVG animations on the Web
  23. 23. Thank you Questions? 23 23/10/2013 Cyril Concolato Streaming SVG animations on the Web
  24. 24. Further reading  SVG Related • HTML 5 Video and SVG Graphics Synchronization • Carriage of SVG and WebVTT in MP4 (Blog, Text Track CG)  Streaming related • Live streaming over HTTP of video and subtitles (Part 1 & Part 2) • WebVTT streaming 24 23/10/2013 Cyril Concolato Streaming SVG animations on the Web

×