Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Master Degree Course in
Cinema and Media Engineering
SUPERVISOR
Giovanni Malnati
INTERNSHIP SUPERVISORS
Raphaël Troncy
Jos...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
4
• Share
• Bookmark
• Save band
• Annotate
• Save time
• Se...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
5
TEMPORAL DIMENSION (T)
SPATIAL DIMENSION (XYWH)
MEDIA FRAG...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
6
MEDIA FRAGMENTS
URI 1.0
RECOMENDATION
TRACK DIMENSION
NAME...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query format
7
Hash format
http://www.example.com/example.og...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
9
CLIENTS VIDEO PLATFORMS
TEMPORAL
NPT (hh:mm:ss)
SMPTE - Cl...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
10
CLIENT
IMPLEMENTATIONS
• Synote Media Fragment Player
• N...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
11
CLIENT IMPLEMENTATIONS
SYNOTE MEDIA FRAGMENT PLAYER
• Cro...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
12
SERVER IMPLEMENTATIONS
NINSUNA MEDIA FRAGMENT
SERVER RAFA...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
13
MEDIAFRAGMENT.JS
http://www.example.com/video.ogv
?t=1:00...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query Fragment
• Time (npt)
• Track (video/audio)
• Xywh
15
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
16
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
17
FRAGMENT QUERY FFMPEG OPTION NOTE
t=10 -ss 10
t=,20 -to 2...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
18
400px
220px
00:21:43
video.mp4
? t= 00:00:10, 00:00:20
& ...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
ALIAS
collection
19
t=10,20 and t=11,20 are byte identical
v...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
20
mediafragment.js
ffmpegmongoDB
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
21
PARTIII
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
22
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes, t, id
...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
23
localhost:3000/video/video.mp4?t=10,20
localhost:3000/vid...
Pasquale Lisena
Developing a Media Fragment Server in Node.JS
24
• We contributed to the generic "media-fragment.js" polyf...
Developing a Media Fragment Node.JS Server
Upcoming SlideShare
Loading in …5
×

Developing a Media Fragment Node.JS Server

1,399 views

Published on

A short presentation of my research work in EURECOM that lets me to develop MaFFiN, a Media Fragment Server in Node.JS.

Published in: Internet
  • Be the first to comment

Developing a Media Fragment Node.JS Server

  1. 1. Master Degree Course in Cinema and Media Engineering SUPERVISOR Giovanni Malnati INTERNSHIP SUPERVISORS Raphaël Troncy José Luis Redondo Garcìa CANDIDATE Pasquale Lisena
  2. 2. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 2
  3. 3. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 3
  4. 4. Pasquale Lisena Developing a Media Fragment Server in Node.JS 4 • Share • Bookmark • Save band • Annotate • Save time • Search • …and more ADVANTAGES PARTI
  5. 5. Pasquale Lisena Developing a Media Fragment Server in Node.JS 5 TEMPORAL DIMENSION (T) SPATIAL DIMENSION (XYWH) MEDIA FRAGMENTS URI 1.0 RECOMENDATION PARTI
  6. 6. Pasquale Lisena Developing a Media Fragment Server in Node.JS 6 MEDIA FRAGMENTS URI 1.0 RECOMENDATION TRACK DIMENSION NAMED DIMENSION (ID) PARTI
  7. 7. Pasquale Lisena Developing a Media Fragment Server in Node.JS Query format 7 Hash format http://www.example.com/example.ogv#t=10,20 http://www.example.com/example.ogv?t=10,20 Server generates the fragment resource HTTP Range request (time) User Agent is in charge of display it correctly PARTI
  8. 8. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 8
  9. 9. Pasquale Lisena Developing a Media Fragment Server in Node.JS 9 CLIENTS VIDEO PLATFORMS TEMPORAL NPT (hh:mm:ss) SMPTE - Clock SPATIAL Only start Not standard syntax PARTII
  10. 10. Pasquale Lisena Developing a Media Fragment Server in Node.JS 10 CLIENT IMPLEMENTATIONS • Synote Media Fragment Player • NinSuna Media Fragment Player SERVER IMPLEMENTATIONS • NinSuna Media Fragment Server • Rafael POLYFILL PARSER LIBRARY • MediaFragment.js PARTII
  11. 11. Pasquale Lisena Developing a Media Fragment Server in Node.JS 11 CLIENT IMPLEMENTATIONS SYNOTE MEDIA FRAGMENT PLAYER • Cross-browser (Flash fallback) • HTML5, YouTube, Daylimotion, Vimeo support • HTML5-like interface https://github.com/pasqLisena/Media-Fragment-Player • JavaScript plugin • Spatial and temporal support • No Time range requests • Highlight of fragment in timeline • Dark mask for spatial fragment OUR CONTRIBUTION PARTII
  12. 12. Pasquale Lisena Developing a Media Fragment Server in Node.JS 12 SERVER IMPLEMENTATIONS NINSUNA MEDIA FRAGMENT SERVER RAFAEL • Pre-processing • Annotation-based • Support for Time range request • Extraction on the fly • Fragment stored on file system • Support only for query fragments http://ninsuna.elis.ugent.be/MediaFragmentsServer https://github.com/Noterik/Rafael PARTII
  13. 13. Pasquale Lisena Developing a Media Fragment Server in Node.JS 13 MEDIAFRAGMENT.JS http://www.example.com/video.ogv ?t=1:00:00#t=npt:10,20 &xywh=percent:25,25,50,50 { "query":{ "t":[ { "value":"1:00:00", "unit":"npt", "start":"1:00:00", "end":"", "startNormalized":3600, "endNormalized":"" } ] }, "hash":{ "t":[ { "value":"npt:10,20", "unit":"npt", https://github.com/tomayac/Media-Fragments-URI/ OUR CONTRIBUTION PARSING + UNIT NORMALIZATION + ERROR DETECTION PARTII
  14. 14. Pasquale Lisena Developing a Media Fragment Server in Node.JS PART I What is a «Media Fragment»? PART II State of the art PART III Implementation 14
  15. 15. Pasquale Lisena Developing a Media Fragment Server in Node.JS Query Fragment • Time (npt) • Track (video/audio) • Xywh 15 Hash fragment • Range request (npt) PARTIII
  16. 16. Pasquale Lisena Developing a Media Fragment Server in Node.JS 16 PARTIII
  17. 17. Pasquale Lisena Developing a Media Fragment Server in Node.JS 17 FRAGMENT QUERY FFMPEG OPTION NOTE t=10 -ss 10 t=,20 -to 20 t=10,20 -ss 10 -to 20 track=video -an no audio track=audio -vn no video xywh=10,10,50,60 -filter:v "crop=50:60:10:10" require transcoding xywh=percent:10,10,50,60 -filter:v "crop=in_w*50/100: in_h*60/100:in_w*10/100: in_h*10/100" require transcoding PARTIII
  18. 18. Pasquale Lisena Developing a Media Fragment Server in Node.JS 18 400px 220px 00:21:43 video.mp4 ? t= 00:00:10, 00:00:20 & xywh=0,0,400,220 PARTIII
  19. 19. Pasquale Lisena Developing a Media Fragment Server in Node.JS ALIAS collection 19 t=10,20 and t=11,20 are byte identical video_10.45-19.41 video_10-20 video_11-20 GridFS Auto-deletion of older resources. PARTIII
  20. 20. Pasquale Lisena Developing a Media Fragment Server in Node.JS 20 mediafragment.js ffmpegmongoDB PARTIII
  21. 21. Pasquale Lisena Developing a Media Fragment Server in Node.JS 21 PARTIII
  22. 22. Pasquale Lisena Developing a Media Fragment Server in Node.JS 22 HTTP/1.1 206 Partial Content Accept-Ranges: bytes, t, id Content-Length: 3795 Content-Type: video/ogg Content-Range-Mapping: { t:npt 9.85-21.16/0.0-653.79;include-setup } = { bytes 0-52,19147-22880/35614993 } Content-type: multipart/byteranges; boundary=BOUNDARY Etag: "b7a60-21f7111-46f3219476580" --BOUNDARY Content-type: video/ogg Content-Range: bytes 0-52/35614993 {binary data} --BOUNDARY Content-type: video/ogg Content-Range: bytes 19147-22880/35614993 {binary data} ---BOUNDARY-- METADATA DATA PARTIII
  23. 23. Pasquale Lisena Developing a Media Fragment Server in Node.JS 23 localhost:3000/video/video.mp4?t=10,20 localhost:3000/video/video.mp4#t=10,20
  24. 24. Pasquale Lisena Developing a Media Fragment Server in Node.JS 24 • We contributed to the generic "media-fragment.js" polyfill and prepare a Node.JS version (open source) • We contributed to the Synote Media Player to build a more generic media fragment player (open source) • We discovered bugs in the W3C specification and proposed amendments (revised text) to be endorsed by W3C • We implemented a media fragment server in Node.JS (using ffmeg) + a smart cache (based on MongoDB) • Future work: finalize the Chrome extension so that the browser understand the response from the server, extend support to all media and test MaFFiN on a more large scale environment.

×