A short presentation of my research work in EURECOM that lets me to develop MaFFiN, a Media Fragment Server in Node.JS.

1. Master Degree Course in
Cinema and Media Engineering
SUPERVISOR
Giovanni Malnati
INTERNSHIP SUPERVISORS
Raphaël Troncy
José Luis Redondo Garcìa
CANDIDATE
Pasquale Lisena

2. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
2

3. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
3

4. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
4
• Share
• Bookmark
• Save band
• Annotate
• Save time
• Search
• …and more
ADVANTAGES
PARTI

5. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
5
TEMPORAL DIMENSION (T)
SPATIAL DIMENSION (XYWH)
MEDIA FRAGMENTS
URI 1.0
RECOMENDATION
PARTI

6. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
6
MEDIA FRAGMENTS
URI 1.0
RECOMENDATION
TRACK DIMENSION
NAMED DIMENSION (ID)
PARTI

7. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query format
7
Hash format
http://www.example.com/example.ogv#t=10,20
http://www.example.com/example.ogv?t=10,20
Server generates the fragment
resource
HTTP Range request (time)
User Agent is in charge of display it
correctly
PARTI

8. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
8

9. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
9
CLIENTS VIDEO PLATFORMS
TEMPORAL
NPT (hh:mm:ss)
SMPTE - Clock
SPATIAL
Only start
Not standard syntax
PARTII

10. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
10
CLIENT
IMPLEMENTATIONS
• Synote Media Fragment Player
• NinSuna Media Fragment Player
SERVER
IMPLEMENTATIONS
• NinSuna Media Fragment Server
• Rafael
POLYFILL PARSER
LIBRARY
• MediaFragment.js
PARTII

11. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
11
CLIENT IMPLEMENTATIONS
SYNOTE MEDIA FRAGMENT PLAYER
• Cross-browser (Flash fallback)
• HTML5, YouTube, Daylimotion,
Vimeo support
• HTML5-like interface
https://github.com/pasqLisena/Media-Fragment-Player
• JavaScript plugin
• Spatial and temporal support
• No Time range requests
• Highlight of fragment in timeline
• Dark mask for spatial fragment
OUR
CONTRIBUTION
PARTII

12. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
12
SERVER IMPLEMENTATIONS
NINSUNA MEDIA FRAGMENT
SERVER RAFAEL
• Pre-processing
• Annotation-based
• Support for Time range
request
• Extraction on the fly
• Fragment stored on file
system
• Support only for query
fragments
http://ninsuna.elis.ugent.be/MediaFragmentsServer https://github.com/Noterik/Rafael
PARTII

13. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
13
MEDIAFRAGMENT.JS
http://www.example.com/video.ogv
?t=1:00:00#t=npt:10,20
&xywh=percent:25,25,50,50
{
"query":{
"t":[
{
"value":"1:00:00",
"unit":"npt",
"start":"1:00:00",
"end":"",
"startNormalized":3600,
"endNormalized":""
}
]
},
"hash":{
"t":[
{
"value":"npt:10,20",
"unit":"npt",
https://github.com/tomayac/Media-Fragments-URI/
OUR
CONTRIBUTION
PARSING
+
UNIT NORMALIZATION
+
ERROR DETECTION
PARTII

14. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
PART I What is a «Media Fragment»?
PART II State of the art
PART III Implementation
14

15. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
Query Fragment
• Time (npt)
• Track (video/audio)
• Xywh
15
Hash fragment
• Range request
(npt)
PARTIII

16. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
16
PARTIII

17. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
17
FRAGMENT QUERY FFMPEG OPTION NOTE
t=10 -ss 10
t=,20 -to 20
t=10,20 -ss 10 -to 20
track=video -an no audio
track=audio -vn no video
xywh=10,10,50,60 -filter:v "crop=50:60:10:10" require transcoding
xywh=percent:10,10,50,60
-filter:v "crop=in_w*50/100:
in_h*60/100:in_w*10/100:
in_h*10/100"
require transcoding
PARTIII

18. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
18
400px
220px
00:21:43
video.mp4
? t= 00:00:10, 00:00:20
& xywh=0,0,400,220
PARTIII

19. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
ALIAS
collection
19
t=10,20 and t=11,20 are byte identical
video_10.45-19.41
video_10-20
video_11-20
GridFS
Auto-deletion of older resources.
PARTIII

20. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
20
mediafragment.js
ffmpegmongoDB
PARTIII

21. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
21
PARTIII

22. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
22
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes, t, id
Content-Length: 3795
Content-Type: video/ogg
Content-Range-Mapping:
{ t:npt 9.85-21.16/0.0-653.79;include-setup } =
{ bytes 0-52,19147-22880/35614993 }
Content-type: multipart/byteranges; boundary=BOUNDARY
Etag: "b7a60-21f7111-46f3219476580"
--BOUNDARY
Content-type: video/ogg
Content-Range: bytes 0-52/35614993
{binary data}
--BOUNDARY
Content-type: video/ogg
Content-Range: bytes 19147-22880/35614993
{binary data}
---BOUNDARY--
METADATA
DATA
PARTIII

23. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
23
localhost:3000/video/video.mp4?t=10,20
localhost:3000/video/video.mp4#t=10,20

24. Pasquale Lisena
Developing a Media Fragment Server in Node.JS
24
• We contributed to the generic "media-fragment.js" polyfill and
prepare a Node.JS version (open source)
• We contributed to the Synote Media Player to build a more
generic media fragment player (open source)
• We discovered bugs in the W3C specification and proposed
amendments (revised text) to be endorsed by W3C
• We implemented a media fragment server in Node.JS (using
ffmeg) + a smart cache (based on MongoDB)
• Future work: finalize the Chrome extension so that the
browser understand the response from the server, extend
support to all media and test MaFFiN on a more large scale
environment.