VideoMR is a map and reduce framework for real-time video processing. It uses bounded memory streams based on LMAX disruptors to manage memory across video frames. The map operation processes each stream and outputs a new stream, while the reduce operation combines multiple streams into one. An example program loads a video, maps pixel differences across three frames to detect motion, and displays the output. Performance scales from SD to 4K resolution. Future work includes generalizing the framework and integrating CPU/GPU backends like OpenCL.

1. VideoMR:
A Map and Reduce Framework for
Real-time Video Processing
Benjamin-Heinz Meier, Matthias Trapp, Jürgen Döllner
Hasso Plattner Institute, University of Potsdam, Germany
1

2. Motivation
For our research we were searching
for a real-time video processing
framework supporting us to develop
GPU-based filter operation.
2

3. 3

4. Video
Processing
3

5. Abstraction
& Usability
Video
Processing
3

6. Abstraction
& Usability
GPU
Program-
ming
Video
Processing
3

7. GPU
Program-
ming
Abstraction
& Usability
Video
Processing
GStreamer,
Videotoolkit,
Premiere,
...
Plug-
Ins
Map & Reduce
MARS, ...
4

8. VideoMR
5
GPU
Program-
ming
Abstraction
& Usability
Video
Processing

9. VideoMR
bounded
memory concept,
new definition of operators
Map & Reduce
transparent
memory management
using streams
6
GPU
Program-
ming
Abstraction
& Usability
Video
Processing

10. Agenda
Memory management using LMAX disruptors as streams
The concept of Map & Reduce
A redefinition for video processing
A small example
Performance Evaluation
Limitations & Improvements
Conclusion
7

11. Memory Management
A video is a sequence of frames
A stream is a subsequence of the video with a fixed size
Implemented using the idea of a disruptor [Tho’11] :
using LMAX disruptors as streams
time
current frame
pointer
0
-1
-2
-3
…
8

12. – [Dean'04]
“Programs written in this functional style are automatically
parallelized and executed on a large cluster of
commodity machines.”
9
Map & Reduce

13. Map & Reduce
concept
10

14. Map Operation
map(key1,value1) → list(key2,value2)
Map & Reduce
concept
10

15. Map Operation
map(key1,value1) → list(key2,value2)
Map & Reduce
concept
for each word:
if word is a noun:
emit („noun“,1)
else if word is a verb:
emit („verb“,1)
10

16. Reduce Operation
reduce(key2,list(value2)) → list(value2)
Map Operation
map(key1,value1) → list(key2,value2)
Map & Reduce
concept
for each word:
if word is a noun:
emit („noun“,1)
else if word is a verb:
emit („verb“,1)
10

17. Reduce Operation
reduce(key2,list(value2)) → list(value2)
Map Operation
map(key1,value1) → list(key2,value2)
Map & Reduce
concept
for each word:
if word is a noun:
emit („noun“,1)
else if word is a verb:
emit („verb“,1)
result = 0
for each value2:
result += 1
emit („number of“+key2
+result)
10

18. Map & Reduce
redefinition for video processing
11

19. Map & Reduce
redefinition for video processing
11
Map
map(streamin,list(streamside)) → streamout
streamin

20. Map & Reduce
redefinition for video processing
11
Map
map(streamin,list(streamside)) → streamout
streamin

21. Map & Reduce
redefinition for video processing
11
Map
map(streamin,list(streamside)) → streamout
streamin
Reduce
reduce(streamin,list(streamside)) → streamout
streamin

22. Map & Reduce
redefinition for video processing
11
Map
map(streamin,list(streamside)) → streamout
streamin
Reduce
reduce(streamin,list(streamside)) → streamout
streamin

23. Example
12

24. Example
Program
12

25. Example
Program
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);

26. Example
Program
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// setup program and add the operation to it
*prog << moveMap;

27. Example
Program
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;

28. Example
Program
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();

29. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();

30. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();
// get position of current pixel
ivec2 pos = vmr_getPosition();

31. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();
// get position of current pixel
ivec2 pos = vmr_getPosition();
// get data from last three frames
vec3 c;
vec3 c1 = vmr_getStreamDataIn(pos, 0);
vec3 c2 = vmr_getStreamDataIn(pos, -1);
vec3 c3 = vmr_getStreamDataIn(pos, -2);

32. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();
// get position of current pixel
ivec2 pos = vmr_getPosition();
// get data from last three frames
vec3 c;
vec3 c1 = vmr_getStreamDataIn(pos, 0);
vec3 c2 = vmr_getStreamDataIn(pos, -1);
vec3 c3 = vmr_getStreamDataIn(pos, -2);
// compute difference
float diff1 = abs(c1.r-c2.r)
+ abs(c1.g-c2.g) + abs(c1.b-c2.b);
float diff2 = abs(c1.r-c3.r)
+ abs(c1.g-c3.g) + abs(c1.b-c3.b);

33. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();
// get position of current pixel
ivec2 pos = vmr_getPosition();
// get data from last three frames
vec3 c;
vec3 c1 = vmr_getStreamDataIn(pos, 0);
vec3 c2 = vmr_getStreamDataIn(pos, -1);
vec3 c3 = vmr_getStreamDataIn(pos, -2);
// compute difference
float diff1 = abs(c1.r-c2.r)
+ abs(c1.g-c2.g) + abs(c1.b-c2.b);
float diff2 = abs(c1.r-c3.r)
+ abs(c1.g-c3.g) + abs(c1.b-c3.b);
// compare with threshold
if (diff1>100 && diff2>100){
c = vec3(255, 255, 255);
} else {
c = vec3(0, 0, 0);
}

34. Example
Program Operation
12
// init program
auto prog = std::make_shared
<vmr::GlfwProgram >();
auto source = std::make_shared
<vmr:: LibavLoader>("./example.mov");
source->init(3);
// move detection
auto move = std::make_shared<vmr::Display>();
auto moveMap = std::make_shared<vmr::Map>
(source,move,“./move.map“);
// setup program and add the operation to it
*prog << moveMap;
// run program
prog->run();
// get position of current pixel
ivec2 pos = vmr_getPosition();
// get data from last three frames
vec3 c;
vec3 c1 = vmr_getStreamDataIn(pos, 0);
vec3 c2 = vmr_getStreamDataIn(pos, -1);
vec3 c3 = vmr_getStreamDataIn(pos, -2);
// compute difference
float diff1 = abs(c1.r-c2.r)
+ abs(c1.g-c2.g) + abs(c1.b-c2.b);
float diff2 = abs(c1.r-c3.r)
+ abs(c1.g-c3.g) + abs(c1.b-c3.b);
// compare with threshold
if (diff1>100 && diff2>100){
c = vec3(255, 255, 255);
} else {
c = vec3(0, 0, 0);
}
// write result to current position
vmr_emitToStreamOut(pos,c);

35. Performance Evaluation
0 ms
10 ms
20 ms
30 ms
40 ms
SD HD Full HD 4K
map reduce complex
13
0 RLOC
125 RLOC
250 RLOC
375 RLOC
500 RLOC
OpenGL VideoMR
program operation

36. Improvements & Future Work
14

37. Improvements & Future Work
14
A general concept for n-dimensional buffers instead of frames

38. Improvements & Future Work
14
A general concept for n-dimensional buffers instead of frames
Extension with template classes to decide which main data type to use

39. Improvements & Future Work
14
A general concept for n-dimensional buffers instead of frames
Extension with template classes to decide which main data type to use
Introduction of an explicit CPU operation with transparent memory
management

40. Improvements & Future Work
14
A general concept for n-dimensional buffers instead of frames
Extension with template classes to decide which main data type to use
Introduction of an explicit CPU operation with transparent memory
management
Considering of OpenCL and CUDA as backend

41. Improvements & Future Work
14
A general concept for n-dimensional buffers instead of frames
Extension with template classes to decide which main data type to use
Introduction of an explicit CPU operation with transparent memory
management
Considering of OpenCL and CUDA as backend
Comparison with other map and reduce frameworks will be part of future
and therefore, a suitable benchmark has to be developed

42. Conclusion
15
GPU
Program-
ming
Abstraction
& Usability
Video
Processing
VideoMR fulfills the requirements of
real-time video processing and
supports with the map and
reduce concept the development
of GPU-based filter operations.

43. Question & Comments
16
Contact:
Benjamin-Heinz Meier/ benjamin-heinz.meier@student.hpi.de
Matthias Trapp / matthias.trapp@hpi.de
Jürgen Döllner / juergen.doellner@hpi.de
Publications: www.4dndvis.de/publikationen.html
This work was funded by the Federal Ministry of Education
and Research (BMBF), Germany within the InnoProfile
Transfer research group "4DnD-Vis".

44. VideoMR:
A Map and Reduce Framework for
Real-time Video Processing
Benjamin-Heinz Meier, Matthias Trapp, Jürgen Döllner
Hasso Plattner Institute, University of Potsdam, Germany
17

45. 18
PAPER
[Dean'04]
Dean, J. & Ghemawat, S. MapReduce: Simplified Data
Processing on Large Clusters OSDI'04: Sixth Symposium on
Operating System Design and Implementation, 2004
[Tho'11]
Thompson, M.; Farley, D.; Barker, M.; Gee,
P. & Stewart, A. DISRUPTOR: High performance
alternative to bounded queues for exchanging data