An introduction to the Pixel Bender Kernel and Graph Languages for highly optimized signal processing in heterogeneous computing environments.

1. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Kevin Goldsmith | Senior Engineering Manager
Bob Archer | Senior Computer Scientist
Pixel Bender

2. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What is Pixel Bender?
A domain specific kernel and graph language for image and video processing
designed to efficiently target current and future heterogeneous hardware.
2

3. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
3
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

4. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
4
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

5. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
5
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

6. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
6
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>
High Pass Texturize
Saturate
Blend
Vacation

7. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
7
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>
•Explicitly Data Parallel
•Explicitly Lock Free
•Implicitly Vectorizable
•Contains Optimization Hints
•Region Reasoning
•Parameter Ranges
•Supports per-frame functions
•JIT
•x86 with SSE
•GLSL
•Implicitly Task Parallel
•Contains Optimization Hints
•Region Reasoning
•Parameter Ranges
•Parameter Use
•Supports per frame and internal
graph logic
•Language/Hardware Agnostic
•Enforces the programming Model

8. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
8

9. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
9

10. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
10

11. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
11

12. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo
12

13. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
13

14. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
14

15. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
15

16. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
16

17. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Warning.
Speculation ahead.
17

18. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How do we make the most of the hardware
we have available?
18

19. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Writing multi threaded code is like juggling
chainsaws; amazing when it works and truly
sucky when it doesn’t.
Andrew Wulf
19

20. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
20

21. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
§ Gflop / watt
21

22. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
§ Gflop / watt
Gflop / developer-hour
22

23. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
23

24. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
24

25. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
25

26. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
26

27. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
27

28. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
28

29. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
29

30. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What cross platform means to Adobe
§ Mac
§ Windows
§ AMD
§ Intel
§ nVidia
§ CPU
§ GPU
§ Mobile hardware
30

31. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
31

32. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
OpenCL
32

33. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
33

34. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
34

35. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
35

36. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
§ Race free by construction
36

37. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
§ Race free by construction
§ Parallelism for the masses
37

38. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Warning.
Increased speculation
ahead.
38

39. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender has certain disadvantages:
§ One fixed programming model
§ Focuses on image processing
§ Not general enough
39

40. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Framework
40
Inputs Framework Outputs

41. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Framework
41
Inputs Framework Outputs

42. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Michael McCool’s parallel programming patterns
§ Gather
§ Map
§ Reduce
§ Superscalar sequences
§ Pipeline
§ Nesting
§ Scans
§ Recurrences
§ Search
§ Subdivision
§ Stencil
§ Scatter
§ Pack
§ Selection
§ Partition
§ Expand
42

43. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What are we going to do about it?
43

44. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Internal customers
44

45. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
External customers
45

46. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
References
Pixel Bender
http://www.adobe.com/devnet/pixelbender.html
http://blogs.adobe.com/pixel-bender/
http://blogs.adobe.com/kevin-goldsmith/tag/pixel-bender-2
Structured Parallel Programming with Deterministic Patterns
Michael D. McCool http://software.intel.com/file/27160
Patterns for Parallel Programming
Mattson, Sanders & Massingill
46

47. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

48. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
48
Pixel Bender programming model
Write a function that produces a single
output pixel

49. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
49
Pixel Bender programming model

50. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
50
Pixel Bender programming model

51. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
51
Pixel Bender programming model

52. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
52
Pixel Bender programming model

53. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

54. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why Pixel Bender?
§ 2001 programmable shading units on the GPU
§ 2005 dual core CPUs
§ 2005 AIF team formed
§ 2006 quad core CPUs
§ 2006 AMD announces CTM
§ 2008 OpenCL
§ 2011 fusion chips
54

55. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender History
§ 2001 programmable shading units on the GPU
§ 2005 dual core CPUs
§ 2005 AIF team formed
§ 2006 quad core CPUs
§ 2006 AMD announces CTM
§ 2007 After Effects CS3 replaces GLSL code with Pixel Bender
§ 2007 Flash Player 10 Announced including Pixel Bender support
§ 2008 OpenCL
§ 2008 After Effects CS4 - first CPU implementation, 3rd party plug-ins, Pixel Bender
Graph language announced
§ 2008 Photoshop CS4 – Pixel Bender plug-in released
§ 2010 Pixel Bender commercial plug-ins start to appear en masse
§ 2010 Flash Player Molehill Announced – Pixel Bender 3D
§ 2011 fusion chips
55