The document provides an overview of Pixel Bender, a domain-specific language developed by Adobe for image and video processing, targeting heterogeneous hardware. It details the structure of Pixel Bender kernels and graphs, including parameters, connections, and runtime scheduling for optimization. Additionally, it shares lessons learned in software development, emphasizing the importance of ease of programmability and just-in-time compilation.

1. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Kevin Goldsmith | Senior Engineering Manager
Bob Archer | Senior Computer Scientist
Pixel Bender

2. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What is Pixel Bender?
A domain specific kernel and graph language for image and video processing
designed to efficiently target current and future heterogeneous hardware.
2

3. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
3
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

4. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
4
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

5. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
5
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>

6. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
6
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>
High Pass Texturize
Saturate
Blend
Vacation

7. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Kernels & Graphs
7
<languageVersion : 1.0;>
kernel pixelate_sample
<
namespace : "AIF Test";
vendor : "Adobe";
version : 1;>
{
parameter int dimension;
input image4 inputImage;
output pixel4 outputPixel;
void evaluatePixel()
{
float dimAsFloat = float(dimension);
float2 sc = floor(outCoord() /
float2(dimAsFloat, dimAsFloat));
sc *= dimAsFloat;
outputPixel = sampleNearest(inputImage, sc);
}
}
<?xml version="1.0" encoding="utf-8"?>
<graph name = "PixelateGraph"xmlns="http://ns.adobe.com/PixelBenderGraph/1.0">
<metadata name = "namespace" value = "AIF"/>
<metadata name = "vendor" value = "Adobe Systems" />
<metadata name = "version" type = "int" value = "1" />
<parameter type = "int" name = "dimension" >
<metadata name = "defaultValue" type = "int" value = "1" />
<metadata name = "minValue" type = "int" value = "1" />
<metadata name = "maxValue" type = "int" value = "100" />
</parameter>
<inputImage type = "image4" name = "inputImage" />
<outputImage type = "image4" name = "outputImage" />
<kernel>
<![CDATA[
<languageVersion : 1.0;>
kernel Pixelate
<
namespace:"AIF";
vendor:"Adobe Systems";
version:1;
>
{
…
}
]]>
</kernel>
<node id = "pixelateFilter" name ="Pixelate" namespace = "AIF" vendor = "Adobe Systems" version ="1"
clientID ="ADBE Pixelate" >
<evaluateParameters>
<![CDATA[
void evaluateParameters()
{
pixelateFilter::dimension = dimension;
}
]]>
</evaluateParameters>
</node>
<!-- Connect the graph -->
<connect fromImage = "inputImage" toNode = "pixelateFilter" toInput = "src" />
<connect fromNode = "pixelateFilter" fromOutput = "dst" toImage = "outputImage" />
</graph>
•Explicitly Data Parallel
•Explicitly Lock Free
•Implicitly Vectorizable
•Contains Optimization Hints
•Region Reasoning
•Parameter Ranges
•Supports per-frame functions
•JIT
•x86 with SSE
•GLSL
•Implicitly Task Parallel
•Contains Optimization Hints
•Region Reasoning
•Parameter Ranges
•Parameter Use
•Supports per frame and internal
graph logic
•Language/Hardware Agnostic
•Enforces the programming Model

8. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
8

9. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
9

10. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
10

11. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender Runtime
§ Run-time scheduling
§ Decide if a Pixel Bender node runs on the GPU or CPU
§ Order the execution of each node
§ Drives Optimization
§ Runs region reasoning pass
§ Concatenate adjacent Pixel Bender nodes
§ Fix parameter values for constant folding and range propagation
§ Caches intra-graph frames when possible to reduce computation
§ Allows host application to aid optimization
§ Lock parameter values
§ Lock inputs
§ Interfaces with host application resource management
§ Application can force CPU or GPU execution
§ Application can provide memory or threading management
11

12. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo
12

13. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
13

14. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
14

15. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
15

16. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Lessons Learned (so far)
§ A DSL allows you to limit the language for optimization purposes while
preserving programmability
§ Just-in-time Compilation allows you not only to use runtime information for
optimization purposes, but also allows you to future-proof algorithms against
later hardware architectures
§ Developers will learn new programming languages if they give them an ability to
do something new, or make something significantly easier
§ A language with only a text representation creates some nice network and
community effects (for a time)
16

17. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Warning.
Speculation ahead.
17

18. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How do we make the most of the hardware
we have available?
18

19. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Writing multi threaded code is like juggling
chainsaws; amazing when it works and truly
sucky when it doesn’t.
Andrew Wulf
19

20. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
20

21. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
§ Gflop / watt
21

22. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
§ Gflop / $
§ Gflop / watt
Gflop / developer-hour
22

23. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
23

24. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
24

25. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
25

26. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
26

27. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
27

28. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
28

29. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
29

30. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What cross platform means to Adobe
§ Mac
§ Windows
§ AMD
§ Intel
§ nVidia
§ CPU
§ GPU
§ Mobile hardware
30

31. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
31

32. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What have we got?
§ DSL for image processing (Pixel Bender)
§ JIT compiler / optimizer
§ Runtime – enforces programming model, allows graphs
§ High performance on parallel hardware (using OpenGL)
§ Future proof against new hardware
§ Cross platform
OpenCL
32

33. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
33

34. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
34

35. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
35

36. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
§ Race free by construction
36

37. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why not just switch entirely to OpenCL?
Pixel Bender has certain advantages:
§ Ease of writing
§ Optimization opportunities
§ Race free by construction
§ Parallelism for the masses
37

38. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Warning.
Increased speculation
ahead.
38

39. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender has certain disadvantages:
§ One fixed programming model
§ Focuses on image processing
§ Not general enough
39

40. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Framework
40
Inputs Framework Outputs

41. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Framework
41
Inputs Framework Outputs

42. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Michael McCool’s parallel programming patterns
§ Gather
§ Map
§ Reduce
§ Superscalar sequences
§ Pipeline
§ Nesting
§ Scans
§ Recurrences
§ Search
§ Subdivision
§ Stencil
§ Scatter
§ Pack
§ Selection
§ Partition
§ Expand
42

43. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
What are we going to do about it?
43

44. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Internal customers
44

45. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
External customers
45

46. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
References
Pixel Bender
http://www.adobe.com/devnet/pixelbender.html
http://blogs.adobe.com/pixel-bender/
http://blogs.adobe.com/kevin-goldsmith/tag/pixel-bender-2
Structured Parallel Programming with Deterministic Patterns
Michael D. McCool http://software.intel.com/file/27160
Patterns for Parallel Programming
Mattson, Sanders & Massingill
46

47. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

48. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
48
Pixel Bender programming model
Write a function that produces a single
output pixel

49. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
49
Pixel Bender programming model

50. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
50
Pixel Bender programming model

51. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
51
Pixel Bender programming model

52. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
52
Pixel Bender programming model

53. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.

54. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Why Pixel Bender?
§ 2001 programmable shading units on the GPU
§ 2005 dual core CPUs
§ 2005 AIF team formed
§ 2006 quad core CPUs
§ 2006 AMD announces CTM
§ 2008 OpenCL
§ 2011 fusion chips
54

55. © 2011 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Pixel Bender History
§ 2001 programmable shading units on the GPU
§ 2005 dual core CPUs
§ 2005 AIF team formed
§ 2006 quad core CPUs
§ 2006 AMD announces CTM
§ 2007 After Effects CS3 replaces GLSL code with Pixel Bender
§ 2007 Flash Player 10 Announced including Pixel Bender support
§ 2008 OpenCL
§ 2008 After Effects CS4 - first CPU implementation, 3rd party plug-ins, Pixel Bender
Graph language announced
§ 2008 Photoshop CS4 – Pixel Bender plug-in released
§ 2010 Pixel Bender commercial plug-ins start to appear en masse
§ 2010 Flash Player Molehill Announced – Pixel Bender 3D
§ 2011 fusion chips
55