1.
Using Language Oriented
Programming to Execute
Computations on the GPU
Robert Pickering, ALTI

2.
About the Presenter
• Using F# for about 6 years
• Oldest F# user outside of Microsoft
• Written a book about F#
(now in its second edition)
• Spend at least 2 years as a professional
functional programmer
• I have 3 cats
Contact me:
robert@strangelights.com
http://strangelights.com/blog
2

3.
ALTI
• Large French services company
• Consulting, engineering and
training
• Believe in helping clients
discover upcoming niches
• Already offer F# training and
consulting

4.
What is Language Oriented
Programming ?
• Creating programs that are an abstract
description of the a problem
• This description can then either be
interpreted, compiled, or analyzed in another
way

5.
LOP & Combinators
• F# has two main approaches to Language
Oriented Programming:
– Reinterpreting the F# syntax though the
quotations system
– Using combinators create a new syntax
… both approaches are very similar

6.
What is a Combinator?
A combinator is a higher-order function that
uses only function application and earlier
defined combinators to define a result from its
arguments.
Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

7.
Combinatory Logic in Computing
In computer science, combinatory logic is used
as a simplified model of computation, used in
computability theory and proof theory. Despite
its simplicity, combinatory logic captures many
essential features of computation.
Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic

8.
Combinator Library
"A combinator library offers functions (the
combinators) that combine functions together to make
bigger functions"[1]. These kinds of libraries are
particularly useful for allowing domain-specific
programming languages to be easily embedded into a
general purpose language by defining a few primitive
functions for the given domain.
Souce: Wikipedia http://en.wikipedia.org/wiki/Combinator_library
[1] “A History of Haskell” Hudak, Hughes, Peyton Jones, Wadler

9.
History of Haskell: Combinator Libraries
What is a combinator library? The reader will
search in vain for a definition of this heavily
used term, but the key idea is this: a combinator
library offers functions (the combinators) that
combine functions together to make bigger
functions.

10.
History of Haskell: Combinator Libraries
What is a combinator library? The reader will
search in vain for a definition of this heavily
used term, but the key idea is this: a combinator
library offers functions (the combinators) that
combine functions together to make bigger
functions.

11.
History of Haskell: Combinator Libraries
Another productive way to think of a
combinator library is as a domain-specific
language (DSL) for describing values of a
particular type.

12.
What is a Domain Specific Language?
A programming language tailored for a particular application
domain, which captures precisely the semantics of the
application domain -- no more, no less.
A DSL allows one to develop software for a particular
application domain quickly, and effectively, yielding
programs that are easy to understand, reason about, and
maintain.
Hudak

13.
Combinators vs DSLs
• Combinartor libraries are a special case of DSLs
– Sometimes called DSELs (Domain Specific Embed languages)
• DSELs have several advantages:
― Inherit non-domain-specific parts of the design.
― Inherit compilers and tools.
― Uniform “look and feel” across many DSLs
― DSLs integrated with full programming language, and with each other.
• DSELs one disadvantage:
― Constrained by host language syntax and type system

14.
What Makes F# a Suitable for DSLs ?
• Union Types / Active Patterns
– type Option<'a> = Some x | None
• Lambda functions
– fun x -> x + 1
• Define and redefine operators
– let (++) x = x + 1
• Define custom numeric literals
– let x : Expression = 1.0N

15.
Union Types – The Option Type
// The pre-defined option type
type Option<'a> =
| Some of 'a
| None
// constructing options
let someValue = Some 1
let noValue = None
// pattern matching over options
let convert value =
match value with
| Some x -> Printf.sprintf "Value: %i" x
| None -> "No value"

16.
Union Types - Trees
// a binary tree definition
type BinaryTree<'a> =
| Node of BinaryTree<'a> * BinaryTree<'a>
| Leaf of 'a
// walk the tree collection values
let rec collectValues acc tree =
match tree with
| Node(ltree, rtree) ->
// recursively walk the left tree
let acc = collectValues acc ltree
// recursively walk the right tree
collectValues acc rtree
| Leaf value -> value :: acc
// add value to accumulator

17.
Using the Tree
// define a tree
let tree =
Node(
Node(Leaf 1, Leaf 2),
Node(Leaf 3, Leaf 4))
// recover all values from the leaves
let values = collectValues [] tree

18.
Union Types
Union types play a key role in language oriented
programming, as they allow the user to easily
define a tree that will form the abstract syntax
tree of the language

19.
Active Patterns
// definition of the active pattern
let (|Bool|Int|Float|String|) input =
// attempt to parse a bool
let sucess, res = Boolean.TryParse input
if sucess then Bool(res)
else
// attempt to parse an int
let sucess, res = Int32.TryParse input
if sucess then Int(res)
else
// attempt to parse a float (Double)
let sucess, res = Double.TryParse input
if sucess then Float(res)
else String(input)

20.
Active Patterns
// function to print the results by pattern
// matching over the active pattern
let printInputWithType input =
match input with
| Bool b -> printfn "Boolean: %b" b
| Int i -> printfn "Integer: %i" i
| Float f -> printfn "Floating point: %f" f
| String s -> printfn "String: %s" s
// print the results
printInputWithType "true"
printInputWithType "12"

21.
Active Patterns
Active patterns play a supporting role in
language oriented programming in F#. They
allow the programmer to create abstractions of
complex operations on the abstract syntax tree

22.
Lambda Functions
fun () ->
lovin'()

23.
Lambda Functions
Lambda functions are important for language
oriented programming. They allow the users of
the language to embed actions within other
language elements

24.
Custom Operators
let (++) x = x + 1

25.
Custom Operators
Custom operators play a supporting role in
language oriented programming. They allow the
language designer to have a more flexible
syntax.

26.
Custom Numeric Literals
type Expression =
| Constant of int
module NumericLiteralN =
let FromZero() = Constant 0
let FromOne() = Constant 1
let FromInt32 = Constant
let expr = 1N

27.
Custom Numeric Literals
Numeric literals play a supporting role in
language oriented programming. They allow
numeric literals be treated in a more natural
within an embedded language

28.
The Anatomy of a DSL
type Expression = Syntax Tree
| Add of Expression * Expression
| Subtract of Expression * Expression
| Multiply of Expression * Expression
| Constant of int Combinators
| Parameter of string
with
static member (+) (x, y) = Add(x, y)
static member (-) (x, y) = Subtract(x, y)
static member (*) (x, y) = Multiply(x, y)
module NumericLiteralN =
let FromZero() = Constant 0
let FromOne() = Constant 1
let FromInt32 = Constant
let param = Parameter

29.
The Anatomy of a DSL
let expr = (1N + 2N) * (5N - 2N)
val expr : Expression =
Multiply (Add (Constant 1,Constant 2),
Subtract (Constant 5,Constant 2))

30.
The Anatomy of a DSL
• Expressions now have an abstract tree like
representation:
― Multiply
― Add
― Constant 1
― Constant 2
― Subtract
― Constant 5
― Constant 2
• This can then be evaluated
• Or we can preform more advanced analysis

31.
The Anatomy of a DSL
let evaluateExpression parameters =
let rec innerEval tree =
match tree with
| Multiply (x, y) -> innerEval x * innerEval y
| Add (x, y) -> innerEval x + innerEval y
| Subtract (x, y) -> innerEval x - innerEval y
| Constant value -> value
| Parameter key -> Map.find key parameters
innerEval
let expr = (1N + 2N) * (5N - 2N)
evaluateExpression Map.empty expr

32.
The Anatomy of a DSL

33.
The Anatomy of a DSL
let rec simplifyExpression exp =
let simpIfPoss op exp1 exp2 =
let exp' = op (simplifyExpression exp1,
simplifyExpression exp2)
if exp' = exp then exp' else simplifyExpression exp'
match exp with
| Multiply(Constant 0, Constant _) -> Constant 0
| Multiply(Constant _, Constant 0) -> Constant 0
| Multiply(Constant n1, Constant n2) -> Constant (n1 * n2)
| Add(Constant n1, Constant n2) -> Constant (n1 + n2)
| Subtract(Constant n1, Constant n2) -> Constant (n1 - n2)
| Multiply(exp1, exp2) -> simpIfPoss Multiply exp1 exp2
| Add(exp1, exp2) -> simpIfPoss Add exp1 exp2
| Subtract(exp1, exp2) -> simpIfPoss Add exp1 exp2
| Constant _ | Parameter _ -> exp

34.
The Anatomy of a DSL

35.
Quotations
let quotation = <@ (1 + 2) * (5 - 2) @>
val quotation : Quotations.Expr<int> =
Call (None, Int32 op_Multiply[Int32,Int32,Int32](Int32, Int32),
[Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32),
[Value (1), Value (2)]),
Call (None, Int32 op_Subtraction[Int32,Int32,Int32](Int32, Int32),
[Value (5), Value (2)])])

36.
Quotations
• Allow you to grab a tree structure that
represents a piece of F# code
• This tree structure can then be:
– Interpreted
– Compiled
– Converted to instructions understood by another
system or runtime

37.
Microsoft Accelerator
• A project from MRS, now available on
Microsoft Connect [1]
• Automatically parallelize code for execution
on the GPU or x64 multicore
• Implemented as an unmanaged library with
.NET wrapper
[1] http://connect.microsoft.com/acceleratorv2

38.
Microsoft Accelerator
• Based on “Parallel Array”
• You define a set of operations to process the
content of the array
• The Accelerator runtime will then process the
operations in parallel

39.
Microsoft Accelerator
User Program – Define Operations
Accelerator
Input data Accelerated Program Input data
DX9Target X64Target

40.
Microsoft Accelerator – Code!
let nums = [| 6; 1; 5; 5; 3 |]
let input = new FloatParallelArray(nums);
let sum = ParallelArrays.Shift(input, 1) + input +
ParallelArrays.Shift(input, -1);
let output = sum / 3.0f;
let target = new DX9Target();
let res = target.ToArray1D(output);

41.
Game of Life

42.
Game of Life
• Green - When an existing cell (green in the middle) has three
or two neighbours it survives to the next round
• Red - When an existing cell has less than two neighbours it
dies, because it is lonely (first red case), when it has more
than three neighbours it dies of overcrowding (second red
case)
• Blue - Finally, a new cell is born in an empty grid location if
there are exactly three neighbours .

43.
The implementation
1 1
1 1
initial rotation 1
Sum of all
etc. ... neighbours

44.
Game of Life
CPU GPU

45.
Game of Life
/// Evaluate next generation of the life game state
let nextGeneration (grid: Matrix<float32>) =
// Shift in each direction, to count the neighbours
let sum = shiftAndSum grid offsets
// Check to see if we're born or remain alive
(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)
CPU GPU

46.
Game of Life
/// Evaluate next generation of the life game state
[<ReflectedDefinition>]
let nextGeneration (grid: Matrix<float32>) =
// Shift in each direction, to count the neighbours
let sum = shiftAndSum grid offsets
// Check to see if we're born or remain alive
(sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid)
CPU GPU

47.
Game of life in F# with Microsoft Accelerator
DEMO

48.
Thanks!
• A big thanks to Tomáš Petříček!
• More info on his blog:
– http://tomasp.net/articles/accelerator-intro.aspx

49.
Books

50.
Shameless Plug!
• Robert Pickering’s Beginning F# Workshop:
– Thursday 9th Sept – 10th September 2010
http://skillsmatter.com/course/open-source-
dot-net/robert-pickerings-beginning-f-workshop