Using Language Oriented Programming to Execute Computations on the GPU

1,662 views

Published on

F# has a number of features that support language oriented programming (LOP) – the ability to create an abstract description of a problem then have this description executed in another environment. In this talk we’ll look at the design of an F# library that uses LOP techniques to a user execute matrix calculations either on the CPU or GPU. We’ll examine the features that F# provides to support this technique. We’ll start by taking a look at union types and active patterns, and then we’ll see how these are used by F#’s quotation system to give access to an abstract description of functions. Finally, we’ll see how these descriptions of functions can then be translated into computations the GPU understands and executed.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,662
On SlideShare
0
From Embeds
0
Number of Embeds
180
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Using Language Oriented Programming to Execute Computations on the GPU

  1. 1. Using Language Oriented Programming to Execute Computations on the GPU Robert Pickering, ALTI
  2. 2. About the Presenter • Using F# for about 6 years • Oldest F# user outside of Microsoft • Written a book about F# (now in its second edition) • Spend at least 2 years as a professional functional programmer • I have 3 cats Contact me: robert@strangelights.com http://strangelights.com/blog 2
  3. 3. ALTI • Large French services company • Consulting, engineering and training • Believe in helping clients discover upcoming niches • Already offer F# training and consulting
  4. 4. What is Language Oriented Programming ? • Creating programs that are an abstract description of the a problem • This description can then either be interpreted, compiled, or analyzed in another way
  5. 5. LOP & Combinators • F# has two main approaches to Language Oriented Programming: – Reinterpreting the F# syntax though the quotations system – Using combinators create a new syntax … both approaches are very similar
  6. 6. What is a Combinator? A combinator is a higher-order function that uses only function application and earlier defined combinators to define a result from its arguments. Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic
  7. 7. Combinatory Logic in Computing In computer science, combinatory logic is used as a simplified model of computation, used in computability theory and proof theory. Despite its simplicity, combinatory logic captures many essential features of computation. Source: Wikipedia, http://en.wikipedia.org/wiki/Combinatory_Logic
  8. 8. Combinator Library "A combinator library offers functions (the combinators) that combine functions together to make bigger functions"[1]. These kinds of libraries are particularly useful for allowing domain-specific programming languages to be easily embedded into a general purpose language by defining a few primitive functions for the given domain. Souce: Wikipedia http://en.wikipedia.org/wiki/Combinator_library [1] “A History of Haskell” Hudak, Hughes, Peyton Jones, Wadler
  9. 9. History of Haskell: Combinator Libraries What is a combinator library? The reader will search in vain for a definition of this heavily used term, but the key idea is this: a combinator library offers functions (the combinators) that combine functions together to make bigger functions.
  10. 10. History of Haskell: Combinator Libraries What is a combinator library? The reader will search in vain for a definition of this heavily used term, but the key idea is this: a combinator library offers functions (the combinators) that combine functions together to make bigger functions.
  11. 11. History of Haskell: Combinator Libraries Another productive way to think of a combinator library is as a domain-specific language (DSL) for describing values of a particular type.
  12. 12. What is a Domain Specific Language? A programming language tailored for a particular application domain, which captures precisely the semantics of the application domain -- no more, no less. A DSL allows one to develop software for a particular application domain quickly, and effectively, yielding programs that are easy to understand, reason about, and maintain. Hudak
  13. 13. Combinators vs DSLs • Combinartor libraries are a special case of DSLs – Sometimes called DSELs (Domain Specific Embed languages) • DSELs have several advantages: ― Inherit non-domain-specific parts of the design. ― Inherit compilers and tools. ― Uniform “look and feel” across many DSLs ― DSLs integrated with full programming language, and with each other. • DSELs one disadvantage: ― Constrained by host language syntax and type system
  14. 14. What Makes F# a Suitable for DSLs ? • Union Types / Active Patterns – type Option<'a> = Some x | None • Lambda functions – fun x -> x + 1 • Define and redefine operators – let (++) x = x + 1 • Define custom numeric literals – let x : Expression = 1.0N
  15. 15. Union Types – The Option Type // The pre-defined option type type Option<'a> = | Some of 'a | None // constructing options let someValue = Some 1 let noValue = None // pattern matching over options let convert value = match value with | Some x -> Printf.sprintf "Value: %i" x | None -> "No value"
  16. 16. Union Types - Trees // a binary tree definition type BinaryTree<'a> = | Node of BinaryTree<'a> * BinaryTree<'a> | Leaf of 'a // walk the tree collection values let rec collectValues acc tree = match tree with | Node(ltree, rtree) -> // recursively walk the left tree let acc = collectValues acc ltree // recursively walk the right tree collectValues acc rtree | Leaf value -> value :: acc // add value to accumulator
  17. 17. Using the Tree // define a tree let tree = Node( Node(Leaf 1, Leaf 2), Node(Leaf 3, Leaf 4)) // recover all values from the leaves let values = collectValues [] tree
  18. 18. Union Types Union types play a key role in language oriented programming, as they allow the user to easily define a tree that will form the abstract syntax tree of the language
  19. 19. Active Patterns // definition of the active pattern let (|Bool|Int|Float|String|) input = // attempt to parse a bool let sucess, res = Boolean.TryParse input if sucess then Bool(res) else // attempt to parse an int let sucess, res = Int32.TryParse input if sucess then Int(res) else // attempt to parse a float (Double) let sucess, res = Double.TryParse input if sucess then Float(res) else String(input)
  20. 20. Active Patterns // function to print the results by pattern // matching over the active pattern let printInputWithType input = match input with | Bool b -> printfn "Boolean: %b" b | Int i -> printfn "Integer: %i" i | Float f -> printfn "Floating point: %f" f | String s -> printfn "String: %s" s // print the results printInputWithType "true" printInputWithType "12"
  21. 21. Active Patterns Active patterns play a supporting role in language oriented programming in F#. They allow the programmer to create abstractions of complex operations on the abstract syntax tree
  22. 22. Lambda Functions fun () -> lovin'()
  23. 23. Lambda Functions Lambda functions are important for language oriented programming. They allow the users of the language to embed actions within other language elements
  24. 24. Custom Operators let (++) x = x + 1
  25. 25. Custom Operators Custom operators play a supporting role in language oriented programming. They allow the language designer to have a more flexible syntax.
  26. 26. Custom Numeric Literals type Expression = | Constant of int module NumericLiteralN = let FromZero() = Constant 0 let FromOne() = Constant 1 let FromInt32 = Constant let expr = 1N
  27. 27. Custom Numeric Literals Numeric literals play a supporting role in language oriented programming. They allow numeric literals be treated in a more natural within an embedded language
  28. 28. The Anatomy of a DSL type Expression = Syntax Tree | Add of Expression * Expression | Subtract of Expression * Expression | Multiply of Expression * Expression | Constant of int Combinators | Parameter of string with static member (+) (x, y) = Add(x, y) static member (-) (x, y) = Subtract(x, y) static member (*) (x, y) = Multiply(x, y) module NumericLiteralN = let FromZero() = Constant 0 let FromOne() = Constant 1 let FromInt32 = Constant let param = Parameter
  29. 29. The Anatomy of a DSL let expr = (1N + 2N) * (5N - 2N) val expr : Expression = Multiply (Add (Constant 1,Constant 2), Subtract (Constant 5,Constant 2))
  30. 30. The Anatomy of a DSL • Expressions now have an abstract tree like representation: ― Multiply ― Add ― Constant 1 ― Constant 2 ― Subtract ― Constant 5 ― Constant 2 • This can then be evaluated • Or we can preform more advanced analysis
  31. 31. The Anatomy of a DSL let evaluateExpression parameters = let rec innerEval tree = match tree with | Multiply (x, y) -> innerEval x * innerEval y | Add (x, y) -> innerEval x + innerEval y | Subtract (x, y) -> innerEval x - innerEval y | Constant value -> value | Parameter key -> Map.find key parameters innerEval let expr = (1N + 2N) * (5N - 2N) evaluateExpression Map.empty expr
  32. 32. The Anatomy of a DSL
  33. 33. The Anatomy of a DSL let rec simplifyExpression exp = let simpIfPoss op exp1 exp2 = let exp' = op (simplifyExpression exp1, simplifyExpression exp2) if exp' = exp then exp' else simplifyExpression exp' match exp with | Multiply(Constant 0, Constant _) -> Constant 0 | Multiply(Constant _, Constant 0) -> Constant 0 | Multiply(Constant n1, Constant n2) -> Constant (n1 * n2) | Add(Constant n1, Constant n2) -> Constant (n1 + n2) | Subtract(Constant n1, Constant n2) -> Constant (n1 - n2) | Multiply(exp1, exp2) -> simpIfPoss Multiply exp1 exp2 | Add(exp1, exp2) -> simpIfPoss Add exp1 exp2 | Subtract(exp1, exp2) -> simpIfPoss Add exp1 exp2 | Constant _ | Parameter _ -> exp
  34. 34. The Anatomy of a DSL
  35. 35. Quotations let quotation = <@ (1 + 2) * (5 - 2) @> val quotation : Quotations.Expr<int> = Call (None, Int32 op_Multiply[Int32,Int32,Int32](Int32, Int32), [Call (None, Int32 op_Addition[Int32,Int32,Int32](Int32, Int32), [Value (1), Value (2)]), Call (None, Int32 op_Subtraction[Int32,Int32,Int32](Int32, Int32), [Value (5), Value (2)])])
  36. 36. Quotations • Allow you to grab a tree structure that represents a piece of F# code • This tree structure can then be: – Interpreted – Compiled – Converted to instructions understood by another system or runtime
  37. 37. Microsoft Accelerator • A project from MRS, now available on Microsoft Connect [1] • Automatically parallelize code for execution on the GPU or x64 multicore • Implemented as an unmanaged library with .NET wrapper [1] http://connect.microsoft.com/acceleratorv2
  38. 38. Microsoft Accelerator • Based on “Parallel Array” • You define a set of operations to process the content of the array • The Accelerator runtime will then process the operations in parallel
  39. 39. Microsoft Accelerator User Program – Define Operations Accelerator Input data Accelerated Program Input data DX9Target X64Target
  40. 40. Microsoft Accelerator – Code! let nums = [| 6; 1; 5; 5; 3 |] let input = new FloatParallelArray(nums); let sum = ParallelArrays.Shift(input, 1) + input + ParallelArrays.Shift(input, -1); let output = sum / 3.0f; let target = new DX9Target(); let res = target.ToArray1D(output);
  41. 41. Game of Life
  42. 42. Game of Life • Green - When an existing cell (green in the middle) has three or two neighbours it survives to the next round • Red - When an existing cell has less than two neighbours it dies, because it is lonely (first red case), when it has more than three neighbours it dies of overcrowding (second red case) • Blue - Finally, a new cell is born in an empty grid location if there are exactly three neighbours .
  43. 43. The implementation 1 1 1 1 initial rotation 1 Sum of all etc. ... neighbours
  44. 44. Game of Life CPU GPU
  45. 45. Game of Life /// Evaluate next generation of the life game state let nextGeneration (grid: Matrix<float32>) = // Shift in each direction, to count the neighbours let sum = shiftAndSum grid offsets // Check to see if we're born or remain alive (sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid) CPU GPU
  46. 46. Game of Life /// Evaluate next generation of the life game state [<ReflectedDefinition>] let nextGeneration (grid: Matrix<float32>) = // Shift in each direction, to count the neighbours let sum = shiftAndSum grid offsets // Check to see if we're born or remain alive (sum =. threeAlive) ||. ((sum =. twoAlive) &&. grid) CPU GPU
  47. 47. Game of life in F# with Microsoft Accelerator DEMO
  48. 48. Thanks! • A big thanks to Tomáš Petříček! • More info on his blog: – http://tomasp.net/articles/accelerator-intro.aspx
  49. 49. Books
  50. 50. Shameless Plug! • Robert Pickering’s Beginning F# Workshop: – Thursday 9th Sept – 10th September 2010 http://skillsmatter.com/course/open-source- dot-net/robert-pickerings-beginning-f-workshop

×