Domain specific languages and Scala

4,054 views
3,829 views

Published on

The presentation was given to Rivier Scala / Clojure User Group meeting on 10.6.2013. It is half-baked presentation. Will upload the final version when ready.
The first part is about DSLs in general, complexities in software engineering and abstraction. The seconds part presents an quick overview about DSLs in Scala and touches some of the technologies used for deep embedding.

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,054
On SlideShare
0
From Embeds
0
Number of Embeds
19
Actions
Shares
0
Downloads
0
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Domain specific languages and Scala

  1. 1. Domain-Specific Languages & Scala Filip Křikava Riviera Scala/Clojure User Group meeting 10.6.2013
  2. 2. Why should I care about Domain-Specific Languages?
  3. 3. Because it is modern?
  4. 4. Because it is cool?
  5. 5. Because everybody talks about it?
  6. 6. No, you should care because of ...
  7. 7. werewolves.
  8. 8. Let’s take a step back
  9. 9. What is programming?
  10. 10. Mapping Problem-level abstractions Implementation-level abstractions mapping
  11. 11. Abstraction Gothic Security & Miss Grant’s Controller2 mapping Event doorClosed = new Event("doorClosed", "D1CL"); Event drawerOpened = new Event("drawerOpened", "D2OP"); Event lightOn = new Event("lightOn", "L1ON"); Event doorOpened = new Event("doorOpened", "D1OP"); Event panelClosed = new Event("panelClosed", "PNCL"); Command unlockPanelCmd = new Command("unlockPanel", "PNUL"); Command lockPanelCmd = new Command("lockPanel", "PNLK"); Command lockDoorCmd = new Command("lockDoor", "D1LK"); Command unlockDoorCmd = new Command("unlockDoor", "D1UL"); State idle = new State("idle"); State activeState = new State("active"); State waitingForLightState = new State("waitingForLight"); State waitingForDrawerState = new State("waitingForDrawer"); State unlockedPanelState = new State("unlockedPanel"); StateMachine machine = new StateMachine(idle); idle.addTransition(doorClosed, activeState); idle.addAction(unlockDoorCmd); idle.addAction(lockPanelCmd); activeState.addTransition(drawerOpened, waitingForLightState); activeState.addTransition(lightOn, waitingForDrawerState); waitingForLightState.addTransition(lightOn, unlockedPanelState); waitingForDrawerState.addTransition(drawerOpened, unlockedPanelState); unlockedPanelState.addAction(unlockPanelCmd); unlockedPanelState.addAction(lockDoorCmd); unlockedPanelState.addTransition(panelClosed, idle); machine.addResetEvents(doorOpened); Implementation-level abstractions2 Fowler, Martin (2010). Domain Specific Languages (1st ed.).Addison-Wesley Professional. http://www.metacase.com/blogs/stevek/blogView?showComments=true&entry=34148345201 2 Problem-level abstractions1 ” “Miss Grant has a secret compartment in her bedroom that is normally locked and concealed....
  12. 12. So?
  13. 13. Contemporary computing systems are very complex
  14. 14. Complex Software Development • Developing these systems using too much code-centric technologies is a herculean effort, because: • wide conceptual gap between the problem and the implementation • bridging this gap with extensive handcrafting gives rise to accidental complexities Problem-level abstractions Implementation-level abstractions wide conceptual gap in this mapping
  15. 15. Two Complexities • Essential Complexity - i.e. complexity that is intrinsic to the problem you are trying to solve. • Accidental Complexity - i.e. complexity that is caused by the approach you have chosen to solve that
  16. 16. ” “We marvel at these software implementations in much the same way that archaeologists marvel at the pyramids:The wonder is mostly based on an appreciation of the effort required to tackle the significant accidental complexities arising from the use of inadequate technologies. France, R., & Rumpe, B. (2007). Model-driven Development of Complex Software:A Research Roadmap. Future of Software Engineering (FOSE ’07) (pp. 37–54). IEEE. doi: 10.1109/FOSE.2007.14
  17. 17. Result Brooks, Frederic P. (1986). No Silver Bullet, Essence and Accidents of Software Engineering, Information Processing, Elsevier Science Publishers. easily and unexpectedly becomes Of all the monster that fill the nightmares of our folklore, non terrify more than werewolves, because they transform unexpectedly from the familiar into horrors.“
  18. 18. Hypothesis: DSLs can help to separate essential complexity from accidental complexity
  19. 19. What is a Domain-Specific Language?
  20. 20. ”“ Domain-Specific Language A computer programming language of limited expressiveness focused on a particular domain. Fowler, Martin (2010). Domain Specific Languages (1st ed.).Addison-Wesley Professional.
  21. 21. ”“ Domain-Specific Language A computer programming language of limited expressiveness focused on a particular domain. Fowler, Martin (2010). Domain Specific Languages (1st ed.).Addison-Wesley Professional. The limited expressiveness might be a bit confusing, DSLs have great expressiveness, but limited applicability.
  22. 22. ” “A language offering expressive power focused on a particular problem domain, such as a specific class of applications or application aspect. Czarnecki, Krzysztof (2005). Overview of Generative Software Development. Unconventional Programming Paradigms, LNCS 3566, Springer-Verlang Domain-Specific Language Problem-level abstractions Implementation-level abstractions Ideally DSL
  23. 23. ”“ They (DSLs) offer substantial gains in expressiveness and ease of use compared with general purpose programming languages in their domain of applications. Mernik, M., Heering, J., & Sloane,A. M. (2005). When and how to develop domain-specific languages. ACM Computer Survey, 37(4), 316–344. doi:10.1145/1118890.1118892 Domain-Specific Language
  24. 24. DSL advantages • Domain-specific abstractions • Domain-specific concrete syntax • Domain-specific error checking • Domain-specific optimizations • Domain-specific tool support Czarnecki, Krzysztof (2005). Overview of Generative Software Development. Unconventional Programming Paradigms, LNCS 3566, Springer-Verlang
  25. 25. Internal or External?
  26. 26. A Simple Case Study
  27. 27. Context • Model-Driven Software Development (MDSD) • models are the primary development artifacts • using systematic transformation to synthesize running system implementation • prerequisites for model transformations are well-formed models • we need to check consistency of models - whether they are consistent to their meta-models
  28. 28. Context
  29. 29. Context Domain Model public interface Library extends EObject { ! // ... ! EList<Book> getBooks(); ! // ... } public interface Book extends EObject { ! // ... ! String getIsbn(); ! void setIsbn(String value); ! int getPages(); ! void setPages(int value); ! // ... } Interfaces as generated by Eclipse Modeling Framework
  30. 30. How to make sure the ISBN codes in the library are unique? Through a constraint Motivation 1
  31. 31. Round 1: GPL
  32. 32. public boolean validateBook_UniqueISBN(Book book, DiagnosticChain diagnostics, Map<Object, Object> context) { boolean violated = false; for (Book e : book.getLibrary().getBooks()) { if (e != book && e.getIsbn().equals(book.getIsbn())) { violated = true; break; } } if (violated) { if (diagnostics != null) { diagnostics.add(createDiagnostic(Diagnostic.ERROR, DIAGNOSTIC_SOURCE, 0, "_UI_GenericConstraint_diagnostic", new Object[] { "UniqueISBN", getObjectLabel(book, context) }, new Object[] { book }, context)); } return false; } return true; } In Java
  33. 33. public boolean validateBook_UniqueISBN(Book book, DiagnosticChain diagnostics, Map<Object, Object> context) { boolean violated = false; for (Book e : book.getLibrary().getBooks()) { if (e != book && e.getIsbn().equals(book.getIsbn())) { violated = true; break; } } if (violated) { if (diagnostics != null) { diagnostics.add(createDiagnostic(Diagnostic.ERROR, DIAGNOSTIC_SOURCE, 0, "_UI_GenericConstraint_diagnostic", new Object[] { "UniqueISBN", getObjectLabel(book, context) }, new Object[] { book }, context)); } return false; } return true; } In Java • Problems • Poor abstraction level • Expressed concern is hidden among Java commands • Verbose • Advantages • Eclipse JDT helps a lot (code completion, templates, ...) • Amenable to dynamic program verification (debugging, profiling) • No extra work, Java is right here
  34. 34. Curving out an apple core
  35. 35. Using a general purpose tool
  36. 36. ...raising accidental complexity But...
  37. 37. Choosing the right tool for the right job
  38. 38. It is all about raising the level of abstraction
  39. 39. Hypothesis: Domain-Specific Languages can raise level of abstraction
  40. 40. Round 2: External DSL
  41. 41. External Domain-Specific Language for model validations http://www.omg.org/spec/OCL/2.3.1/
  42. 42. In OCL context Book: invariant UniqueISBN: self.library.books->forAll(book | book <> self implies book.isbn <> self.isbn); • Higher-level of abstraction • Very expressive and concise • But, ...
  43. 43. Motivation 2 How to make sure that ISBN is correctly formatted?
  44. 44. Motivation 2 How to make sure that ISBN is correctly formatted? ISBNx20(?=.{13}$)d{1,5}([- ])d{1,7}1d{1,6}1(d|X)$ By using regular expressions e.g.
  45. 45. In OCL There is no support for RE in OCL Fall back to Java
  46. 46. Problem of Language Evolution There is no support for RE in OCL Could be slow as it is interpreted No debugger / profiler and other shortcomings Limited editor support On the Use of an Internal DSL for Enriching EMF Models Filip Kˇrikava Université Nice Sophia Antipolis, France I3S - CNRS UMR 7271 filip.krikava@i3s.unice.fr Philippe Collet Université Nice Sophia Antipolis, France I3S - CNRS UMR 7271 philippe.collet@unice.fr ABSTRACT The Object Constraint Language (OCL) is widely used to enrich modeling languages with structural constraints, side effect free query operations implementation and contracts. OCL was designed to be small and compact language with appealing short “to-the-point” expressions. When trying to apply it to larger EMF models some shortcomings appear in the language expressions, the invariant con- structs as well as in the supporting tools. In this paper we argue that some of these shortcomings are mainly related to the scalability of the OCL language and its trade-offs be- tween domain-specificity and general-purpose. We present an al- ternative approach based on an internal DSL in Scala. By using this modern multi-paradigm programing language we can realize an internal DSL with similar features found in OCL while taking full advantage of the host language including state-of-the-art tool support. In particular, we discuss the mapping between the OCL and Scala concepts together with some additional constructs for better scalability in both expressiveness and reusability of the ex- pressions. 1. INTRODUCTION OCL is used to complement the limited expressiveness of the structural constraints of the modeling languages like UML (Uni- fied Modeling Language) or EMF (Eclipse Modeling Framework). Such model constraints are captured as state invariants using a side- effect free expression language that supports first order predicate logic with model querying and navigation facilities [18, 15]. More- over, these expressions can be further used to include additional information to the model such as operation contracts in form of pre and post conditions, and implementation of derived features and operation bodies. OCL is also embedded in context of other tools such as the Object Management Group QVT model transformation. OCL is an appealing and expressive language, but when applied to larger EMF models using Eclipse OCL1, we found a number of shortcomings in the language expressions, the invariant constructs as well as in the supporting tools. While some of these problems are already well identified in the literature either as research agenda or accompanied with some solutions (c.f. Section 2), the lack of an overall integration eventually led us to look for alternatives. Ac- cording to us, some of these shortcomings are in general related to some scalability issues as a result of trade-offs between domain- specificity and general-purpose. In this paper we present an alternative approach based on an in- ternal DSL in Scala2, a statically typed object-oriented and func- tional programming language that runs on top of a Java Virtual Machine (JVM). Besides the seamless integration with EMF, some 1http://goo.gl/TECuz 2http://www.scala-lang.org/ Scala features, such as support for higher-order functions, rich col- lection libraries and implicit type conversions, allow us to write very similar OCL-like expressions, but also leverage from the many libraries found in the Java and Scala ecosystems. Besides Scala also comes with state-of-the-art tool support. The rest of the paper is organized as follows. In Section 2 we describe the main shortcomings of OCL based on our experience. Section 3 shows how we use Scala as an alternative approach to enrich EMF models. It is followed by Section 4 where the im- plementation details are presented together with an evaluation. In Section 5, we discuss related work. We briefly conclude and outline future work in Section 6. 2. SOME SHORTCOMINGS OF OCL In this section we present a list of shortcomings we came across while using OCL in an EMF based MDE toolchain, but various usages of OCL reveal different pros and cons. In our study we were concerned with the practical side of OCL rather than a formal one like in [14] or [4]. For each of the point we also report on works in which similar problems were reported. Despite the fact that many of these issues have already been identified or addressed, the lack of an overall integration is a crucial issue, which, according to us, influences the slow adoption of OCL in the industry. 2.1 OCL Expressions One of the key points that Anders Ivner mentions in the foreword to the second edition of The Object Constraint Language [18] is “Second, it is compact, yet powerful. You can write short and to- the-point expressions that do a lot”. While this is true for many of the short and straight-forward expressions, when the complex- ity grows our ease of reading and writing of these expressions de- creases radically. This might be especially hard for the new users when they move from the tutorial like expressions to real world ones. Complex expressions are hard to write and maintain OCL constraints and queries are defined in form of expressions that can be chained together. The underlying linearity of this chaining often leads to long and complex expressions that are difficult to understand and maintain. Since the debugging support in OCL is rather limited, mostly only simple logging or tracing is possible, maintaining of these expressions is particularly hard. In [10] Correa et al. provide some refactoring techniques to sim- plify some of these complex expressions. Ackermann et al. pro- pose in [1] to utilize specification patterns for which OCL con- straints can be generated automatically, together with a collection of OCL specification patterns formally defined. These techniques Křikava F. and Collet P. (2012) On the Use of an Internal DSL for Enriching EMF Models, OCL Workshop, MODELS 2012, Innsbruck Austria
  47. 47. Growing a Language http://www.youtube.com/watch?v=_ahvzDzKdB0 Growing a Language, by Guy Steele at ACM OOPSLA 1998
  48. 48. Round 3: Internal DSL
  49. 49. In Scala context Book: invariant UniqueISBN: self.library.books->forAll(book | book <> self implies book.isbn <> self.isbn); def validateUniqueISBN(self: Book) = self.library.books forall { book => book != self implies book.isbn != self.isbn } • in OCL • in Scala
  50. 50. In Scala def validateUniqueISBN(self: Book) = self.library.books forall { e => e != self implies e.isbn != self.isbn } trait LibraryPackageScalaSupport { implicit class LibraryScalaSupport(that: Library) { def books = that.getBooks } implicit class BookScalaSupport(that: Book) { def library = that.getLibrary def isbn = that.getIsbn } } .books .library .isbn implicit class ExtendedBoolean(a: Boolean) { def implies(b: => Boolean) = !a || b } implies
  51. 51. Scala as a host language • flexible • infix operator syntax for method calls • omitting parenthesis • no need for “;” • extensible • operator overloading • higher-order functions • by-name parameter evaluation • implicit definitions and parameters • traits
  52. 52. General Purpose Languages Should be Metalanguages Jeremy G. Siek University of Colorado at Boulder Internal DSLs Invited keynote at Partial Evaluation and Program Manipulation workshop of the ACM SIGPLAN 2010
  53. 53. Motivation 3 library.books exists { book => book.pages > 300 } library.books nonEmpty ... ... ... library.books forall { book => book.pages < 300 } • Imagine we have successfully used our DSL • Now we have a large collection of constraints for our library
  54. 54. Motivation 3 library.books exists { book => book.pages > 300 } library.books forall { book => book.pages < 300 } • Some constraints might be are contradictory How can we ensure satisfiability of all the constraints?
  55. 55. • In external DSL we have the abstract syntax • We can transform it into formal model and use formal model checkers • Domain-specific verification through formal analysis Motivation 3 VoidType DataType Class TupleType SetTypeSequenceType BagTypeOrderedSetType Classifier CollectionType PrimitiveType InvalidType AnyType Operation Signal MessageType +elementType 1 * +referredOperation 0..1 * +referredSignal 0..1 * TemplateParameterType +specification: String
  56. 56. • In external DSL we have the abstract syntax • We can transform it into formal model and use formal model checkers • Domain-specific verification through formal analysis • In the case of OCL we can use first order and logic1 and SMT solver Motivation 3 Figure 8.1 - Abstract Syntax Kernel Metamodel for OCL Types AnyType AnyType is the metaclass of the special type OclAny, which is the type to which all other types conform. OclAny is the sole instance of AnyType. This metaclass allows defining the special property of being the generalization of all other VoidType DataType Class TupleType SetTypeSequenceType BagTypeOrderedSetType Classifier CollectionType PrimitiveType InvalidType AnyType Operation Signal MessageType +elementType 1 * +referredOperation 0..1 * +referredSignal 0..1 * TemplateParameterType +specification: String 1Clavel, M., Egea, M., Garcia de Dios, M.A. (2009) Checking unsatisfiability for OCL constraints. OCL Workshop, MODELS 2009 library.books->exists(book | book.pages > 300) library.books->forAll(book | book.pages < 300)
  57. 57. • In the case of Scala how can we do the same? Motivation 3 library.books exists { book => book.pages > 300 } library.books forall { book => book.pages < 300 } • It turns out that it can be realized using deep embedding
  58. 58. DSL Embedding • shallow - embeddings use the host language’s features directly to implement their semantics • deep - embeddings have a separate internal representation (IR - an abstract syntax) that is evaluated in a separate step and then either interpreted or compiled
  59. 59. In Scala • Polymorphic DSL embedding1 • Scala virtualization2 • Lightweight Modular Staging3 • Deep DSL embedding in Scala is supported through 1Holfer, C. et al. (2008) Polymorphic Embedding of DSLs. GPCE’08 2Moors,A. et al. (2011), Tool Demo: Scala-Virtualized, ICSE’11 3Rompf,T. and Odersky, M. (2010) Lightweight Modular Staging:A Pragmatic Approach to Runtime Code Generation and Compiled DSLs, GPCE’10
  60. 60. Polymorphic Embedding • Provide multiple semantics for the same embedded syntax • The programs then become polymorphic over the semantics • DSEL is defined modularly enabling composition through traits mixins
  61. 61. ScalaVirtualization • Fully compatible Scala derivate that additionally • allows to override default constructs by virtualizing them into methods • allows to define new infix-methods on existing types (implicits with less boilerplate) • lifts static source information https://github.com/tiarkrompf/scala-virtualized scalaOrganization := "org.scala-lang.virtualized"
  62. 62. Infix methods • More readable • Less boilerplate • Infix methods win over build-in methods scala> def infix +(a: Int, b: Int): Int = 0 infix_$plus: (a: Int, b: Int)Int scala> 5+5 res1: Int = 0 Don’t work straight in REPL - wrap into a type
  63. 63. Virtualize Language Concepts if (c) a else b __ifThenElse(c, a, b) while(c) b __whileDo(c, b) do b while(c) __doWhile(b, c) var x = i val x = __newVar(i) x = a __assign(x, a) return a __return(a) a == b __equal(a, b) a == (b_1,..., b_n) __equal(a, b_1, ..., b_n)
  64. 64. Virtualize Language Concepts def __ifThenElse[T](cond: => Boolean, thenp: => T, elsep: => T): T = { println("if: "+cond) thenp } scala> if(false) 1 else 2 // virtualized to `__ifThenElse(false, 1, 2)` if: false res0: Int = 1 trait Conditional { def switch(thn: => Any, els: => Any): Any } def __ifThenElse[T <% Conditional](cond: T, thenp: Any, elsep: Any): Any = { cond.switch(thenp, elsep) }
  65. 65. Virtualize Language Concepts Puzzle: replace ??? so you don’t loose def puzzle(x: Int, msg: String) { ! if (x) { ! ! println(msg) ! } else { ! println("You loose!") ! } } puzzle(???, "I win") type mismatch; found : Int required: Boolean
  66. 66. Virtualize Language Concepts We don’t want to loose implicit class IntCond(that: Int) extends Conditional { def switch(thenp: => Any, elsep: => Any): Any = if (that != 0) thenp else elsep } trait Conditional { def switch(thn: => Any, els: => Any): Any } def __ifThenElse[T <% Conditional](cond: T, thenp: Any, elsep: Any): Any = { cond.switch(thenp, elsep) } scala> puzzle(1, “I win”) I win
  67. 67. Lightweight Modular Staging def prog1(b: Boolean, x: Rep[Int]) = if (b) x else x + 1 def prog2(b: Rep[Boolean], x: Rep[Int]) = if (b) x else x + 1 prog1(true, x) //=> x prog2(b, x) // If(b, x, Plus(x, Const(1))) • Lightweight - purely library based • Modular - modular fashion and separates DSL specification and implementation • Staging - program is split into multiple stages http://skillsmatter.com/podcast/agile-testing/javascript-embedded-dsl-scala
  68. 68. • LMS-core framework • reusable component for performing LMS • smart-constructor representation for constant loops, conditions • goes with scala-virtualized • A code generator framework based on an internal AST Lightweight Modular Staging http://github.com/TiarkRompf/virtualization-lms-core
  69. 69. LMS - Linear Algebra Example Implement a DSL for vector multiplication val u = v * 3.14 Concrete Syntax https://github.com/julienrf/lms-tutorial
  70. 70. LMS Example - Concepts import scala.virtualization.lms.common._ // Concepts and concrete syntax trait LinearAlgebra extends Base { // Concepts type Vector def vector_scale(v: Rep[Vector], k: Rep[Double]): Rep[Vector] // Concrete syntax implicit class VectorOps(v: Rep[Vector]) { def * (k: Rep[Double]): Rep[Vector] = vector_scale(v, k) }
  71. 71. LMS Example - Concepts import scala.virtualization.lms.common._ // Concepts and concrete syntax trait LinearAlgebra extends Base { // Concepts type Vector def vector_scale(v: Rep[Vector], k: Rep[Double]): Rep[Vector] // Concrete syntax implicit class VectorOps(v: Rep[Vector]) { def * (k: Rep[Double]): Rep[Vector] = vector_scale(v, k) } // Alternatively final def infix_*(v: Rep[Vector], k: Rep[Double]): Rep[Vector] = vector_scale(v, k) }
  72. 72. LMS Example - Shallow Embedding // does not build an abstract intermediate representation of expressions trait Interpreter extends Base { // Use a value of type T to represent a value of type T override type Rep[+T] = T // Lifting a constant to its representation is the identity function override def unit[T : Manifest](a: T) = a } trait LinearAlgebraInterpreter extends LinearAlgebra with Interpreter { // Concrete types override type Vector = Seq[Double] // Concrete operation override def vector_scale(v: Seq[Double], k: Double) = v map (_ * k) }
  73. 73. LMS Example - Shallow Embedding // does not build an abstract intermediate representation of expressions trait Interpreter extends Base { // Use a value of type T to represent a value of type T override type Rep[+T] = T // Lifting a constant to its representation is the identity function override def unit[T : Manifest](a: T) = a } trait LinearAlgebraInterpreter extends LinearAlgebra with Interpreter { // Concrete types override type Vector = Seq[Double] // Concrete operation override def vector_scale(v: Seq[Double], k: Double) = v map (_ * k) } val p = new Prog with LinearAlgebraInterpreter println(p.f(Seq(1.0, 2.0))) // => Seq(3.14, 6.28) trait Prog extends LinearAlgebra { def f(u: Rep[Vector]): Rep[Vector] = u * unit(3.14) }
  74. 74. LMS Example - Deep Embedding // Deep embedding: use an abstract IR to model concepts of the DSL trait LinearAlgebraExp extends LinearAlgebra with BaseExp { // Concretely define what Vector is override type Vector = Seq[Double] // Reification of the vector scaling operation into an IR case class VectorScale(v: Exp[Vector], k: Exp[Double]) extends Def[Vector] // Supply a new implementation override def vector_scale(v: Exp[Vector], k: Exp[Double]) = // A composite expression Def VectorScale(v, k) }
  75. 75. LMS Example - Code Generator // Scala code generator trait ScalaGenLinearAlgebra extends ScalaGenBase { val IR: LinearAlgebraExp // Import our Internal Representation import IR._ override def emitNode(sym: Sym[Any], node: Def[Any]): Unit = node match { // This is our only definition case VectorScale(v, k) => { // exactly what is done in the interpreter // v.map(x => x * k) emitValDef(sym, quote(v) + ".map(x => x * " + quote(k) + ")") } case _ => super.emitNode(sym, node) } }
  76. 76. LMS Example - Usage // code generation val prog = new Prog with LinearAlgebraExp with EffectExp with CompileScala { // ... } // instantiate the function - generates the code, compiles and load val f = prog.compile(prog.f) // execute println(f(Seq(1.0, 2.0))) trait Prog extends LinearAlgebra { def f(u: Rep[Vector]): Rep[Vector] = u * unit(3.14) } [info] Running tutorial.Main /***************************************** Emitting Generated Code *******************************************/ class F extends ((scala.collection.Seq[Double])=>(scala.collection.Seq[Double])) { def apply(x0:scala.collection.Seq[Double]): scala.collection.Seq[Double] = { val x1 = x0.map(x => x * 3.14) x1 } } /***************************************** End of Generated Code *******************************************/ compilation: ok List(3.14, 6.28)
  77. 77. LMS Example - Interpreted vs Generated // code generation val prog = new Prog with LinearAlgebraExp with EffectExp with CompileScala { // ... } println(prog.compile(prog.f)(Seq(1.0, 2.0))) // interpreter val prog = new Prog with LinearAlgebraInterpreter println(prog.f(Seq(1.0, 2.0)))
  78. 78. LMS Example - Optimization // Optimizations working on the intermediate representation trait LinearAlgebraOpt extends LinearAlgebraExp { override def vector_scale( v: Exp[Vector], k: Exp[Double] ) = k match { // identity vector case Const(1.0) => v case _ => super.vector_scale(v, k) } }
  79. 79. LMS Example - Optimization Usage /***************************************** Emitting Generated Code *******************************************/ class G extends ((scala.collection.Seq[Double])=>(scala.collection.Seq[Double])) { def apply(x2:scala.collection.Seq[Double]): scala.collection.Seq[Double] = { x2 } } /***************************************** End of Generated Code *******************************************/ compilation: ok List(1.0, 2.0) // code generation val prog = new Prog with LinearAlgebraOpt with EffectExp with CompileScala { // ... } // instantiate the function - generates the code, compiles and load val g = prog.compile(prog.g) // execute println(g(Seq(1.0, 2.0))) trait Prog extends LinearAlgebra { def f(v: Rep[Vector]): Rep[Vector] = v * unit(3.14) def g(v: Rep[Vector]): Rep[Vector] = v * unit(1.0) // use optimization }
  80. 80. LMS Example - Optimization Usage val prog = new Prog with LinearAlgebraOpt with EffectExp with CompileScala val prog = new Prog with LinearAlgebraExp with EffectExp with CompileScala To use the optimization - only need to switch to LinearAlgebraOpt instead of LinearAlgebraExp That’s way it is modular!
  81. 81. LMS - Why it is useful? • Abstraction without regret • Allow to generate high-performance code from high-level code • Particularly useful for highly-parallel and distributed code • e.g. generates code for: C, CUDA, MPI1 • Allow to target different platforms • e.g. generate client/sever side Javascript validation code from the same code base2 • Allow to formally analyze DSLs • e.g. generate First-Order Logic from structural constraints in Sigma3 2 http://stanford-ppl.github.io/Delite/ http://infoscience.epfl.ch/record/179888/files/js-scala-ecoop.pdf https://github.com/fikovnik/Sigma 1 3
  82. 82. Who is using all this • OptiML:A DSL for machine learning developed at Stanford PPL • Scala Integrated Query (SIQ): LINQ-style database interface in Scala • Delite: Framework/Runtime for heterogeneous parallelism • LMS-Core: Compiler framework for embedded DSLs
  83. 83. Conclusions
  84. 84. No Silver Bullet1 1Brooks, Frederic P. (1986). No Silver Bullet, Essence and Accidents of Software Engineering, Information Processing, Elsevier Science Publishers. • There are pitfalls • External DSL • It might not be easy to define a usable language • Tools are important and costly to build • Internal DSL • DSLs are leaky abstraction • Syntax is not quire right • Difficulty in debugging and error messages (low level) • LMS takes some effort do right
  85. 85. • Look behind your desk • Scala is fun • LMS is even more fun • but ... ”“If all you have is a hammer, everything looks like a nail.
  86. 86. Xtext • Software language engineering framework • Particularly suitable for DSLs • Generates complete DSL workbenches for Eclipse IDE (compiler, debugger, ...)
  87. 87. MPS http://www.slideshare.net/schogglad/gtsl
  88. 88. MetaEdit+ http://www.metacase.com
  89. 89. Abstraction Gothic Security & Miss Grant’s Controller2 mapping Event doorClosed = new Event("doorClosed", "D1CL"); Event drawerOpened = new Event("drawerOpened", "D2OP"); Event lightOn = new Event("lightOn", "L1ON"); Event doorOpened = new Event("doorOpened", "D1OP"); Event panelClosed = new Event("panelClosed", "PNCL"); Command unlockPanelCmd = new Command("unlockPanel", "PNUL"); Command lockPanelCmd = new Command("lockPanel", "PNLK"); Command lockDoorCmd = new Command("lockDoor", "D1LK"); Command unlockDoorCmd = new Command("unlockDoor", "D1UL"); State idle = new State("idle"); State activeState = new State("active"); State waitingForLightState = new State("waitingForLight"); State waitingForDrawerState = new State("waitingForDrawer"); State unlockedPanelState = new State("unlockedPanel"); StateMachine machine = new StateMachine(idle); idle.addTransition(doorClosed, activeState); idle.addAction(unlockDoorCmd); idle.addAction(lockPanelCmd); activeState.addTransition(drawerOpened, waitingForLightState); activeState.addTransition(lightOn, waitingForDrawerState); waitingForLightState.addTransition(lightOn, unlockedPanelState); waitingForDrawerState.addTransition(drawerOpened, unlockedPanelState); unlockedPanelState.addAction(unlockPanelCmd); unlockedPanelState.addAction(lockDoorCmd); unlockedPanelState.addTransition(panelClosed, idle); machine.addResetEvents(doorOpened); Implementation-level abstractions2 Fowler, Martin (2010). Domain Specific Languages (1st ed.).Addison-Wesley Professional. http://www.metacase.com/blogs/stevek/blogView?showComments=true&entry=34148345201 2 Problem-level abstractions1 ” “Miss Grant has a secret compartment in her bedroom that is normally locked and concealed....
  90. 90. ThankYou Filip Křikava https://github.com/fikovnik
  91. 91. http://www.fanpop.com/clubs/pencils/images/24173416/title/colored-pencils-wallpaper http://good-wallpapers.com/places/21435 http://ecofrenglobal.blog.com/2011/02/04/this-is-cool/ http://www.topnews.in/health/gossip-lies-eye-beholder-212114 http://browse.deviantart.com/art/Werewolf-54903310 http://www.geektech.in/archives/12122 http://www.arabacademy.com/arabic-blog/arabic-language/egypts-pyramids/ http://www.thezooom.com/wp-content/uploads/2012/08/Trinity-College-Library-Dublin.jpg http://artnail.altervista.org/picture-of-hammer-and-nails/ Acknowledgement
  92. 92. ”“ A DSL is a focused, processable language for describing a specific domain.The abstraction and notation used are natural/suitable for the stakeholders who specify that particular concern. MarkusVoelter, Domain Specific Language Design, http://www.slideshare.net/schogglad/ domain-specific-language-design Domain-Specific Language
  93. 93. Lightweight Modular Staging • Lightweight - purely library based • Modular • constructs in DSL are specified in a modular fashion • separates DSL specification and implementation • Staging - program is split into multiple stages • Each stage reads part of the input and calculates part of the result • Later stages can be optimized based on earlier stages • Eventually they are interpreted or generated

×