Domain Specific Language Design

2,231 views

Published on

Published in: Technology

Domain Specific Language Design

  1. 1. DSL DesignA conceptual frameworkfor building good DSLs Markus Voelter based on material independent/itemis from a paper written voelter@acm.org with Eelco Visser www.voelter.de E.Visser@tudelft.nl voelterblog.blogspot.de http://eelcovisser.org/ @markusvoelter +Markus Voelter
  2. 2. Introduction
  3. 3. A DSL is a focussed, processable language for describing aspecific concern when building asystem in a specific domain. Theabstractions and notations used are natural/suitable for the stakeholders who specify that particular concern.
  4. 4. more in GPLs more in DSLDomain Size large and complex smaller and well-definedDesigned by guru or committee a few engineers and domain expertsLanguage Size large smallTuring- almost always often notcompletenessUser Community large, anonymous and small, accessible and widespread localIn-language sophisticated limitedabstractionLifespan years to decades months to years (driven by context)Evolution slow, often fast-paced standardizedIncompatible almost impossible feasibleChanges
  5. 5. General PurposeC Components Domain State Machines Specific Sensor Access LEGO Robot Control
  6. 6. Modular Language  a b c d e f my L g h i j k l  with many optional,composable modules
  7. 7. Case Studies
  8. 8. ExampleCompo nents
  9. 9. ExampleCompo nents
  10. 10. ExampleRefrigerators Refrige rators
  11. 11. ExampleRefrigerators
  12. 12. ExampleRefrigerators
  13. 13. ExampleExtended C
  14. 14. ExamplePension Plans
  15. 15. ExamplePension Plans
  16. 16. Example Web DSL
  17. 17. Terms &Concepts
  18. 18. Model A schematic description of a system, theory, orphenomenon that accounts for its known or inferred properties and may be used for further study of its characteristics www.answers.com/topic/mode l
  19. 19. ModelA representation of a set of components of a process, system, or subject area, generally developed for understanding, analysis, improvement, and/orreplacement of the process www.ichnet.org/glossary.htm
  20. 20. Model which ones? an abstraction or simplification of realitywhat shouldwe leave out? ecosurvey.gmu.edu/glossary.ht m
  21. 21. Model Purpose… code generation… analysis and checking… platform independence… stakeholder integration… drives design of language!
  22. 22. Model Purpose… code generation… analysis and checking… platform independence… stakeholder integration Example Compo nents
  23. 23. Model Purpose… code generation… analysis and checking… platform independence… stakeholder integration Example Refrigerators Refrige rators
  24. 24. Model Purpose… code generation… analysis and checking… platform independence… stakeholder integration Example Extended C Exten ded C
  25. 25. Model Purpose… code generation… analysis and checking… platform independence… stakeholder integration Example Pension Plans
  26. 26. body of knowledge deductive top down in the real world Domain inductive existing bottom upsoftware (family)
  27. 27. body of knowledge deductive top down in the real world Domain Example ExampleRefrigerators Penion Plans Refrige rators
  28. 28. Domain Domain inductive existing bottom upsoftware Example Example (family) Exten Compo ded C netns
  29. 29. A DSL LD for D is a language that is specialized to en- coding PD programs.more efficientsmaller
  30. 30. Programsare trees elemen t
  31. 31. Fragments aresubtrees w/ root fragment root ffragment
  32. 32. Parent-ChildRelation f
  33. 33. Programsand Fragments f
  34. 34. Programs aregraphs, really. reference
  35. 35. Programs aregraphs, really.
  36. 36. Languages aresets of concepts C2 C1 Cn C3 L
  37. 37. Languages aresets of concepts C2 C1 Cn C3 L
  38. 38. Programsand languages C2 C1 Cn C3
  39. 39. Language:concept inheritance C2 L C1 1 L2 C3 C1
  40. 40. Languagedoes not depend onany other language IndependenceFragmentdoes not depend onany other fragment
  41. 41. Independence Example Refrige rators
  42. 42. HomogeneousFragmenteverything expressedwith one language
  43. 43. HeterogeneousHeterogeneous CStatemachines Testing Example Exten ded C
  44. 44. Domain Hierarchy
  45. 45. Domain Hierarchy avionics automotiveembeddedsoftware Example all programs Exten ded C
  46. 46. Design Dimensionsexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  47. 47. Expressivityexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  48. 48. ShorterPrograms MoreAccessibleSemantics
  49. 49. For a limited Domain! Domain Knowledgeencapsulated in language
  50. 50. Smaller DomainMore Specialized Language Shorter Programs
  51. 51. The do-what-I-want language Ѱ
  52. 52. Single Programvs. Class/Domain No Variability! Ѱ
  53. 53. Domain Hierarchy more specialized domainsmore specialized languages
  54. 54. Reification Dn+1 == Dn
  55. 55. Reification Transformation / Generation ==LanguageDefinition
  56. 56. Overspecification!Requires Semantic Analysis!
  57. 57. LinguisticAbstraction Declarative! Directly represents Semantics.
  58. 58. Def: DSLA DSL is a language at D thatprovides linguistic abstractionsfor common patterns and idiomsof a language at D-1 when usedwithin the domain D.
  59. 59. Def: DSL cont’dA good DSL does not require theuse of patterns and idioms toexpress semantically interestingconcepts in D.Processing tools do not have todo “semantic recovery” on Dprograms.Declarative!
  60. 60. Another Example Example Exten ded C
  61. 61. Another Example Example Exten Turing Complete! Requires Semantic Analysis! ded C
  62. 62. LinguisticAbstraction Example Exten ded C
  63. 63. LinguisticAbstraction Example Exten ded C
  64. 64. LinguisticAbstraction In-Language Abstraction Libraries Classes Frameworks
  65. 65. LinguisticAbstractionAnalyzableBetter IDE Support In-Language Abstraction User-Definable Simpler Language
  66. 66. LinguisticAbstraction SpecialAnalyzable Treatment!Better IDE Support In-Language Abstraction User-Definable Simpler Language
  67. 67. LinguisticAbstraction Std Lib In-Language Abstraction
  68. 68. Std Lib Example Refrige rators
  69. 69. Coverageexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  70. 70. Domain DL definedinductively by L(the domain that can be expressed by L) CL(L) == 1 (by definition)not very interesting!
  71. 71. Def: Coverageto what extend can a languageL cover a domain D
  72. 72. Def: Coveragewhy would CD(L) be != 1? 1) L is deficient 2) L is intended to cover only a subset of D, corner cases may make L too complex Rest must be expressed in D-1
  73. 73. Def: CoverageCoverage is full.You call always write C. Example Exten ded C
  74. 74. Def: CoverageOnly a particular style ofweb apps are suported.Many more are conceivable. Example WebDSL
  75. 75. Def: CoverageDSLs are continuously evolvedso the relevant parts of thedeductive domain aresupported. Example Example Example Compo Refrige Pension nents rators Plans
  76. 76. Semantics & Executionexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  77. 77. Static SemanticsExecution Semantics
  78. 78. Static SemanticsExecution Semantics
  79. 79. Static Semantics Constraints Type Systems
  80. 80. Unique State NamesUnreachable States Dead End States … Example Exten ded C
  81. 81. Unique State NamesUnreachable States Dead End States … Easier to do on a declarative Level! Example Exten ded C
  82. 82. Unique State Names Unreachable States Dead End States … Easier to do on a declarative Level!Thinking of all Exampleconstraints is a Extencoverage problem! ded C
  83. 83. Assign fixed typesDerive Types Calculate Common Types Check Type Consistency What does a type system do?
  84. 84. Intent + DeriveCheckMore code More convenientBetter error More complexmessages checkersBetter Performance Harder to understand for users
  85. 85. ExampleRefrigerators
  86. 86. What does it all mean?Execution Semantics
  87. 87. Def: Semantics … via mapping to lower levelOB: Observable Behaviour (Test Cases)
  88. 88. Def: Semantics … via mapping to lower level LD Transformation Interpretation LD-1
  89. 89. Transformation Dn+1 Dn
  90. 90. Transformation Example Exten ded C
  91. 91. Transformation LD Transformation LD-1 Known Semantics!
  92. 92. Transformation LD Correct!? Transformation LD-1
  93. 93. TransformationTests (D) LD TransformationTests (D-1) LD-1Run tests on both levels; all pass.Coverage Problem!
  94. 94. ExampleRefrigerators
  95. 95. Transformation Example Pension Plans
  96. 96. Behavior Example Refrige rators
  97. 97. Multi-Stage L3 L2 Modularization L1 L0
  98. 98. Multi-Stage: Reuse L3 L2 L5 L1 Reusing Later Stages L0 Optimizations!
  99. 99. Multi-Stage: Reuse L3 Robot Control State Machine L2 L5 Components L1 C (MPS tree) Example L0 C Text Exten ded C
  100. 100. Multi-Stage: Reuse L3 Robot Control State Machine Consistency Model Checking L2 L5 Components Efficient Mappings C Type System L1 C (MPS tree) Syntactic Correctness, Example Headers L0 C Text Exten ded C
  101. 101. Multi-Stage: Reuse L3 Reusing Early Stages L2 Portability L1 L1b L0 L0b
  102. 102. Multi-Stage: Reuse L3 L2 L1 L1b Java C# Example L0 L0b Exten ded C
  103. 103. Multi-Stage:Preprocess Adding an optional, modular emergency stop feature
  104. 104. Platform
  105. 105. Platform Temporal Data Versioning Database Access Transactions Distribution (JEE) Example Pension Plans
  106. 106. Platform Data Collection HAL Device Drivers Example Refrige rators
  107. 107. InterpretationA program at D0 thatacts on the structureof an input program at D>0
  108. 108. InterpretationA program at D0 thatacts on the structureof an input program at D>0 imperative  step through functional  eval recursively declarative ? solver ?
  109. 109. ExamplePension Plans
  110. 110. ExampleRefrigerators
  111. 111. Behavior Example Refrige rators
  112. 112. InterpretationA program at D0 thatacts on the structureof an input program at D>0
  113. 113. InterpretationAn interpreter :-)
  114. 114. Transformation Interpretation+ Code Inspection + Turnaround Time+ Debugging + Runtime Change+ Performance & Optimization+ Platform Con- formance
  115. 115. Def: Semantics … via mapping to lower level LD Transformation Interpretation LD-1
  116. 116. Multiple Mappings … at the same time LD Similar Semantics? Lx Ly Lz
  117. 117. Multiple Mappings … at the same time LD T Similar Semantics? Lx Ly Lz T T T all green!
  118. 118. ExamplePension Plans
  119. 119. ExampleRefrigerators
  120. 120. Multiple Mappings … alternatively, selectably Extend LD to include LD explicit data that determines transformation Lx Ly Lz
  121. 121. Multiple Mappings … alternatively, selectably Extend LD to include LD explicit data that determines transformation Lx Ly Lz Example Restricted Port leads to reduced overhead Exten C ded C
  122. 122. Multiple Mappings … alternatively, selectably External Data: LD - Switches - Annotation Model Lx Ly Lz
  123. 123. Multiple Mappings … alternatively, selectably External Data: LD - Switches - Annotation Model Switch Control Java vs. C Lx Ly Lz Example Pension Plans
  124. 124. Multiple Mappings … alternatively, selectably Heuristics: Analyze LD model to try to decide Lx Ly Lz
  125. 125. Multiple Mappings … alternatively, selectably T LD TESTING! Lx Ly Lz T T T
  126. 126. Reduced Expressiveness bad? maybe. good? maybe! Model Checking SAT Solving Exhaustive Search, Proof!
  127. 127. Unique State Names Unreachable States Dead End States Guard Decidability ReachabilityExhaustive Search, Proof!
  128. 128. ExampleExtended C
  129. 129. ExampleExtended C
  130. 130. Separation of Concernsexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  131. 131. Several Concerns … in one domain
  132. 132. Several Concerns … in one domain integrated into separated into one fragment several fragments
  133. 133. Viewpoints independent dependent
  134. 134. Viewpoints Hardware Behavior Tests Example Refrige rators
  135. 135. Viewpoints: Why? Sufficiency Different Stakeholders Different Steps in Proces – VCS unit!
  136. 136. Viewpoints Hardware Product Management Behaviour Thermo- dynamics- Experts Tests Example Refrige rators
  137. 137. Viewpoints independent sufficient? contains all the data for running a meaningful transformation
  138. 138. Viewpoints sufficient implementation code sufficient Hardware structure documentation Example Refrige rators
  139. 139. Viewpoints: Why? 1:n Relationships
  140. 140. Viewpoints 1:n Hardware Behaviour Tests Example Refrige rators
  141. 141. Viewpoints Well-defined Dependencies No Cycles! Avoid Synchronization! (unless you use a projectional editor)
  142. 142. Viewpoints: Why? LD LAnnotation LD-1
  143. 143. Progressive Refinement
  144. 144. Views on Programs Example Pension Plans
  145. 145. Completenessexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  146. 146. Can you generate 100% ofthe code from the DSLprogram?More generally: all of D-1
  147. 147. Semantics:Introduce G (“generator”): complete incomplete
  148. 148. Incomplete: What to do? FD OB(FD) != FD-1
  149. 149. Incomplete: What to do?Manually written code! FD OB(FD) == FD-1 + FD-1, man
  150. 150. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD program
  151. 151. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD program Embed C statements in components, state machines or decision tables Example Refrige rators
  152. 152. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD programUse composition mechanisms ofLD-1 (inheritance, patterns, aspects, …)
  153. 153. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD programUse composition mechanisms ofLD-1 (inheritance, patterns, aspects, …)Generate base classes Examplewith abstract methods; Compoimplement in subclass nents
  154. 154. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD programUse composition mechanisms ofLD-1 (inheritance, patterns, aspects, …)Use protected regions (if youreally have to…)
  155. 155. Manually written code?Call “black box” code(foreign functions)Embed LD-1 code in LD programUse composition mechanisms ofLD-1 (inheritance, patterns, aspects, …)Use protected regions (if youreally have to…) DON’T!
  156. 156. Incomplete: When?Good for technical Example ExampleDSLs: Devs write Exten CompoLD-1 code ded C nentsBad for business DSLs.Maybe use a LD-1 std lib that LDcode can call into?Example Example ExampleRefrige Pension Webrators Plans DSL
  157. 157. Prevent Breaking Promises! class B extens BBase { public void doSomething() { Registry.get(„A“).someMethod(); } }
  158. 158. Prevent Breaking Promises!Better: Dependency Injection Example Static analysis tools Compo nents
  159. 159. Roundtripping LD L’D LD-1 ……… D-1 L’
  160. 160. Roundtripping – Don’t! LD L’D Semantic Recovery! LD-1 ……… D-1 L’
  161. 161. Fundamental Paradigmsexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  162. 162. StructureModularization, Visibility Namespaces, public/private importing
  163. 163. StructureModularization, Visibility Namespaces, public/private importing divide & conquer reuse stakeholder integration
  164. 164. StructurePartitioning (Files) VCS Unit Unit of sharing Unit of IP != logical modules may influence language design
  165. 165. StructureModularization, Visibility Example Exten ded C
  166. 166. StructureModularization, Visibility Example Compo nents
  167. 167. StructurePartitioning (Files) change impact link storage model organization tool integration
  168. 168. StructureSpec vs. Implementation plug in different Impls different stakeholders
  169. 169. StructureSpec vs. Impl. Example Exten ded C
  170. 170. StructureSpecialization Liskov substitution P leaving holes (“abstract”)
  171. 171. StructureSpecialization Liskov substitution P leaving holes (“abstract”) variants (in space) evolution (over time)
  172. 172. StructureSpecialization Pension Plans can inherit from other plans. Rules can be abstract; Plans with abstract Example rules are abstact Pension Plans
  173. 173. StructureSuperposition, Aspects merging overlay AOP modularize cross-cuts
  174. 174. StructureSuperposition, Aspects Example Compo nents
  175. 175. BehaviorNot all DSLs specify behaviorSome just declare behavior This section is not for those!
  176. 176. BehaviorImperative sequence of statements changes program state write understand debug analyze performance simple simple - simpl hard good e
  177. 177. BehaviorImperative sequence of statements changes program state Example Refrige rators
  178. 178. BehaviorFunctional functions call other functions. no state. No aliasing. write understand debug analyze performance simple - simple simpl good good - e
  179. 179. BehaviorFunctional functions call other functions. no state. No aliasing. Example Pension Plans
  180. 180. BehaviorFunctional Example Pension Plans
  181. 181. BehaviorFunctional pure expressions are a subset of functional (operators hard-wired) guards preconditions derived attributes
  182. 182. BehaviorDeclarative only facts and goals. no control flow. eval engine, solver (several) write understand debug analyze performance simple simple - hard depends often bad
  183. 183. BehaviorDeclarative concurrency constraint programming solving logic programming
  184. 184. BehaviorDeclarative only facts and goals. no control flow. eval engine, solver (several) Example Web DSL
  185. 185. BehaviorDeclarative Example Exten ded C
  186. 186. BehaviorReactive reactions to events, more events are produced. Communication via events and channels/queues write understand debu analyze performance g simple simple/har hard simple can be good
  187. 187. Behavior Reactive Example Refrige rators
  188. 188. BehaviorData Flow chained blocks consume continuous data that flows from block to blockwrite understand debu analyze performance gsimple - simple/har hard simple can be good
  189. 189. BehaviorData Flow continuous, calc on change quantized, calc on new data time triggered, calc every x
  190. 190. BehaviorData Flow Embedded Programming Enterprise ETL & CEP
  191. 191. BehaviorState Based states, transitions, guards, reactions event driven, timed write understand debu analyze performance g simple - simple/har s/h simple + can be good
  192. 192. BehaviorState Based Example Refrige rators
  193. 193. BehaviorCombinations data flow uses functional, imperative or declarative lang inside block
  194. 194. BehaviorCombinations state machines use expressions in guards and often an imperative lang in actions
  195. 195. BehaviorCombinations Example Refrige rators
  196. 196. Behavior
  197. 197. BehaviorCombinations purely structural languages often use expressions to specify constraints Example Exten ded C
  198. 198. Language Modularityexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  199. 199. Language Modularity,Behavior and Reuse (LMR&C)Composition increase efficiency of DSL development
  200. 200. Language Modularity,Behavior and Reuse (LMR&C)Composition increase efficiency of DSL development4 ways of composition: Referencing Reuse Extension Reuse
  201. 201. Language Modularity,Behavior and Reuse (LMR&C)Composition increase efficiency of DSL development4 ways of composition: distinguished regarding dependencies and fragment structure
  202. 202. Dependencies:Behavior do we have to know about the reuse when designing the languages?
  203. 203. Dependencies:Behavior do we have to know about the reuse when designing the languages?Fragment Structure: homogeneous vs. heterogeneous („mixing languages“)
  204. 204. Dependencies &BehaviorFragment Structure:
  205. 205. Dependencies &BehaviorFragment Structure:
  206. 206. ReferencingReferencing
  207. 207. ReferencingReferencing Dependen t No containment
  208. 208. Referencing Used in Viewpoints
  209. 209. Referencing Fragment Fragment Fragment
  210. 210. Referencing Example Refrige rators
  211. 211. Extension Dependen t Containment
  212. 212. ExtensionExtension more specialized domains more specialized languages
  213. 213. Extension Dn+1 == Dn
  214. 214. Extension Dn+1 == Dn
  215. 215. Extension Good for bottom-up (inductive) domains, and for use by technical DSLs (people) == Dn
  216. 216. ExtensionDrawbacksBehavior tightly bound to base potentially hard to analyze the combined program
  217. 217. Extension Example Exten ded C
  218. 218. ExtensionExtension Example Exten ded C
  219. 219. Reuse Reuse Independent No containment
  220. 220. Reuse ReuseBehavior referenced language Often the is built expecting it will be reused. Hooks may be added.
  221. 221. Embedding Independent Containment
  222. 222. Embedding Example Pension Plans
  223. 223. EmbeddingBehavior Embedding often uses Extension to extend the embedded language to adapt it to its new context.
  224. 224. Challenges - SyntaxBehavior and Embedding Extension requires modular concrete syntax Many tools/formalisms cannot do that
  225. 225. Challenges – Type SystemsBehavior the type system of Extension: the base language must be designed to be extensible/ overridable
  226. 226. Challenges – Type SystemsBehavior Reuse and Embedding: Rules that affect the interplay can reside in the adapter language.
  227. 227. Challenges – Trafo & GenReferencing (I)BehaviorWrittenspecifically Can befor the Reusedcombination Two separate, dependent single-source transformations
  228. 228. Challenges – Trafo & GenReferencing (II) A single multi-sourced transformation
  229. 229. Challenges – Trafo & GenReferencing (III)A preprocessing trafo that changes the referenced frag in a way specified by the referencing frag
  230. 230. Challenges – Trafo & GenExtension Transformation by assimiliation, i.e. generating code in the host lang from code expr in the extension lang.
  231. 231. Challenges – Trafo & GenExtension Example Exten ded C
  232. 232. Challenges – Trafo & GenReuse (I) Reuse of existing transformations for both fragments plus generation of adapter code
  233. 233. Challenges – Trafo & GenReuse (II) composing transformations
  234. 234. Challenges – Trafo & GenReuse (III) generating separate artifacts plus a weaving specification
  235. 235. Challenges – Trafo & GenEmbedding (I) a purely embeddable language may not come with a generator. Assimilation (as with Extension)
  236. 236. Challenges – Trafo & GenEmbedding (II) Adapter language can coordinate thetransformations for the host and for the emebedded languages.
  237. 237. Concrete Syntaxexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  238. 238. UI for the language!Important for acceptance by users! Textual Symbolic Tabular Graphical
  239. 239. Reuse existingsyntax ofdomain, if any!Tools let youfreely combineall kinds.
  240. 240. Default: Text Editors simple to build Productive Easy to integrate w/ tools Easy to evolve programs
  241. 241. Default: Text Editors simple to build Productive Easy to integrate w/ tools Easy to evolve programs… then add other forms,if really necessary
  242. 242. Graphical in case… Relationships
  243. 243. Graphical in case… Flow and Depenency
  244. 244. Graphical in case… Causality and Timing
  245. 245. Symbolic EitherMathematical, or often highly inspired by domain
  246. 246. Tables
  247. 247. Combinations
  248. 248. Combinations
  249. 249. Combinations
  250. 250. Combinations
  251. 251. Processexpressivity completenesscoverage paradigmssemantics modularityseparation of concrete concerns syntax process
  252. 252. Domain Analysis Interview Experts Get Structure their Feedback Knowledge Create Build the Examples Language
  253. 253. Iterate to goal
  254. 254. Documentation Create example- based tutorials!
  255. 255. Domain FolksProgramming? Precision vs. Algorithmics!
  256. 256. Domain FolksProgramming? DU Coding DU/Dev Paired Dev Coding DU Reviewing
  257. 257. DSL as a Product Release Plan Bug Tracker Testing! Support Documentation
  258. 258. Reviews Reviews become easier --- less code, more domain-specific
  259. 259. The End. This material is part of my upcoming (early 2013) book DSL Engineering with Language WorkbenchesStay in touch; it will be cheap or maybe even free :-)www.voelter.de/dslbook @markusvoelter +Markus Voelter

×