Successfully reported this slideshow.

Generic Tools, Specific Laguages

1

Share

Loading in …3
×
1 of 184
1 of 184

Generic Tools, Specific Laguages

1

Share

Developing software often requires using a number of tools and languages. In embedded software, for example, it is common to use C and its IDE, Matlab/Simulink, a number of custom XML files, a requirements management tool such as DOORS and possibly a UML tool and a variant management tool. The integration of such a zoo of tools is often a major source of (accidental) complexity in development projects.

Contrast that with the "good old days" when everything was text files and command line executables running on the unix shell. This approach had two important properties: the infrastructure was extremely generic (unix shell, pipes, text editors) and the actual contents were easily extensible and composable (new text file formats/languages and new command line tools); a productive environment for a given project or domain could easily be built from the generic infrastructure plus a few custom extensions.

In this talk I want to propose that creating (domain-specific) development environments based on language workbenches results in many of the same advantages that we all valued in the unix shell-world. A language workbench is an extremely generic infrastructure that is easily extensible with new languages. It is easy to create domain-specific development tools that can address different aspects of the system with suitable abstractions, but are nonetheless very well integrated in terms of semantics and tooling.

Developing software often requires using a number of tools and languages. In embedded software, for example, it is common to use C and its IDE, Matlab/Simulink, a number of custom XML files, a requirements management tool such as DOORS and possibly a UML tool and a variant management tool. The integration of such a zoo of tools is often a major source of (accidental) complexity in development projects.

Contrast that with the "good old days" when everything was text files and command line executables running on the unix shell. This approach had two important properties: the infrastructure was extremely generic (unix shell, pipes, text editors) and the actual contents were easily extensible and composable (new text file formats/languages and new command line tools); a productive environment for a given project or domain could easily be built from the generic infrastructure plus a few custom extensions.

In this talk I want to propose that creating (domain-specific) development environments based on language workbenches results in many of the same advantages that we all valued in the unix shell-world. A language workbench is an extremely generic infrastructure that is easily extensible with new languages. It is easy to create domain-specific development tools that can address different aspects of the system with suitable abstractions, but are nonetheless very well integrated in terms of semantics and tooling.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Generic Tools, Specific Laguages

  1. 1. Generic Tools - Specific Languages On the Art of Building Software Development Tools http://voelter.de voelter@acm.org @markusvoelter Markus Voelter independent/itemis
  2. 2. 1 2 3 Tools Extensibility Challenges 4 5 6 GTSL An Example Wrap Up
  3. 3. 1 Tools
  4. 4. Tools Command-Line Tools
  5. 5. Tools UI Tools
  6. 6. 2 Tool Extensibility
  7. 7. Tool Extensibility Study Findings I The majority of our interviewees were very successful with MDE but all of them either built their own modeling tools, made heavy adaptations of off-the- shelf tools, or spent a lot of time finding ways to work around tools. The only accounts of easy-to- use, intuitive tools came from those who had developed tools themselves for bespoke purposes. Indeed, this suggests that current tools are a barrier to success rather than an enabler.
  8. 8. Tool Extensibility Study Findings II Complexity problems are typically associated with off- the- shelf tools. Of particular note is accidental complexity – which can be introduced due to poor consideration of other categories, such as lack of flexibility to adapt the tools to a company’s own context [..]
  9. 9. Tool Extensibility Study Findings III Our interviews point to a strong need for tailoring of some sort: either tailor the tool to the process, tailor the process to the tool, or build your own tool that naturally fits your own process. Based on our data, it seems that, on balance, it is currently much easier to do the latter.
  10. 10. Tool Extensibility Command-Line Tools New File Formats New Processors
  11. 11. Tool Extensibility Command-Line Tools New File Formats New Processors Assemble Components (Pipes & Filters)
  12. 12. Tool Extensibility UI Tools Buttons Menus Views Actions
  13. 13. Tool Extensibility UI Tools Buttons Menus Views Actions New Languages New Editors
  14. 14. Tool Extensibility UI Tools Buttons Menus Views Actions New Languages New Editors Platform/Plugin Systems
  15. 15. 3 Challenges
  16. 16. Overview Context: embedded programming Examples are from Embedded Programming and use C as the „data format“ But applies similarly to other domains or other data formats/languages
  17. 17. Extensibility Examples Physical Units: The Challenge How do you work with physical units in your code?
  18. 18. Extensibility Examples Physical Units: The Challenge
  19. 19. Extensibility Examples Physical Units: The Challenge How do you detect this error?
  20. 20. Extensibility Examples Physical Units: The Challenge How do you detect this error? Tool (to do the checking) Data (the units in the code) You need:
  21. 21. Extensibility Examples Physical Units via Comments Bad!
  22. 22. Extensibility Examples Physical Units via Macros Bad!
  23. 23. Extensibility Examples Physical Units via external XML Bad!
  24. 24. Extensibility Examples Physical Units via Extension Good!
  25. 25. Extensibility Examples Physical Units via Extension Good! Type Checker (to do the checking) Program Code (the units in the code) You get:
  26. 26. Extensibility Examples State Machines: The Challenge How do you represent state-based behavior in code and support analyses?
  27. 27. Extensibility Examples State Machines: The Challenge red red yellow greenyellow ... the TL get green eventually? yellow blinking ... if the TL is turned off/on, it starts in yellow/bl. Is it guaranteed that ... ... the TL never goes from yellow to green?
  28. 28. Extensibility Examples State Machines via C idioms Bad!
  29. 29. Extensibility Examples State Machines via C idioms Bad!
  30. 30. Extensibility Examples State Machines via External Tool Bad!
  31. 31. Extensibility Examples State Machines: The Challenge Tool (to do the checking) Data („clean“ state machines) You need: How do you perform anlyses on state machines?
  32. 32. Extensibility Examples State Machines via Extensions Good!
  33. 33. Extensibility Examples State Machines via Extensions Good! Constraint Checker (to do the checking) Program Code (the units in the code) You get:
  34. 34. Extensibility Examples Tracing: The Challenge How do you add traces to requirements anywhere in your code, robustly?
  35. 35. Extensibility Examples Tracing via Macros Bad!
  36. 36. Extensibility Examples Tracing via Macros Tool (to create trace reports) Data (robust trace annotations) You need:
  37. 37. Extensibility Examples Tracing via Language Extensions You get the idea :-)
  38. 38. Extensibility Examples Combinations How do you combine these (and other) extenions?
  39. 39. Extensibility Examples Combinations
  40. 40. Extensibility Examples Combinations C
  41. 41. Extensibility Examples Combinations C Statemachines
  42. 42. Extensibility Examples Combinations C Statemachines Tracing
  43. 43. Extensibility Examples Combinations C Statemachines TracingUnits
  44. 44. Extensibility Examples Combinations (in an actual tool)
  45. 45. Key Takeaway Point Tool Extension is not enough!
  46. 46. Key Takeaway Point Tool Extension is not enough! Focus on the data first!
  47. 47. Key Takeaway Point Tool Extension is not enough! These both do not explicitly support extending the underlying data!
  48. 48. Key Takeaway Point Tool Extension is not enough! Relatively high effort to reimple- ment editors and „synchronizers“
  49. 49. Key Takeaway Point Tool Extension is not enough! Focus on the data first!
  50. 50. 4 GTSLGeneric Tools Specific Languages
  51. 51. Thought Process From Data Formats To Languages
  52. 52. Thought Process From Data Formats To Languages Structure, Constraints, Semantics Data Format
  53. 53. Thought Process From Data Formats To Languages Structure, Constraints, Semantics + SyntaxData Format Language
  54. 54. Thought Process From Data Formats To Languages Languages
  55. 55. Thought Process Language Engineering Languages Language Reuse Language Modularization Language Composition
  56. 56. Thought Process Language Engineering Languages Language Reuse Language Modularization Language Composition Language Engineering
  57. 57. Thought Process Language Engineering Languages Language Engineering
  58. 58. Thought Process Language Engineering Languages Language Engineering Text Tables Math Symbols Graphics Forms
  59. 59. Thought Process Language Engineering Languages Language Engineering Syntactic Diversity Text Tables Math Symbols Graphics Forms
  60. 60. Thought Process Language Engineering Languages Language Engineering Syntactic Diversity
  61. 61. Thought Process Language Workbenches Languages Language Engineering Syntactic Diversity But does this really work?
  62. 62. Thought Process Language Workbenches Languages Language Engineering Syntactic Diversity But does this really work? Language Workbenches
  63. 63. Thought Process Language Workbenches Languages Language Engineering Syntactic Diversity Language Workbenches
  64. 64. Generic Tools, Specific Languages Ingredients Languages Language Engineering Syntactic Diversity Language Workbenches
  65. 65. Generic Tools, Specific Languages Ingredients Languages Language Engineering Syntactic Diversity Language WorkbenchesGeneric Tools Specific Languages
  66. 66. Generic Tools, Specific Languages Ingredients Languages Language Engineering Syntactic Diversity Language WorkbenchesGeneric Tools Specific Languages (we don‘t have to reimplement editors and synchronizers)
  67. 67. Generic Tools, Specific Languages Ingredients Languages Language Engineering Syntactic Diversity Language WorkbenchesGeneric Tools Specific Languages support
  68. 68. Language Workbenches Typical Features
  69. 69. Language Workbenches Typical Features Language Definition, Reuse, Extension, Composition
  70. 70. Language Workbenches Typical Features Language Definition, Reuse, Extension, Comp osition Mixing Notations
  71. 71. Language Workbenches Typical Features Language Definition, Reuse, Extension, Composition Mixing Notations Type Systems, Constraints, Transformation, Interpretation
  72. 72. Language Workbenches Typical Features Goto Definition/Find Usages
  73. 73. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes
  74. 74. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting
  75. 75. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion
  76. 76. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace
  77. 77. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring
  78. 78. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring Debugging
  79. 79. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring Debugging Reporting
  80. 80. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring Debugging Reporting Visualization
  81. 81. Language Workbenches Typical Features Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring Debugging Reporting Visualization Version Control
  82. 82. Language Workbenches Typical Features for any Language!
  83. 83. Language Workbenches Typical Features Language Workbenches are IDEs for arbitrary languages.
  84. 84. Languages / Language Extensions Contribute Customizations
  85. 85. Languages / Language Extensions Contribute Customizations Language Definition, Reuse, Extension, Composition Mixing Notations Type Systems, Constraints, Transformation, Interpretation
  86. 86. Languages / Language Extensions Contribute Customizations Goto Definition/Find Usages Error Markup/Quick Fixes Syntax Highlighting Code Completion Search/Replace Refactoring Debugging Reporting Visualization Version Control
  87. 87. Languages / Language Extensions Additional Stuff Buttons Menus Views Actions
  88. 88. Tool Extensibility Study Findings I The majority of our interviewees were very successful with MDE but all of them either built their own modeling tools, made heavy adaptations of off-the- shelf tools, or spent a lot of time finding ways to work around tools. The only accounts of easy-to- use, intuitive tools came from those who had developed tools themselves for bespoke purposes. Indeed, this suggests that current tools are a barrier to success rather than an enabler.
  89. 89. Tool Extensibility Study Findings I The majority of our interviewees were very successful with MDE but all of them either built their own modeling tools, made heavy adaptations of off-the- shelf tools, or spent a lot of time finding ways to work around tools. The only accounts of easy-to-use, intuitive tools came from those who had developed tools themselves for bespoke purposes. Indeed, this suggests that current tools are a barrier to success rather than an enabler.
  90. 90. Tool Extensibility Study Findings II Complexity problems are typically associated with off- the- shelf tools. Of particular note is accidental complexity – which can be introduced due to poor consideration of other categories, such as lack of flexibility to adapt the tools to a company’s own context [..]
  91. 91. Tool Extensibility Study Findings II Complexity problems are typically associated with off- the- shelf tools. Of particular note is accidental complexity – which can be introduced due to poor consideration of other categories, such as lack of flexibility to adapt the tools to a company’s own context [..]
  92. 92. Language Workbenches Typical Features Used by the tool vendor to build the initial tool (languages).
  93. 93. Language Workbenches Typical Features Used by the tool vendor to build the initial tool (languages). Used by the end user to adapt the tool (lang extensions)!
  94. 94. Language Workbenches Typical Features Used by the tool vendor to build the initial tool (languages). Used by the end user to adapt the tool (lang extensions)! Extensions are first-class!
  95. 95. Generic Tools, Specific Languages Adaptability is built-in! Extensions are first-class!
  96. 96. Generic Tools, Specific Languages Adaptability is built-in! Extensions are first-class! Fundamentally different from Today‘s State-of-the-Art in Tools
  97. 97. 5 An Example
  98. 98. An Example System Language Engineering Embedded Software Specific Languages
  99. 99. An Example System Language Engineering Embedded Software Specific Languages A collection of integrated languages for embedded software engineering.
  100. 100. An Example System Language Engineering Embedded Software A collection of integrated languages Specific Languages for embedded software engineering.
  101. 101. An Example System Language Engineering Embedded Software Specific Languages An IDE for all Of them
  102. 102. An Example System Language Engineering Embedded Software Open Source Eclipse Public License http://mbeddr.com Specific Languages
  103. 103. An Example System Built on JetBrains MPS Generic Tool
  104. 104. An Example System Built on JetBrains MPS Generic Tool Projectional Editing Textual/Symbolic/Tabular/(soon Graphical) Multiple projections for the same language (in 3.0, due soon) Modular language development, extension and embedding
  105. 105. An Example System Built on JetBrains MPS Generic Tool Support for language aspects such as type system, scopes, code completion, find usages, dataflow Template-based approach for transformation and code generation with IDE support for target language in templates Support for building extensible debuggers
  106. 106. An Example System Built on JetBrains MPS Generic Tool
  107. 107. An Example System Built on JetBrains MPS Generic Tool Open Source Apache 2.0 http://jetbrains.com/mps
  108. 108. D E M O
  109. 109. Features Hello, World Messages abstract over IO. Report statements output them.
  110. 110. Features Function Types and Function Pointers Better notation for function types and function references.
  111. 111. Features Function Types and Lambdas And yes, we have lambdas (it‘s 2013 after all )
  112. 112. Features Reporting Reporting has conditions. All of it removed, when disabled!
  113. 113. Features Test Cases Special expression to run test cases and collect failure count.
  114. 114. Features Physical Units Types can have units; additional units can be defined.
  115. 115. Features Physical Units II Literals can have units; type system calculates w/ units.
  116. 116. Features Physical Units III Convertible units support *value* conversions!
  117. 117. Features Interfaces and Components I Interfaces define operations. Components provide interfaces.
  118. 118. Features Interfaces and Components II Components can be instantiated and wired.
  119. 119. Features Interfaces and Components III Interfaces can have pre- and postconditions.
  120. 120. Features Interfaces and Components IV In addition, interfaces can have protocol state machines.
  121. 121. Features Interfaces and Components V Components can also require ports (dependency injection)
  122. 122. Features Interfaces and Components VI Interfaces and components can be visualized.
  123. 123. Features Interfaces and Components VIII Mock components specify expectations in context of a test.
  124. 124. Features Interfaces and Components IX Mocks can be instantiated and validated in tests.
  125. 125. Features Interfaces and Components X Interface contracts can be verified statically!
  126. 126. Features Decision Tables Decision tables nicely exploit the projectional editor.
  127. 127. Features Combinable Extensions! C, components, units and decision tables comhined!
  128. 128. Features Decision Tables II Decision Tables are analyzed f. consistency and completeness
  129. 129. Features State Machines I State machines fundamentally consist of states.
  130. 130. Features State Machines II States contain transitions with guards, and actions.
  131. 131. Features State Machines III State machines can be instantiated; code can interact.
  132. 132. Features State Machines IV + special support for testing state machines.
  133. 133. Features State Machines V Outgoing interactins via function calls or out events.
  134. 134. Features State Machines VI Hierarchical state machines (composite states)
  135. 135. Features State Machines VII State Machines can be visualized in various way.
  136. 136. Features State Machines VIII Symbolic Model Checking for State Machines
  137. 137. Features Documentation Rich, Structured Comments (note the embedded nodes)
  138. 138. Features Documentation II Documentation Language w/ Embeddable Code (LaTeX/HTML)
  139. 139. Features Documentation III Format Text, Reference Code, Define Glossary Terms
  140. 140. Features Documentation IV Embeddable Expressions with Real Type Checking
  141. 141. Features Requirements Structured and Hierarchical Requirements. Color = TraceStatus
  142. 142. Features Requirements II Relationships between Requirements (downstream, upstream)
  143. 143. Features Requirements III Ubiquitous Tracing from Arbitrary Program Elements
  144. 144. Features Requirements IV Requirements are Extensible, e.g. with Scenarios
  145. 145. Features Requirements V Scenarios can be Visualized
  146. 146. Features Requirements VI Live (interpreted) Business Rules can be Embedded in Req.
  147. 147. Features Requirements VII These Business Rules can be „called“ from C Code
  148. 148. Features Product Line Variability Feature Models (and Checked Configs) to Express Variability
  149. 149. Features Product Line Variability II Runtime Variability based on Feature Models
  150. 150. Features Product Line Variability III Static Variability for any Program w/ Variant Editing
  151. 151. Features Debugging Debugging on the DSL Level (Extensible!)
  152. 152. Features VCS Diff/Merge Diff/Merge on the Projected Syntax
  153. 153. Features CI Server Integration Building Programs on Command Line/CI Server
  154. 154. 6 Summing up
  155. 155. Summing Up Key Points To build meaningful tools, the data must be extended. Extending the tool (buttons, views, ...) is not enough!
  156. 156. Summing Up Key Points Structured Data can be expressed as languages. Languages are data plus syntax.
  157. 157. Summing Up Key Points Language Engineering supports extension and composition This supports adapting tools for specific domains easily.
  158. 158. Summing Up Key Points IDE-style tools are very good for editing data/programs. We‘ve got a lot of experience from regular programming.
  159. 159. Summing Up Key Points Language Workbenches are the key enabling technology. MPS is IMHO the most power- ful, but it‘s not the only one!
  160. 160. Summing Up Key Points Let‘s build new classes of tools! ... which make meaningful extensibility a reality!
  161. 161. Ω The End.
  162. 162. voelter.de dslbook.org mbeddr.com jetbrains.com/mps The End.

×