Generic Tools, Specific Laguages

2,057 views

Published on

Developing software often requires using a number of tools and languages. In embedded software, for example, it is common to use C and its IDE, Matlab/Simulink, a number of custom XML files, a requirements management tool such as DOORS and possibly a UML tool and a variant management tool. The integration of such a zoo of tools is often a major source of (accidental) complexity in development projects.

Contrast that with the "good old days" when everything was text files and command line executables running on the unix shell. This approach had two important properties: the infrastructure was extremely generic (unix shell, pipes, text editors) and the actual contents were easily extensible and composable (new text file formats/languages and new command line tools); a productive environment for a given project or domain could easily be built from the generic infrastructure plus a few custom extensions.

In this talk I want to propose that creating (domain-specific) development environments based on language workbenches results in many of the same advantages that we all valued in the unix shell-world. A language workbench is an extremely generic infrastructure that is easily extensible with new languages. It is easy to create domain-specific development tools that can address different aspects of the system with suitable abstractions, but are nonetheless very well integrated in terms of semantics and tooling.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,057
On SlideShare
0
From Embeds
0
Number of Embeds
705
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Generic Tools, Specific Laguages

  1. 1. Generic Tools -Specific LanguagesOn the Art of BuildingSoftware Development Toolshttp://voelter.devoelter@acm.org@markusvoelterMarkus Voelterindependent/itemis
  2. 2. 123ToolsExtensibilityChallenges456GTSLAn ExampleWrap Up
  3. 3. 1Tools
  4. 4. ToolsCommand-Line Tools
  5. 5. ToolsUI Tools
  6. 6. 2Tool Extensibility
  7. 7. Tool ExtensibilityStudy Findings IThe majority of our interviewees were very successfulwith MDE but all of them either built their ownmodeling tools, made heavy adaptations of off-the-shelf tools, or spent a lot of time finding ways to workaround tools. The only accounts of easy-to-use, intuitive tools came from those who haddeveloped tools themselves for bespoke purposes.Indeed, this suggests that current tools are a barrierto success rather than an enabler.
  8. 8. Tool ExtensibilityStudy Findings IIComplexity problems are typically associated with off-the- shelf tools. Of particular note is accidentalcomplexity – which can be introduced due to poorconsideration of other categories, such as lack offlexibility to adapt the tools to a company’s owncontext [..]
  9. 9. Tool ExtensibilityStudy Findings IIIOur interviews point to a strong need for tailoring ofsome sort: either tailor the tool to the process, tailorthe process to the tool, or build your own tool thatnaturally fits your own process. Based on our data, itseems that, on balance, it is currently much easier todo the latter.
  10. 10. Tool ExtensibilityCommand-Line ToolsNew File FormatsNew Processors
  11. 11. Tool ExtensibilityCommand-Line ToolsNew File FormatsNew ProcessorsAssemble Components (Pipes & Filters)
  12. 12. Tool ExtensibilityUI ToolsButtonsMenusViewsActions
  13. 13. Tool ExtensibilityUI ToolsButtonsMenusViewsActionsNew LanguagesNew Editors
  14. 14. Tool ExtensibilityUI ToolsButtonsMenusViewsActionsNew LanguagesNew EditorsPlatform/Plugin Systems
  15. 15. 3Challenges
  16. 16. OverviewContext: embedded programmingExamples are from EmbeddedProgramming and use C as the„data format“But applies similarly to other domainsor other data formats/languages
  17. 17. Extensibility ExamplesPhysical Units: The ChallengeHow do you work withphysical units in your code?
  18. 18. Extensibility ExamplesPhysical Units: The Challenge
  19. 19. Extensibility ExamplesPhysical Units: The ChallengeHow do you detect this error?
  20. 20. Extensibility ExamplesPhysical Units: The ChallengeHow do you detect this error?Tool (to do the checking)Data (the units in the code)You need:
  21. 21. Extensibility ExamplesPhysical Units via CommentsBad!
  22. 22. Extensibility ExamplesPhysical Units via MacrosBad!
  23. 23. Extensibility ExamplesPhysical Units via external XMLBad!
  24. 24. Extensibility ExamplesPhysical Units via ExtensionGood!
  25. 25. Extensibility ExamplesPhysical Units via ExtensionGood!Type Checker (to do the checking)Program Code (the units in the code)You get:
  26. 26. Extensibility ExamplesState Machines: The ChallengeHow do you representstate-based behavior in codeand support analyses?
  27. 27. Extensibility ExamplesState Machines: The Challengeredredyellowgreenyellow... the TL get green eventually?yellowblinking... if the TL is turned off/on, it starts in yellow/bl.Is it guaranteed that ...... the TL never goes from yellow to green?
  28. 28. Extensibility ExamplesState Machines via C idiomsBad!
  29. 29. Extensibility ExamplesState Machines via C idiomsBad!
  30. 30. Extensibility ExamplesState Machines via External ToolBad!
  31. 31. Extensibility ExamplesState Machines: The ChallengeTool (to do the checking)Data („clean“ state machines)You need:How do you performanlyses on state machines?
  32. 32. Extensibility ExamplesState Machines via ExtensionsGood!
  33. 33. Extensibility ExamplesState Machines via ExtensionsGood!Constraint Checker (to do the checking)Program Code (the units in the code)You get:
  34. 34. Extensibility ExamplesTracing: The ChallengeHow do you add traces torequirements anywhere inyour code, robustly?
  35. 35. Extensibility ExamplesTracing via MacrosBad!
  36. 36. Extensibility ExamplesTracing via MacrosTool (to create trace reports)Data (robust trace annotations)You need:
  37. 37. Extensibility ExamplesTracing via Language ExtensionsYou get the idea :-)
  38. 38. Extensibility ExamplesCombinationsHow do you combine these(and other) extenions?
  39. 39. Extensibility ExamplesCombinations
  40. 40. Extensibility ExamplesCombinationsC
  41. 41. Extensibility ExamplesCombinationsCStatemachines
  42. 42. Extensibility ExamplesCombinationsCStatemachinesTracing
  43. 43. Extensibility ExamplesCombinationsCStatemachinesTracingUnits
  44. 44. Extensibility ExamplesCombinations (in an actual tool)
  45. 45. Key Takeaway PointTool Extension is not enough!
  46. 46. Key Takeaway PointTool Extension is not enough!Focus on thedata first!
  47. 47. Key Takeaway PointTool Extension is not enough!These both do not explicitly supportextending the underlying data!
  48. 48. Key Takeaway PointTool Extension is not enough!Relatively high effort to reimple-ment editors and „synchronizers“
  49. 49. Key Takeaway PointTool Extension is not enough!Focus on thedata first!
  50. 50. 4GTSLGenericToolsSpecificLanguages
  51. 51. Thought ProcessFrom Data Formats To Languages
  52. 52. Thought ProcessFrom Data Formats To LanguagesStructure, Constraints, SemanticsData Format
  53. 53. Thought ProcessFrom Data Formats To LanguagesStructure, Constraints, Semantics+ SyntaxData FormatLanguage
  54. 54. Thought ProcessFrom Data Formats To LanguagesLanguages
  55. 55. Thought ProcessLanguage EngineeringLanguagesLanguage ReuseLanguage ModularizationLanguage Composition
  56. 56. Thought ProcessLanguage EngineeringLanguagesLanguage ReuseLanguage ModularizationLanguage CompositionLanguage Engineering
  57. 57. Thought ProcessLanguage EngineeringLanguagesLanguage Engineering
  58. 58. Thought ProcessLanguage EngineeringLanguagesLanguage EngineeringTextTablesMathSymbolsGraphicsForms
  59. 59. Thought ProcessLanguage EngineeringLanguagesLanguage EngineeringSyntactic DiversityTextTablesMathSymbolsGraphicsForms
  60. 60. Thought ProcessLanguage EngineeringLanguagesLanguage EngineeringSyntactic Diversity
  61. 61. Thought ProcessLanguage WorkbenchesLanguagesLanguage EngineeringSyntactic DiversityBut does this really work?
  62. 62. Thought ProcessLanguage WorkbenchesLanguagesLanguage EngineeringSyntactic DiversityBut does this really work?Language Workbenches
  63. 63. Thought ProcessLanguage WorkbenchesLanguagesLanguage EngineeringSyntactic DiversityLanguage Workbenches
  64. 64. Generic Tools, Specific LanguagesIngredientsLanguagesLanguage EngineeringSyntactic DiversityLanguage Workbenches
  65. 65. Generic Tools, Specific LanguagesIngredientsLanguagesLanguage EngineeringSyntactic DiversityLanguage WorkbenchesGeneric ToolsSpecificLanguages
  66. 66. Generic Tools, Specific LanguagesIngredientsLanguagesLanguage EngineeringSyntactic DiversityLanguage WorkbenchesGeneric ToolsSpecificLanguages(we don‘t have to reimplementeditors and synchronizers)
  67. 67. Generic Tools, Specific LanguagesIngredientsLanguagesLanguage EngineeringSyntactic DiversityLanguage WorkbenchesGeneric ToolsSpecificLanguagessupport
  68. 68. Language WorkbenchesTypical Features
  69. 69. Language WorkbenchesTypical FeaturesLanguage Definition, Reuse,Extension, Composition
  70. 70. Language WorkbenchesTypical FeaturesLanguageDefinition, Reuse, Extension, CompositionMixing Notations
  71. 71. Language WorkbenchesTypical FeaturesLanguage Definition, Reuse,Extension, CompositionMixing NotationsType Systems, Constraints,Transformation, Interpretation
  72. 72. Language WorkbenchesTypical FeaturesGoto Definition/Find Usages
  73. 73. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick Fixes
  74. 74. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax Highlighting
  75. 75. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode Completion
  76. 76. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/Replace
  77. 77. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoring
  78. 78. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoringDebugging
  79. 79. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoringDebuggingReporting
  80. 80. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoringDebuggingReportingVisualization
  81. 81. Language WorkbenchesTypical FeaturesGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoringDebuggingReportingVisualizationVersion Control
  82. 82. Language WorkbenchesTypical Featuresfor anyLanguage!
  83. 83. Language WorkbenchesTypical FeaturesLanguageWorkbenches areIDEs for arbitrarylanguages.
  84. 84. Languages / Language ExtensionsContribute Customizations
  85. 85. Languages / Language ExtensionsContribute CustomizationsLanguage Definition, Reuse,Extension, CompositionMixing NotationsType Systems, Constraints,Transformation, Interpretation
  86. 86. Languages / Language ExtensionsContribute CustomizationsGoto Definition/Find UsagesError Markup/Quick FixesSyntax HighlightingCode CompletionSearch/ReplaceRefactoringDebuggingReportingVisualizationVersion Control
  87. 87. Languages / Language ExtensionsAdditional StuffButtonsMenusViewsActions
  88. 88. Tool ExtensibilityStudy Findings IThe majority of our interviewees were very successfulwith MDE but all of them either built their ownmodeling tools, made heavy adaptations of off-the-shelf tools, or spent a lot of time finding ways to workaround tools. The only accounts of easy-to-use, intuitive tools came from those who haddeveloped tools themselves for bespoke purposes.Indeed, this suggests that current tools are a barrierto success rather than an enabler.
  89. 89. Tool ExtensibilityStudy Findings IThe majority of our interviewees were very successfulwith MDE but all of them either built their ownmodeling tools, made heavy adaptations of off-the-shelf tools, or spent a lot of time finding ways to workaround tools. The only accounts of easy-to-use,intuitive tools came from those who had developedtools themselves for bespoke purposes. Indeed, thissuggests that current tools are a barrier to successrather than an enabler.
  90. 90. Tool ExtensibilityStudy Findings IIComplexity problems are typically associated with off-the- shelf tools. Of particular note is accidentalcomplexity – which can be introduced due to poorconsideration of other categories, such as lack offlexibility to adapt the tools to a company’s owncontext [..]
  91. 91. Tool ExtensibilityStudy Findings IIComplexity problems are typically associated with off-the- shelf tools. Of particular note is accidentalcomplexity – which can be introduced due to poorconsideration of other categories, such as lack offlexibility to adapt the tools to a company’s owncontext [..]
  92. 92. Language WorkbenchesTypical FeaturesUsed by the tool vendor tobuild the initial tool (languages).
  93. 93. Language WorkbenchesTypical FeaturesUsed by the tool vendor tobuild the initial tool (languages).Used by the end user toadapt the tool (lang extensions)!
  94. 94. Language WorkbenchesTypical FeaturesUsed by the tool vendor tobuild the initial tool (languages).Used by the end user toadapt the tool (lang extensions)!Extensions are first-class!
  95. 95. Generic Tools, Specific LanguagesAdaptability is built-in!Extensions arefirst-class!
  96. 96. Generic Tools, Specific LanguagesAdaptability is built-in!Extensions arefirst-class!Fundamentally different fromToday‘s State-of-the-Art in Tools
  97. 97. 5An Example
  98. 98. An Example SystemLanguage Engineering Embedded SoftwareSpecific Languages
  99. 99. An Example SystemLanguage Engineering Embedded SoftwareSpecific LanguagesA collection of integrated languagesfor embedded software engineering.
  100. 100. An Example SystemLanguage Engineering Embedded SoftwareA collection of integrated languagesSpecific Languagesfor embedded software engineering.
  101. 101. An Example SystemLanguage Engineering Embedded SoftwareSpecific LanguagesAnIDEforallOfthem
  102. 102. An Example SystemLanguage Engineering Embedded SoftwareOpen SourceEclipse Public Licensehttp://mbeddr.comSpecific Languages
  103. 103. An Example SystemBuilt on JetBrains MPSGeneric Tool
  104. 104. An Example SystemBuilt on JetBrains MPSGeneric ToolProjectional EditingTextual/Symbolic/Tabular/(soon Graphical)Multiple projections for the same language(in 3.0, due soon)Modular language development, extensionand embedding
  105. 105. An Example SystemBuilt on JetBrains MPSGeneric ToolSupport for language aspects such as typesystem, scopes, code completion, findusages, dataflowTemplate-based approach for transformationand code generation with IDE support fortarget language in templatesSupport for building extensible debuggers
  106. 106. An Example SystemBuilt on JetBrains MPSGeneric Tool
  107. 107. An Example SystemBuilt on JetBrains MPSGeneric ToolOpen SourceApache 2.0http://jetbrains.com/mps
  108. 108. D E M O
  109. 109. FeaturesHello, WorldMessages abstract over IO. Report statements output them.
  110. 110. FeaturesFunction Types and Function PointersBetter notation for function types and function references.
  111. 111. FeaturesFunction Types and LambdasAnd yes, we have lambdas (it‘s 2013 after all )
  112. 112. FeaturesReportingReporting has conditions. All of it removed, when disabled!
  113. 113. FeaturesTest CasesSpecial expression to run test cases and collect failure count.
  114. 114. FeaturesPhysical UnitsTypes can have units; additional units can be defined.
  115. 115. FeaturesPhysical Units IILiterals can have units; type system calculates w/ units.
  116. 116. FeaturesPhysical Units IIIConvertible units support *value* conversions!
  117. 117. FeaturesInterfaces and Components IInterfaces define operations. Components provide interfaces.
  118. 118. FeaturesInterfaces and Components IIComponents can be instantiated and wired.
  119. 119. FeaturesInterfaces and Components IIIInterfaces can have pre- and postconditions.
  120. 120. FeaturesInterfaces and Components IVIn addition, interfaces can have protocol state machines.
  121. 121. FeaturesInterfaces and Components VComponents can also require ports (dependency injection)
  122. 122. FeaturesInterfaces and Components VIInterfaces and components can be visualized.
  123. 123. FeaturesInterfaces and Components VIIIMock components specify expectations in context of a test.
  124. 124. FeaturesInterfaces and Components IXMocks can be instantiated and validated in tests.
  125. 125. FeaturesInterfaces and Components XInterface contracts can be verified statically!
  126. 126. FeaturesDecision TablesDecision tables nicely exploit the projectional editor.
  127. 127. FeaturesCombinable Extensions!C, components, units and decision tables comhined!
  128. 128. FeaturesDecision Tables IIDecision Tables are analyzed f. consistency and completeness
  129. 129. FeaturesState Machines IState machines fundamentally consist of states.
  130. 130. FeaturesState Machines IIStates contain transitions with guards, and actions.
  131. 131. FeaturesState Machines IIIState machines can be instantiated; code can interact.
  132. 132. FeaturesState Machines IV+ special support for testing state machines.
  133. 133. FeaturesState Machines VOutgoing interactins via function calls or out events.
  134. 134. FeaturesState Machines VIHierarchical state machines (composite states)
  135. 135. FeaturesState Machines VIIState Machines can be visualized in various way.
  136. 136. FeaturesState Machines VIIISymbolic Model Checking for State Machines
  137. 137. FeaturesDocumentationRich, Structured Comments (note the embedded nodes)
  138. 138. FeaturesDocumentation IIDocumentation Language w/ Embeddable Code (LaTeX/HTML)
  139. 139. FeaturesDocumentation IIIFormat Text, Reference Code, Define Glossary Terms
  140. 140. FeaturesDocumentation IVEmbeddable Expressions with Real Type Checking
  141. 141. FeaturesRequirementsStructured and Hierarchical Requirements. Color = TraceStatus
  142. 142. FeaturesRequirements IIRelationships between Requirements (downstream, upstream)
  143. 143. FeaturesRequirements IIIUbiquitous Tracing from Arbitrary Program Elements
  144. 144. FeaturesRequirements IVRequirements are Extensible, e.g. with Scenarios
  145. 145. FeaturesRequirements VScenarios can be Visualized
  146. 146. FeaturesRequirements VILive (interpreted) Business Rules can be Embedded in Req.
  147. 147. FeaturesRequirements VIIThese Business Rules can be „called“ from C Code
  148. 148. FeaturesProduct Line VariabilityFeature Models (and Checked Configs) to Express Variability
  149. 149. FeaturesProduct Line Variability IIRuntime Variability based on Feature Models
  150. 150. FeaturesProduct Line Variability IIIStatic Variability for any Program w/ Variant Editing
  151. 151. FeaturesDebuggingDebugging on the DSL Level (Extensible!)
  152. 152. FeaturesVCS Diff/MergeDiff/Merge on the Projected Syntax
  153. 153. FeaturesCI Server IntegrationBuilding Programs on Command Line/CI Server
  154. 154. 6Summing up
  155. 155. Summing UpKey PointsTo build meaningfultools, the data mustbe extended.Extending the tool (buttons,views, ...) is not enough!
  156. 156. Summing UpKey PointsStructured Data canbe expressed aslanguages.Languages are data plus syntax.
  157. 157. Summing UpKey PointsLanguage Engineeringsupports extensionand compositionThis supports adapting toolsfor specific domains easily.
  158. 158. Summing UpKey PointsIDE-style tools arevery good for editingdata/programs.We‘ve got a lot of experiencefrom regular programming.
  159. 159. Summing UpKey PointsLanguage Workbenchesare the key enablingtechnology.MPS is IMHO the most power-ful, but it‘s not the only one!
  160. 160. Summing UpKey PointsLet‘s build new classesof tools!... which make meaningfulextensibility a reality!
  161. 161. ΩThe End.
  162. 162. voelter.dedslbook.orgmbeddr.comjetbrains.com/mpsThe End.

×