Dynamic Language Embedding With Homogeneous Tool Support
•
1 like•1,212 views
Domain-specific languages (DSLs) are increasingly used as embedded languages within general-purpose host languages. DSLs provide a compact, dedicated syntax for specifying parts of an application related to specialized domains. Unfortunately, such language extensions typically do not integrate well with existing development tools. Editors, compilers and debuggers are either unaware of the extensions, or must be adapted at a non-trivial cost. Furthermore, these embedded languages typically conflict with the grammar of the host language and make it difficult to write hybrid code; few mechanisms exist to control the scope and usage of multiple tightly interconnected embedded languages. In this dissertation we present Helvetia, a novel approach to embed languages into an existing host language by leveraging the underlying representation of the host language used by these tools. We introduce Language Boxes, an approach that offers a simple, modular mechanism to encapsulate (i) compositional changes to the host language, (ii) transformations to address various concerns such as compilation and syntax highlighting, and (iii) scoping rules to control visibility of fine-grained language changes. We describe the design and implementation of Helvetia and Language Boxes, discuss the required infrastructure of a host language enabling language embedding, and validate our approach by case studies that demonstrate different ways to extend or adapt the host language syntax and semantics.
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to Dynamic Language Embedding With Homogeneous Tool Support
Similar to Dynamic Language Embedding With Homogeneous Tool Support (20)
More from Lukas Renggli
More from Lukas Renggli (11)
Recently uploaded
Recently uploaded (20)
Dynamic Language Embedding With Homogeneous Tool Support
- 1. Dynamic Language Embedding With Homogeneous Tool Support PhD Defense Lukas Renggli Advisor Oscar Nierstrasz 1
- 2. 2 SELECT email FROM users WHERE username = 'lr'
- 3. 3 SELECT email FROM users WHERE username = 'lr' Syntax
- 4. 4 SELECT email FROM users WHERE username = 'lr' Semantics
- 5. General Purpose Host Language 5 + SELECT email FROM users WHERE username = 'lr'
- 6. General Purpose Host Language 6 ? SELECT email FROM users WHERE username = 'lr' SyntaxSQL SemanticsSQL SyntaxHost SemanticsHost ToolsHost
- 7. 7
- 9. 9
- 11. 11
- 14. 14
- 15. 15 !" #"$%&
- 21. Homogeneous Code and Data Abstraction 21
- 24. Thesis To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation. 24
- 25. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 25
- 26. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 26
- 27. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 27
- 28. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 28
- 29. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 29
- 30. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 30
- 31. 31 !" #"$%&
- 33. 33 ◦ ◦ ◦ ◦ ◦ ● ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
- 34. 34 Host Language ◦ ◦ ◦ ◦ ◦ ● ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
- 35. 35 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
- 36. 36 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
- 37. 37 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
- 38. 38 Host Language ◦ ◦ ◦ Argot ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
- 39. 39 Host Language ◦ ◦ ◦ Argot ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● — ● ◦ ◦ — ● ◦ ● — ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
- 40. 40 Pidgin ◦ ● ● Creole ● ● ● Argot ◦ ◦ ● Syntax Vocabulary Semantics
- 41. 41
- 42. 42
- 43. 43
- 44. 44
- 45. 45
- 46. 46 Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2
- 47. Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 47 aBuilder row grow. aBuilder row fill. aBuilder column grow. aBuilder column fill. aBuilder x: 1 y: 1 add: (LabelShape new text: [ :each | each name ]; borderColor: #black; borderWidth: 1; yourself). aBuilder x: 1 y: 2 w: 2 h: 1 add: (RectangleShape new borderColor: #black; borderWidth: 1; width: 200; height: 100; yourself)
- 48. Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 48 row = grow. row = fill. column = grow. column = fill. (1 , 1) = label text: [ :each | each name ]; borderColor: #black; borderWidth: 1. (1 , 2) -‐ (2 , 1) = rectangle borderColor: #black; borderWidth: 1; width: 200; height: 100.
- 49. 49
- 50. shape { cols: #grow, #fill; rows: #grow, #fill; } label { position: 1 , 1; text: [ :each | each name ]; borderColor: #black; borderWidth: 1; } rectangle { position: 1 , 2; colspan: 2; borderColor: #black; borderWidth: 1; width: 200; height: 100; } Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 50
- 51. 51
- 57. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 57 Renggli et al.ECOOP 2010
- 65. 65 SELECT * FROM users
- 66. 66 | r | r := ^ User fromRow: r .SELECT * FROM users
- 70. ‣ System ‣ Packages ‣ Classes ‣ Methods Language Scope 70
- 73. Language Concern 73 Context Menus Navigation Search Code Expansion Code Completion Error Correction Custom Inspector Refactorings Code Folding Highlighting
- 76. Language Change 76 Host Language + SQL Language
- 77. Language Change 77 Host Language + SQL Language + ...
- 78. Host Language + SQL Language + ... = Custom Host Language Language Change 78
- 79. 79
- 80. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 80 Renggli et al.SLE 2009
- 81. Language Boxes Host Language Dynamic Grammars 81 Language and Tool Extensions
- 86. scanIdentifier self step. ((currentCharacter between: $A and: $Z) or: [ currentCharacter between: $a and: $z ]) ifTrue: [ [ self recordMatch: #IDENTIFIER. self step. (currentCharacter between: $0 and: $9) or: [ (currentCharacter between: $A and: $Z) or: [ currentCharacter between: $a and: $z ] ] ] whileTrue. ^ self reportLastMatch ] 86
- 87. #( #[1 0 9 0 25 0 13 0 34 0 17 0 40 0 21 0 41] #[1 0 9 0 25 0 13 0 34 0 93 0 76 0 157 0 112] #[1 2 38 0 21 2 38 0 25 2 38 0 26 0 13 0 34] #[0 1 154 0 16 0 21 0 25 0 26 0 34 0 40 0 41] #[0 1 210 0 76 0 81] #[0 1 214 0 76 0 81] #[1 0 173 0 76 0 177 0 81] #[0 1 134 0 16 0 21 0 25 0 26 0 34 0 40 0 41] #[1 1 46 0 21 1 46 0 25 1 46 0 26 1 69] #[1 1 54 0 21 1 54 0 25 1 54 0 26 1 54 0 34] #[0 2 102 0 21 0 25 0 26 0 34 0 40 0 41 0 76] #[0 2 50 0 21 0 25 0 26 0 76 0 79] #[1 1 13 0 76 2 85 0 124 1 21 0 125] #[1 2 89 0 17 2 30 0 21 2 30 0 82] #[1 2 93 0 21 2 97 0 82] ) 87
- 89. a..z a..z 0..9 ID ::= letter { letter | digit } ; 89
- 90. letter letter digit sequence choice many ID ::= letter { letter | digit } ; 90
- 92. letter letter digit sequence choice many letter ! letter | "_" 92
- 93. letter digit sequence choice manychoice _ letter choice _ letter ! letter | "_" 93
- 96. Insert grammar fragment before/after grammar production as a choice/sequence/replacement. 96 Language Change
- 97. Insert SQL grammar after expression production as an additional choice 97 Language Change
- 101. SELECT * FROM users 101
- 102. SELECT * FROM users 102 | r | r := ^ User fromRow: r .
- 103. expr sql| 103
- 104. SELECT * FROM users<SQL: > 104
- 111. expr sql| 111 !sql expr / !expr sql
- 112. expr sql$ 112 !sql expr / !expr sql / ui
- 113. expr sql$ 113 !sql expr / !expr sql / ui
- 114. Language and Tool Extensions Language Boxes Host Language Dynamic Grammars Renggli et al.DYLA 2010 114
- 115. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 115
- 116. Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes Mondrian Object Relationships Positional Arguments Program Checking Quasiquoting Regular Expression Roman Numbers SPath Expression SQL Schematic Tables String Interpolation Transactional Memory Tuple Space 116 [http://scg.unibe.ch/research/helvetia/examples]
- 117. Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes ✓Mondrian Object Relationships Positional Arguments Program Checking Quasiquoting Regular Expression Roman Numbers SPath Expression ✓SQL Schematic Tables String Interpolation Transactional Memory Tuple Space 117 [http://scg.unibe.ch/research/helvetia/examples]
- 118. Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes ✓Mondrian Object Relationships Positional Arguments ‣ Program Checking ‣ Quasiquoting Regular Expression Roman Numbers SPath Expression ✓SQL Schematic Tables String Interpolation ‣ Transactional Memory Tuple Space 118 [http://scg.unibe.ch/research/helvetia/examples]
- 119. apply hasChanged hasConflict Change object * changes Process 0..1 currentTransaction do: aBlock retry: aBlock checkpoint abort: anObject escapeContext Transaction previousCopy workingCopy ObjectChange applyBlock conflictTestBlock CustomChange * Transactional Memory 119
- 122. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions Renggli et al. CLSS 2009 Renggli et al. IWST 2009 Nierstrasz et al. LNCS 2009 Renggli et al. TOOLS 2010 122
- 123. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 123 To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation.
- 124. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 124 To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation.
- 125. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 125 To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation.
- 126. Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 126 To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation.
- 127. Language Boxes Host Language Dynamic Grammars 127 Language and Tool Extensions