Dynamic Language Embedding With Homogeneous Tool Support

Dynamic Language Embedding With Homogeneous Tool Support
Dynamic Language Embedding With Homogeneous Tool Support PhD Defense Lukas Renggli Advisor Oscar Nierstrasz 1
2 SELECT email FROM users WHERE username = 'lr'
3 SELECT email FROM users WHERE username = 'lr' Syntax
4 SELECT email FROM users WHERE username = 'lr' Semantics
General Purpose Host Language 5 + SELECT email FROM users WHERE username = 'lr'
General Purpose Host Language 6 ? SELECT email FROM users WHERE username = 'lr' SyntaxSQL SemanticsSQL SyntaxHost SemanticsHost ToolsHost
7
8 Host Language External Language
9
10 Host Language Internal Language
11
12 Host Language Embedded Language
13 Non-Standard Host Language Embedded Language
14
15 !" #"$%&
MarcoZanoli,cc-by-sa,2.5,www.wikipedia.de 16
Conventional Language 17
18 MarcoZanoli,cc-by-sa,2.5,www.wikipedia.de
Multiple Context Specific Languages 19
Homogeneous Code and Data Abstraction 21
§ § § 22 MarcoZanoli,cc-by-sa,2.5,www.wikipedia.de
§ § § Homogeneous Tool Support 23
Thesis To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation. 24
Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 25
31 !" #"$%&
32 Types of Embedded Languages
33 ◦ ◦ ◦ ◦ ◦ ● ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
34 Host Language ◦ ◦ ◦ ◦ ◦ ● ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
35 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
36 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ ● ● ●Syntax Vocabulary Semantics
37 Host Language ◦ ◦ ◦ ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
38 Host Language ◦ ◦ ◦ Argot ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● ● ◦ ◦ ● ◦ ● ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
39 Host Language ◦ ◦ ◦ Argot ◦ ◦ ● Internal Language ◦ ● ◦ Pidgin ◦ ● ● — ● ◦ ◦ — ● ◦ ● — ● ● ◦ Creole ● ● ●Syntax Vocabulary Semantics
40 Pidgin ◦ ● ● Creole ● ● ● Argot ◦ ◦ ● Syntax Vocabulary Semantics
41
42
43
44
45
46 Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2
Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 47 aBuilder row grow. aBuilder row fill. aBuilder column grow. aBuilder column fill. aBuilder x: 1 y: 1 add: (LabelShape new text: [ :each | each name ]; borderColor: #black; borderWidth: 1; yourself). aBuilder x: 1 y: 2 w: 2 h: 1 add: (RectangleShape new borderColor: #black; borderWidth: 1; width: 200; height: 100; yourself)
Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 48 row = grow. row = fill. column = grow. column = fill. (1 , 1) = label text: [ :each | each name ]; borderColor: #black; borderWidth: 1. (1 , 2) -‐ (2 , 1) = rectangle borderColor: #black; borderWidth: 1; width: 200; height: 100.
49
shape { cols: #grow, #fill; rows: #grow, #fill; } label { position: 1 , 1; text: [ :each | each name ]; borderColor: #black; borderWidth: 1; } rectangle { position: 1 , 2; colspan: 2; borderColor: #black; borderWidth: 1; width: 200; height: 100; } Package Name x = 1 y = 1 (2, 1) (2, 2)(1, 2) x = 2 y = 2 50
51
52 Conventional Language
53 Conventional Language Context Specific
54 Conventional Language Context Specific Homogeneous Code & Data
55 Conventional Language Context Specific Homogeneous Code & Data Homogeneous Tool Support
56 Conventional Language Context Specific Homogeneous Code & Data Homogeneous Tool Support
Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 57 Renggli et al.ECOOP 2010
Editor Compiler Debugger Syntax Language 1 60
Editor Compiler Debugger Syntax Language 1 61 Language 2
Language Boxes 64
65 SELECT * FROM users
66 | r | r := ^ User fromRow: r .SELECT * FROM users
Language Scope Language Concern Language Change Language Box 67
Language Scope Active? 68
Language Scope 69
‣ System ‣ Packages ‣ Classes ‣ Methods Language Scope 70
Language Concern Semantics 71
Language Concern Transformation 72
Language Concern 73 Context Menus Navigation Search Code Expansion Code Completion Error Correction Custom Inspector Refactorings Code Folding Highlighting
Language Change 74 Syntax
Language Change 75 Host Language
Language Change 76 Host Language + SQL Language
Language Change 77 Host Language + SQL Language + ...
Host Language + SQL Language + ... = Custom Host Language Language Change 78
79
Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 80 Renggli et al.SLE 2009
Language Boxes Host Language Dynamic Grammars 81 Language and Tool Extensions
Language Change Language Concern Language Scope Language Box 82
Language Change = Host Language δ 83
Language Change = Host Language Grammarδ 84
Language Change = Grammar Transformation 85
scanIdentifier self step. ((currentCharacter between: $A and: $Z) or: [ currentCharacter between: $a and: $z ]) ifTrue: [ [ self recordMatch: #IDENTIFIER. self step. (currentCharacter between: $0 and: $9) or: [ (currentCharacter between: $A and: $Z) or: [ currentCharacter between: $a and: $z ] ] ] whileTrue. ^ self reportLastMatch ] 86
#( #[1 0 9 0 25 0 13 0 34 0 17 0 40 0 21 0 41] #[1 0 9 0 25 0 13 0 34 0 93 0 76 0 157 0 112] #[1 2 38 0 21 2 38 0 25 2 38 0 26 0 13 0 34] #[0 1 154 0 16 0 21 0 25 0 26 0 34 0 40 0 41] #[0 1 210 0 76 0 81] #[0 1 214 0 76 0 81] #[1 0 173 0 76 0 177 0 81] #[0 1 134 0 16 0 21 0 25 0 26 0 34 0 40 0 41] #[1 1 46 0 21 1 46 0 25 1 46 0 26 1 69] #[1 1 54 0 21 1 54 0 25 1 54 0 26 1 54 0 34] #[0 2 102 0 21 0 25 0 26 0 34 0 40 0 41 0 76] #[0 2 50 0 21 0 25 0 26 0 76 0 79] #[1 1 13 0 76 2 85 0 124 1 21 0 125] #[1 2 89 0 17 2 30 0 21 2 30 0 82] #[1 2 93 0 21 2 97 0 82] ) 87
Scannerless Parser Combinator 88
a..z a..z 0..9 ID ::= letter { letter | digit } ; 89
letter letter digit sequence choice many ID ::= letter { letter | digit } ; 90
Grammar Transformation 91
letter letter digit sequence choice many letter ! letter | "_" 92
letter digit sequence choice manychoice _ letter choice _ letter ! letter | "_" 93
letter digit sequence choice manychoice _ Optimizations 94
Grammar Composition 95
Insert grammar fragment before/after grammar production as a choice/sequence/replacement. 96 Language Change
Insert SQL grammar after expression production as an additional choice 97 Language Change
98 Language Change Variable Literal Parens Expr:
99 Language Change Variable Literal Parens Expr: SQL
Conflicts & Ambiguities 100
SELECT * FROM users 101
SELECT * FROM users 102 | r | r := ^ User fromRow: r .
expr sql| 103
SELECT * FROM users<SQL: > 104
Parsing Expression Grammars 105
expr sql/ ordered 106
expr sql/ no conflict 107
expr sql/ 108 surprise
/ exprsql 109 surprise
expr sql| 110 unordered
expr sql| 111 !sql expr / !expr sql
expr sql$ 112 !sql expr / !expr sql / ui
expr sql$ 113 !sql expr / !expr sql / ui
Language and Tool Extensions Language Boxes Host Language Dynamic Grammars Renggli et al.DYLA 2010 114
Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes Mondrian Object Relationships Positional Arguments Program Checking Quasiquoting Regular Expression Roman Numbers SPath Expression SQL Schematic Tables String Interpolation Transactional Memory Tuple Space 116 [http://scg.unibe.ch/research/helvetia/examples]
Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes ✓Mondrian Object Relationships Positional Arguments Program Checking Quasiquoting Regular Expression Roman Numbers SPath Expression ✓SQL Schematic Tables String Interpolation Transactional Memory Tuple Space 117 [http://scg.unibe.ch/research/helvetia/examples]
Assignments and Swapping Asynchronous Messages Automaton Brainfuck Language Functional Pattern Matching Grammar Definition Message Pipes ✓Mondrian Object Relationships Positional Arguments ‣ Program Checking ‣ Quasiquoting Regular Expression Roman Numbers SPath Expression ✓SQL Schematic Tables String Interpolation ‣ Transactional Memory Tuple Space 118 [http://scg.unibe.ch/research/helvetia/examples]
apply hasChanged hasConflict Change object * changes Process 0..1 currentTransaction do: aBlock retry: aBlock checkpoint abort: anObject escapeContext Transaction previousCopy workingCopy ObjectChange applyBlock conflictTestBlock CustomChange * Transactional Memory 119
Meta-Programming Facilities ``(`,(aString) asRegex) 120
Domain-Specific Program Checking 121
Language Boxes Host Language Dynamic Grammars Language and Tool Extensions Renggli et al. CLSS 2009 Renggli et al. IWST 2009 Nierstrasz et al. LNCS 2009 Renggli et al. TOOLS 2010 122
Language Boxes Host Language Dynamic Grammars Language and Tool Extensions 123 To support seamless integration of context-dependent languages without breaking the tools, we need 1. a host-language grammar that can be changed by language extensions, 2. a first-class language description used by the development environment, and 3. a transformation mechanism of the embedded language into a common executable representation.
Language Boxes Host Language Dynamic Grammars 127 Language and Tool Extensions

Dynamic Language Embedding With Homogeneous Tool Support

Check these out next

Dynamic Language Embedding With Homogeneous Tool Support

Recommended

More Related Content

Viewers also liked(20)

Similar to Dynamic Language Embedding With Homogeneous Tool Support(20)

More from Lukas Renggli(11)

Recently uploaded(20)

Dynamic Language Embedding With Homogeneous Tool Support