The document discusses several challenges in designing domain-specific languages (DSLs). It notes that while reusing the target platform's expressions is pragmatic, it can lead to dispersed information and complexity. An alternative proposed is developing expressions as a reusable library, offering advantages like non-dispersed information and good tooling while requiring additional learning. Concrete syntax challenges include grammar mixing and ambiguity, which can be addressed with techniques like scannerless parsing and contextual lexing. Abstract syntax poses challenges for meta-model reuse and linking elements. Validation rules may not be weakened and compilers/interpreters require contract specifications. Customizing generated artifacts requires non-invasive changes.

1. Challenges in DSL Design
Sebastian Zarnekow - Sven Efftinge

2. What do today’s DSL
projects look like?

3. Lots of problems are solved

4. Lots of problems are solved
textual
DSL frameworks

5. Lots of problems are solved
textual
DSL frameworks
graphical
DSL frameworks

6. Lots of problems are solved
textual
DSL frameworks
graphical
DSL frameworks
nice transformation
languages

7. Lots of problems are solved
textual
DSL frameworks
graphical
DSL frameworks
nice transformation
languages
open source IDEs

8. Lots of problems are solved
textual
DSL frameworks
graphical
DSL frameworks
nice validation nice transformation
languages languages
open source IDEs

9. Lots of problems are solved
textual
DSL frameworks
nice template graphical
languages DSL frameworks
nice validation nice transformation
languages languages
open source IDEs

10. Lots of problems are solved
textual
DSL frameworks
nice template graphical
languages DSL frameworks
proven infrastructure
nice validation nice transformation
languages languages
open source IDEs

11. In most cases
DSLs cover only 80%

12. ... but what’s missing?

13. Expressions

14. Why are expressions
missing?

15. Hiding
implementation details?

16. Information hiding in computer science is the
principle of the hiding of design decisions in a
computer program that are most likely to change,
thus protecting other parts of the program from
change if the design decision is changed. The
protection involves providing a stable interface which
shields the remainder of the program from the
implementation (the details that are most likely to
change).
Hiding
implementation details?

17. ation of c
Separ once
rns?

18. atio n of c
Se par once
rns?
In computer science, separation of concerns (SoC) is
the process of breaking a computer program into
distinct features that overlap in functionality as little as
possible. A concern is any piece of interest or focus in a
program. Typically, concerns are synonymous with
features or behaviors.

20. • expressions are complicated to
implement

21. • expressions are complicated to
implement
• expressions of target platform
language are already understood

22. • expressions are complicated to
implement
• expressions of target platform
language are already understood
• tooling and libraries can be reused

23. What’s the price we pay?

25. Protected Regions

26. Protected Regions
• generated and manually written code mixed
up

27. Protected Regions
• generated and manually written code mixed
up
• manual deletion of artifacts

28. Protected Regions
• generated and manually written code mixed
up
• manual deletion of artifacts
• platform dependent information

29. Protected Regions
• generated and manually written code mixed
up
• manual deletion of artifacts
• platform dependent information
• information is dispersed!

30. Generation Gap Pattern

31. Generation Gap Pattern
technically required type hierarchies

32. Generation Gap Pattern
technically required type hierarchies
additional complexity

33. Generation Gap Pattern
technically required type hierarchies
additional complexity
platform dependent information

34. Generation Gap Pattern
technically required type hierarchies
additional complexity
platform dependent information
information dispersed!

35. Black box
target language literals

36. Black box
target language literals
• bound to target platform

37. Black box
target language literals
• bound to target platform
• no option for MDA friends

38. Black box
target language literals
• bound to target platform
• no option for MDA friends
• no tooling!

39. Black box
target language literals
• bound to target platform
• no option for MDA friends
• no tooling!
• long turnarounds!

41. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

42. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

43. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

44. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

45. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

46. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

47. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

48. reuse target platform
developers know it
libraries
very good tooling
proven
dispersed information
complexity in code generation
hard to built interpreters

49. reuse target platform invent expressions
developers know it non-dispersed information
libraries very complex
very good tooling very expensive
relatively low potential for
proven
further abstraction
dispersed information
complexity in code generation
hard to built interpreters

50. reuse target platform invent expressions
developers know it non-dispersed information
libraries very complex
very good tooling very expensive
relatively low potential for
proven
further abstraction
dispersed information
complexity in code generation
hard to built interpreters

51. reuse target platform invent expressions
developers know it non-dispersed information
libraries very complex
very good tooling very expensive
relatively low potential for
proven
further abstraction
dispersed information
complexity in code generation
hard to built interpreters

52. reuse target platform invent expressions
developers know it non-dispersed information
libraries very complex
very good tooling very expensive
relatively low potential for
proven
further abstraction
dispersed information
complexity in code generation
hard to built interpreters

53. reuse target platform invent expressions
developers know it non-dispersed information
libraries very complex
very good tooling very expensive
relatively low potential for
proven
further abstraction
dispersed information
complexity in code generation
hard to built interpreters

54. Reusing Target Platform
• seems reasonable & pragmatic
• good compromise

55. But what if ...

56. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

57. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

58. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

59. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

60. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

61. But what if ...
reuse target platform expression as library
developers know it non-dispersed information
libraries nicer expression language
- closures
very good tooling
- type inference
proven - operator overloading
dispersed information extendable (e.g. custom literals)
complexity in code generation good tooling
hard to built interpreters additional learning curve

62. What are the
challenges?

64. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

65. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

66. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

67. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

68. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

69. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

70. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

71. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

72. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

73. concrete syntax grammar mixing
abstract syntax meta model reuse
constraints static analysis and validation
execution time compiler and interpreter
development time IDE support

74. Concrete syntax

75. Concrete syntax
• scan the document for tokens
• build a parse result from tokens

76. Concrete syntax
• scan the document for tokens
• build a parse result from tokens

77. Concrete syntax
• scan the document for tokens contextless
context-
• build a parse result from tokens aware

78. Concrete syntax
• scan the document for tokens contextless
context-
• build a parse result from tokens aware

79. package org.foo;

80. package org.foo;
Keyword
package

81. package org.foo;
Keyword WS
package

82. package org.foo;
Keyword WS ID
package org

83. package org.foo;
Keyword WS ID K eyword
package org .

84. package org.foo;
Keyword WS ID K eyword ID
package org . foo

85. package org.foo;
o rd
Keyword WS ID K eyword ID yw
Ke
package org . foo ;

86. package org.foo;
o rd
Keyword WS ID K eyword ID yw
Ke
package org . foo ;
PackDecl: “package” ID (‘.’ ID)*’;’

87. What’s reusable?

88. What’s reusable?
• use rules of other language

89. What’s reusable?
• use rules of other language
• specialize rules of other language

90. What’s reusable?
• use rules of other language
• specialize rules of other language
• introduce new rules

91. Extensible syntax?
Ambiguity!

92. Extensible syntax?
Ambiguity!
• alternatives may be conflicting

93. Extensible syntax?
Ambiguity!
• alternatives may be conflicting
• static analysis can help

94. package org.foo;
o rd
Keyword WS ID K eyword ID yw
Ke
package org . foo ;
PackDecl: “package” ID (‘.’ ID)*’;’

95. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword ID yw
Ke
package org . foo ;
PackDecl: “package” ID (‘.’ ID)*’;’

96. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword yw
Ke
package org . ;
PackDecl: “package” ID (‘.’ ID)*’;’

97. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword Keyword eyw
K
package org . assert ;
PackDecl: “package” ID (‘.’ ID)*’;’

98. What can we do?

99. What can we do?
• scannerless parsing

100. What can we do?
• scannerless parsing
• contextual lexing

101. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword Keyword eyw
K
package org . assert ;
PackDecl: “package” ID (‘.’ ID)*’;’

102. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword Keyword eyw
K
package org . assert ;
PackDecl: “package” ID (‘.’ ID)*’;’

103. package org.foo;
package org.assert;
o rd
Keyword WS ID K eyword Keyword eyw
ID! K
package org . assert ;
PackDecl: “package” ID (‘.’ ID)*’;’

104. Context-aware scanning
can be confusing

105. Context-aware scanning
can be confusing
import “http://namespace/” as
generate import “http://namespace/” as alias
import “http://namespace/” as generate
import “http://namespace/” as alias

106. Context-aware scanning
can be confusing
Import:
import “http://namespace/” as lias
generate import ort ‘U RI’ as a
imp “http://namespace/” as alias
nerate:
“http://namespace/” s algenerate
Ge import as ias
te name ‘URI’ a as alias
import “http://namespace/”
genera

107. Context-aware scanning
can be confusing
import “http://namespace/” as
generate import “http://namespace/” as alias
import “http://namespace/” as generate
import “http://namespace/” as alias

108. Be cautious!
• be careful with new keywords
• minimize set of terminals
• think about extensibility

109. Abstract syntax

110. Abstract syntax
meta models
are frozen

111. Linking

112. Linking
• inter & intra language

113. Linking
• inter & intra language
• to platform elements

114. Linking
• inter & intra language
• to platform elements
• reachable elements

116. Validation

117. Validation
• inherited

118. Validation
• inherited
• invariants may not be weakened

119. Validation
• inherited
• invariants may not be weakened
• contract for compiler and
interpreter

120. And then there are...
• compilers
• interpreters
• IDE infrastructure
• code completion
• formatting
• label provider

121. How can we add and change
implementation non-invasively?

122. Example:
Customizing the LabelProvider
public class MySpecialLabelProvider
extends BaseLanguageLabelProvider {
public String label(Entity e) {
return e.getName();
}
public String label(Feature f) {
return f.getName()+":"+f.getType().getName();
}
}
‣naming convention
‣polymorphic dispatch

123. Available June 24th.

124. Thank you very much!
www.itemis.com www.xtext.org