How Domains Shape Languages

Markus Völter
voelter@acm.org
www.voelter.de
@markusvoelter
Domain Requirements
Languages
How
shape
Language Design Experience from
10 Years of Building DSLs

Health & Medical
AutomoFve
Aerospace
RoboFcs
Finance
Embedded
Government
DOMAINS SoNware Engineering
Requirements Engineering
SpecificaFon and VerificaFon
ImplementaFon and Test
Mixed NotaFons and End User Programming
Informal -> Semiformal -> Formal
Languages + VerificaFon
OptmizaFons, Performance and Concurrency
Fundamentals: Editors, Type Systems, Trafos
User-Friendly IDEs, Tools
Safety and Security
Architecture
Science
Methodology
Language Engineering

What is a
Language
1
How
Domain
Requirements
Shape
Languages

Metamodel for
Business Logic
Clearly deﬁned data structure to express all
business-relevant structures, behaviors and
non-funcFonal concerns.

Metamodel for
Business Logic
{
Data Structures
Behavioral Rules
Expressions
ValidaFons
Special Types (eg. temporal)
...

Metamodel for
Business Logic
SemanFcs
Well-deﬁned meaning of this data structure

Metamodel for
Business Logic
SemanFcs
IDE Support is possible
EvoluFon is possible
Portability is possible

Metamodel for
Business Logic
SemanFcs
IDE Support is possible
EvoluFon is possible
Portability is possible
Type Checking
Solver-IntegraFon
Model Checking
Contracts

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
ExecuFon
Engine
Technical Pla_orm for correct, eﬃcient
and scalable execuFon

Metamodel for
Business Logic
SemanFcs
Technical Pla_orm for correct, eﬃcient
and scalable execuFon
Tech Infrastructure
generate code,
deploy
transfer data,
interpret

Tech Infrastructure
generate code,
deploy
transfer data,
interpret
+ Code Inspec9on
+ Debugging
+ Performance &
Op9miza9on
+ PlaBorm Con-
formance
GeneraFon InterpretaFon
+ Turn around Time
+ Run9me Change

Metamodel for
Business Logic
SemanFcs
Tech Infrastructure
generate code,
deploy
transfer data,
interpret

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
generate code,
deploy
transfer data,
interpret

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
generate code,
deploy
transfer data,
interpret
Syntax is criFcally important for
ProducFvity
CommunicaFon and Review
Domain Expert IntegraFon
Only Buaons and Forms
don‘t work!

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
generate code,
deploy
transfer data,
interpret
IDE is criFcally important for acceptance IDE

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
generate code,
deploy
transfer data,
interpret
IDE is criFcally important for acceptance
Learning
ExploraFon
ProducFvity
Tool IntegraFon
IDE

Tech Infrastructure
Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
generate code,
deploy
transfer data,
interpret
IDE
L a n g u a g e
W o r b e n c h
Language Workbenches let
you eﬃciently implement
languages and IDEs.
They are essenFal for the
language implementor.

What is
Language Design
2
How
Domain
Requirements
Shape
Languages

L a n g u a g e
W o r b e n c h
Requirements
from Domain

Language
ImplementaFon
Other
Requirements
ImplementaFon
R
D
Iterate!
Language
Design

Language
Design
Forces
Constraints
Tradeoﬀs
CommunicaFon
Compromises
ValidaFon
{
Done by people.
Supported by Tools.

Domain
Structure
Model
Purpose
Analyze, Generate
User
Skills
SoNware
Engineering
PracFces
Non
FuncFonals
Permissions,
IP, Sharing
Tool
CapabiliFes
NotaFons,
EdiFng, Scale
Sep. of Concerns
Diﬀerent Views
Get a beaer tool :-)
Refactor towards
Structure
Educate,
Put results in context
Language Design::Inﬂuences

Domain
Structure
Model
Purpose
Analyze, Generate
User
Skills
SoNware
Engineering
PracFces
Non
FuncFonals
Permissions,
IP, Sharing
Tool
CapabiliFes
NotaFons,
EdiFng, Scale
Sep. of Concerns
Diﬀerent Views
Get a beaer tool :-)
Refactor towards
Structure
Educate,
Put results in context
Language Design::Inﬂuences
Style!

Good
Language
Design
Fitness for purpose
EvaluaFon.

Balancing the Forces
RaFonales.

Conceptual Frameworks
e.g. CogniFve Dimensions of NotaFons

?
AbstracFon gradient Ÿ Closeness of mapping Ÿ Consistency Ÿ
Diﬀuseness/Terseness Ÿ Error-proneness Ÿ Hard mental operaFons Ÿ
Hidden dependencies Ÿ Juxtaposability Ÿ Premature commitment Ÿ
Progressive evaluaFon Ÿ Role-expressiveness Ÿ Secondary notaFon
and escape from formalism Ÿ Viscosity Ÿ Visibility

Language Design::Main Paaerns
New Language
GPL Extension
ExisFng
Domain NotaFon
(Informal)
FormalizaFon
Formalized
Language
Reuse GPL incl. Expressions and TS
Add/Embed DS-extensions
CompaFble notaFonal style
Reduce to GPL
Analyze Domain to ﬁnd AbstracFons
Deﬁne suitable, new notaFons
Rely on exisFng behavioral paradigm
Reuse standard expression language
Interpret/Generate to one or more GPLs
Use exisFng notaFon from domain
Clean up and formalize
Generate/Interpret
ONen import exisFng „models“

Language Design::Granularity
Contract Structure?
Inheritance?
State Machine?
Graphical or Textual?
ImperaFve?
Temporal Types?
Error Handling?
Syntax f. OpFon Types?
vs.
vs.
?

Language
Great IDE
Analyses
Refactorings
TesFng
Debuggers
AbstracFons
NotaFons
Syntax Coloring
Code CompleFon
Goto DeﬁniFon
Relevant
Good Errors
Aligned with Processes
Write Tests
Run them
Report Back
Animate ExecuFon
Simulators
GOOD
GREAT
Language Design::More than Lang

Language Great IDE Analyses Refactorings TesFng Debuggers
Language Design::Cross Inﬂuences
Some Syntaxes can be
beaer supported by IDEs
More ﬁrst-class abstracFons
make analyses simpler
Can only refactor what
can be analyzed
Syntax to express tests
Some abstracFons are easier to debug
Good test support may
limit need for debugging

Example Domains
& Languages
3
How
Domain
Requirements
Shape
Languages

Stakeholder integraFon, Scalable Business, Document GeneraFon + CerFﬁcaFon
Code Complexity, Frameworks (Autosar), Product Lines
ReducFon of Accidental Complexity in Code, Process Conformance (Docs)
A powerful language and IDE for exisFng frameworks (Industry Robots, ROS)
Precise SpeciﬁcaFon and ImplementaFon of Insurance Products („Rules“)
MulF-Paradigm Programming, not just Simulink and C
Consistent Derived Documents
Changing RegulaFons, Fast ImplementaFon, End User Empowerment

Mixed NotaFons and End User Programming
Informal -> Semiformal -> Formal
Languages + VeriﬁcaFon
OptmizaFons, Performance and Concurrency
Fundamentals: Editors, Type Systems, Trafos
User-Friendly IDEs, Tools
SimulaFon, ExecuFon, Test
Consistent Derived Documents
Language vs. Library, Code Gen
ObservaFons:
1)  Every domain has diﬀerent reasons
why language engineering is used.
2)  No domain has only one reason – it is
always a combina9on
3)  Languages, Editors, Type Systems and
good IDEs are always important!

F I N A N C E
Customer had „coded“ in Word
Needed a real IDE
Syntax to be retained if possible

Insurance
Programs
Specify/Program
Write formal code in a DSL
mixed with tables and text
No tool support whatsoever
No tesFng (except inspecFon)
No reuse
No modularity
No varibility

Insurance
Programs
„Pixelcrap“
Specify/Program
Printed, PDF
No reuse
No modularity
No varibility

Insurance
Programs
„Pixelcrap“
C Code
Implement
Specify/Program
Printed, PDF Developer reads „spec“
Very idiomaFc implementaFon

Dev acts as a human compiler and implements it in C
No reuse
No modularity
No varibility

Insurance
Programs
„Pixelcrap“
C Code
Debug
Implement
Specify/Program
Printed, PDF Developer reads „spec“
Very idiomaFc implementaFon

Dev acts as a human compiler and implements it in C
Debugging directly in C
Search-for-use by text search
Don‘t trust the documents –
may be outdated!
No reuse
No modularity
No varibility

Insurance
Programs
Now with IDE support and executable tests

The same notaFon!
Specify/Program/Test/Debug

Insurance
Programs
C Code
Now with IDE support and executable tests

The same notaFon!
Exactly the same C code.
Generate

Insurance
Programs
C Code
Generate
Incremental Reﬁnement/Refactoring of languages:

ParFally automated migraFon of models
Add model natural notaFons (insurance-speciﬁc, math)
Add Support for modularity, reuse, variants

SFll exactly the same C code,
or improved as needed.

Small-scale syntacFc decisions

Small-scale syntacFc decisions
Tool support via paleaes is important for
things that „cannot be typed“.

Structure/Guid.
NotaFon
Views
IDE/Tool
Learn/EﬀecFve
+
Mixed
*
Clean
L
-
Text
1
Powerful
E

Syntax is the „UI“ of the language.

ONen perceived as the most important aspect
from the customer‘s perspecFve.
ReplicaFng exisFng syntax is oNen a
good starFng point, if your tool can do it.

EMBEDDED SOFTWARE
„General Purpose DSL“
Based on C
Improved Robustness, ProducFvity
Designed for end-user extensibility

An extensible set of integrated languages
for embedded soNware engineering.

Composable extensions, Diverse notaFons

Diﬀerent kinds of languages, as
illustrated by the diﬀerent
distribuFons of aspect code.

Relying on modular language extension
and extension composiFon.

Typical OO paaerns to support extensibility

Design for Extensibility

The internal design decisions may not be relevant to
the end user, but they are decisive wrt. the ability to
modularly extend a language.
This may in itself be a goal!

Integrated VeriﬁcaFons
Bounded Model Checking
for C using the CBMC tool.

Formal VeriﬁcaFon is simpliﬁed if the
language has the right abstracFons.

... because no semanFc recovery is required.

A E R O S P A C E
C-based satellite on-board soNware
Standardized Architecture
Lots of infrastructure code
Lots of documentaFon

Further domain-speciﬁc extensions to C.
Developed by end-user lang engineer.

Infrastructure Speciﬁcs in C

An exisFng meta model / standard
is oNen a great starFng point for a DSL.
There is more to generate/derive
than just executable soNware.

„ “I am comminng myself to
develop the next satellite‘s on
board soNware in one year
instead of the usual 50.

The more infrastructure you have to deal
with, the more can be generated.
Remote communicaFon, protocols,
buﬀering/serializaFon, mulF-pla_orm interfaces...

AUTOMOTIVE
SpeciﬁcaFon of FuncFons and Interfaces
And the Data communicated betw. Them
Incrementally more speciﬁc

Start with a few
ontological data
deﬁniFons

Deﬁne components
that produce or
consume the data
items (allocate
responsibiliFes).

Establish
reﬁnement
relaFonships
between
data items.

Only then
do you start
deﬁning
types and
value
constraints.

Then decompose
hierarchical system
structures.

Workﬂow may impact the language.

Not deﬁning a type for a variable is unintuiFve for
the soNware engineer. But it is the way a systems
engineer works. Make sure you accomodate this.

HEALTH/MEDICINE
SoNware Medical Devices
Accessible to Doctors
Robustness/Correctness Required
To be FDA-cerFﬁed

Prose-like call syntax
Extension funcFon can be called in dot-notaFon,
perfectly suitable for developers.

Prose-like call syntax
For non-programmers, a more prose-like notaFon is
helpful. NoFce the prose-call facility is a modular
extension of the expression language.

Once again, be creaFve in the notaFons
you use to accomodate end users.

They might not like your fancy higher-order,
lambda-based pure funcFonal language.

Test + Monitoring Aspects
System Model MS
T & V Model MT
ensures
LWB
System Implementa9on IS
T&V Implementa9on IT
ensures
transform
transform

TS
TT
RTE
Established approaches for developing reliable
and safe systems applied to languages.

System Model MS
T & V Model MT
ensures
LWB
Behavior
Constraints
monitors System Implementa9on IS
ensures
transform
TS
TT
RTE

System Model MS
T & V Model MT
ensures
LWB
Behavior
Constraints
ensures
transform
TS
TT
RTE
During TesFng

System Model MS
T & V Model MT
ensures
LWB
Behavior
Constraints
ensures
transform
TS
TT
RTE
During Regular
OperaFon

Redundancy in the Tool Chain
The basic idea is that the system model and the test
model are disFnct, and are transformed separately.

Redundancy in the Tool Chain
A number of addiFonal steps must be taken to address
possible error scenarios; many rely on redundancy.

Integrated VeriﬁcaFons
Integrated SMT solver checks for logical and arithmeFc
inconsistencies in decision tables.
Solver also for decision trees, state machines and
protocol/contract checking in other projects.

Non-FuncFonal Concerns can have a
huge impact on language design and
transformaFon architecture.

Safety, Security, Performance

End Users vs. SemanFcs
Estsblished NotaFons
End-user Happyness
Expressive Formalism
VeriﬁcaFon feasible
Ask user for X
... ...
reply aYer 2 minutes otherwise
Some kind of decision
tree/diagram
?
S1
S2
S3
/entry Ask
aNer 2 min
on Reply [data.isValid]
State Machine
Asynchronicity (User InteracFon)
Time-based decisions (Fmeouts)
Checking for completeness and other
criteria (model checking)
No problem for semanFc issues.
But „strange“ way of expressing things.
Hard to get accepted.

GeneraFng „nice“ reports
Start with this ... ... derive this.
Editable. Analyzable.
Read-only.
Traceback.
Conﬁgurable.

GeneraFng „nice“ reports
Read-only.
Traceback.
Conﬁgurable.
{
Generate parFally.
Avoid some details.
Mark ExcepFonal Paths.
Show only Happy Path.
Highlight Safety-Checks.
...

Generate derived arFfacts
Review the DSL sources
Pair programming
Independent Development
Levels of
Domain Expert IntegraFon
User sophisFcaFon
Development over Fme.

SemanFcs First Language Design!
Robust, reusable, embeddable.
Established paradigms and
semanFcs, but custom language
Completely Custom
LWB must support language composiFon!

E X P R E S S I O N S
Reusable Expression Language
Embedded in > 10 DSLs
What is a good way to do opFon types?

Basic Expressions and types
OpFon Types

What is a good way to do opFon types?
Good == Safe

Checkable

Concise

Non-Surprising

Implementable
?can the value only be used in the then part?
does type checking work?
how writable and readable is it?
is the syntax surprising or obvious?
is it implementable with MPS and easily
extensible for language extension developers?
StarFng Point: a ﬁrst-class expression ... Safe
Checkable
Concise
Non-Surprising
Implementable

Problem: what about nesFng?
one has specify names for the value
otherwise the val is ambiguous
Safe
Checkable
Concise
Non-Surprising
Implementable
SoluFon: two styles for opFonal names
Safe
Checkable
Concise
Non-Surprising
Implementable
We opted for the 2nd version, because
IDE support is beaer: write post-facto
we already had a framework for the as notaFon

NesFng: one can now write this Safe
Checkable
Concise
Non-Surprising
Implementable
This is ugly!
Beaer: support comma-separated mulFple tests
Safe
Checkable
Concise
Non-Surprising
Implementable

NesFng: one can now write this Safe
Checkable
Concise
Non-Surprising
Implementable
This is ugly!
Beaer: support comma-separated mulFple tests
Safe
Checkable
Concise
Non-Surprising
Implementable
But: why not just use an if?

Why not just use an if? Safe
Checkable
Concise
Non-Surprising
Implementable
Problem: how to analyze this?
SoluFon: enfore some to be topmost expression
Safe
Checkable
Concise
Non-Surprising
Implementable

Use an opFon as bool? Safe
Checkable
Concise
Non-Surprising
Implementable
Discarded: too much magic.
Use the expression in the then part? Safe
Checkable
Concise
Non-Surprising
Implementable
Challenging to implement with MPS because
- one cannot change the type of a node (x)
depending on its context.
- alternaFvely, the x in the then part must be a
diﬀerent lang concept with negaFve consequences
for scoping.
Adopted, accepFng the scoping challenge.

Final issue: comma in expressions Safe
Checkable
Concise
Non-Surprising
Implementable
SoluFon: allow && but not other operators
Safe
Checkable
Concise
Non-Surprising
Implementable

Final, adopted soluFon: Safe
Checkable
Concise
Non-Surprising
Implementable
We accept the tradeoﬀ between end-user syntax and the
somewhat more complicated implementaFon (scoping).

Many concerns affect language design;
tradeoffs are oNen required.

Specifically, if the language is expected to be
extended or embedded, since then the customers‘
language engineers are also stakeholders.

How to deal with numbers?
?
Seems to be obvious: why think about it?
We have int and real. Why bother?
But many domains have to deal with
-  numbers bigger than int or long
-  ﬁxed precision (e.g., currencies)
-  numbers with speciﬁc ranges
-  or other constraints on the values.
This should be part of the language!

Ranges and Operators
Ranges are checked staFcally by the TS.
Other constraints are checked at runFme.
(Later maybe with a solver).

Even „obvious“ issues may be
worth revisiFng for DSLs.

„Normal people“ don‘t really care about
integer and real, they just want to work
with numbers.

Science
High-Performance CompuFng
OpFmized Math on Specialized H/W
Parallel CompuFng
High-Throughput

Math computaFon with sliding windows

Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
IDE
Diﬀerent Data Layouts
OpFmizing f. Registers
Minimizing Mem Accs
CPU GPU
OpFmizing Generators.

„
“ANer making an algorithm
eﬃcient (for eﬃcient execuFon),
we cannot recognize the science
or math anymore. We cannot
maintain the code.

Metamodel for
Business Logic
SemanFcs
Syntax
L a n g u a g e
IDE
Diﬀerent Data Layouts
OpFmizing f. Registers
Minimizing Mem Accs
CPU GPU
AbstracFons
that contain enough
semanFcs to enable
generators!
NotaFons
that make „the power“
accessible to scienFtsts.

Meta
4
How
Domain
Requirements
Shape
Languages

MPS is bootstrapped,
i.e. it is mostly built with itself.
The MPS team and us are eaFng our own dogfood.
Java Expressions and Statements (BL)
Refactorings
Inten9ons
Ac9ons
Structure
Type System
Transforma9on
Editor
f(x)
Dataﬂow
ImperaFve/OO DeclaraFve

MPS is bootstrapped,
i.e. it is mostly built with itself.
The MPS team and us are eaFng our own dogfood.
Refactorings
Inten9ons
Ac9ons
Structure
Type System
Transforma9on
Editor
Interpreter
Math
Tables
Diagrams
f(x)
IncA
Dataﬂow
Pa]erns
Tree

Interpreter
DeclaraFve type mappings
Type Guards
Concise syntax, especially for recursive calls
Composable (just as languages)
AbstracFons for environments and stacks

Program Analysis (IncA)
Restricted, DeclaraFve Language
Transformed to Paaerns
Executed incrementally
Very fast!

New Editor NotaFons
New primiFve editor cells for math, diagram, tables
More query-oriented than default MPS notaFons
NotaFon independent from Language

Java as an imperaFve base language is problemaFc
Because of side eﬀects, it is very hard to analyze and opFmize
Refactorings
Inten9ons
Ac9ons
Structure
Type System
Transforma9on
Editor
Interpreter
Math
Tables
Diagrams
f(x)
IncA
Dataﬂow
Pa]erns

Java as an imperaFve base language is problemaFc
Because of side eﬀects, it is very hard to analyze and opFmize
f(x)
Refactorings
Inten9ons
Ac9ons
Structure
Type System
Transforma9on
Editor
Interpreter
Math
Tables
Diagrams
IncA
Dataﬂow
Pa]erns
We are invesFgaFng how to get a
more funcFonal core into MPS

Summing up
5
How
Domain
Requirements
Shape
Languages

All languages shown in this talk
are built with the open source
JetBrains MPS language workbench.

Lots of further reading is available.

NotaFon is crucial for users.
Be open to requirements and be creaFve.
Rely on a solid semanFc foundaFon.
Otherwise you cannot deliver the promises of DSLs.
SimulaFon, VeriﬁcaFon and TesFng
the models is a major part of the value-add of a DSL.
Try to incorporate SE best pracFces
(modularity, SoC) to keep overall complexity in check.
Use a powerful language workbench
to make sure it does not unduly constrain the design.

A personal note.
1999
UML Modeling with RaFonal Rose and „Skeleton Code“ generaFon.
Classes, Stereotypes, Tagged Values, Simple OCL constraints.

A personal note.
2016
MPS with full code generaFon.

Mixed notaFons, mulFple integrated languages, type checking and
inferencing, data ﬂow checking, refactoring, version control, diﬀ/
merge, non-trivial transformaFons, languages for non-programmers.
Modeling 2.0 „ “I cannot understand how developers sFll deal with modeling 1.0
– and then complain that it does not work.
Try 2.0!

How Domains Shape Languages

More Related Content

More from Markus Voelter

Recently uploaded

How Domains Shape Languages