Building DSLs with Eclipse
                                Peter Friese, itemis

                                     @peterfriese
                                       @xtext
                                     www.itemis.de



    (c) 2009 Peter Friese. Distributed under the EDL V1.0 - http://www.eclipse.org/org/documents/edl-v10.php
                           More info: http://www.peterfriese.de / http://www.itemis.com
Think about your job!
Feel like this?
Because
Because
you need to write
Because
you need to write
lots of boring code?
Wrong Level of Abstraction!
http://www.flickr.com/photos/rykerstribe/3222969466/
DSL
o   p   a
m   e   n
a   c   g
i   i   u
n   f   a
    i   g
    c   e
DSL
 A language...

     ...that lets you express your intention
     ... that solves one problem
     ... that doesn’t get in your way
     ... that fits your situation
     ... that’s formal and processable
     ... that’s simple and easy to learn
Some DSLs
you might have
   heard of
select name, salary
 from employees
 where salary > 2000
 order by salary
<project name="MyProject" default="dist" basedir=".">
  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

  <target name="init">
    <mkdir dir="${build}"/>
  </target>

  <target name="compile" depends="init">
    <javac srcdir="${src}" destdir="${build}"/>
  </target>

  <target name="dist" depends="compile">
    <mkdir dir="${dist}/lib"/>
    <jar jarfile="${dist}/lib/MyProject.jar"
         basedir="${build}"/>
  </target>

  <target name="clean">
    <delete dir="${build}"/>
    <delete dir="${dist}"/>
  </target>
</project>
<project name="MyProject" default="dist" basedir=".">
  <property name="src" location="src"/>
  <property name="build" location="build"/>
  <property name="dist" location="dist"/>

  <target name="init">
    <mkdir dir="${build}"/>
  </target>

  <target name="compile" depends="init">
    <javac srcdir="${src}" destdir="${build}"/>
  </target>

  <target name="dist" depends="compile">
    <mkdir dir="${dist}/lib"/>
    <jar jarfile="${dist}/lib/MyProject.jar"
         basedir="${build}"/>
  </target>

  <target name="clean">
    <delete dir="${build}"/>
    <delete dir="${dist}"/>
  </target>
</project>
External
   or
Internal
Internal DSLs
Mailer.mail()
 .to(“you@gmail.com”)
 .from(“me@gmail.com”)
 .subject(“Writing DSLs in Java”)
 .body(“...”)
 .send();
Simple
Limited
External DSLs
1)Create ANTLR grammar
2)Generate lexer / parser
3)Parser will create parse tree
4)Transform parse tree to semantic model

5)Iterate model
6)Pass model element(s) to template
Lack of symbolic integration


Writing parsers / generators is complicated


IDE support is inferior / non-existent
Flexible
Adaptable
Complicated
Why not use a DSL...




... for building DSLs?
http://www.eclipse.org/Xtext/

           @xtext
ar
Model




                                    m
                                   m
                               ra
                               G
                             Generator



        Runtime
                  Superclass




                  Subclass              Class




 LL(*) Parser   ECore meta model                Editor
ar
Model




                                    m
                                   m
                               ra
                               G
                             Generator



        Runtime
                  Superclass




                  Subclass              Class




 LL(*) Parser   ECore meta model                Editor
Grammar (similar to EBNF)
 grammar org.xtext.example.Entity with org.eclipse.xtext.common.Terminals

 generate entity "http://www.xtext.org/example/Entity"

 Model:
   (types+=Type)*;

 Type:
   TypeDef | Entity;

 TypeDef:
   "typedef" name=ID ("mapsto" mappedType=JAVAID)?;

 JAVAID:
   name=ID("." ID)*;

 Entity:
   "entity" name=ID ("extends" superEntity=[Entity])?
   "{"
     (attributes+=Attribute)*
   "}";

 Attribute:
   type=[Type] (many?="*")? name=ID;
grammar org.xtext.example.Entity
    with org.eclipse.xtext.common.Terminals            Meta model inference
generate entity
    "http://www.xtext.org/example/Entity"
                                              entity                   Rules -> Classes
Model:
  (types+=Type)*;
                                                                       Model
Type:
  TypeDef | Entity;
                                                                          types
TypeDef:                                                                *
                                                                    Type
  "typedef" name=ID
                                                                 name: EString
  ("mapsto" mappedType=JAVAID)?;
                                                                                                superEntity
JAVAID:
  name=ID("." ID)*;                                      TypeDef                    Entity


Entity:                                       mappedType                                attributes
  "entity" name=ID
                                                                                  Attribute
  ("extends" superEntity=[Entity])?                       JAVAID
                                                                               name: EString
  "{"                                                  name: EString
                                                                               many: EBoolean
    (attributes+=Attribute)*
  "}";                                                                                  type

Attribute:
  type=[Type] (many?="*")? name=ID;
grammar org.xtext.example.Entity
    with org.eclipse.xtext.common.Terminals            Meta model inference
generate entity
    "http://www.xtext.org/example/Entity"
                                              entity            Alternatives -> Hierarchy
Model:
  (types+=Type)*;
                                                                       Model
Type:
  TypeDef | Entity;
                                                                          types
TypeDef:                                                                *
                                                                    Type
  "typedef" name=ID
                                                                 name: EString
  ("mapsto" mappedType=JAVAID)?;
                                                                                                superEntity
JAVAID:
  name=ID("." ID)*;                                      TypeDef                    Entity


Entity:                                       mappedType                                attributes
  "entity" name=ID
                                                                                  Attribute
  ("extends" superEntity=[Entity])?                       JAVAID
                                                                               name: EString
  "{"                                                  name: EString
                                                                               many: EBoolean
    (attributes+=Attribute)*
  "}";                                                                                  type

Attribute:
  type=[Type] (many?="*")? name=ID;
grammar org.xtext.example.Entity
    with org.eclipse.xtext.common.Terminals            Meta model inference
generate entity
    "http://www.xtext.org/example/Entity"
                                              entity               Assignment -> Feature
Model:
  (types+=Type)*;
                                                                       Model
Type:
  TypeDef | Entity;
                                                                          types
TypeDef:                                                                *
                                                                    Type
  "typedef" name=ID
                                                                 name: EString
  ("mapsto" mappedType=JAVAID)?;
                                                                                                superEntity
JAVAID:
  name=ID("." ID)*;                                      TypeDef                    Entity


Entity:                                       mappedType                                attributes
  "entity" name=ID
                                                                                  Attribute
  ("extends" superEntity=[Entity])?                       JAVAID
                                                                               name: EString
  "{"                                                  name: EString
                                                                               many: EBoolean
    (attributes+=Attribute)*
  "}";                                                                                  type

Attribute:
  type=[Type] (many?="*")? name=ID;
Let’s build a DSL
for Mobile Apps
Building DSLs

1.   Analyze problem domain
2.   Map concepts of problem domain to language
3.   (Optional: write interpreter)
4.   (Optional: write generator)
Analyzing the problem domain




http://www.flickr.com/photos/minifig/3174009125/
Anatomy of an iPhone app

                          View title
    Speaker
                          Table view
Name
Image


Title
      Session
                          Table cell
Location




                          Tab bar
    Entity
                          Tab bar button
    Data Provider
Mapping concepts
                  tabbarApplication jazoonApp {
                  	 button {
Entity            	 	 title= "Schedule"
                  	 	 icon= "83-calendar.png"
Data Provider     	 	 view= ScheduleList( AllSessions() )
                  	 }
                  	 button {
Tab bar           	 	 title= "Speakers"
                  	 	 icon= "112-group.png"
Tab bar button    	 	 view= SpeakerList( AllSpeakers() )
                  	 }
Table view        	 button {
                  	 	 title= "Topics"
View title        	 	 icon= "44-shoebox.png"
                  	 	 view= TopicList( AllTopics() )
Table cell        	 }
                  }
Mapping concepts
                   entity Speaker {
                   	   String speakerId
                   	   String name
Entity             	
                   	
                       String bio
                       String organization
                   	   String country
Data Provider      	   String smallImageURL
                   	   String largeImageURL

Tab bar            	
                   }
                       Session[] talks



Tab bar button     entity Session {
                   	   String talkId
                   	   String title
Table view         	   String abstract
                   	   String ^type
View title         	
                   	
                       String location
                       String startTime
                   	   String endTime
Table cell         	   Speaker[] speakers
                   	   Topic[] topics
                   }

                   entity Topic {
                   	   String topicId
                   	   String themeId
                   	   String name
                   }
Mapping concepts

Entity           contentprovider AllSessions
                 	 returns Session[]
Data Provider    	 fetches XML
                 	 	 from "http://jipa.netcetera.ch/talks.xml"
Tab bar          	 	 selects "feed.entry"
                 	 	
Tab bar button   contentprovider AllSpeakers
                 	 returns Speaker[]
                 	 fetches XML
Table view       	 	 from "http://jipa.netcetera.ch/speakers.xml"
                 	 	 selects "feed.entry"
View title
Table cell
Mapping concepts
                  contentprovider AllSessions
                  	 returns Session[]
                  	 fetches XML
Entity            	 	 from "http://query.yahooapis.com/v1/public/
                  yql?q=use%20%22http%3A%2F%2FHeikoBehrens.net
Data Provider     %2Fjazoon%2Ftalks.xml%22%20as%20talks%3B%0Aselect
                  %20*%20from%20talks%20where%20omitDetails
Tab bar           %3D1&debug=true"
                  	 	 selects "query.results.talks.talk"

Tab bar button    contentprovider SessionsByTopic(String id)
                  	 returns Session[]
Table view        	 fetches XML
                  	 	 from ("http://query.yahooapis.com/v1/public/
View title        yql?q=use%20%22http%3A%2F%2FHeikoBehrens.net
                  %2Fjazoon%2Ftalks.xml%22%20as%20s%3B%0Aselect%20*
Table cell        %20from%20s%20where%20topicId%20%3D
                  %20%22"id"%22&diagnostics=true&debug=true&debug=true
                  ")	 	
                  	 	 selects "query.results.talks.talk"
Mapping concepts

Entity
Data Provider     tableview ScheduleList(Session[] sessions) {
                  	 title= "Schedule"
Tab bar           	 section {
                  	 	 cell Subtitle foreach sessions as session {
Tab bar button    	 	 	 text= session.title
                  	 	 	 details= session.startTime
                  	 	 	 action= SessionDetails(
Table view                     SessionById(session.talkId))
                  	 	 }
View title        	 }
                  }
Table cell
Demo 1
Mapping concepts to code
«Xpand»
Protected regions
   Debugger      Editor
                                   Outlets

                                       Profiler
Cartridges
                                   Polymorphism

                                       Type safe



Produces any                  Can run standalone
 kind of text   Eclipse-based  (ANT / Maven)
Mapping concepts to code
Mapping concepts to code
 tableview SpeakerList(
      Speaker[] speakers)
{
  title= "Speakers"
  section
  {
    cell Default foreach
         speakers as speaker
    {
       text= speaker.name
       image= speaker.smallImageURL
       action= SpeakerDetails
         (SpeakerById(
           speaker.speakerId))
    }
  }
}
Mapping concepts to code
 tableview SpeakerList(                «DEFINE viewModule FOR
      Speaker[] speakers)             SectionedView»
{                                     «FILE filenameModule()»
  title= "Speakers"                   #import "«filenameHeader()»"
  section                             #import "NSObject+Applause.h"
  {                                   «EXPAND imports»
    cell Default foreach
         speakers as speaker          @implementation «className()»
    {
       text= speaker.name             «EXPAND sectionCount»
       image= speaker.smallImageURL   «EXPAND sectionTitleHeader»
       action= SpeakerDetails         «EXPAND rowCounts»
         (SpeakerById(                «EXPAND cellDescriptions»
           speaker.speakerId))        «EXPAND cellSelections»
    }                                 «EXPAND staticData»
  }                                   @end
}                                     «ENDFILE»
                                      «ENDDEFINE»
Demo 2
3 things to take home
3 things to take home

1    Implementing DSLs is easy with Xtext

2    Xtext delivers great IDE support

3    Symbolic integration is possible
More Info?
            Xtext Webinar:
  http://live.eclipse.org/node/886

          www.xtext.org

            Consulting:
      http://www.itemis.com


@peterfriese | http://peterfriese.de

Jazoon 2010 - Building DSLs with Eclipse

  • 1.
    Building DSLs withEclipse Peter Friese, itemis @peterfriese @xtext www.itemis.de (c) 2009 Peter Friese. Distributed under the EDL V1.0 - http://www.eclipse.org/org/documents/edl-v10.php More info: http://www.peterfriese.de / http://www.itemis.com
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
    Because you need towrite lots of boring code?
  • 7.
    Wrong Level ofAbstraction! http://www.flickr.com/photos/rykerstribe/3222969466/
  • 8.
    DSL o p a m e n a c g i i u n f a i g c e
  • 9.
    DSL A language... ...that lets you express your intention ... that solves one problem ... that doesn’t get in your way ... that fits your situation ... that’s formal and processable ... that’s simple and easy to learn
  • 14.
    Some DSLs you mighthave heard of
  • 15.
    select name, salary from employees where salary > 2000 order by salary
  • 16.
    <project name="MyProject" default="dist"basedir="."> <property name="src" location="src"/> <property name="build" location="build"/> <property name="dist" location="dist"/> <target name="init"> <mkdir dir="${build}"/> </target> <target name="compile" depends="init"> <javac srcdir="${src}" destdir="${build}"/> </target> <target name="dist" depends="compile"> <mkdir dir="${dist}/lib"/> <jar jarfile="${dist}/lib/MyProject.jar" basedir="${build}"/> </target> <target name="clean"> <delete dir="${build}"/> <delete dir="${dist}"/> </target> </project>
  • 17.
    <project name="MyProject" default="dist"basedir="."> <property name="src" location="src"/> <property name="build" location="build"/> <property name="dist" location="dist"/> <target name="init"> <mkdir dir="${build}"/> </target> <target name="compile" depends="init"> <javac srcdir="${src}" destdir="${build}"/> </target> <target name="dist" depends="compile"> <mkdir dir="${dist}/lib"/> <jar jarfile="${dist}/lib/MyProject.jar" basedir="${build}"/> </target> <target name="clean"> <delete dir="${build}"/> <delete dir="${dist}"/> </target> </project>
  • 18.
    External or Internal
  • 19.
  • 20.
    Mailer.mail() .to(“you@gmail.com”) .from(“me@gmail.com”) .subject(“Writing DSLs in Java”) .body(“...”) .send();
  • 21.
  • 22.
  • 23.
  • 25.
    1)Create ANTLR grammar 2)Generatelexer / parser 3)Parser will create parse tree 4)Transform parse tree to semantic model 5)Iterate model 6)Pass model element(s) to template
  • 26.
    Lack of symbolicintegration Writing parsers / generators is complicated IDE support is inferior / non-existent
  • 27.
  • 28.
  • 29.
  • 30.
    Why not usea DSL... ... for building DSLs?
  • 31.
  • 32.
    ar Model m m ra G Generator Runtime Superclass Subclass Class LL(*) Parser ECore meta model Editor
  • 33.
    ar Model m m ra G Generator Runtime Superclass Subclass Class LL(*) Parser ECore meta model Editor
  • 34.
    Grammar (similar toEBNF) grammar org.xtext.example.Entity with org.eclipse.xtext.common.Terminals generate entity "http://www.xtext.org/example/Entity" Model: (types+=Type)*; Type: TypeDef | Entity; TypeDef: "typedef" name=ID ("mapsto" mappedType=JAVAID)?; JAVAID: name=ID("." ID)*; Entity: "entity" name=ID ("extends" superEntity=[Entity])? "{" (attributes+=Attribute)* "}"; Attribute: type=[Type] (many?="*")? name=ID;
  • 35.
    grammar org.xtext.example.Entity with org.eclipse.xtext.common.Terminals Meta model inference generate entity "http://www.xtext.org/example/Entity" entity Rules -> Classes Model: (types+=Type)*; Model Type: TypeDef | Entity; types TypeDef: * Type "typedef" name=ID name: EString ("mapsto" mappedType=JAVAID)?; superEntity JAVAID: name=ID("." ID)*; TypeDef Entity Entity: mappedType attributes "entity" name=ID Attribute ("extends" superEntity=[Entity])? JAVAID name: EString "{" name: EString many: EBoolean (attributes+=Attribute)* "}"; type Attribute: type=[Type] (many?="*")? name=ID;
  • 36.
    grammar org.xtext.example.Entity with org.eclipse.xtext.common.Terminals Meta model inference generate entity "http://www.xtext.org/example/Entity" entity Alternatives -> Hierarchy Model: (types+=Type)*; Model Type: TypeDef | Entity; types TypeDef: * Type "typedef" name=ID name: EString ("mapsto" mappedType=JAVAID)?; superEntity JAVAID: name=ID("." ID)*; TypeDef Entity Entity: mappedType attributes "entity" name=ID Attribute ("extends" superEntity=[Entity])? JAVAID name: EString "{" name: EString many: EBoolean (attributes+=Attribute)* "}"; type Attribute: type=[Type] (many?="*")? name=ID;
  • 37.
    grammar org.xtext.example.Entity with org.eclipse.xtext.common.Terminals Meta model inference generate entity "http://www.xtext.org/example/Entity" entity Assignment -> Feature Model: (types+=Type)*; Model Type: TypeDef | Entity; types TypeDef: * Type "typedef" name=ID name: EString ("mapsto" mappedType=JAVAID)?; superEntity JAVAID: name=ID("." ID)*; TypeDef Entity Entity: mappedType attributes "entity" name=ID Attribute ("extends" superEntity=[Entity])? JAVAID name: EString "{" name: EString many: EBoolean (attributes+=Attribute)* "}"; type Attribute: type=[Type] (many?="*")? name=ID;
  • 38.
    Let’s build aDSL for Mobile Apps
  • 40.
    Building DSLs 1. Analyze problem domain 2. Map concepts of problem domain to language 3. (Optional: write interpreter) 4. (Optional: write generator)
  • 41.
    Analyzing the problemdomain http://www.flickr.com/photos/minifig/3174009125/
  • 42.
    Anatomy of aniPhone app View title Speaker Table view Name Image Title Session Table cell Location Tab bar Entity Tab bar button Data Provider
  • 43.
    Mapping concepts tabbarApplication jazoonApp { button { Entity title= "Schedule" icon= "83-calendar.png" Data Provider view= ScheduleList( AllSessions() ) } button { Tab bar title= "Speakers" icon= "112-group.png" Tab bar button view= SpeakerList( AllSpeakers() ) } Table view button { title= "Topics" View title icon= "44-shoebox.png" view= TopicList( AllTopics() ) Table cell } }
  • 44.
    Mapping concepts entity Speaker { String speakerId String name Entity String bio String organization String country Data Provider String smallImageURL String largeImageURL Tab bar } Session[] talks Tab bar button entity Session { String talkId String title Table view String abstract String ^type View title String location String startTime String endTime Table cell Speaker[] speakers Topic[] topics } entity Topic { String topicId String themeId String name }
  • 45.
    Mapping concepts Entity contentprovider AllSessions returns Session[] Data Provider fetches XML from "http://jipa.netcetera.ch/talks.xml" Tab bar selects "feed.entry" Tab bar button contentprovider AllSpeakers returns Speaker[] fetches XML Table view from "http://jipa.netcetera.ch/speakers.xml" selects "feed.entry" View title Table cell
  • 46.
    Mapping concepts contentprovider AllSessions returns Session[] fetches XML Entity from "http://query.yahooapis.com/v1/public/ yql?q=use%20%22http%3A%2F%2FHeikoBehrens.net Data Provider %2Fjazoon%2Ftalks.xml%22%20as%20talks%3B%0Aselect %20*%20from%20talks%20where%20omitDetails Tab bar %3D1&debug=true" selects "query.results.talks.talk" Tab bar button contentprovider SessionsByTopic(String id) returns Session[] Table view fetches XML from ("http://query.yahooapis.com/v1/public/ View title yql?q=use%20%22http%3A%2F%2FHeikoBehrens.net %2Fjazoon%2Ftalks.xml%22%20as%20s%3B%0Aselect%20* Table cell %20from%20s%20where%20topicId%20%3D %20%22"id"%22&diagnostics=true&debug=true&debug=true ") selects "query.results.talks.talk"
  • 47.
    Mapping concepts Entity Data Provider tableview ScheduleList(Session[] sessions) { title= "Schedule" Tab bar section { cell Subtitle foreach sessions as session { Tab bar button text= session.title details= session.startTime action= SessionDetails( Table view SessionById(session.talkId)) } View title } } Table cell
  • 48.
  • 50.
  • 51.
  • 52.
    Protected regions Debugger Editor Outlets Profiler Cartridges Polymorphism Type safe Produces any Can run standalone kind of text Eclipse-based (ANT / Maven)
  • 53.
  • 54.
    Mapping concepts tocode tableview SpeakerList( Speaker[] speakers) { title= "Speakers" section { cell Default foreach speakers as speaker { text= speaker.name image= speaker.smallImageURL action= SpeakerDetails (SpeakerById( speaker.speakerId)) } } }
  • 55.
    Mapping concepts tocode tableview SpeakerList( «DEFINE viewModule FOR Speaker[] speakers) SectionedView» { «FILE filenameModule()» title= "Speakers" #import "«filenameHeader()»" section #import "NSObject+Applause.h" { «EXPAND imports» cell Default foreach speakers as speaker @implementation «className()» { text= speaker.name «EXPAND sectionCount» image= speaker.smallImageURL «EXPAND sectionTitleHeader» action= SpeakerDetails «EXPAND rowCounts» (SpeakerById( «EXPAND cellDescriptions» speaker.speakerId)) «EXPAND cellSelections» } «EXPAND staticData» } @end } «ENDFILE» «ENDDEFINE»
  • 56.
  • 57.
    3 things totake home
  • 58.
    3 things totake home 1 Implementing DSLs is easy with Xtext 2 Xtext delivers great IDE support 3 Symbolic integration is possible
  • 59.
    More Info? Xtext Webinar: http://live.eclipse.org/node/886 www.xtext.org Consulting: http://www.itemis.com @peterfriese | http://peterfriese.de