Practical DSL Design
                       London Groovy & Grails User Group
                               May 17th, 2010

                                 Peter Bell
                            CEO/CTO SystemsForge



Monday, May 17, 2010
Overview
  • What is a DSL?
  • Key Concepts
  • DSL Design
  • Implementing DSLs
  • Additional Considerations
           •      Testing, evolution, documentation,
                  error handling , IDE support




Monday, May 17, 2010
Goals:
  • Understand:
    • What is a DSL
    • Where use one
    • Engineering considerations
  • NOT about syntax



Monday, May 17, 2010
Division of Labor

    •       My Job:
        •      Present ideas


    •       Your Job:
        •      Discriminate, select, adapt, experiment




Monday, May 17, 2010
About Me
   • Programmer: 50-80 projects/yr
   • Entrepreneur: profitable/practical
   • Research:
     • Domain Specific Modeling/DSLs
     • Software Product Lines
   • Writer: GroovyMag, InfoQ, IEEE
              Software, JSMag, Methods & Tools,
              Fusion Authority, Flex Authority . . .
      •       Presenter: ooPSLA, Code
              Generation, BCS SPA . . .

Monday, May 17, 2010
What is a DSL?
      •      Specific to single domain - generally not
             Turing complete (excl. XSLT)
      •      More expressive than GPL within the
             domain
      •      Executable
      •      Examples: SQL, CSS, Regex, PC
             configurator, document workflow rules




Monday, May 17, 2010
What is a DSL? - DSL vs. API
   • Subjective
   • Often facade to API
   • Focus on language nature
   • “Fluent” interface
     API: new event( startTime: "16:00", endTime: "18:00" )
     DSL: new event.from 4.pm, to: 6.pm



Monday, May 17, 2010
What is a DSL? - Benefits
   • Expressive (clear intent, concise)
   • Separate business logic from code
   • Separate lifecycle
   • SME readable (not writable)




Monday, May 17, 2010
What is a DSL? - When Use?
      •       Frequently changing business rules
      •       Interactions with stakeholders
      •       Relatively well understood domain
      •       Typical use cases:
            •      Product family
            •      Platform based development
            •      Configuration
            •      Business rule definitions

Monday, May 17, 2010
Key Concepts
   • Overview:
     • Vertical vs horizontal
     • Abstract vs Concrete
     • Projections
     • Three types of DSL
     • Internal vs. External


Monday, May 17, 2010
Vertical vs. Horizontal
      •       Horizontal: technical domain
            •      SQL, CSS, GORM . . .
            •      More maintainable/readable code
            •      Good enough is good enough
      •       Vertical: business domain
            •      Insurance, PC configurator . . .
            •      Share “code” with SME’s
            •      Readability is key

Monday, May 17, 2010
Abstract vs. Concrete
   • Abstract grammar       <person>
     • what say                <FirstName>Paul</FirstName>
                               <LastName>King</LastName>
   • Concrete syntax        </person>
     • how say new person( firstName: "Paul", lastName: "King" )
                                      person called “Paul “, “ King “

                                                          Paul King




Monday, May 17, 2010
Projections
   • Multiple projections
     • Textual
     • Spreadsheet
     • Visual
     • Form based



Monday, May 17, 2010
Three Broad Types of DSL
   • Internal (embedded)
   • External (parser)
   • Language workbench




Monday, May 17, 2010
Internal vs. External
   • Internal:              • External:
     • Easier to implement • Easier to evolve
     • Power of GPL:         • End user writable (forms)?
       • Loops               • Projectional:
       • Math                  • Forms
       • Conditionals . . .    • Visual
     • Refactor to external    • Spreadsheet

Monday, May 17, 2010
Top Down/Bottom Up
   • Bottom up:
     • Evolve from API (e.g.
                  Hibernate)
          •       Not very expressive, but
                  quick starting point
     •      Top down:
          •       Sketch out UL with
                  stakeholders
          •       Generally better language

Monday, May 17, 2010
General Hints
   • Learn from DDD/UL:
     • Common, rigorous understanding
     • Driven by domain model
     • Involve to SME’s
   • Lots of little languages (e.g. Grails)
     • Bounded contexts
   • Risk:
     • DSL eventually becomes badly designed GPL
     • Think: Ant
Monday, May 17, 2010
Implementing DSLs
   • API
   • Fluent interface
   • Reducing visual noise
   • Config file/XML/Json
   • Simple parser
   • Nested parser


Monday, May 17, 2010
API
   • Good starting point
   • Easy to evolve
   • Often readable enough . . .
     Event salesMeet = new event( startTime: "16:00", endTime: "18:00" )
     Participant Joe = new Participant( firstName: “Joe”, lastName: “Smith” )
     Participant Fred = new Participant( firstName: “Fred”, lastName: “Jones” )
     salesMeet.addParticipant( Joe )
     salesMeet.addParticipant( Fred )




Monday, May 17, 2010
Fluent interface
   • The start of DSLs
   • Language nature
   • Focus on sentence like syntax
                       Event salesMeet = new event( from: 4.pm, to: 6.pm )
                                                  .with( “Joe”, “Smith” )
                                                  .and( “Fred”, “Jones” )




Monday, May 17, 2010
Reducing Visual Noise
   • Flexible and malleable
              syntax
      •       Metaprogramming
      •       AST transformations




     © Guillaume Laforge
Monday, May 17, 2010
Flexible and Malleable Syntax
   • Scripts
   • Optional typing
   • Native syntax constructs
     • (lists, maps)
   • Parentheses/semi omission
   • Named arguments
   • Operator overloading
   • Closures
     © Guillaume Laforge
Monday, May 17, 2010
Metaprogramming
   • PoGo
   • Categories
   • Builders
   • Custom metaclass
   • ExpandoMetaClass


     © Guillaume Laforge
Monday, May 17, 2010
AST Transformations
      •       AST traversal
      •       Local transformations
      •       Global transformations
      •       Hooks into ANTLR




     © Guillaume Laforge
Monday, May 17, 2010
Outcome:
               event from 4.pm to 6.pm
                        .with “Joe” “Smith”
                        .and “Fred” “Jones”




     © Guillaume Laforge
Monday, May 17, 2010
Config File/XML/Json
   • Introduction to external DSLs
   • Easy to load/parse
   • Supports projections and tooling
          <events>
            <event startTime=”16:00” endTime=”18:00”>
              <participants>
                <participant firstname=”Joe” lastName=”Smith” />
                <participant firstname=”Joe” lastName=”Smith” />
              </participants>
            </event>
          <events>


Monday, May 17, 2010
Simple Parser
   • Delimiter directed translation
     • Regex or similar
     • Doesn’t support nesting
     • Only for simplest languages




Monday, May 17, 2010
Nested Parser
   • Syntax directed translation
     • Grammar/parser
     • ANTLR




Monday, May 17, 2010
Additional Considerations
   • Testing
   • Evolution
   • Documentation
   • Error handling
   • IDE support



Monday, May 17, 2010
Testing DSLs
   • Test domain model
   • Test fluent interface
              (expression builder)
            •      Can Compile
            •      Scenarios
            •      Orthogonal test DSLs




Monday, May 17, 2010
DSL Evolution
   • The problem: success!
   • Approaches:
     • No changes
     • Backwards compatibility
     • Versioning
     • Model migrations


Monday, May 17, 2010
Documentation
   • Getting started guide
   • Samples
   • FAQs for common tasks
   • Language reference
   • Some debugging suggestions
   • Not more than API requires
   • Source/regression tests help but
              aren't enough


Monday, May 17, 2010
Error Handling
   • Throw while process = easy
   • Meaningful messages hard




Monday, May 17, 2010
IDE Support
   • Syntax highlighting
   • Code completion
   • Static verification
   • IntelliJ helps . . .
   • If important, consider Xtext/EMF



Monday, May 17, 2010
Conclusions
   • It’s all about the domain
   • Evolve to DSLs
   • API to internal . . .
   • . . . external if required
   • Check Guillaume’s preso
              for syntax
      •       Think about additional
              considerations from day 1



Monday, May 17, 2010
Resources

   History:
   http://www.faqs.org/docs/artu/minilanguageschapter.html

   DSM:
   JP Tolvanen: http://www.metacase.com/blogs/jpt/blogView
   Steve Kelly: http://www.metacase.com/blogs/stevek/blogView
   Markus Voelter: http://www.voelter.de/

   DDD:
   http://domaindrivendesign.org/



Monday, May 17, 2010
Resources (2)
  DSLs:
  Martin Fowler: http://martinfowler.com/dslwip/

  Groovy DSLs:
  Google - Guillaume Laforge, Paul King, Hamlet D’Arcy,Venkat Subramaniam,
  Neil Ford

  DSL Evolution:
  http://www.infoq.com/articles/dsl-evolution

  Books:
  Domain Specific Modeling - Kelly/Tolvanen
  Generative Programming - Czarnecki/Eisenecker
  Domain Driven Design - Evans
Monday, May 17, 2010
The End!
  • http://gettinggroovy.com
  • Email: peter@pbell.com
  • Twitter: peterbell




Monday, May 17, 2010

GGUG:Practical DSL Design

  • 1.
    Practical DSL Design London Groovy & Grails User Group May 17th, 2010 Peter Bell CEO/CTO SystemsForge Monday, May 17, 2010
  • 2.
    Overview •What is a DSL? • Key Concepts • DSL Design • Implementing DSLs • Additional Considerations • Testing, evolution, documentation, error handling , IDE support Monday, May 17, 2010
  • 3.
    Goals: •Understand: • What is a DSL • Where use one • Engineering considerations • NOT about syntax Monday, May 17, 2010
  • 4.
    Division of Labor • My Job: • Present ideas • Your Job: • Discriminate, select, adapt, experiment Monday, May 17, 2010
  • 5.
    About Me • Programmer: 50-80 projects/yr • Entrepreneur: profitable/practical • Research: • Domain Specific Modeling/DSLs • Software Product Lines • Writer: GroovyMag, InfoQ, IEEE Software, JSMag, Methods & Tools, Fusion Authority, Flex Authority . . . • Presenter: ooPSLA, Code Generation, BCS SPA . . . Monday, May 17, 2010
  • 6.
    What is aDSL? • Specific to single domain - generally not Turing complete (excl. XSLT) • More expressive than GPL within the domain • Executable • Examples: SQL, CSS, Regex, PC configurator, document workflow rules Monday, May 17, 2010
  • 7.
    What is aDSL? - DSL vs. API • Subjective • Often facade to API • Focus on language nature • “Fluent” interface API: new event( startTime: "16:00", endTime: "18:00" ) DSL: new event.from 4.pm, to: 6.pm Monday, May 17, 2010
  • 8.
    What is aDSL? - Benefits • Expressive (clear intent, concise) • Separate business logic from code • Separate lifecycle • SME readable (not writable) Monday, May 17, 2010
  • 9.
    What is aDSL? - When Use? • Frequently changing business rules • Interactions with stakeholders • Relatively well understood domain • Typical use cases: • Product family • Platform based development • Configuration • Business rule definitions Monday, May 17, 2010
  • 10.
    Key Concepts • Overview: • Vertical vs horizontal • Abstract vs Concrete • Projections • Three types of DSL • Internal vs. External Monday, May 17, 2010
  • 11.
    Vertical vs. Horizontal • Horizontal: technical domain • SQL, CSS, GORM . . . • More maintainable/readable code • Good enough is good enough • Vertical: business domain • Insurance, PC configurator . . . • Share “code” with SME’s • Readability is key Monday, May 17, 2010
  • 12.
    Abstract vs. Concrete • Abstract grammar <person> • what say <FirstName>Paul</FirstName> <LastName>King</LastName> • Concrete syntax </person> • how say new person( firstName: "Paul", lastName: "King" ) person called “Paul “, “ King “ Paul King Monday, May 17, 2010
  • 13.
    Projections • Multiple projections • Textual • Spreadsheet • Visual • Form based Monday, May 17, 2010
  • 14.
    Three Broad Typesof DSL • Internal (embedded) • External (parser) • Language workbench Monday, May 17, 2010
  • 15.
    Internal vs. External • Internal: • External: • Easier to implement • Easier to evolve • Power of GPL: • End user writable (forms)? • Loops • Projectional: • Math • Forms • Conditionals . . . • Visual • Refactor to external • Spreadsheet Monday, May 17, 2010
  • 16.
    Top Down/Bottom Up • Bottom up: • Evolve from API (e.g. Hibernate) • Not very expressive, but quick starting point • Top down: • Sketch out UL with stakeholders • Generally better language Monday, May 17, 2010
  • 17.
    General Hints • Learn from DDD/UL: • Common, rigorous understanding • Driven by domain model • Involve to SME’s • Lots of little languages (e.g. Grails) • Bounded contexts • Risk: • DSL eventually becomes badly designed GPL • Think: Ant Monday, May 17, 2010
  • 18.
    Implementing DSLs • API • Fluent interface • Reducing visual noise • Config file/XML/Json • Simple parser • Nested parser Monday, May 17, 2010
  • 19.
    API • Good starting point • Easy to evolve • Often readable enough . . . Event salesMeet = new event( startTime: "16:00", endTime: "18:00" ) Participant Joe = new Participant( firstName: “Joe”, lastName: “Smith” ) Participant Fred = new Participant( firstName: “Fred”, lastName: “Jones” ) salesMeet.addParticipant( Joe ) salesMeet.addParticipant( Fred ) Monday, May 17, 2010
  • 20.
    Fluent interface • The start of DSLs • Language nature • Focus on sentence like syntax Event salesMeet = new event( from: 4.pm, to: 6.pm ) .with( “Joe”, “Smith” ) .and( “Fred”, “Jones” ) Monday, May 17, 2010
  • 21.
    Reducing Visual Noise • Flexible and malleable syntax • Metaprogramming • AST transformations © Guillaume Laforge Monday, May 17, 2010
  • 22.
    Flexible and MalleableSyntax • Scripts • Optional typing • Native syntax constructs • (lists, maps) • Parentheses/semi omission • Named arguments • Operator overloading • Closures © Guillaume Laforge Monday, May 17, 2010
  • 23.
    Metaprogramming • PoGo • Categories • Builders • Custom metaclass • ExpandoMetaClass © Guillaume Laforge Monday, May 17, 2010
  • 24.
    AST Transformations • AST traversal • Local transformations • Global transformations • Hooks into ANTLR © Guillaume Laforge Monday, May 17, 2010
  • 25.
    Outcome: event from 4.pm to 6.pm .with “Joe” “Smith” .and “Fred” “Jones” © Guillaume Laforge Monday, May 17, 2010
  • 26.
    Config File/XML/Json • Introduction to external DSLs • Easy to load/parse • Supports projections and tooling <events> <event startTime=”16:00” endTime=”18:00”> <participants> <participant firstname=”Joe” lastName=”Smith” /> <participant firstname=”Joe” lastName=”Smith” /> </participants> </event> <events> Monday, May 17, 2010
  • 27.
    Simple Parser • Delimiter directed translation • Regex or similar • Doesn’t support nesting • Only for simplest languages Monday, May 17, 2010
  • 28.
    Nested Parser • Syntax directed translation • Grammar/parser • ANTLR Monday, May 17, 2010
  • 29.
    Additional Considerations • Testing • Evolution • Documentation • Error handling • IDE support Monday, May 17, 2010
  • 30.
    Testing DSLs • Test domain model • Test fluent interface (expression builder) • Can Compile • Scenarios • Orthogonal test DSLs Monday, May 17, 2010
  • 31.
    DSL Evolution • The problem: success! • Approaches: • No changes • Backwards compatibility • Versioning • Model migrations Monday, May 17, 2010
  • 32.
    Documentation • Getting started guide • Samples • FAQs for common tasks • Language reference • Some debugging suggestions • Not more than API requires • Source/regression tests help but aren't enough Monday, May 17, 2010
  • 33.
    Error Handling • Throw while process = easy • Meaningful messages hard Monday, May 17, 2010
  • 34.
    IDE Support • Syntax highlighting • Code completion • Static verification • IntelliJ helps . . . • If important, consider Xtext/EMF Monday, May 17, 2010
  • 35.
    Conclusions • It’s all about the domain • Evolve to DSLs • API to internal . . . • . . . external if required • Check Guillaume’s preso for syntax • Think about additional considerations from day 1 Monday, May 17, 2010
  • 36.
    Resources History: http://www.faqs.org/docs/artu/minilanguageschapter.html DSM: JP Tolvanen: http://www.metacase.com/blogs/jpt/blogView Steve Kelly: http://www.metacase.com/blogs/stevek/blogView Markus Voelter: http://www.voelter.de/ DDD: http://domaindrivendesign.org/ Monday, May 17, 2010
  • 37.
    Resources (2) DSLs: Martin Fowler: http://martinfowler.com/dslwip/ Groovy DSLs: Google - Guillaume Laforge, Paul King, Hamlet D’Arcy,Venkat Subramaniam, Neil Ford DSL Evolution: http://www.infoq.com/articles/dsl-evolution Books: Domain Specific Modeling - Kelly/Tolvanen Generative Programming - Czarnecki/Eisenecker Domain Driven Design - Evans Monday, May 17, 2010
  • 38.
    The End! • http://gettinggroovy.com • Email: peter@pbell.com • Twitter: peterbell Monday, May 17, 2010