0
Test-DrivenLanguage DevelopmentLennart Kats   Delft University of TechnologyRob Vermaas    LogicBloxEelco Visser   Delft U...
Language Workbenches
Testing“DOES THE TYPE CHECKER CATCH THIS?”
Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
SyntaxSemantics            IDE
parsing                                 abstract syntax                                 syntax error marking              ...
How to testlanguage services?
IMPLEMENTFEATURE, TRY WITH                   DOESN’T WORK    EXAMPLE                                       DISCARD        ...
How can wesystematically testlanguage definitions?
General-Purpose Testing Tools?
A Test Inputmodule Examplefunction foo() {  bar();}function bar() {}
Another Test Inputmodule Examplefunction foo() {  foo();}function bar() {}
A Negative Test Casemodule Examplefunction foo() {  notfoo();}function bar() {}
Automated Testing     Infrastructure                  ParserTest    Test     Compilerinput   script                   IDE
Automated Testing            Infrastructure                                Parser      Test            Test     Compiler  ...
Can we design a    general, reusablesolution for language testing?
Yes We CanGeneric test specification language                +         Parametrization
Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[ module ...
LPTL Opportunities     Expressiveness     Tool support
LPTL Opportunities (1)Module system      +GUI Test Runner
LPTL Opportunities (2) IDE support   for testspecifications
LPTL Opportunities (3)Immediate   testevaluation
LPTL Opportunities (4)IDE support  for test   inputs
LPTL Opportunities (5)              setup [[               module Example               imports stuff Reduced        funct...
LPTL Opportunities (6)                  1 error                  2 warnings                  /expected here/              ...
Testing Syntax (1) test Proper declaration [[   var s : String = "a"; ]] parse test Java-like declaration [[   String s = ...
Testing Syntax (2) test Proper declaration [[   var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[   3+1...
Testing Error Markers   test Variable declaration [[     var s : String = "a";   ]] 0 errors   test Bad variable declarati...
Testing Referencestest [[ module Example function foo() {   bar(); } function bar() {  }]] resolve ... to ...
Testing Referencestest [[ module Example function foo() {   bar();   [[bar]](); } function [[bar]]() {          bar() {  }...
Testing CodeGeneration..?
Testing Code      Generation..?test [[  function foo() {    return 3;  }]] build generate-javascript to [[  [[bar]](); fun...
Testing Code     Generation..?test [[   function foo() {     return 3;   }                    to [[]] build generate-javas...
Testing Executionsetup [[ application execution  function main() : Num {    // init    [[...]]  }]]test Arithmetic [[  ret...
Implementation Spoofax Testing Language      (spoofax.org)
Implementation       TechniquesLanguage embeddingDynamic instantiation of language services
Language Embedding:      Syntax  module tests  language mobl  test Java-like declaration [[    String s = "a";  ]] parse f...
Language Embedding:      Syntax 1 module tests    language mobl    test Java-like declaration [[      String s = "a";    ]...
Language Embedding:      Syntax 1 module tests 2 language mobl    test Java-like declaration [[      String s = "a";    ]]...
Language Embedding:      Syntax 1 module tests 2 language mobl   test Java-like declaration [[ 3 ]]String sfails      pars...
Language Embedding: Semantics & IDE (1)  test A function call [[   function foo() {   }    fo|  ]]          Content complete
Language Embedding: Semantics & IDE (1)  test A function call [[   function foo() {                             Mobl   }  ...
Language Embedding: Semantics & IDE (2)  test A function call [[   function foo() {                            Mobl   }   ...
Dynamic Language Service     Instantiation                      Mobl
Dynamic Language Service     Instantiation   Language consists of separate services     Services have functional interfaces
Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
WRITE 1 TEST                    IMPLEMENT FEATURE       TESTS FAIL                       TESTS SUCCEED!
ADDITIONAL  SLIDES
Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[  language mobl  test Testing [[    module y  ]]]]]
Dynamic Language Service     Instantiation                           Test  “mobl”                   input  Language      M...
Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
The Spoofax Language      Workbench [OOPSLA 2010]• Integrated environment for language  definition• Define syntax, semantics...
Upcoming SlideShare
Loading in...5
×

Test-driven language development

215

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
215
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • Language workbenches combine interactive definition and use of DSLs\n
  • Scenario: scoping rules!\n
  • Scenario: scoping rules!\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • not recording intentions, not avoiding regressions\n
  • \n
  • \n
  • \n
  • Other variations: nested functions, closures, methods, inheritance\n
  • Other variations: nested functions, closures, methods, inheritance\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Module system\n GUI Test Runner\n Avoids boilerplate\n Testing conditions for syntax, semantics, IDE\n
  • \n
  • \n
  • Generic language and infrastructure\nspecialized with embedding\n
  • Generic language and infrastructure\nspecialized with embedding\n- module system\n- parametrization\n- general format for specifying tests\n- and conditions\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Test will use parser and compare to expectation\n
  • Test will use parser and compare to expectation\n
  • Test will use parser and compare to expectation\n
  • Test will simulate reference resolve and compare to expectation\n
  • Test will simulate reference resolve and compare to expectation\n
  • Test will simulate reference resolve and compare to expectation\n
  • Test will simulate reference resolve and compare to expectation\n
  • Test will simulate reference resolve and compare to expectation\n
  • \n
  • \n
  • Next:\n\n
  • Next:\n\n
  • \n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • 1. Parse testing language\n 2. Determine tested language\n 3. Parse tested language\n(might be syntax incorrect!)\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • expressiveness + tool support\n
  • \n
  • does not \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Oracle: baseline compiler, specification, execution\n\n
  • \n
  • Transcript of "Test-driven language development"

    1. 1. Test-DrivenLanguage DevelopmentLennart Kats Delft University of TechnologyRob Vermaas LogicBloxEelco Visser Delft University of Technology
    2. 2. Language Workbenches
    3. 3. Testing“DOES THE TYPE CHECKER CATCH THIS?”
    4. 4. Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
    5. 5. SyntaxSemantics IDE
    6. 6. parsing abstract syntax syntax error marking Syntax Semantics IDEtype checking errors/warningscompilation reference resolvinginterpretation content completionexecution refactoring views
    7. 7. How to testlanguage services?
    8. 8. IMPLEMENTFEATURE, TRY WITH DOESN’T WORK EXAMPLE DISCARD CLOSE ENOUGH. EXAMPLE
    9. 9. How can wesystematically testlanguage definitions?
    10. 10. General-Purpose Testing Tools?
    11. 11. A Test Inputmodule Examplefunction foo() { bar();}function bar() {}
    12. 12. Another Test Inputmodule Examplefunction foo() { foo();}function bar() {}
    13. 13. A Negative Test Casemodule Examplefunction foo() { notfoo();}function bar() {}
    14. 14. Automated Testing Infrastructure ParserTest Test Compilerinput script IDE
    15. 15. Automated Testing Infrastructure Parser Test Test Compiler input script IDE+ simple– language-specific script– limited expressiveness– boilerplate in tests– ...
    16. 16. Can we design a general, reusablesolution for language testing?
    17. 17. Yes We CanGeneric test specification language + Parametrization
    18. 18. Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[ module Example module Example <test input> function test() { var s s : String = 1; var : String = 1; }}]] 1 error
    19. 19. LPTL Opportunities Expressiveness Tool support
    20. 20. LPTL Opportunities (1)Module system +GUI Test Runner
    21. 21. LPTL Opportunities (2) IDE support for testspecifications
    22. 22. LPTL Opportunities (3)Immediate testevaluation
    23. 23. LPTL Opportunities (4)IDE support for test inputs
    24. 24. LPTL Opportunities (5) setup [[ module Example imports stuff Reduced function test() { [[...]]boilerplate } ]] test Cannot assign ... [[ var s : String = 1; ]] 1 error
    25. 25. LPTL Opportunities (6) 1 error 2 warnings /expected here/ parse fails Wide set oftest conditions complete ... to ... resolve ... to ... refactor ... to ... build ... run ...
    26. 26. Testing Syntax (1) test Proper declaration [[ var s : String = "a"; ]] parse test Java-like declaration [[ String s = "a"; ]] parse fails
    27. 27. Testing Syntax (2) test Proper declaration [[ var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[ 3+1*2 ]] parse to [[ 3 + (1 * 2) ]]
    28. 28. Testing Error Markers test Variable declaration [[ var s : String = "a"; ]] 0 errors test Bad variable declaration [[ var s : String = 25; ]] 1 error /wrong type/
    29. 29. Testing Referencestest [[ module Example function foo() { bar(); } function bar() { }]] resolve ... to ...
    30. 30. Testing Referencestest [[ module Example function foo() { bar(); [[bar]](); } function [[bar]]() { bar() { }]] resolve ... to ... #1 to #2
    31. 31. Testing CodeGeneration..?
    32. 32. Testing Code Generation..?test [[ function foo() { return 3; }]] build generate-javascript to [[ [[bar]](); function bar() { }]]
    33. 33. Testing Code Generation..?test [[ function foo() { return 3; } to [[]] build generate-javascript to [[ var foo = function() { [[bar]](); return 3; }; function bar() {]] }]]
    34. 34. Testing Executionsetup [[ application execution function main() : Num { // init [[...]] }]]test Arithmetic [[ return 1 + 1;]] run run-test to 2
    35. 35. Implementation Spoofax Testing Language (spoofax.org)
    36. 36. Implementation TechniquesLanguage embeddingDynamic instantiation of language services
    37. 37. Language Embedding: Syntax module tests language mobl test Java-like declaration [[ String s = "a"; ]] parse fails
    38. 38. Language Embedding: Syntax 1 module tests language mobl test Java-like declaration [[ String s = "a"; ]] parse fails
    39. 39. Language Embedding: Syntax 1 module tests 2 language mobl test Java-like declaration [[ String s = "a"; ]] parse fails
    40. 40. Language Embedding: Syntax 1 module tests 2 language mobl test Java-like declaration [[ 3 ]]String sfails parse = "a";
    41. 41. Language Embedding: Semantics & IDE (1) test A function call [[ function foo() { } fo| ]] Content complete
    42. 42. Language Embedding: Semantics & IDE (1) test A function call [[ function foo() { Mobl } fo| ]] Content complete
    43. 43. Language Embedding: Semantics & IDE (2) test A function call [[ function foo() { Mobl } notfoo(); ]] No condition; error not expected
    44. 44. Dynamic Language Service Instantiation Mobl
    45. 45. Dynamic Language Service Instantiation Language consists of separate services Services have functional interfaces
    46. 46. Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
    47. 47. WRITE 1 TEST IMPLEMENT FEATURE TESTS FAIL TESTS SUCCEED!
    48. 48. ADDITIONAL SLIDES
    49. 49. Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[ language mobl test Testing [[ module y ]]]]]
    50. 50. Dynamic Language Service Instantiation Test “mobl” input Language Mobl Parser registry language Completion service
    51. 51. Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
    52. 52. The Spoofax Language Workbench [OOPSLA 2010]• Integrated environment for language definition• Define syntax, semantics, IDE• Based on Eclipse www.spoofax.org
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×