0
Integrated Language     Definition TestingLennart Kats   Delft University of TechnologyRob Vermaas    LogicBloxEelco Visser...
Language Workbenches
Testing“DOES THE TYPE CHECKER CATCH THIS?”
Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
parsing                                 abstract syntax                                 syntax error marking              ...
How to testlanguage services?
IMPLEMENTFEATURE, TEST WITH                               DOESN’T WORK     EXAMPLE                                   DISCA...
How can wesystematically testlanguage definitions?
General-Purpose Testing Tools?
A Test Inputmodule Examplefunction foo() {  bar();}function bar() {}
Another Test Inputmodule Examplefunction foo() {  foo();}function bar() {}
A Negative Test Casemodule Examplefunction foo() {  baz();}function bar() {}
Automated Testing            Infrastructure                                Parser      Test            Test     Compiler  ...
Can we design a general solution for specifying     language tests?
Yes We CanGeneric test specification language                +         Parametrization
Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[  module...
LPTL Opportunities     Expressiveness     Tool support
LPTL Opportunities (1)Module system      +GUI Test Runner
LPTL Opportunities (2) IDE support   for testspecifications
LPTL Opportunities (3)Immediate   testevaluation
LPTL Opportunities (4)IDE support  for test   inputs
LPTL Opportunities (5)              setup [[                module Example               imports stuff Reduced        func...
LPTL Opportunities (6)                  1 error                  2 warnings                  /expected here/              ...
Testing Syntax (1) test Proper declaration [[   var s : String = "a"; ]] parse test Java-like declaration [[   String s = ...
Testing Syntax (2) test Proper declaration [[   var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[   3 +...
Testing Error Markers   test Variable declaration [[     var s : String = "a";   ]] 0 errors   test Bad variable declarati...
Testing Referencestest [[  module Example function foo() {   bar();   [[bar]](); } function [[bar]]() {          bar() {  ...
Testing Code    Generation..?test [[   function foo() {     return 3;   }]] build generate-javascript to [[  var foo = fun...
Testing Executionsetup [[  application execution  function test() : Num {    // init    [[...]]  }]]test Arithmetic [[  re...
Implementation Spoofax Testing Language      (spoofax.org)
Implementation       TechniquesLanguage embeddingDynamic instantiation of language services
Language Embedding:      Syntax 1 module tests 2 language mobl   test Java-like declaration [[ 3 ]]String s = "a";      pa...
Language Embedding: Semantics & IDE (1)  test A function call [[    function foo() {                            Mobl    } ...
Language Embedding: Semantics & IDE (2)  test A function call [[    function foo() {                            Mobl    } ...
Dynamic Language Service     Instantiation                      Mobl
Dynamic Language Service     InstantiationLanguage registryLanguage consists of servicesServices have functional interfaces
Dynamic Language Service     Instantiation                           Test  “mobl”                   input  Language      M...
Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
Conclusions• General abstraction for language testing• Explored opportunities in expressiveness  and tool support         ...
ADDITIONAL  SLIDES
Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[   language Mobl   test Testing [[     module y   ]]]]]
The Spoofax Language      Workbench [OOPSLA 2010]• Integrated environment for language  definition• Define syntax, semantics...
Upcoming SlideShare
Loading in...5
×

Integrated Language Definition Testing: Enabling Test-Driven Language Development (SPLASH 2012)

1,678

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,678
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Integrated Language Definition Testing: Enabling Test-Driven Language Development (SPLASH 2012)"

  1. 1. Integrated Language Definition TestingLennart Kats Delft University of TechnologyRob Vermaas LogicBloxEelco Visser Delft University of Technology
  2. 2. Language Workbenches
  3. 3. Testing“DOES THE TYPE CHECKER CATCH THIS?”
  4. 4. Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
  5. 5. parsing abstract syntax syntax error marking Syntax Semantics IDEtype checking errors/warningscompilation reference resolvinginterpretation content completionexecution refactoring views
  6. 6. How to testlanguage services?
  7. 7. IMPLEMENTFEATURE, TEST WITH DOESN’T WORK EXAMPLE DISCARD CLOSE ENOUGH. TEST
  8. 8. How can wesystematically testlanguage definitions?
  9. 9. General-Purpose Testing Tools?
  10. 10. A Test Inputmodule Examplefunction foo() { bar();}function bar() {}
  11. 11. Another Test Inputmodule Examplefunction foo() { foo();}function bar() {}
  12. 12. A Negative Test Casemodule Examplefunction foo() { baz();}function bar() {}
  13. 13. Automated Testing Infrastructure Parser Test Test Compiler input script IDE+ simple– language-specific script– limited expressiveness– boilerplate code– ...
  14. 14. Can we design a general solution for specifying language tests?
  15. 15. Yes We CanGeneric test specification language + Parametrization
  16. 16. Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[ module Example <test input> function test() { var s : String = 1; }]] 1 error
  17. 17. LPTL Opportunities Expressiveness Tool support
  18. 18. LPTL Opportunities (1)Module system +GUI Test Runner
  19. 19. LPTL Opportunities (2) IDE support for testspecifications
  20. 20. LPTL Opportunities (3)Immediate testevaluation
  21. 21. LPTL Opportunities (4)IDE support for test inputs
  22. 22. LPTL Opportunities (5) setup [[ module Example imports stuff Reduced function test() { [[...]]boilerplate } ]] test Cannot assign ... [[ var s : String = 1; ]] 1 error
  23. 23. LPTL Opportunities (6) 1 error 2 warnings /expected here/ parse fails Wide set oftest conditions complete ... to ... resolve ... to ... refactor ... to ... build ... run ...
  24. 24. Testing Syntax (1) test Proper declaration [[ var s : String = "a"; ]] parse test Java-like declaration [[ String s = "a"; ]] parse fails
  25. 25. Testing Syntax (2) test Proper declaration [[ var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[ 3 + 1 * 2 ]] parse to [[ 3 + (1 * 2) ]]
  26. 26. Testing Error Markers test Variable declaration [[ var s : String = "a"; ]] 0 errors test Bad variable declaration [[ var s : String = 25; ]] 1 error /wrong type/
  27. 27. Testing Referencestest [[ module Example function foo() { bar(); [[bar]](); } function [[bar]]() { bar() { }]] resolve ...to #2 #1 to ...
  28. 28. Testing Code Generation..?test [[ function foo() { return 3; }]] build generate-javascript to [[ var foo = function foo() { [[bar]](); return 3; }; function bar() {]] }]]
  29. 29. Testing Executionsetup [[ application execution function test() : Num { // init [[...]] }]]test Arithmetic [[ return 1 + 1;]] run run-test to 2
  30. 30. Implementation Spoofax Testing Language (spoofax.org)
  31. 31. Implementation TechniquesLanguage embeddingDynamic instantiation of language services
  32. 32. Language Embedding: Syntax 1 module tests 2 language mobl test Java-like declaration [[ 3 ]]String s = "a"; parse fails
  33. 33. Language Embedding: Semantics & IDE (1) test A function call [[ function foo() { Mobl } fo| ]] Content complete
  34. 34. Language Embedding: Semantics & IDE (2) test A function call [[ function foo() { Mobl } notfoo(); ]] No condition; error not expected
  35. 35. Dynamic Language Service Instantiation Mobl
  36. 36. Dynamic Language Service InstantiationLanguage registryLanguage consists of servicesServices have functional interfaces
  37. 37. Dynamic Language Service Instantiation Test “mobl” input Language Mobl Parser registry language Completion service
  38. 38. Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
  39. 39. Conclusions• General abstraction for language testing• Explored opportunities in expressiveness and tool support www.spoofax.org
  40. 40. ADDITIONAL SLIDES
  41. 41. Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
  42. 42. Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[ language Mobl test Testing [[ module y ]]]]]
  43. 43. The Spoofax Language Workbench [OOPSLA 2010]• Integrated environment for language definition• Define syntax, semantics, IDE• Based on Eclipse www.spoofax.org
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×