Integrated Language Definition Testing: Enabling Test-Driven Language Development (SPLASH 2012)

1,920
-1

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,920
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Integrated Language Definition Testing: Enabling Test-Driven Language Development (SPLASH 2012)

  1. 1. Integrated Language Definition TestingLennart Kats Delft University of TechnologyRob Vermaas LogicBloxEelco Visser Delft University of Technology
  2. 2. Language Workbenches
  3. 3. Testing“DOES THE TYPE CHECKER CATCH THIS?”
  4. 4. Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
  5. 5. parsing abstract syntax syntax error marking Syntax Semantics IDEtype checking errors/warningscompilation reference resolvinginterpretation content completionexecution refactoring views
  6. 6. How to testlanguage services?
  7. 7. IMPLEMENTFEATURE, TEST WITH DOESN’T WORK EXAMPLE DISCARD CLOSE ENOUGH. TEST
  8. 8. How can wesystematically testlanguage definitions?
  9. 9. General-Purpose Testing Tools?
  10. 10. A Test Inputmodule Examplefunction foo() { bar();}function bar() {}
  11. 11. Another Test Inputmodule Examplefunction foo() { foo();}function bar() {}
  12. 12. A Negative Test Casemodule Examplefunction foo() { baz();}function bar() {}
  13. 13. Automated Testing Infrastructure Parser Test Test Compiler input script IDE+ simple– language-specific script– limited expressiveness– boilerplate code– ...
  14. 14. Can we design a general solution for specifying language tests?
  15. 15. Yes We CanGeneric test specification language + Parametrization
  16. 16. Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[ module Example <test input> function test() { var s : String = 1; }]] 1 error
  17. 17. LPTL Opportunities Expressiveness Tool support
  18. 18. LPTL Opportunities (1)Module system +GUI Test Runner
  19. 19. LPTL Opportunities (2) IDE support for testspecifications
  20. 20. LPTL Opportunities (3)Immediate testevaluation
  21. 21. LPTL Opportunities (4)IDE support for test inputs
  22. 22. LPTL Opportunities (5) setup [[ module Example imports stuff Reduced function test() { [[...]]boilerplate } ]] test Cannot assign ... [[ var s : String = 1; ]] 1 error
  23. 23. LPTL Opportunities (6) 1 error 2 warnings /expected here/ parse fails Wide set oftest conditions complete ... to ... resolve ... to ... refactor ... to ... build ... run ...
  24. 24. Testing Syntax (1) test Proper declaration [[ var s : String = "a"; ]] parse test Java-like declaration [[ String s = "a"; ]] parse fails
  25. 25. Testing Syntax (2) test Proper declaration [[ var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[ 3 + 1 * 2 ]] parse to [[ 3 + (1 * 2) ]]
  26. 26. Testing Error Markers test Variable declaration [[ var s : String = "a"; ]] 0 errors test Bad variable declaration [[ var s : String = 25; ]] 1 error /wrong type/
  27. 27. Testing Referencestest [[ module Example function foo() { bar(); [[bar]](); } function [[bar]]() { bar() { }]] resolve ...to #2 #1 to ...
  28. 28. Testing Code Generation..?test [[ function foo() { return 3; }]] build generate-javascript to [[ var foo = function foo() { [[bar]](); return 3; }; function bar() {]] }]]
  29. 29. Testing Executionsetup [[ application execution function test() : Num { // init [[...]] }]]test Arithmetic [[ return 1 + 1;]] run run-test to 2
  30. 30. Implementation Spoofax Testing Language (spoofax.org)
  31. 31. Implementation TechniquesLanguage embeddingDynamic instantiation of language services
  32. 32. Language Embedding: Syntax 1 module tests 2 language mobl test Java-like declaration [[ 3 ]]String s = "a"; parse fails
  33. 33. Language Embedding: Semantics & IDE (1) test A function call [[ function foo() { Mobl } fo| ]] Content complete
  34. 34. Language Embedding: Semantics & IDE (2) test A function call [[ function foo() { Mobl } notfoo(); ]] No condition; error not expected
  35. 35. Dynamic Language Service Instantiation Mobl
  36. 36. Dynamic Language Service InstantiationLanguage registryLanguage consists of servicesServices have functional interfaces
  37. 37. Dynamic Language Service Instantiation Test “mobl” input Language Mobl Parser registry language Completion service
  38. 38. Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
  39. 39. Conclusions• General abstraction for language testing• Explored opportunities in expressiveness and tool support www.spoofax.org
  40. 40. ADDITIONAL SLIDES
  41. 41. Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
  42. 42. Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[ language Mobl test Testing [[ module y ]]]]]
  43. 43. The Spoofax Language Workbench [OOPSLA 2010]• Integrated environment for language definition• Define syntax, semantics, IDE• Based on Eclipse www.spoofax.org

×