Integrated Language Definition Testing: Enabling Test-Driven Language Development (SPLASH 2012)

  • 1,510 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,510
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
9
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Integrated Language Definition TestingLennart Kats Delft University of TechnologyRob Vermaas LogicBloxEelco Visser Delft University of Technology
  • 2. Language Workbenches
  • 3. Testing“DOES THE TYPE CHECKER CATCH THIS?”
  • 4. Testing“DOES THIS HYPERLINK POINT TO THE RIGHT PLACE?”
  • 5. parsing abstract syntax syntax error marking Syntax Semantics IDEtype checking errors/warningscompilation reference resolvinginterpretation content completionexecution refactoring views
  • 6. How to testlanguage services?
  • 7. IMPLEMENTFEATURE, TEST WITH DOESN’T WORK EXAMPLE DISCARD CLOSE ENOUGH. TEST
  • 8. How can wesystematically testlanguage definitions?
  • 9. General-Purpose Testing Tools?
  • 10. A Test Inputmodule Examplefunction foo() { bar();}function bar() {}
  • 11. Another Test Inputmodule Examplefunction foo() { foo();}function bar() {}
  • 12. A Negative Test Casemodule Examplefunction foo() { baz();}function bar() {}
  • 13. Automated Testing Infrastructure Parser Test Test Compiler input script IDE+ simple– language-specific script– limited expressiveness– boilerplate code– ...
  • 14. Can we design a general solution for specifying language tests?
  • 15. Yes We CanGeneric test specification language + Parametrization
  • 16. Language-ParametricTesting Language (LPTL)module my-testslanguage mobltest Cannot assign an integer to a string [[ module Example <test input> function test() { var s : String = 1; }]] 1 error
  • 17. LPTL Opportunities Expressiveness Tool support
  • 18. LPTL Opportunities (1)Module system +GUI Test Runner
  • 19. LPTL Opportunities (2) IDE support for testspecifications
  • 20. LPTL Opportunities (3)Immediate testevaluation
  • 21. LPTL Opportunities (4)IDE support for test inputs
  • 22. LPTL Opportunities (5) setup [[ module Example imports stuff Reduced function test() { [[...]]boilerplate } ]] test Cannot assign ... [[ var s : String = 1; ]] 1 error
  • 23. LPTL Opportunities (6) 1 error 2 warnings /expected here/ parse fails Wide set oftest conditions complete ... to ... resolve ... to ... refactor ... to ... build ... run ...
  • 24. Testing Syntax (1) test Proper declaration [[ var s : String = "a"; ]] parse test Java-like declaration [[ String s = "a"; ]] parse fails
  • 25. Testing Syntax (2) test Proper declaration [[ var s : String = "a"; ]] parse to VarDecl("s", _) test Precedence [[ 3 + 1 * 2 ]] parse to [[ 3 + (1 * 2) ]]
  • 26. Testing Error Markers test Variable declaration [[ var s : String = "a"; ]] 0 errors test Bad variable declaration [[ var s : String = 25; ]] 1 error /wrong type/
  • 27. Testing Referencestest [[ module Example function foo() { bar(); [[bar]](); } function [[bar]]() { bar() { }]] resolve ...to #2 #1 to ...
  • 28. Testing Code Generation..?test [[ function foo() { return 3; }]] build generate-javascript to [[ var foo = function foo() { [[bar]](); return 3; }; function bar() {]] }]]
  • 29. Testing Executionsetup [[ application execution function test() : Num { // init [[...]] }]]test Arithmetic [[ return 1 + 1;]] run run-test to 2
  • 30. Implementation Spoofax Testing Language (spoofax.org)
  • 31. Implementation TechniquesLanguage embeddingDynamic instantiation of language services
  • 32. Language Embedding: Syntax 1 module tests 2 language mobl test Java-like declaration [[ 3 ]]String s = "a"; parse fails
  • 33. Language Embedding: Semantics & IDE (1) test A function call [[ function foo() { Mobl } fo| ]] Content complete
  • 34. Language Embedding: Semantics & IDE (2) test A function call [[ function foo() { Mobl } notfoo(); ]] No condition; error not expected
  • 35. Dynamic Language Service Instantiation Mobl
  • 36. Dynamic Language Service InstantiationLanguage registryLanguage consists of servicesServices have functional interfaces
  • 37. Dynamic Language Service Instantiation Test “mobl” input Language Mobl Parser registry language Completion service
  • 38. Reflection+ simple+ no scripting required+ IDE helps avoid errors+ little boilerplate code+ expressiveness
  • 39. Conclusions• General abstraction for language testing• Explored opportunities in expressiveness and tool support www.spoofax.org
  • 40. ADDITIONAL SLIDES
  • 41. Related:Automatic Test Generation• Generate tests from grammar• Requires oracle• Complementary to our approach
  • 42. Self-Applicationlanguage Spoofax-Testingtest Testing testing [[[ language Mobl test Testing [[ module y ]]]]]
  • 43. The Spoofax Language Workbench [OOPSLA 2010]• Integrated environment for language definition• Define syntax, semantics, IDE• Based on Eclipse www.spoofax.org