Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Exposing Test Analyses with DrTests

79 views

Published on

Talk from IWST at ESUG19, Cologne, Germany

Published in: Software
  • Be the first to comment

  • Be the first to like this

Exposing Test Analyses with DrTests

  1. 1. Exposing Test Analyses with DrTests Dayne Guerra, Julien Deplanque, Stephane Ducasse guerra.dayne@gmail.com , {first}.{last}@inria.fr
  2. 2. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 2
  3. 3. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 3
  4. 4. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Outpu t Results of tests run Variety of analyses on tests 4
  5. 5. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases … Outpu t Results of tests run Percentage of methods covered … Variety of analyses on tests 5
  6. 6. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases Test cases … Outpu t Results of tests run Percentage of methods covered Time taken for each test run … Variety of analyses on tests 6
  7. 7. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases Test cases Test cases
 +
 parameters Outpu t Results of tests run Percentage of methods covered Time taken for each test run Results of tests run depending on parameter Variety of analyses on tests 7
  8. 8. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases Test cases Test cases
 +
 parameters Test cases
 +
 mutation s Outpu t Results of tests run Percentage of methods covered Time taken for each test run Results of tests run depending on parameter Mutants to be killed Variety of analyses on tests 8
  9. 9. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases Test cases Test cases
 +
 parameters Test cases
 +
 mutation s Comments Outpu t Results of tests run Percentage of methods covered Time taken for each test run Results of tests run depending on parameter Mutants to be killed Comments containing failing examples Variety of analyses on tests 9
  10. 10. Runner Coverage Profile Parametrisable
 test Mutation testing Run examples in comment Rotten Tests Finder … Input Test cases Test cases Test cases Test cases
 +
 parameters Test cases
 +
 mutation s Comments Test cases … Outpu t Results of tests run Percentage of methods covered Time taken for each test run Results of tests run depending on parameter Mutants to be killed Comments containing failing examples Rotten tests … Variety of analyses on tests 10
  11. 11. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 11
  12. 12. How to design an
 infrastructure dealing
 with all these analyses? Different 
 Inputs Different 
 Outputs 12 Test Runner Test Coverage Test profiling …
  13. 13. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 13
  14. 14. Test Runner Test Coverage Test profiling Configuration Plugin Result 14 An architecture to handle test analyses variety: DrTests
  15. 15. Dr Tests Model Plugin Configuration Plugin Result Plugin Plugin run,
 provides updates
 through announcements Created from UI
 or by scripting Can be queried from
 UI or by scripting 15
  16. 16. Dr Tests UI Packages under analysis Plugin selected Logging label 16
  17. 17. Dr Tests UI Plugin input 17
  18. 18. Dealing with different inputs Packages with comments Tests Test package
 &
 Package under analysis 18
  19. 19. Dr Tests UI Start plugin 19
  20. 20. Dr Tests UI Logging label Results
 tree 20
  21. 21. Different results Comments Percentage & Uncovered methods Rotten Tests 21
  22. 22. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 22
  23. 23. We implemented 3 plugins with various input and output
  24. 24. Dr Tests model validation Test Runner Rotten Test Test Coverage INPUT Tests Cases
 (packages and classes) Tests Cases
 (packages and classes) Tests and Package under analysis ACTION Run the test cases Find Rotten Green Tests Run tests and verify that methods inside the package were executed RESULT Test result sorting by groups A list of rotten tests Percentage of covering and list of uncovered methods 24
  25. 25. Test Runner Rotten Test Test Coverage INPUT Tests Cases
 (packages and classes) Tests Cases
 (packages and classes) Tests and Package under analysis ACTION Run the test cases Find Rotten Green Tests Run tests and verify that methods inside the package were executed RESULT Test result sorting by groups A list of rotten tests Percentage of covering and list of uncovered methods Dr Tests model validation 25 These analyses are implemented inside DrTests architecture and it works!
  26. 26. Roadmap • Variety of test analyses available • What architecture to handle these analyses? • DrTests • Evaluating DrTests on various analyses • Conclusion 26
  27. 27. Conclusion ‣ Extensible via plugins ‣ Provides good model to configure, run and gather results from plugins ‣ Be integrated in Pharo 8.0 ‣Future work: ‣ Extend the validation by implementing more
 plugins @dayneStorm_ GDayne juliendelplanque/DrTests 27
  28. 28. TestRunner UI Packages containing tests TestCases Results summary Failed tests Errors Run tests Profile test execution Analyse code coverage Re-run failures or errors only Export results 28
  29. 29. TestRunner UI: coverage and profile Extra windows generated 
 for results 29
  30. 30. Dr Tests Packages under analysis Plugin input Plugin selected Results
 tree Kind of visualization Start pluginLogging label Plugin-defined action(s)30
  31. 31. Dr Tests UI Kind of visualization Skip if no time 31
  32. 32. Dr Tests UISkip if no time 32
  33. 33. Dr Tests UI Plugin-defined action(s) Skip if no time 33

×