Recommended
More Related Content
Similar to A Unit Test Metamodel for Test Generation
Similar to A Unit Test Metamodel for Test Generation (20)
More from ESUG
More from ESUG (20)
Recently uploaded
Recently uploaded (20)
A Unit Test Metamodel for Test Generation
- 1. 1 1 1 Evref fervE A Unit Test Metamodel for Test Generation Gabriel Darbord¹ Anne Etien¹ Nicolas Anquetil¹ Benoit Verhaeghe² Mustapha Derras² ¹Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRIStAL, F-59000 Lille, France ²Berger-Levrault, France IWST 2023 Evref fervE
- 2. 2 2 2 Evref fervE The Importance of Testing § Nowadays, when developping new software systems: § 20-50% time spent on testing § We test because we want: § Bug detection and prevention § Quality assurance § User satisfaction § Non-regression § Confidence § Etc.
- 3. 3 3 3 Evref fervE Legacy Software System Lack Tests § In 2022, Berger-Levrault owns 150 software programs § Three-tier architectures (client, server, database) § Millions of lines of code § Different legacy technologies that can be up to 25 years old § Severe lack of tests § Developers fear changes
- 4. 4 4 4 Evref fervE Towards Automated Test Generation § Berger-Levrault wants tests for their legacy software § Impossible to write them manually § Time consuming § Difficult and error-prone § Lack of resources
- 5. 5 5 5 Evref fervE Our Test Generation Approach § Using software models and execution traces § static and dynamic analysis § Our objective is to generate tests that are: § Relevant § Readable § Maintainable § Not relying on existing tests
- 6. 6 6 6 Evref fervE Different Criteria Paper Relevant Readable Maintainable J. Pires et al. ~ ~ + G. Fraser et al. ~ - ~ A. C. R. Paiva et al. + ~ ~ M. Tufano et al. ~ + ~ LLM approaches need to be trained on codebase and existing tests
- 7. 7 7 7 Evref fervE About the Moose Platform § Moose is a platform for software analysis § It allows to: § Represent a software system in a model § Query, manipulate, transform, and visualize models
- 8. 8 8 8 Evref fervE An Approach Based on Metamodels Codebase Traces Value Model Code Model Unit Test Model Unit Test M0 Reality M1 Models M2 Metamodels Value Metamodel Code Metamodel Unit Test Metamodel analyze extract generate build conformsTo conformsTo conformsTo produce
- 9. 9 9 9 Evref fervE Unit Test Metamodel TestSuite TestCase TestMethod Fixture SetUp TearDown Assertion before beforeAll beforeEach cleanups arrangements * Value * * * * * expected arguments valuesToHandle Class Method * testedClass testedMethod * unitTests Act after afterAll afterEach * * *
- 10. 10 10 10 Evref fervE Value Metamodel Value Dictionary Primitive Collection Object EnumValue Unknown Association dictionaryValue dictionaryKey * * * * attributes * references * * * * capturedValues UnknownType Type TypedEntity EnumValue * elements source
- 11. 11 11 11 Evref fervE Example Value Model [{"name": "John Doe"}] Set<User> getUsers() Produces trace Modelized Code Value "John Doe" :Primitive :Collection :Object HashSet<User> :Type getUsers() :TypedEntity User :Type String :Type name source type type type
- 12. 12 12 12 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 13. 13 13 13 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 14. 14 14 14 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 15. 15 15 15 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 16. 16 16 16 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 17. 17 17 17 Evref fervE Build Unit Test Using Model Transformations Code Model Method Parameter Class Unit Test Model TestCase TestMethod Arrange Act Assert Value Model Argument Result
- 18. 18 18 18 Evref fervE Export Code Using Abstract Syntax Trees M0 Reality M1 Models M2 Metamodels Unit Test Model Unit Test Metamodel conformsTo AST Metamodel AST Model Unit Test conformsTo export refactor generate
- 19. 19 19 19 Evref fervE Refactoring Abstract Syntax Trees Statements VarDecl Variable name Primitive "John Doe" MethodCall setName Variable user Variable name String name = "John Doe"; user.setName(name);
- 20. 20 20 20 Evref fervE Refactoring Abstract Syntax Trees Statements VarDecl Variable name Primitive "John Doe" MethodCall setName Variable user Variable name Statements Primitive "John Doe" MethodCall setName Variable user Inline constants String name = "John Doe"; user.setName(name); user.setName("John Doe");
- 21. 21 21 21 Evref fervE Example of Generated Test for JUnit class UserManager { Set<User> users; Set<User> usersByName(String name) { Set<User> result = new HashSet<>(); for (User user: users) if (user.getName().equals(name)) result.add(user); return result; } } class UserManagerTest { UserManager manager = new UserManager(); @Test void testUsersByName() { Set<User> expected = new HashSet<>(); User user = new User(); user.setName("John Doe"); expected.add(user); Set<User> actual = manager .usersByName("John Doe"); assertEquals(expected, actual); } } Application code Generated test code
- 22. 22 22 22 Evref fervE Future Work § Pursue test quality § Relevant, readable, maintainable… § Evaluate our approach in Pharo, thanks to our community § Compare existing tests to generated tests § Generate unit tests for our industrial partner Berger-Levrault UT
- 23. 23 23 23 Evref fervE 8 8 8 Evref fervE An Approach Based on Metamodels Codebase Traces Value Model Code Model Unit Test Model Unit Test M0 Reality M1 Models M2 Metamodels Value Metamodel Code Metamodel Unit Test Metamodel analyze extract generate build conformsTo conformsTo conformsTo produce 21 21 21 Evref fervE Example of Generated Test for JUnit class UserManager { Set<User> users; Set<User> usersByName(String name) { Set<User> result = new HashSet<>(); for (User user: users) if (user.getName().equals(name)) result.add(user); return result; } } class UserManagerTest { UserManager manager = new UserManager(); @Test void testUsersByName() { Set<User> expected = new HashSet<>(); User user = new User(); user.setName("John Doe"); expected.add(user); Set<User> actual = manager .usersByName("John Doe"); assertEquals(expected, actual); } } Application code Generated test code 5 5 5 Evref fervE Our Test Generation Approach § Using software models and execution traces § static and dynamic analysis § Our objective is to generate tests that are: § Relevant § Readable § Maintainable § Not relying on existing tests 9 9 9 Evref fervE Unit Test Metamodel TestSuite TestCase TestMethod Fixture SetUp TearDown Assertion before beforeAll beforeEach cleanups arrangements * Value * * * * * expected arguments valuesToHandle Class Method * testedClass testedMethod * unitTests Act after afterAll afterEach * * * Conclusion