Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lorentz workshop - 2018

4 views

Published on

Behavioral Model Seeding for Search-based Crash Reproduction, Lorentz workshop: In-Vivo Analytics for Big Software Quality

Published in: Software
  • Be the first to comment

  • Be the first to like this

Lorentz workshop - 2018

  1. 1. Behavioral Model Seeding for Search-based Crash Reproduction Pouria Derakhshanfar, Xavier Devroey, 
 Gilles Perrouin, Andy Zaidman and Arie van Deursen
  2. 2. !2
  3. 3. !3 java.lang.ClassCastException: […] at org…..SolrEntityReferenceResolver.getWikiReference(....java:93) at org…..SolrEntityReferenceResolver.getEntityReference(….java:70) at org…..SolrEntityReferenceResolver.resolve(….java:63) at org…..SolrDocumentReferenceResolver.resolve(….java:48) at … Crash!?!
  4. 4. Java Stack Trace (Issue XWIKI-13031) !4 java.lang.ClassCastException: […] at org…..SolrEntityReferenceResolver.getWikiReference(....java:93) at org…..SolrEntityReferenceResolver.getEntityReference(….java:70) at org…..SolrEntityReferenceResolver.resolve(….java:63) at org…..SolrDocumentReferenceResolver.resolve(….java:48) at … Exception Frames {Target→
  5. 5. !5 Issue XWIKI-13031
  6. 6. !6 a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() Random initial test suite a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() c() e()d() e() c() b() e() a() c() e()d() e() c() b() e() Evolutionary search a() e()d() Exception: at x(…) at y(…) at e(…) Exception: at x(…) at y(…) at e(…) Crash reproducing test case Stack trace Soltani, M., Panichella, A. and van Deursen, A. 2018. Search-Based Crash Reproduction and Its Impact on Debugging. Software Engineering, IEEE Transactions on. (2018).
  7. 7. !7 a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() Random initial test suite a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() c() e()d() e() c() b() e() a() c() e()d() e() c() b() e() Evolutionary search a() e()d() Exception: at x(…) at y(…) at e(…) Exception: at x(…) at y(…) at e(…) Crash reproducing test case Stack trace Isn’t an initial test suite close to actual usage of the classes more likely to lead to a crash reproduction?
  8. 8. Test Seeding • Use existing tests to generate the initial test suite • J. M. Rojas, et al. , “Seeding strategies in search- based unit test generation,” STVR, 2016. • Applied to crash replication !8 a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() c() c() e()c() e() c() a() Exception: at x(…) at y(…) at e(…) Stack trace a() c() c() c() b() a() e() Existing tests e() Random initial test suite Existing tests subset
  9. 9. Behavioral Model Seeding • Use a model of method usage to generate objects used in the initial test suite • Relies on Model-based Testing • Select abstract test cases • Concretized into objects !9 Random initial test suite Exception: at x(…) at y(…) at e(…) Stack trace a()b() c() e() d() Model Model-driven initial test suite a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() a() b() c() e()d() e() c() e() b() a() c() e()d() e() c() e() a()
  10. 10. size() add(Object)iterator() S0 get(int) remove(int) S1 S2S3 remove(int) S4 S5 SX size() add(Object) size() add(Object) !10 Abstract test case selection <add(Object), add(Object)>Abstract test case int[] t = new int[7]; t[3] = (-2147483647); EuclideanIntegerPoint ep = new […](t); LinkedList<[...]> lst = new LinkedList<>(); lst.add(ep); lst.add(ep); Concrete object
  11. 11. Test MATH-79b, Frame 2 !11 java.lang.NullPointerException at ...KMeansPlusPlusClusterer.assignPointsToClusters() at ...KMeansPlusPlusClusterer.cluster() … Input stack trace public void testCluster() throws Exception{
 int[] t = new int[7];
 t[3] = (-2147483647);
 EuclideanIntegerPoint ep = new EuclideanIntegerPoint(t);
 LinkedList<[...]> lst = new LinkedList<>(); 
 lst.add(ep);
 lst.add(ep);
 KMeansPlusPlusClusterer<[...]> kmean = new KMeansPlusPlusClusterer<>(12);
 lst.offerFirst(ep); 
 kmean.cluster(lst, 1, (-1357));
 } Crash reproducing test
  12. 12. Behavioral Model Inference • Inference from sequences of method calls • Coming from • Source code • Static analysis • Test cases • Dynamic analysis • Operations logs (future work) • Online analysis !12 a()b() c() e() d() N-gram inference [b(), a(), e()] [c(), d(), a(), e()] [b(), a(), d(), a(), d(), a(), e()] …
  13. 13. Behavioral Model Inference • Inference from sequences of method calls • Coming from • Source code • Static analysis • Test cases • Dynamic analysis • Operations logs (future work) • Online analysis !12 a()b() c() e() d() N-gram inference [b(), a(), e()] [c(), d(), a(), e()] [b(), a(), d(), a(), d(), a(), e()] …
  14. 14. Behavioral Model Inference • Inference from sequences of method calls • Coming from • Source code • Static analysis • Test cases • Dynamic analysis • Operations logs (future work) • Online analysis !12 a()b() c() e() d() N-gram inference [b(), a(), e()] [c(), d(), a(), e()] [b(), a(), d(), a(), d(), a(), e()] …
  15. 15. Evaluation on 45 crashes !13 0 25 50 75 not started failed line reached ex. thrown reproduced Numberofframes Configurations no s. test s. 0.2 test s. 0.5 test s. 0.8 test s. 1.0 model s. 0.2 model s. 0.5 model s. 0.8 model s. 1.0
  16. 16. Evaluation on 45 crashes !13 0 25 50 75 not started failed line reached ex. thrown reproduced Numberofframes Configurations no s. test s. 0.2 test s. 0.5 test s. 0.8 test s. 1.0 model s. 0.2 model s. 0.5 model s. 0.8 model s. 1.0
  17. 17. Evaluation on 45 crashes !13 0 25 50 75 not started failed line reached ex. thrown reproduced Numberofframes Configurations no s. test s. 0.2 test s. 0.5 test s. 0.8 test s. 1.0 model s. 0.2 model s. 0.5 model s. 0.8 model s. 1.0
  18. 18. Execution Time !14 ● ● 653 653 641 638 629 699 690 697 704 1e+01 1e+03 1e+05 nos. tests.0.2 tests.0.5 tests.0.8 tests.1.0 models.0.2 models.0.5 models.0.8 models.1.0 Numberoffitnessevaluations
  19. 19. Execution Time !14 ● ● 653 653 641 638 629 699 690 697 704 1e+01 1e+03 1e+05 nos. tests.0.2 tests.0.5 tests.0.8 tests.1.0 models.0.2 models.0.5 models.0.8 models.1.0 Numberoffitnessevaluations
  20. 20. Influencing Factors • Seeding abstract test cases • <add(Object), add(Object)> • Having dissimilar abstract test cases • Multiple information sources • Static analysis • Dynamic analysis • Prioritising abstract test cases selection • Select abstract test cases for classes in the stack trace !15
  21. 21. https://www.stamp-project.eu https://github.com/STAMP-project

×