CSMR13b.ppt

289 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
289
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CSMR13b.ppt

  1. 1. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e ACRE: An Automated Aspect Creator for Gu´h´neuc and e e Bram Adams Testing C++ ApplicationsBackgroundRelated Work Etienne Duclos, S´bastien Le Digabel, eApproachDSL Yann-Ga¨l Gu´h´neuc and Bram Adams e e eACREHypotheses Department of Computer and Software EngineeringResults GERAD and Department of Mathematics and Industrial EngineeringH1 ´ Ecole Polytechnique de Montr´al, Qu´bec, Canada e eH2H3Threats to Validity CSMRConclusion March 07, 2013 Pattern Trace Identification, Detection, and Enhancement in Java Groupe d’Etudes et de Recherche en Analyse des D´cisions e
  2. 2. ACRE: AnAutomated Aspect ProblemCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram AdamsBackgroundRelated WorkApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 2 / 16
  3. 3. ACRE: AnAutomated Aspect OutlineCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Background Digabel, Yann-Ga¨l e Gu´h´neuc and e e Related Work Bram AdamsBackground ApproachRelated Work DSLApproach ACREDSLACRE HypothesesHypothesesResults ResultsH1H2 H1H3 H2Threats to Validity H3Conclusion Threats to Validity Conclusion 3 / 16
  4. 4. ACRE: AnAutomated Aspect BackgroundCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Aspect-Oriented Programming Digabel, Yann-Ga¨l e Based on the concept of separation of concerns Gu´h´neuc and e e Bram Adams Used with OOP thanks to a weaverBackground Different execution models availableRelated Work Two major languages: AspectJ and AspectC++ApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 4 / 16
  5. 5. ACRE: AnAutomated Aspect BackgroundCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Aspect-Oriented Programming Digabel, Yann-Ga¨l e Based on the concept of separation of concerns Gu´h´neuc and e e Bram Adams Used with OOP thanks to a weaverBackground Different execution models availableRelated Work Two major languages: AspectJ and AspectC++ApproachDSLACREHypotheses TestingResultsH1 Memory ManagementH2 Difficult in C++ [Boehm 12], high risk of memory leakH3Threats to ValidityConclusion 4 / 16
  6. 6. ACRE: AnAutomated Aspect BackgroundCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Aspect-Oriented Programming Digabel, Yann-Ga¨l e Based on the concept of separation of concerns Gu´h´neuc and e e Bram Adams Used with OOP thanks to a weaverBackground Different execution models availableRelated Work Two major languages: AspectJ and AspectC++ApproachDSLACREHypotheses TestingResultsH1 Memory ManagementH2 Difficult in C++ [Boehm 12], high risk of memory leakH3Threats to Validity Invariant TestingConclusion Allow to verify that a property is always true 4 / 16
  7. 7. ACRE: AnAutomated Aspect BackgroundCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Aspect-Oriented Programming Digabel, Yann-Ga¨l e Based on the concept of separation of concerns Gu´h´neuc and e e Bram Adams Used with OOP thanks to a weaverBackground Different execution models availableRelated Work Two major languages: AspectJ and AspectC++ApproachDSLACREHypotheses TestingResultsH1 Memory ManagementH2 Difficult in C++ [Boehm 12], high risk of memory leakH3Threats to Validity Invariant TestingConclusion Allow to verify that a property is always true Interference Bug Testing Very common bug, and one of the most difficult to remove [Software Quality Research Group 12] 4 / 16
  8. 8. ACRE: AnAutomated Aspect Related WorkCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram Adams AOP in testBackground AOP used in Java for unit and integrationRelated Work testing [Xu 04, Knauber 04, Li 09]ApproachDSL AOP used to test embedded C++ programsACRE [Pesonen 05, Pesonen 06, Mets¨ 07] aHypothesesResults AOP used for memory management in C++ programsH1H2 [Sioud 06]H3 Some disadvantages for using AOP in testThreats to ValidityConclusion [Mets¨ 07, Farhat 10] a 5 / 16
  9. 9. ACRE: AnAutomated Aspect ApproachCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram Adams Domain Specific LanguageBackgroundRelated Work DSL used to describe the test to createApproach Embedded in the source code, but does not modify itsDSLACRE behavior nor its compilationHypothesesResults Four types of aspects supported : counter, checking,H1H2 timing and loggingH3 Few simple rules (see Table II named DSL Synthesis)Threats to ValidityConclusion 6 / 16
  10. 10. ACRE: AnAutomated Aspect ApproachCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e ACRE: What is it ? Gu´h´neuc and e e Bram Adams Automated aspeCt cREatorBackground Input: C++ source code containing aspect definitionsRelated Work Output: generated aspectsApproachDSLACRE ACRE: How does it work ?HypothesesResults Implements a parser for each aspect typeH1H2 Easy to add new types of aspectH3Threats to Validity ACRE: Where to find ?Conclusion http://web.soccerlab.polymtl.ca/~ducloset/ACRE Under the GPL license 7 / 16
  11. 11. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram AdamsBackgroundRelated WorkApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 8 / 16
  12. 12. ACRE: AnAutomated Aspect ApproachCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Hypotheses Digabel, Yann-Ga¨l e H1: Generated aspects can be used to do memory Gu´h´neuc and e e Bram Adams testing for C++ programs.Background H2: Generated aspects can be used to do invariantRelated Work testing for C++ programs.ApproachDSL H3: Generated aspects can be used to help findingACREHypotheses interference bugs in C++ programs.ResultsH1H2H3Threats to ValidityConclusion 9 / 16
  13. 13. ACRE: AnAutomated Aspect ApproachCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Hypotheses Digabel, Yann-Ga¨l e H1: Generated aspects can be used to do memory Gu´h´neuc and e e Bram Adams testing for C++ programs.Background H2: Generated aspects can be used to do invariantRelated Work testing for C++ programs.ApproachDSL H3: Generated aspects can be used to help findingACREHypotheses interference bugs in C++ programs.ResultsH1H2 Tested program: NOMADH3 C++ program [Audet 06, Le Digabel 11] used in bothThreats to Validity research and industryConclusion 49 classes, 48k LOC Under the LGPL license, downloaded more than 3,000 times since 2008 http://www.gerad.ca/nomad 9 / 16
  14. 14. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Results: H1 Yann-Ga¨l e Gu´h´neuc and e e H1: Generated aspects can be used to do memory Bram Adams testing for C++ programsBackground Bug report: memory leak on NOMADRelated WorkApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 10 / 16
  15. 15. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Results: H1 Yann-Ga¨l e Gu´h´neuc and e e H1: Generated aspects can be used to do memory Bram Adams testing for C++ programsBackground Bug report: memory leak on NOMADRelated WorkApproachDSL Injected linesACREHypotheses //// name : EVPCGResultsH1 //// type : counterH2 //// class : Eval PointH3 //// namespace : NOMADThreats to ValidityConclusion 10 / 16
  16. 16. ACRE: AnAutomated Aspect Generated counter aspect for memory testingCreator for TestingC++ Applications a s p e c t EVPCG{ Etienne Duclos, S´bastien Le e public : static int Eval PointCount ; Digabel, Yann-Ga¨l e pointcut Eval PointCounted () = Gu´h´neuc and e e ”NOMAD: : E v a l P o i n t ” ; Bram AdamsBackground advice Eval PointCounted () : s l i c e struct {Related Work class Eval PointCount { public : Eval PointCount (){ApproachDSL EVPCG : : E v a l P o i n t C o u n t ++;}ACRE public : ˜ Eval PointCount (){Hypotheses EVPCG : : E v a l P o i n t C o u n t −−;}Results } Eval Point counter ;H1H2 };H3 a d v i c e e x e c u t i o n ( ”% main ( . . . ) ”) : a f t e r ( ) {Threats to Validity p r i n t f ( ”F i n a l count of E v a l P o i n t :Conclusion %dn ” , E v a l P o i n t C o u n t ) ; i f ( Eval PointCount > 0) p r i n t f ( ”Memory l e a k ! ! n ” ) ; } }; 11 / 16
  17. 17. ACRE: AnAutomated Aspect Results: H1Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram AdamsBackgroundRelated WorkApproachDSLACREHypothesesResults Figure: NOMAD output with a counter aspect weavedH1H2H3Threats to ValidityConclusion 12 / 16
  18. 18. ACRE: AnAutomated Aspect Results: H1Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram AdamsBackgroundRelated WorkApproachDSLACREHypothesesResults Figure: NOMAD output with a counter aspect weavedH1H2H3Threats to Validity Memory leak foundedConclusion Leak fixed by the NOMAD team of developers Generated aspects can be used to do memory testing for C++ programs 12 / 16
  19. 19. ACRE: AnAutomated Aspect Results: H2Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H2: Generated aspects can be used to do invariant Gu´h´neuc and e e Bram Adams testing for C++ programsBackground Goal: to verify a crucial mathematical point of theRelated Work MADS algorithmApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 13 / 16
  20. 20. ACRE: AnAutomated Aspect Results: H2Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H2: Generated aspects can be used to do invariant Gu´h´neuc and e e Bram Adams testing for C++ programsBackground Goal: to verify a crucial mathematical point of theRelated Work MADS algorithmApproachDSLACREHypotheses (Un)fortunately no bug found for the 37 tested problemsResults A developer from the NOMAD team mutated NOMADH1H2 source code to have an errorH3Threats to Validity Bug founded on the mutated versionConclusion 13 / 16
  21. 21. ACRE: AnAutomated Aspect Results: H2Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H2: Generated aspects can be used to do invariant Gu´h´neuc and e e Bram Adams testing for C++ programsBackground Goal: to verify a crucial mathematical point of theRelated Work MADS algorithmApproachDSLACREHypotheses (Un)fortunately no bug found for the 37 tested problemsResults A developer from the NOMAD team mutated NOMADH1H2 source code to have an errorH3Threats to Validity Bug founded on the mutated versionConclusion Generated aspects can be used to do invariant testing for C++ programs 13 / 16
  22. 22. ACRE: AnAutomated Aspect Results: H3Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H3: Generated aspects can be used to help finding Gu´h´neuc and e e Bram Adams interference bugs in C++ programsBackground Goal: to verify the parallel version of NOMADRelated WorkApproachDSLACREHypothesesResultsH1H2H3Threats to ValidityConclusion 14 / 16
  23. 23. ACRE: AnAutomated Aspect Results: H3Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H3: Generated aspects can be used to help finding Gu´h´neuc and e e Bram Adams interference bugs in C++ programsBackground Goal: to verify the parallel version of NOMADRelated WorkApproachDSL Timing aspects give access times for both read andACREHypotheses write eventsResults Times used by Bhattacharya et al. approachH1H2 [Bhattacharya 12]H3Threats to Validity Result: it is possible that an interference bug occursConclusion 14 / 16
  24. 24. ACRE: AnAutomated Aspect Results: H3Creator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e H3: Generated aspects can be used to help finding Gu´h´neuc and e e Bram Adams interference bugs in C++ programsBackground Goal: to verify the parallel version of NOMADRelated WorkApproachDSL Timing aspects give access times for both read andACREHypotheses write eventsResults Times used by Bhattacharya et al. approachH1H2 [Bhattacharya 12]H3Threats to Validity Result: it is possible that an interference bug occursConclusion Generated aspects can be used to help finding interference bugs in C++ programs 14 / 16
  25. 25. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Threats to Validity Digabel, Yann-Ga¨l e Internal validity : Founded bugs may be caused by the Gu´h´neuc and e e Bram Adams added aspectsBackground External validity : Approach used only under LinuxRelated Work Reliability validity : All the programs are available onApproachDSL the InternetACREHypothesesResultsH1H2H3Threats to ValidityConclusion 15 / 16
  26. 26. ACRE: AnAutomated AspectCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Threats to Validity Digabel, Yann-Ga¨l e Internal validity : Founded bugs may be caused by the Gu´h´neuc and e e Bram Adams added aspectsBackground External validity : Approach used only under LinuxRelated Work Reliability validity : All the programs are available onApproachDSL the InternetACREHypotheses Future WorkResultsH1 Generate more types of aspectsH2H3 Extend ACRE to test other languagesThreats to ValidityConclusion Use ACRE on bigger programs 15 / 16
  27. 27. ACRE: AnAutomated Aspect ConclusionCreator for TestingC++ Applications Allow the use of AOP to do memory, invariant and Etienne Duclos, S´bastien Le e interference testing in C++ programs Digabel, Yann-Ga¨l e No modifications in the source code of the tested Gu´h´neuc and e e Bram Adams programBackground Allow the use of aspect without knowledge in AOPRelated WorkApproachDSLACRE Thank you for your attentionHypothesesResults Any questions ?H1H2H3Threats to ValidityConclusion 16 / 16
  28. 28. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Bram Adams C. Audet & J.E. Dennis, Jr.Background Mesh Adaptive Direct Search Algorithms for ConstrainedRelated Work Optimization.Approach SIAM Journal on Optimization, vol. 17, no. 1, pagesDSLACRE 188–217, 2006.HypothesesResults Yosi Ben-Asher, Eitan Farchi & Yaniv Eytani.H1H2 Heuristics for Finding Concurrent Bugs.H3 In Proceedings of the 17th International Symposium onThreats to Validity Parallel and Distributed Processing, pages 288.1–, 2003.Conclusion 17 / 16
  29. 29. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Neelesh Bhattacharya, Olfat El-Mahi, Etienne Duclos, Gu´h´neuc and e e Bram Adams Giovanni Beltrame, Giuliano Antoniol, S´bastien eBackground Le Digabel & Yann-Ga¨l Gu´h´neuc. e e eRelated Work Optimizing Threads Schedule Alignments to Expose theApproach Interference Bug Pattern.DSLACRE In Proceedings of the 4th edition of the Symposium onHypotheses Search Based Software Engineering (SSBSE), 2012.ResultsH1H2 Hans-J. Boehm.H3 Advantages and Disadvantages of Conservative GarbageThreats to Validity Collection, 2012.Conclusion tir´ de http://www.hpl.hp.com/personal/Hans_ e Boehm/gc/issues.html, consult´ le 06 juin 2012. e 18 / 16
  30. 30. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Orit Edelstein, Eitan Farchi, Evgeny Goldin, Yarden Nir, Bram Adams Gil Ratsaby & Shmuel Ur.Background Framework for testing multi-threaded Java programs.Related Work Concurrency and Computation: Practice andApproachDSL Experience, pages 485–499, 2003.ACREHypotheses Michael D. Ernst, Jeff H. Perkins, Philip J. Guo,Results Stephen Mccamant, Carlos Pacheco, Matthew S.H1H2 Tschantz & Chen Xiao.H3Threats to Validity The Daikon system for dynamic detection of likelyConclusion invariants. In Science of Computer Programming, 2006. 19 / 16
  31. 31. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Salam Farhat, Greg Simco & Frank J. Mitropoulos. Bram Adams Using Aspects for Testing Nonfunctional RequirementsBackground in Object-Oriented Systems.Related Work In Proceedings of the IEEE SoutheastCon 2010, pagesApproachDSL 356–359, 2010.ACREHypotheses Pallavi Joshi, Mayur Naik, Chang-seo Park & KoushikResults Sen.H1H2 CalFuzzer: An Extensible Active Testing Framework forH3 Concurrent Programs.Threats to ValidityConclusion In Proceedings of the 21st International Conference on Computer Aided Verification, pages 675–681, 2009. 20 / 16
  32. 32. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e Gu´h´neuc and e e Peter Knauber & Johannes Schneider. Bram Adams Tracing Variability from Implementation to Test UsingBackground Aspect-Oriented Programming.Related Work International Workshop on Software Product LineApproachDSL Testing (SPLiT 2004), Boston, Massachusetts, USA,ACREHypotheses 2004.Results S. Le Digabel.H1H2 Algorithm 909: NOMAD: Nonlinear Optimization withH3 the MADS algorithm.Threats to Validity ACM Transactions on Mathematical Software, vol. 44,Conclusion pages 1–15, 2011. 21 / 16
  33. 33. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Xiaofei Li & Xusheng Xie. Yann-Ga¨l e Gu´h´neuc and e e Research of software testing based on AOP. Bram Adams In Proceedings of the 3rd International Conference onBackground Intelligent Information Technology Application, pagesRelated Work 187–189, 2009.ApproachDSL Jani Mets¨, Mika Katara & Tommi Mikkonen. aACREHypotheses Testing Non-Functional Requirements with Aspects: AnResults Industrial Case Study.H1H2 In Proceedings of the Seventh International ConferenceH3Threats to Validity on Quality Software, pages 5–14, 2007.Conclusion Madanlal Musuvathi, Shaz Qadeer & Thomas Ball. CHESS: A Systematic Testing Tool for Concurrent Software, 2007. 22 / 16
  34. 34. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Jani Pesonen, Mika Katara & Tommi Mikkonen. Yann-Ga¨l e Gu´h´neuc and e e Production-Testing of Embedded Systems with Aspects. Bram Adams In Haifa Verification Conference, pages 90–102, 2005.Background Jani Pesonen.Related Work Extending Software Integration Testing Using Aspects inApproachDSL Symbian OS.ACREHypotheses In Proceedings of the Testing: Academic & IndustrialResults Conference on Practice and Research Techniques, pagesH1H2 147–151, 2006.H3Threats to Validity Aymen Sioud.Conclusion Gestion de cycle de vie des objets par aspects pour C++. Master’s thesis, Universit´ de Qu´bec ` Chicoutimi, e e a 2006. 23 / 16
  35. 35. ACRE: AnAutomated Aspect ReferencesCreator for TestingC++ Applications Etienne Duclos, S´bastien Le e Digabel, Yann-Ga¨l e UOIT Software Quality Research Group Faculty Gu´h´neuc and e e Bram Adams of Science.Background Concurrency Anti-pattern Catalog for Java, 2012.Related Work tir´ de http: eApproach //faculty.uoit.ca/bradbury/concurr-catalog/,DSLACRE consult´ le 06 juin 2012. eHypothesesResults Guoqing Xu, Zongyuan Yang, Haitao Huang, QianH1H2 Chen, Ling Chen & Fengbin Xu.H3 JAOUT: Automated Generation of Aspect-Oriented UnitThreats to Validity Test.Conclusion In Proceedings of the 11th Asia-Pacific Software Engineering Conference, pages 374–381, 2004. 24 / 16

×