1. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
ACRE: An Automated Aspect Creator for
Gu´h´neuc and
e e
Bram Adams Testing C++ Applications
Background
Related Work
Etienne Duclos, S´bastien Le Digabel,
e
Approach
DSL
Yann-Ga¨l Gu´h´neuc and Bram Adams
e e e
ACRE
Hypotheses
Department of Computer and Software Engineering
Results GERAD and Department of Mathematics and Industrial Engineering
H1
´
Ecole Polytechnique de Montr´al, Qu´bec, Canada
e e
H2
H3
Threats to Validity
CSMR
Conclusion
March 07, 2013
Pattern Trace Identification, Detection, and Enhancement in Java
Groupe d’Etudes et de Recherche en Analyse des D´cisions
e

2. ACRE: An
Automated Aspect Problem
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
Background
Related Work
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
2 / 16

3. ACRE: An
Automated Aspect Outline
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Background
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e Related Work
Bram Adams
Background
Approach
Related Work DSL
Approach ACRE
DSL
ACRE
Hypotheses
Hypotheses
Results Results
H1
H2
H1
H3
H2
Threats to Validity
H3
Conclusion
Threats to Validity
Conclusion
3 / 16

4. ACRE: An
Automated Aspect Background
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Aspect-Oriented Programming
Digabel,
Yann-Ga¨l
e Based on the concept of separation of concerns
Gu´h´neuc and
e e
Bram Adams
Used with OOP thanks to a weaver
Background Different execution models available
Related Work
Two major languages: AspectJ and AspectC++
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
4 / 16

5. ACRE: An
Automated Aspect Background
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Aspect-Oriented Programming
Digabel,
Yann-Ga¨l
e Based on the concept of separation of concerns
Gu´h´neuc and
e e
Bram Adams
Used with OOP thanks to a weaver
Background Different execution models available
Related Work
Two major languages: AspectJ and AspectC++
Approach
DSL
ACRE
Hypotheses Testing
Results
H1
Memory Management
H2 Difficult in C++ [Boehm 12], high risk of memory leak
H3
Threats to Validity
Conclusion
4 / 16

6. ACRE: An
Automated Aspect Background
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Aspect-Oriented Programming
Digabel,
Yann-Ga¨l
e Based on the concept of separation of concerns
Gu´h´neuc and
e e
Bram Adams
Used with OOP thanks to a weaver
Background Different execution models available
Related Work
Two major languages: AspectJ and AspectC++
Approach
DSL
ACRE
Hypotheses Testing
Results
H1
Memory Management
H2 Difficult in C++ [Boehm 12], high risk of memory leak
H3
Threats to Validity
Invariant Testing
Conclusion Allow to verify that a property is always true
4 / 16

7. ACRE: An
Automated Aspect Background
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Aspect-Oriented Programming
Digabel,
Yann-Ga¨l
e Based on the concept of separation of concerns
Gu´h´neuc and
e e
Bram Adams
Used with OOP thanks to a weaver
Background Different execution models available
Related Work
Two major languages: AspectJ and AspectC++
Approach
DSL
ACRE
Hypotheses Testing
Results
H1
Memory Management
H2 Difficult in C++ [Boehm 12], high risk of memory leak
H3
Threats to Validity
Invariant Testing
Conclusion Allow to verify that a property is always true
Interference Bug Testing
Very common bug, and one of the most difficult to
remove [Software Quality Research Group 12]
4 / 16

8. ACRE: An
Automated Aspect Related Work
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams AOP in test
Background AOP used in Java for unit and integration
Related Work testing [Xu 04, Knauber 04, Li 09]
Approach
DSL
AOP used to test embedded C++ programs
ACRE [Pesonen 05, Pesonen 06, Mets¨ 07]
a
Hypotheses
Results AOP used for memory management in C++ programs
H1
H2
[Sioud 06]
H3
Some disadvantages for using AOP in test
Threats to Validity
Conclusion
[Mets¨ 07, Farhat 10]
a
5 / 16

9. ACRE: An
Automated Aspect Approach
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
Domain Specific Language
Background
Related Work
DSL used to describe the test to create
Approach Embedded in the source code, but does not modify its
DSL
ACRE behavior nor its compilation
Hypotheses
Results
Four types of aspects supported : counter, checking,
H1
H2
timing and logging
H3
Few simple rules (see Table II named DSL Synthesis)
Threats to Validity
Conclusion
6 / 16

10. ACRE: An
Automated Aspect Approach
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
ACRE: What is it ?
Gu´h´neuc and
e e
Bram Adams Automated aspeCt cREator
Background Input: C++ source code containing aspect definitions
Related Work Output: generated aspects
Approach
DSL
ACRE ACRE: How does it work ?
Hypotheses
Results
Implements a parser for each aspect type
H1
H2
Easy to add new types of aspect
H3
Threats to Validity
ACRE: Where to find ?
Conclusion
http://web.soccerlab.polymtl.ca/~ducloset/ACRE
Under the GPL license
7 / 16

11. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
Background
Related Work
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
8 / 16

12. ACRE: An
Automated Aspect Approach
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Hypotheses
Digabel,
Yann-Ga¨l
e H1: Generated aspects can be used to do memory
Gu´h´neuc and
e e
Bram Adams testing for C++ programs.
Background H2: Generated aspects can be used to do invariant
Related Work testing for C++ programs.
Approach
DSL
H3: Generated aspects can be used to help finding
ACRE
Hypotheses
interference bugs in C++ programs.
Results
H1
H2
H3
Threats to Validity
Conclusion
9 / 16

13. ACRE: An
Automated Aspect Approach
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Hypotheses
Digabel,
Yann-Ga¨l
e H1: Generated aspects can be used to do memory
Gu´h´neuc and
e e
Bram Adams testing for C++ programs.
Background H2: Generated aspects can be used to do invariant
Related Work testing for C++ programs.
Approach
DSL
H3: Generated aspects can be used to help finding
ACRE
Hypotheses
interference bugs in C++ programs.
Results
H1
H2
Tested program: NOMAD
H3
C++ program [Audet 06, Le Digabel 11] used in both
Threats to Validity
research and industry
Conclusion
49 classes, 48k LOC
Under the LGPL license, downloaded more than 3,000
times since 2008 http://www.gerad.ca/nomad
9 / 16

14. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Results: H1
Yann-Ga¨l
e
Gu´h´neuc and
e e H1: Generated aspects can be used to do memory
Bram Adams
testing for C++ programs
Background Bug report: memory leak on NOMAD
Related Work
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
10 / 16

15. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Results: H1
Yann-Ga¨l
e
Gu´h´neuc and
e e H1: Generated aspects can be used to do memory
Bram Adams
testing for C++ programs
Background Bug report: memory leak on NOMAD
Related Work
Approach
DSL
Injected lines
ACRE
Hypotheses
//// name : EVPCG
Results
H1 //// type : counter
H2
//// class : Eval Point
H3
//// namespace : NOMAD
Threats to Validity
Conclusion
10 / 16

16. ACRE: An
Automated Aspect
Generated counter aspect for memory testing
Creator for Testing
C++ Applications
a s p e c t EVPCG{
Etienne Duclos,
S´bastien Le
e public : static int Eval PointCount ;
Digabel,
Yann-Ga¨l
e
pointcut Eval PointCounted () =
Gu´h´neuc and
e e ”NOMAD: : E v a l P o i n t ” ;
Bram Adams
Background advice Eval PointCounted () : s l i c e struct {
Related Work
class Eval PointCount {
public : Eval PointCount (){
Approach
DSL
EVPCG : : E v a l P o i n t C o u n t ++;}
ACRE public : ˜ Eval PointCount (){
Hypotheses
EVPCG : : E v a l P o i n t C o u n t −−;}
Results
} Eval Point counter ;
H1
H2 };
H3
a d v i c e e x e c u t i o n ( ”% main ( . . . ) ”) : a f t e r ( ) {
Threats to Validity p r i n t f ( ”F i n a l count of E v a l P o i n t :
Conclusion %dn ” , E v a l P o i n t C o u n t ) ;
i f ( Eval PointCount > 0)
p r i n t f ( ”Memory l e a k ! ! n ” ) ;
}
};
11 / 16

17. ACRE: An
Automated Aspect Results: H1
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
Background
Related Work
Approach
DSL
ACRE
Hypotheses
Results Figure: NOMAD output with a counter aspect weaved
H1
H2
H3
Threats to Validity
Conclusion
12 / 16

18. ACRE: An
Automated Aspect Results: H1
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
Background
Related Work
Approach
DSL
ACRE
Hypotheses
Results Figure: NOMAD output with a counter aspect weaved
H1
H2
H3
Threats to Validity
Memory leak founded
Conclusion
Leak fixed by the NOMAD team of developers
Generated aspects can be used to do memory testing
for C++ programs
12 / 16

19. ACRE: An
Automated Aspect Results: H2
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H2: Generated aspects can be used to do invariant
Gu´h´neuc and
e e
Bram Adams testing for C++ programs
Background
Goal: to verify a crucial mathematical point of the
Related Work MADS algorithm
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
13 / 16

20. ACRE: An
Automated Aspect Results: H2
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H2: Generated aspects can be used to do invariant
Gu´h´neuc and
e e
Bram Adams testing for C++ programs
Background
Goal: to verify a crucial mathematical point of the
Related Work MADS algorithm
Approach
DSL
ACRE
Hypotheses
(Un)fortunately no bug found for the 37 tested problems
Results A developer from the NOMAD team mutated NOMAD
H1
H2 source code to have an error
H3
Threats to Validity
Bug founded on the mutated version
Conclusion
13 / 16

21. ACRE: An
Automated Aspect Results: H2
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H2: Generated aspects can be used to do invariant
Gu´h´neuc and
e e
Bram Adams testing for C++ programs
Background
Goal: to verify a crucial mathematical point of the
Related Work MADS algorithm
Approach
DSL
ACRE
Hypotheses
(Un)fortunately no bug found for the 37 tested problems
Results A developer from the NOMAD team mutated NOMAD
H1
H2 source code to have an error
H3
Threats to Validity
Bug founded on the mutated version
Conclusion
Generated aspects can be used to do invariant testing
for C++ programs
13 / 16

22. ACRE: An
Automated Aspect Results: H3
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H3: Generated aspects can be used to help finding
Gu´h´neuc and
e e
Bram Adams interference bugs in C++ programs
Background
Goal: to verify the parallel version of NOMAD
Related Work
Approach
DSL
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
14 / 16

23. ACRE: An
Automated Aspect Results: H3
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H3: Generated aspects can be used to help finding
Gu´h´neuc and
e e
Bram Adams interference bugs in C++ programs
Background
Goal: to verify the parallel version of NOMAD
Related Work
Approach
DSL
Timing aspects give access times for both read and
ACRE
Hypotheses
write events
Results Times used by Bhattacharya et al. approach
H1
H2 [Bhattacharya 12]
H3
Threats to Validity
Result: it is possible that an interference bug occurs
Conclusion
14 / 16

24. ACRE: An
Automated Aspect Results: H3
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e H3: Generated aspects can be used to help finding
Gu´h´neuc and
e e
Bram Adams interference bugs in C++ programs
Background
Goal: to verify the parallel version of NOMAD
Related Work
Approach
DSL
Timing aspects give access times for both read and
ACRE
Hypotheses
write events
Results Times used by Bhattacharya et al. approach
H1
H2 [Bhattacharya 12]
H3
Threats to Validity
Result: it is possible that an interference bug occurs
Conclusion
Generated aspects can be used to help finding
interference bugs in C++ programs
14 / 16

25. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Threats to Validity
Digabel,
Yann-Ga¨l
e Internal validity : Founded bugs may be caused by the
Gu´h´neuc and
e e
Bram Adams added aspects
Background External validity : Approach used only under Linux
Related Work Reliability validity : All the programs are available on
Approach
DSL
the Internet
ACRE
Hypotheses
Results
H1
H2
H3
Threats to Validity
Conclusion
15 / 16

26. ACRE: An
Automated Aspect
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e Threats to Validity
Digabel,
Yann-Ga¨l
e Internal validity : Founded bugs may be caused by the
Gu´h´neuc and
e e
Bram Adams added aspects
Background External validity : Approach used only under Linux
Related Work Reliability validity : All the programs are available on
Approach
DSL
the Internet
ACRE
Hypotheses
Future Work
Results
H1 Generate more types of aspects
H2
H3
Extend ACRE to test other languages
Threats to Validity
Conclusion
Use ACRE on bigger programs
15 / 16

27. ACRE: An
Automated Aspect Conclusion
Creator for Testing
C++ Applications Allow the use of AOP to do memory, invariant and
Etienne Duclos,
S´bastien Le
e
interference testing in C++ programs
Digabel,
Yann-Ga¨l
e No modifications in the source code of the tested
Gu´h´neuc and
e e
Bram Adams program
Background
Allow the use of aspect without knowledge in AOP
Related Work
Approach
DSL
ACRE Thank you for your attention
Hypotheses
Results
Any questions ?
H1
H2
H3
Threats to Validity
Conclusion
16 / 16

28. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e
Bram Adams
C. Audet & J.E. Dennis, Jr.
Background Mesh Adaptive Direct Search Algorithms for Constrained
Related Work Optimization.
Approach SIAM Journal on Optimization, vol. 17, no. 1, pages
DSL
ACRE 188–217, 2006.
Hypotheses
Results Yosi Ben-Asher, Eitan Farchi & Yaniv Eytani.
H1
H2
Heuristics for Finding Concurrent Bugs.
H3
In Proceedings of the 17th International Symposium on
Threats to Validity
Parallel and Distributed Processing, pages 288.1–, 2003.
Conclusion
17 / 16

29. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e Neelesh Bhattacharya, Olfat El-Mahi, Etienne Duclos,
Gu´h´neuc and
e e
Bram Adams Giovanni Beltrame, Giuliano Antoniol, S´bastien
e
Background
Le Digabel & Yann-Ga¨l Gu´h´neuc.
e e e
Related Work
Optimizing Threads Schedule Alignments to Expose the
Approach Interference Bug Pattern.
DSL
ACRE
In Proceedings of the 4th edition of the Symposium on
Hypotheses
Search Based Software Engineering (SSBSE), 2012.
Results
H1
H2
Hans-J. Boehm.
H3 Advantages and Disadvantages of Conservative Garbage
Threats to Validity
Collection, 2012.
Conclusion
tir´ de http://www.hpl.hp.com/personal/Hans_
e
Boehm/gc/issues.html, consult´ le 06 juin 2012.
e
18 / 16

30. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e Orit Edelstein, Eitan Farchi, Evgeny Goldin, Yarden Nir,
Bram Adams
Gil Ratsaby & Shmuel Ur.
Background Framework for testing multi-threaded Java programs.
Related Work Concurrency and Computation: Practice and
Approach
DSL
Experience, pages 485–499, 2003.
ACRE
Hypotheses Michael D. Ernst, Jeff H. Perkins, Philip J. Guo,
Results Stephen Mccamant, Carlos Pacheco, Matthew S.
H1
H2 Tschantz & Chen Xiao.
H3
Threats to Validity
The Daikon system for dynamic detection of likely
Conclusion
invariants.
In Science of Computer Programming, 2006.
19 / 16

31. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e Salam Farhat, Greg Simco & Frank J. Mitropoulos.
Bram Adams
Using Aspects for Testing Nonfunctional Requirements
Background in Object-Oriented Systems.
Related Work In Proceedings of the IEEE SoutheastCon 2010, pages
Approach
DSL
356–359, 2010.
ACRE
Hypotheses Pallavi Joshi, Mayur Naik, Chang-seo Park & Koushik
Results Sen.
H1
H2 CalFuzzer: An Extensible Active Testing Framework for
H3
Concurrent Programs.
Threats to Validity
Conclusion
In Proceedings of the 21st International Conference on
Computer Aided Verification, pages 675–681, 2009.
20 / 16

32. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e
Gu´h´neuc and
e e Peter Knauber & Johannes Schneider.
Bram Adams
Tracing Variability from Implementation to Test Using
Background Aspect-Oriented Programming.
Related Work
International Workshop on Software Product Line
Approach
DSL
Testing (SPLiT 2004), Boston, Massachusetts, USA,
ACRE
Hypotheses
2004.
Results S. Le Digabel.
H1
H2 Algorithm 909: NOMAD: Nonlinear Optimization with
H3
the MADS algorithm.
Threats to Validity
ACM Transactions on Mathematical Software, vol. 44,
Conclusion
pages 1–15, 2011.
21 / 16

33. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel, Xiaofei Li & Xusheng Xie.
Yann-Ga¨l
e
Gu´h´neuc and
e e Research of software testing based on AOP.
Bram Adams
In Proceedings of the 3rd International Conference on
Background Intelligent Information Technology Application, pages
Related Work
187–189, 2009.
Approach
DSL Jani Mets¨, Mika Katara & Tommi Mikkonen.
a
ACRE
Hypotheses Testing Non-Functional Requirements with Aspects: An
Results Industrial Case Study.
H1
H2 In Proceedings of the Seventh International Conference
H3
Threats to Validity
on Quality Software, pages 5–14, 2007.
Conclusion Madanlal Musuvathi, Shaz Qadeer & Thomas Ball.
CHESS: A Systematic Testing Tool for Concurrent
Software, 2007.
22 / 16

34. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Jani Pesonen, Mika Katara & Tommi Mikkonen.
Yann-Ga¨l
e
Gu´h´neuc and
e e
Production-Testing of Embedded Systems with Aspects.
Bram Adams In Haifa Verification Conference, pages 90–102, 2005.
Background
Jani Pesonen.
Related Work
Extending Software Integration Testing Using Aspects in
Approach
DSL
Symbian OS.
ACRE
Hypotheses
In Proceedings of the Testing: Academic & Industrial
Results Conference on Practice and Research Techniques, pages
H1
H2
147–151, 2006.
H3
Threats to Validity
Aymen Sioud.
Conclusion
Gestion de cycle de vie des objets par aspects pour
C++.
Master’s thesis, Universit´ de Qu´bec ` Chicoutimi,
e e a
2006.
23 / 16

35. ACRE: An
Automated Aspect References
Creator for Testing
C++ Applications
Etienne Duclos,
S´bastien Le
e
Digabel,
Yann-Ga¨l
e UOIT Software Quality Research Group Faculty
Gu´h´neuc and
e e
Bram Adams of Science.
Background
Concurrency Anti-pattern Catalog for Java, 2012.
Related Work tir´ de http:
e
Approach //faculty.uoit.ca/bradbury/concurr-catalog/,
DSL
ACRE
consult´ le 06 juin 2012.
e
Hypotheses
Results
Guoqing Xu, Zongyuan Yang, Haitao Huang, Qian
H1
H2
Chen, Ling Chen & Fengbin Xu.
H3 JAOUT: Automated Generation of Aspect-Oriented Unit
Threats to Validity Test.
Conclusion
In Proceedings of the 11th Asia-Pacific Software
Engineering Conference, pages 374–381, 2004.
24 / 16