Understanding Plug-in Test Suites from an Integration Perspective         Michaela Greiler, Arie van Deursen        Softwa...
Understanding Plug-in Test Suites
Need more Information?  Which Information?
Code Dependencies   which plug-ins are using each other Service Relationswhich plug-ins are using services each otherExten...
Interviewed 25 Eclipse practitioners                                  Understanding                                    tes...
Interviewed 25 Eclipse practitioners                                            Understanding                             ...
Interviewed 25 Eclipse practitioners                                Understanding                                  test su...
Interviewed 25 Eclipse practitioners                                     Understanding                                    ...
Interviewed 25 Eclipse practitioners                                    Understanding                                     ...
Which plug-ins are                             How are the plug-intested by which test-                            extensi...
Manual Inspection!
Look at…     … extension relations       Only Static           …code
Understanding Plug-in Test Suites from an Integration Perspective
5 Architectural View                                                        Extension        Service   Plug-in         Tes...
Ingredients: Data Behind                   Static Data & Dynamic Trace Data        plugin.xml & schema                    ...
Recipe: Reconstructing the Extension Usage ViewSTATIC: plug-in.xml, schema, .class                        DYNAMIC: Method ...
ETSE: Eclipse Test Suite Explorer
Empirical Evaluation
Case Study Set-Up                                             Connector     27 plug-ins       5 main plug-ins    3 main pl...
ETSE: Eclipse Test Suite Explorer
Static                        Dynamic Which plug-ins aretested by which test-     component?
Dynamic & Static            System Level                              How are the plug-in                           extens...
Plug-in Level                                Dynamic & Static                             How are the plug-in             ...
When and how have                         the extensions been                                used?Which extensions areused...
ETSE: Eclipse Test Suite Explorer
Want to use ETSE in your project?                 Contactus!                m.s.greiler@tudelft.nl             arie.vandeu...
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011
Upcoming SlideShare
Loading in …5
×

Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011

1,039 views
952 views

Published on

Testing plug-in based systems entails testing a complex net of multiple plug-ins
which not only interact with, but also enhance the functionality of each other by extension mechanisms.

In the Eclipse Testing Study, testing practices of successful Eclipse projects have been studied. Among others, Eclipsers stress integration testing as an important testing activity for plug-in systems, and explain that understanding complex plugin test suites can become a challenging task.

To remedy the problem of understanding plug-in test suites, we developed the
Eclipse Plug-in Test Suite Exploration (ETSE) tool. ETSE combines static and
dynamic information on plug-in dependencies, extension initialization, and
extension or service usage during a test run. This information is used to create five
architectural views of the system under test and its test suite, which developers
can use to understand how the integration of multiple plug-ins is tested.

In this presentation, we will talk about plug-in test suites and present the architectural views, which help answering questions like: “which plug-ins are tested by this test suite?”, “where are test harness and test utilities located?”, “which extensions influence the state of the test system?”, and many more.

We will end this session with a short demonstration of ETSE by visualizing the test suite of Mylyn.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,039
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Understanding Eclipse Plug-in Test Suites @ The Eclipse Testing Day 2011

  1. 1. Understanding Plug-in Test Suites from an Integration Perspective Michaela Greiler, Arie van Deursen Software Engineering Research Group Delft University of Technology
  2. 2. Understanding Plug-in Test Suites
  3. 3. Need more Information? Which Information?
  4. 4. Code Dependencies which plug-ins are using each other Service Relationswhich plug-ins are using services each otherExtension Relations which plug-ins are extending each other
  5. 5. Interviewed 25 Eclipse practitioners Understanding test cases Understanding test suites
  6. 6. Interviewed 25 Eclipse practitioners Understanding test cases Coding standards, naming conventions. Explanation and Motivation. “Tests are also like other parts of code. Sometimes people give bad names to their methods and variables. Then it is hard to understand.”
  7. 7. Interviewed 25 Eclipse practitioners Understanding test suites“What one specific test does, that’s quiteeasy to understand. What’s difficult is to seewhere the blank spots on the map are.”
  8. 8. Interviewed 25 Eclipse practitioners Understanding test suites Challenges keeping an Overview: “It is difficult. You have to read it all. If you skip one test, you do not know one part of the system.” Test Organization and Structure. Nested Test Suites.
  9. 9. Interviewed 25 Eclipse practitioners Understanding test suites “We realized: either we work on a particular point, and we run only one test case, or we run all of them. We worry very often about not being able to run the test suite in a more fine-grained way..”
  10. 10. Which plug-ins are How are the plug-intested by which test- extensions and services component? tested? How are the plug-ins extending each other? Which extensions are used during the tests?
  11. 11. Manual Inspection!
  12. 12. Look at… … extension relations Only Static …code
  13. 13. Understanding Plug-in Test Suites from an Integration Perspective
  14. 14. 5 Architectural View Extension Service Plug-in Test-Suite Extension Usage View Usage ViewModularization Modularization Initialization View View ViewPlug-in Level Test Case Level Extension Level Method Level Method Level Plug-in A Plug-in A Plug-in A Plug-in A Plug-in A is tested by loads invokes invokes uses test case t extension e method m of method m of Plug-in B of Plug-in B of Plug-in B extension e service s in plug-in B in plug-in B
  15. 15. Ingredients: Data Behind Static Data & Dynamic Trace Data plugin.xml & schema Test suite manifest class files class files Fact Extraction Instrumentation Facts Traces Reconstruction VIEWS Recipe: Reconstructing The Views
  16. 16. Recipe: Reconstructing the Extension Usage ViewSTATIC: plug-in.xml, schema, .class DYNAMIC: Method Calls Fact Extraction Instrumentation Facts Traces Reconstruction Extension Method Set: Heuristic (all methods of an extension) Inspect the Trace File Extension Usage View which “extension methods” are invoked
  17. 17. ETSE: Eclipse Test Suite Explorer
  18. 18. Empirical Evaluation
  19. 19. Case Study Set-Up Connector 27 plug-ins 5 main plug-ins 3 main plug-ins11 test-components 2 test components 1 test component 2 test suites 4 test suites 2 test suites 200,000 LOCs 100,000 LOCs 8,500 LOCs30,000 LOCs tests 14,000 lines tests 3,400 lines tests
  20. 20. ETSE: Eclipse Test Suite Explorer
  21. 21. Static Dynamic Which plug-ins aretested by which test- component?
  22. 22. Dynamic & Static System Level How are the plug-in extension-relations tested?How are the plug-insextending each other?
  23. 23. Plug-in Level Dynamic & Static How are the plug-in extension-relations tested?How are the plug-insextending each other?
  24. 24. When and how have the extensions been used?Which extensions areused during the tests?
  25. 25. ETSE: Eclipse Test Suite Explorer
  26. 26. Want to use ETSE in your project? Contactus! m.s.greiler@tudelft.nl arie.vandeursen@tudelft.nl Update site:http://swerl.tudelft.nl/etse/ETSE_UpdateSite/site.xml

×