Ensuring the sustainability of Open Source projects is important, especially when organized in ecosystems of interrelated projects. Generic methodologies like SQALE or QualOSS describe how to assess maturity covering product, process and community aspects. Specific ecosystems have defined their own quality model, like OW2’s SQUaT Mature Transition Checklist. Such quality assessment of Open Source Ecosystems should be efficiently supported. In this talk we will present the CHOOSE toolset we have developed to help in automating and uniformly managing the quality of a whole Open Source Ecosystems. Our toolset is based on 35 key indicators related to maintainability, reliability and community which are used to implement and automate the production of OW2’s Mature Transition Checklist. The generic architecture of the toolset will be presented how it is instantiated to implement the OW2 Mature Transition Checklist and to efficiently interface with the OW2 forge. The toolset implementation will also be detailed stressing the use of Open Source components for the workflows' orchestration (Bonita), project/code analysis tools (HtmlUnit, QualOSS) and automatic report generation tools (JasperReports). Finally, a validation on a subset of 21 OW2 projects with an in-depth focus on a few projects (Chameleon, Spago and Orchestra) to produce maturity reports.

1. Automating OW2 SQuAT Assessment and Reporting using Workflow and Metric Collection Tools
6/11/2014
Jean-Christophe Deprez, Durieux Dimitri

2. CETIC in a few words
Research projects
Feasibility studies
Innovation
Applied research
Proof-of-concept
Technology transfer

3. CHOOSE in few words
CHOOSE (2012) :
–
Application of QualOSS Methodology
–
Targeting OW2’s Mature Transition Checklist (MTC)
–
Automated analysis and reporting
–
For Java only
–
Workflow oriented approach

4. Maturity Transition Checklist
Technical
•Source code, Documentation and Binaries
•Build maturity
•Source code repository and management
SQuAT (Software Quality Assurance Trustworthiness)
Community
•Commiters activity
•Dashboard completeness
•Project Activity

5. Mature Transition Checklist
Technical
•Source code, Documentation and Binaries
•Build maturity
•Source code repository and management
QualOSS adapted to SQuAT (with Community)
Community
•Commiters activity
•Dashboard completeness
•Project Activity

6. QualOSS Assessment Method
QualOSS is methodology to assess F/OSS endeavor
=> to construct assessment model :
–
Benchmarking of F/OSS for integration or services
–
Deciding of the migration frequency
–
Deciding of the setup of new endeavor to increase chance of success
–
Assess maturity of an F/OSS endeavor

7. QualOSS Assessment Method

8. OW2 QualOSS Instance
•
QualOSS defines a set of 35 indicators
–
Releases
–
Mailing List
–
Commits log
–
Bugs List
•
Evolution based indicators
–
Multiple major and minor releases
•
Indicators’ scoring system depend on thresholds
–
Calibration on 400 SourceForge projects
–
Specific calibrations made according to MTC

9. Analysis Process
Information and source code gathering
Analysis of source code and community
Reporting
or
Benchmarking

10. Report sample with Chameleon

11. Report sample with Chameleon

12. Conclusion
•
Tools support 35 maturity indicators
•
Tools validated on some OW2’s projects
•
The information gathering process has been validated on a larger number of projects
•
Generated reports are the same as OW2 MTC

13. Furtherworks
•
Continue QualOSS characteristics automation
•
Improve technology coverage
•
Increase number of indicators

14. Aéropôle de Charleroi-Gosselies
Rue des Frères Wright, 29/3
B-6041 Gosselies
info@cetic.be
www.cetic.be
Thank you
Merci
Jean-christophe.deprez@cetic.be dimitri.durieux@cetic.be