Open BQR a framework for the assessment of Open Source Software Davide Taibi - Luigi Lavazza - Sandro Morasca University of Insubria

Goals

O.S.S. Comparison Ideology Opinion “ religion” issue Assessment Errors Big Projects +

Experience: inaccurate Existing methods: very recent No methods yet consider the following: Internal qualities (eg. code size, complexity) External qualities (eg. fault proneness) Support availability in the future Current situation

Experience: inaccurate

Existing methods: very recent

No methods yet consider the following:

Internal qualities (eg. code size, complexity)

External qualities (eg. fault proneness)

Support availability in the future

Goals COMPARISON MODEL simple “ formal” Fast qualification and selection FUNDAMENTAL ASPECTS - Functional coverage - Number of bugs & time needed to solve - Availability of future support - Other aspects

COMPARISON MODEL

simple

“ formal”

FUNDAMENTAL ASPECTS

- Functional coverage

- Number of bugs & time needed to solve - Availability of future support

- Other aspects

Preliminary analysis O.S. Communities Contacts GUFPI-ISMA Software Metrics Java Open Business International Function Point User Group (IFPUG) CMMi Process Improvement Community Software Metrics Function Points Cosmic FFP COCOMO Existing OSS comparison methods Open Source Maturity Model Open Business Readiness Ratings Qualification and Selection of O.S. Software Forum Workshop Surveys

O.S. Communities Contacts

GUFPI-ISMA Software Metrics

Java Open Business

International Function Point User Group (IFPUG)

CMMi Process Improvement Community

Software Metrics

Function Points

Cosmic FFP

COCOMO

Existing OSS comparison methods

Open Source Maturity Model

Open Business Readiness Ratings

Qualification and Selection of O.S. Software

Forum

Workshop

Surveys

Open BQR OSMM Open Source Maturity Model Open BRR Open Business Readiness Rating + QSOS Qualification and Selection of O.S. Software + Open BQR Open Business Quality Rating

OPEN SOURCE SOFTWARE COMPARISON Selection Parameter evaluation Classification v v v x x x x Open BQR

OPEN SOURCE SOFTWARE COMPARISON

Selection

Parameter evaluation

Classification

Open BQR – method 1. Quick Assessment Filter 2. Data Collection & Processing 3. Data Translation

1. Quick Assessment Filter

2. Data Collection & Processing

3. Data Translation

Open BQR – method License Compliance with standards Implementation language Internationalization support User’s manual availability Supervised by analysts Specification analysis Functional coverage assessment Indicator Analysis Target Usage parameters 1.Quick Assessment Filter

License

Compliance with standards

Implementation language

Internationalization support

User’s manual availability

Supervised by analysts

Specification analysis

Functional coverage assessment

Indicator Analysis

Target Usage parameters

Open BQR – method Specification analysis Functional coverage assessment Indicator Analysis Target Usage parameters Bug database analysis Closed bugs/ bug number Bug time correction Donations/bug number 1.Quick Assessment Filter

Specification analysis

Functional coverage assessment

Indicator Analysis

Target Usage parameters

Bug database analysis

Closed bugs/ bug number

Bug time correction

Donations/bug number

Open BQR – method Specification analysis Functional coverage assessment Indicator Analysis Target Usage parameters Bug database analysis Internal qualities analysis Complexity (Mc Cabe) Reuse Dependencies 1.Quick Assessment Filter

Specification analysis

Functional coverage assessment

Indicator Analysis

Target Usage parameters

Bug database analysis

Internal qualities analysis

Complexity (Mc Cabe)

Reuse

Dependencies

Open BQR – method Specification analysis Functional coverage assessment Indicator Analysis Target Usage parameters Bug database analysis Internal qualities analysis Community activity analysis Number of release Number of company supporting the product Average number of developers per company Independent developers 1.Quick Assessment Filter

Specification analysis

Functional coverage assessment

Indicator Analysis

Target Usage parameters

Bug database analysis

Internal qualities analysis

Community activity analysis

Number of release

Number of company supporting the product

Average number of developers per company

Independent developers

Open BQR – method Quick Assessment Filter Data Collection & Processing Data Translation Target usage parameters External qualities Functionality Cost Support Specification analysis Rank of indicators Weight normalization Indicator assessment Final score calculation Result plotting Internal qualities

Quick Assessment Filter

Data Collection & Processing

Data Translation

Specification analysis

Rank of indicators

Weight normalization

Indicator assessment

Final score calculation

Result plotting

Test Drupal Mambo WebGUI CMS ERP Compiere

Open BQR – CMS Custom requirements Personal web site: User-defined layout Creation of new public or hidden page by the user Image gallery Donwloads Upload of files and images via browser Administration interface, better if in the user’s language

Personal web site:

User-defined layout

Creation of new public or hidden page by the user

Image gallery

Donwloads

Upload of files and images via browser

Administration interface, better if in the user’s language

Open BQR – CMS INITIAL CHOICE MAMBO DRUPAL WEB GUI FUNCTIONAL COVERAGE CHECK Customer indicator weight 1. User-defined layout 10/10 2. CRUD (Create, Read, Update, Delete) pages 10/10 3. Image gallery 7/10 4. CRUD (Create, Read, Update, Delete) files and download page 5/10 5. Italian interface supported 5/10

INITIAL CHOICE

MAMBO

DRUPAL

WEB GUI

FUNCTIONAL COVERAGE CHECK

Customer indicator weight

1. User-defined layout 10/10

2. CRUD (Create, Read, Update, Delete) pages 10/10

3. Image gallery 7/10

4. CRUD (Create, Read, Update, Delete) files and download page 5/10

5. Italian interface supported 5/10

Open BQR – CMS Parameter Assessment - Mambo Indicator Weight Normalized weight Score Final Score Target usage assessment License 10 7,35 10 7,35% Respect of standards 5 3,68 8 2,94% Implementation language 8 5,88 20 11,76% Internationalization support 5 3,68 8 2,94% Technical documentation availability 10 7,35 7 5,15% Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS Parameter Assessment - Mambo Indicator Weight Normalized weight Score Final Score Target usage assessment License 10 7,35 10 7,35% Respect of standards 5 3,68 8 2,94% Implementation language 8 5,88 20 11,76% Internationalization support 5 3,68 8 2,94% Technical documentation availability 10 7,35 7 5,15% Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS Parameter Assessment - Mambo Indicator Weight Normalized weight Score Final Score Target usage assessment License 10 7,35 10 7,35% Respect of standards 5 3,68 8 2,94% Implementation language 8 5,88 20 11,76% Internationalization support 5 3,68 8 2,94% Technical documentation availability 10 7,35 7 5,15% Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS Parameter Assessment - Mambo Indicator Weight Normalized weight Score Final Score Target usage assessment License 10 7,35 10 7,35% Respect of standards 5 3,68 8 2,94% Implementation language 8 5,88 20 11,76% Internationalization support 5 3,68 8 2,94% Technical documentation availability 10 7,35 7 5,15% Supervised by analysts 2 1,47 0 0,00%

Open BQR – CMS Functionality Support availability External qualities Evaluation Target usage indicators

Tool

Conclusions & future work Apply Open BQR to CMS projects Refine/extend the set of parameters that are relevant for the evaluation Reduce subjectivity as much as possible Disseminate the method among the O.S. users and developers

Apply Open BQR to CMS projects

Refine/extend the set of parameters that are relevant for the evaluation

Reduce subjectivity as much as possible

Disseminate the method among the O.S. users and developers

Q & A THANKS. QUESTIONS? Open BQR