The document proposes using the ratio of communication (mailing list posts) to development activity (code commits) as an intensive metric to study the evolution of open source projects. It analyzes this ratio for 11 Apache Software Foundation projects of varying sizes, technologies, and maturity levels. Healthy projects generally had ratios between 3-8, while ratios outside this indicated potential stagnation. The ratio metric was found to be independent of project size and thus useful for comparing evolution across projects.

1. Intensive Metrics for the Study of the Evolution
of Open Source Projects: Case Studies from the
ASF
Santiago Gala-Pérez (ASF), Gregorio Robles (URJC),
Jesús M. González-Barahona (URJC), Israel Herraiz (UPM)
10th Working Conference on Mining Software Repositories
SF, California, May 18th, 2013
Preprint available at http://oa.upm.es/14698/
Slides at http://slideshare.net/herraiz/intensive-metrics-software-evolution
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
1/13

2. Metrics for Software Evolution
Common metrics are extensive
Difficult to compare projects of different size
Successful projects undergo large size changes over their lifetime
Intensive metrics in natural sciences
Metrics not depending on the size of system
Scale invariant
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
2/13

3. Metrics for Software Evolution
Common metrics are extensive
Difficult to compare projects of different size
Successful projects undergo large size changes over their lifetime
Intensive metrics in natural sciences
Metrics not depending on the size of system
Scale invariant
Are there any intensive metric for software?
Can we find intensive metrics to study software evolution?
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
2/13

4. The case of the Apache Software Foundation
ASF members mailing list, November 29 2008
Joe Schaeffer says
something IMO interesting about the ASF: the fact that the number of
commits and the number of mailing list posts have grown in linear
relationship [...] over the years.
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
3/13

5. Goal of the paper
Ratio Communication flow / development activity
Hypothesis: the ratio is an intensive metric for software evolution
It varies with
Maturity, technology, community composition
But not with project source code size
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
4/13

6. Goal of the paper
Ratio Communication flow / development activity
Hypothesis: the ratio is an intensive metric for software evolution
It varies with
Maturity, technology, community composition
But not with project source code size
Case study: the ASF
Broad and diverse range of projects
Size, scope, technology, maturity
If it didn’t happen on-list, it didn’t happen
Communications between developers (decisions)
Issue trackers
Code review tools, automated builds, wiki page edits
Commits
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
4/13

7. ASF projects under study
Project
HTTPD
APR
Lucene
Turbine
Tomcat
Jackrabbit
Hadoop
Geronimo
SpamAssassin
Portals
Beehive
,
kSLOC
156
66
414
41
213
344
1270
370
54
202
88
Technology
Web server
Library
Index & search
Java web fwork
Servlet API
JSR-170 ref. impl.
Big Data
JavaEE app. srv.
Spam filter
Web fwork
J2EE Struts
Maturity
Active, long-lived
Active, long-lived
Active, long-lived
Stagnated
Active, long-lived
Active
Very active
Active, long-lived
Mature
Nearly dead
Attic
Intensive metrics for open source evolution – http://oa.upm.es/14698/
Scope
Users
Devs
Users
Devs
Devs
Devs
Devs
Devs
End users
Devs
Devs
5/13

8. ASF projects under study
Project
HTTPD
APR
Lucene
Turbine
Tomcat
Jackrabbit
Hadoop
Geronimo
SpamAssassin
Portals
Beehive
kSLOC
156
66
414
41
213
344
1270
370
54
202
88
Technology
Web server
Library
Index & search
Java web fwork
Servlet API
JSR-170 ref. impl.
Big Data
JavaEE app. srv.
Spam filter
Web fwork
J2EE Struts
Maturity
Active, long-lived
Active, long-lived
Active, long-lived
Stagnated
Active, long-lived
Active
Very active
Active, long-lived
Mature
Nearly dead
Attic
Scope
Users
Devs
Users
Devs
Devs
Devs
Devs
Devs
End users
Devs
Devs
Ratio
What’s the ratio evolution for these projects?
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
5/13

9. Apache httpd
156 kSLOC, active and long lived web server
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
6/13

10. Apache Portable Runtime (APR)
66 kSLOC, active and long lived library used by httpd and Subversion
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
7/13

11. Apache Hadoop
1270 kSLOC, very active development and community, higher presence of
non-human emails
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
8/13

12. Apache SpamAssassin
54 kSLOC, spam filter, intended for end users, maturing project
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
9/13

13. Apache Beehive
88 kSLOC, project in the Attic (no longer under development)
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
10/13

14. Overall comparison
Allows for comparison of projects with large differences in size, scope,
technology, maturity
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
11/13

15. Overall comparison
Lessons learned
Healthy Apache projects have smooth ratios
Projects with little activity, or small core group, are noisier
Peaks to infinity are evidence of stagnation
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
12/13

16. Overall comparison
Lessons learned
Healthy Apache projects have smooth ratios
Projects with little activity, or small core group, are noisier
Peaks to infinity are evidence of stagnation
User-oriented projects
Evolution:
Starts with high values
Stabilize and matures with 3 <ratio< 8
Developer-oriented projects
Evolution:
Smaller community, no peaks
Always within 3 <ratio< 8
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
12/13

17. Conclusions and further work
Metric
Intensive and expressive metric.
Not depending on size, maturity,
scope or technology.
End-users
More suitable for users-oriented
projects. Ratio works better with
large and active communities.
,
Stagnation
Other ratios, other cases
Devel-only messages, issues,
commits complexity.
Study beyond the ASF.
Can identify stagnated projects.
Can signal potential stagnation
threats.
Intensive metrics for open source evolution – http://oa.upm.es/14698/
13/13

18. Conclusions and further work
Metric
Intensive and expressive metric.
Not depending on size, maturity,
scope or technology.
Stagnation
End-users
More suitable for users-oriented
projects. Ratio works better with
large and active communities.
Other ratios, other cases
Devel-only messages, issues,
commits complexity.
Study beyond the ASF.
Can identify stagnated projects.
Can signal potential stagnation
threats.
Get a preprint of the paper at http://oa.upm.es/14698
Replication package
http://gsyc.es/∼grex/repro/2013-apache-intensive/
,
Intensive metrics for open source evolution – http://oa.upm.es/14698/
13/13