MSR Cookbook
2 likes
views
Check these out next
MSR Cookbook
2 likes
views
The Mining Software Repositories (MSR) research community has grown significantly since the first MSR workshop was held in 2004. As the community continues to broaden its scope and deepens its expertise, it is worthwhile to reflect on the best practices that our community has developed over the past decade of research. We identify these best practices by surveying past MSR conferences and workshops. To that end, we review all 117 full papers published in the MSR proceedings between 2004 and 2012. We extract 268 comments from these papers, and categorize them using a grounded theory methodology. From this evaluation, four high-level themes were identified: data acquisition and preparation, synthesis, analysis, and sharing/replication. Within each theme we identify several common recommendations, and also examine how these recommendations have evolved over the past decade. In an effort to make this survey a living artifact, we also provide a public forum that contains the extracted recommendations in the hopes that the MSR community can engage in a continuing discussion on our evolving best practices.
Recommended
MSR Cookbook
- The MSR Cookbook Mining a Decade of Research Hadi Hemmati, Sarah Nadi, Olga Baysal, Oleksii Kononenko, Wei Wang, Reid Holmes, Michael W. Godfrey David R. Cheriton School of Computer Science University of Waterloo, Canada MSR-2013, May 19, 2013
- Why Do We Need a Cookbook? MSR Cookbook 2
- Idea Came from MSR Vision 2020 3
- Methodology Review Open coding MSR 2004–2012 (9 yrs) 270 papers 117 full papers Comments: Generalizable observations or suggestions supported by evidence in the paper 268 comments 4 themes with 16 recommendations Recommendation: Comments supported by evidence from at least 5 papers 4
- Themes 5
- Top 5 Recommendations Validate your assumptions and heuristics; repos are noisy [22 c, 17 p] Watch out for collinearities and skewness when synthesizing models from data [22 c, 17 p] [20 c,16 p] Manually verify outputs [12 c, 10 p] Sometimes other measures work better than precision/ recall Sometimes practicality trumps statistical rigour [16 c, 15 p] 6
- Theme/Recommendation Trends • MSR research is maturing as a field • Apparent shift from data extraction toward deeper analysis of the results and their practical use 7
- Online Forum of MSR Cookbook !"#$%%&'()*+'(,-./00*1(%2030404-4%5&.%6 8
- Take-away Message 9 • Newcomers to MSR have questions • Recommendations of best practices for conducting MSR research • Online forum to learn, discuss and contribute • Check the paper for the full list of guidelines!
- 10