SoundSoftware.ac.uk:Software sustainability for the audioand music researcher            Chris Cannam, Mark Plumbley, Luís...
Who (and why) are we?EPSRC-funded four-year project, 2010–2014:   Run from the Centre for Digital Music at QMUL   Serving ...
Reproducible researchResearch used to be “reproducible” from the paper alone.This is no longer possible: too complex, the ...
Survey 2010–2011                   82% develop code
Survey 2010–2011                   of whom 39% report                     taking steps to                     reproducibil...
Survey 2010–2011                   of whom 35% report                     publishing any code
Survey 2010–2011                   Thats 11% of the whole
Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articleshttp://bit....
Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articles   7 had em...
Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articles   7 had em...
Why dont we publish code & data?Our survey suggested:   – Lack of time   – Copyright restrictions   – Potential for future...
Reasons we dont like to admit?J M Wicherts, M Bakker and D Molenaar, 2011, Willingness to Share   Research Data Is Related...
Reasons we dont like to admit?J M Wicherts, M Bakker and D Molenaar, 2011, Willingness to Share   Research Data Is Related...
What can we do to help?We’re taking a bottom-up approach:   – aid incremental improvements to development practice   by   ...
Barriers to publication and reuse –   Lack of education and confidence with code –   Lack of facilities and tools –   Lack...
Barrier: Lack of confidence in codeIssue:Researchers largely self-trained in software developmentOur approach:  – Training...
Barrier: Lack of facilities and toolsIssue:Researchers don’t use code hosting / version controlOur approach:  – Code site:...
Projects and users of the code site
Barrier: Lack of incentiveIssue:Software not well recognised as research outputOur approach:  – Link publications to code ...
Barrier: Platform incompatibilitiesIssue:Many different platforms and development tools in useSome are not available to al...
Suggestions for research groupsAim at easy training targets; researchers may know less about  coding than youd think!    –...
LinksOur site       –   http://soundsoftware.ac.ukOur code site      – http://code.soundsoftware.ac.ukThese slides, and no...
SoundSoftware: Software Sustainability for audio and Music Researchers
Upcoming SlideShare
Loading in …5
×

SoundSoftware: Software Sustainability for audio and Music Researchers

653 views
620 views

Published on

Presented at the SoundSoftware 2012 Workshop: http://soundsoftware.ac.uk/soundsoftware2012

Sustainable and reusable software and data are becoming increasingly important in today's research environment. Methods for processing audio and music have become so complex they cannot fully be described in a research paper. Even if really useful research is being done in one research group, other researchers may find it hard to build on this research - or even to know it exists. Researchers are becoming increasingly aware of the need to publish and maintain software code alongside their results, but practical barriers often prevent this from happening. We will describe the Sound Software project, an effort to support software development practice in the UK audio and music research community. We examine some of the the barriers to software reuse, and suggest an incremental approach to overcoming some of them. Finally we make some recommendations for research groups seeking to improve their own researchers' software practice.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
653
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

SoundSoftware: Software Sustainability for audio and Music Researchers

  1. 1. SoundSoftware.ac.uk:Software sustainability for the audioand music researcher Chris Cannam, Mark Plumbley, Luís Figueira Centre for Digital Music Queen Mary, University of London
  2. 2. Who (and why) are we?EPSRC-funded four-year project, 2010–2014: Run from the Centre for Digital Music at QMUL Serving the whole UK audio and music research community Supporting the sustainable development and use of software and data to enable high quality research Better science through better handling of software and data
  3. 3. Reproducible researchResearch used to be “reproducible” from the paper alone.This is no longer possible: too complex, the paper is not enoughHence “Reproducible Research”: The paper – ideally Open Access The software – ideally Open Source The data – ideally Open Data all bundled and published togetherIn audio and music research, few people do this. Why?
  4. 4. Survey 2010–2011 82% develop code
  5. 5. Survey 2010–2011 of whom 39% report taking steps to reproducibility
  6. 6. Survey 2010–2011 of whom 35% report publishing any code
  7. 7. Survey 2010–2011 Thats 11% of the whole
  8. 8. Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articleshttp://bit.ly/6otJMx
  9. 9. Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articles 7 had empty entries in the journal archivehttp://bit.ly/6otJMx
  10. 10. Its not just us!McCullough, 2007Surveying economics journal with a data+code archive policy9 empirical articles 7 had empty entries in the journal archive The other two had code, but it didnt work! None of them could be replicated without authors helphttp://bit.ly/6otJMx
  11. 11. Why dont we publish code & data?Our survey suggested: – Lack of time – Copyright restrictions – Potential for future commercial useOther factors (UK Research Information Network, 2010): – Lack of evidence of benefits – Culture of independence or competition – Quality concerns (self-taught programmers)Also: it takes effort early in the research cycle; hard to find time/motivation after the paper is published
  12. 12. Reasons we dont like to admit?J M Wicherts, M Bakker and D Molenaar, 2011, Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results, PLoS ONEhttp://bit.ly/vaU435
  13. 13. Reasons we dont like to admit?J M Wicherts, M Bakker and D Molenaar, 2011, Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results, PLoS ONE Does this cut both ways? Can we improve quality by helping people prepare to share?http://bit.ly/vaU435
  14. 14. What can we do to help?We’re taking a bottom-up approach: – aid incremental improvements to development practice by – identifying specific barriers to publication and reuse, that are relatively straightforward to addressSo we hope to: – increase perception among researchers that code is something you can work on together, that can be reused – prepare the ground for reproducible publication
  15. 15. Barriers to publication and reuse – Lack of education and confidence with code – Lack of facilities and tools – Lack of incentive for publication – Platform incompatibilities These are barriers to publication of code. Pilot data-management project at C4DM (to be discussed this afternoon) to feed into future work on data.
  16. 16. Barrier: Lack of confidence in codeIssue:Researchers largely self-trained in software developmentOur approach: – Training in software development specifically for researchers – Relatively small amounts of training can pay offAutumn School, Software CarpentryWorkshops and tutorials around ISMIR and DAFx this year
  17. 17. Barrier: Lack of facilities and toolsIssue:Researchers don’t use code hosting / version controlOur approach: – Code site: http://code.soundsoftware.ac.uk – Focus on audio and music research – Public and private projects – Link publications with code – Simplified tools, e.g. EasyMercurial, http://easyhg.org
  18. 18. Projects and users of the code site
  19. 19. Barrier: Lack of incentiveIssue:Software not well recognised as research outputOur approach: – Link publications to code on the code site – Increase likelihood of code users discovering your papers – Ensure users know how to cite your work – Increase take-up / impact of your research
  20. 20. Barrier: Platform incompatibilitiesIssue:Many different platforms and development tools in useSome are not available to all possible users (e.g. MATLAB)Our approach: – Plugins where possible (e.g. Vamp for SV) – Take advantage of existing ecosystems
  21. 21. Suggestions for research groupsAim at easy training targets; researchers may know less about coding than youd think! – Program structure, arranging code across files etcInsist on use of version control code.soundsoftware.ac.uk – Use what you have available, or code.soundsoftware.ac.ukTurn code into plugins or components in modular systems – Latch onto the existing ecosystems of popular applicationsEncourage collaborative development – Papers often co-authored, why not code? – Create an environment of confidence about sharing
  22. 22. LinksOur site – http://soundsoftware.ac.ukOur code site – http://code.soundsoftware.ac.ukThese slides, and notes on them – http://code.soundsoftware.ac.uk/documents/40

×