Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software Sustainability in e-Research: Dying for a Change


Published on

Closing keynote from e-Research Australasia 2010

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Software Sustainability in e-Research: Dying for a Change

  1. 1. Software Sustainability Institute Software Sustainability in e-Research: Dying for a Change? Neil Chue Hong e-Research Australasia 11 November 2010
  2. 2. Software Sustainability Institute
  3. 3. Software Sustainability Institute UK e-Science Programme: Preparing the Ground • “e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it”  John Taylor, D-G RCUK • e-Science Centres • e-Science Pilot Projects
  4. 4. Software Sustainability Institute OMII: Sowing the first seeds 11 initial projects funded by Managed Programme Many projects flourished But some wilted and decayed OMII setup to harvest and maintain software output of UK e-Science Core Programme
  5. 5. Software Sustainability Institute OMII-UK: Cultivating and Nurturing • Emphasis on helping existing software grow  Extra gardeners brought in (Edinburgh and Manchester) with their own plant stock  Making the garden public through initiatives like Google Summer of Code and ENGAGE  Inviting specialists through the PALs scheme Cultivate and sustain community software important to research
  6. 6. Software Sustainability Institute Software Sustainability Institute: pruning, staking, grafting • Working with research software users and developers  Helping review and refactor  Providing support and skills  Identifying areas of convergence • Producing strong, capable software able to live long and be successfully built on
  7. 7. Software Sustainability Institute SSI Pilot Projects Pilot collaborators: • Fusion Energy • Climate Policy • Geospatial Linked Data • Crystal Structure • Brain Imaging • Scholarly Journals
  8. 8. Software Sustainability Institute No one startsNo one starts writing softwarewriting software intending tointending to abandon itabandon it completelycompletely
  9. 9. Software Sustainability Institute … and yet … • “Sustainability is not an issue for researchers, we want others to take the software over subject to IP issues” • “Sustainability is a big issue, we are producing complex tools we want to continue to use but it’s not clear how they’ll be sustained”  Quotes taken from ENGAGE interviews
  10. 10. Software Sustainability Institute Just the Nature of the problem? Maintenance is not fun Hacking is fun Statistics courtesy of Greg Wilson, Software Carpentry, from Nature article Published online 13 October 2010 | Nature 467, 775-777 (2010) doi:10.1038/467775a
  11. 11. Software Sustainability Institute A quick comparison • Basic training for kitchen chef: 3-4 years • Head chef: 10 years • Basic training for s/w engineer: 3-4 years • Architect: 10 years PhotobyZagatBuzz • Training in S/W Dev in UG Physics: 140 hours • Training in S/W Dev in UG Geography: 0 hours
  12. 12. Software Sustainability Institute Software development comes in stages • Bridging criteria: strength of team; strength of market; proximity of software to market Idea Prototype Research Idea Prototype Idea Idea Prototype Research Supported Product Idea Prototype Research Supported An idea to solve a problem Scaling to work for others Understand the functionality Allow others to participate
  13. 13. Software Sustainability Institute e-Research is multidisciplinary, timescales don’t synchronise 14 Gap in Interest? Cutting Edge Research Applied Research Timescales vary: -ARIES (1989 – 1994) -Giant Magnetoresistance (1988 – 1999) -Frequency Hopping (1903/1942 – 1976) -Bayesian statistics (1763 – 1996)
  14. 14. Software Sustainability Institute Software is noSoftware is no longer easy tolonger easy to define, let alonedefine, let alone sustainsustain
  15. 15. Software Sustainability Institute What do we sustain: - Workflow? - Software that runs workflow? - Software referenced by workflow?
  16. 16. Software Sustainability Institute Novel reuse of public sector data What do we sustain: - Map? - Software that creates map?
  17. 17. Software Sustainability Institute Sustaining Function or Form 19 What do we sustain: - Function? - Form?
  18. 18. Software Sustainability Institute Repositories areRepositories are not the solution…not the solution… … only a mechanism… only a mechanism
  19. 19. Software Sustainability Institute re pos i tor y⋅ ⋅ ⋅ ⋅ /noun/ [ri-poz-i-tawr-ee] • 1. a receptacle or place where things are deposited, stored, or offered for sale. • 2. a burial place; sepulchre.
  20. 20. Software Sustainability Institute Preserving something …
  21. 21. Software Sustainability Institute … is not the same as sustaining it
  22. 22. Software Sustainability Institute The Zombie Effect • Software not always fully alive when you reanimate it! • Complex set of dependencies  Significant Properties of Software  Purposes and benefits of software preservation programmes/preservation/significantpro pertiesofsoftware-final.doc
  23. 23. Software Sustainability Institute Open Source isOpen Source is not the solution…not the solution… … only a mechanism… only a mechanism
  24. 24. Software Sustainability Institute Open Source Software is Free...
  25. 25. Software Sustainability Institute Free as in Puppy... • Long term costs • Needs love and attention • May lose charm after growing up • Occasional clean- ups required • Many left abandoned by their owners Inspired by Scott McNealy Photos of Great Pyrenees from Jen Schopf
  26. 26. Software Sustainability Institute Open Source Process is rarely completed Committer Policies Contributor Agreements Project Infrastructure Licence Resolution Open Source is not just a licence
  27. 27. Software Sustainability Institute Exploiting software for sustainability Models • Grant Mosaic • Institutional support • Fully Costed Service • External Enterprise / Consultancy • Royalties and Fees • Donations • Advertising • T-shirt (spinoff merchandising) Vehicles  University based  Spin out company  Consultancy and Customisation  Industrial knowledge transfer  Contracts  Licensing  Certification  Support services / training  Software as a Service  Software Foundation
  28. 28. Software Sustainability Institute Software sustainability is part of the process • Comparable to risk management  No one right “solution” but many examples of best practice and process • Plan from before the start if possible  But must be reviewed regularly • No longer considering timescales bounded by a project, but considering the product
  29. 29. Software Sustainability Institute Sustainability in Context Product Management Funding/ Effort Community Engagement Software Engineering Software Sustainability Support / Contributions Market Development
  30. 30. Software Sustainability Institute Communities areCommunities are important…important… youyou are the solution!are the solution!
  31. 31. Software Sustainability Institute Sustainable Communities • Cohesion and Identity: Creating a community • Tolerance and Diversity: Smart growth through collaboration • Efficient use of resources: Leveraging infrastructure • Adaptability to change: Governing sustainably
  32. 32. Software Sustainability Institute A: Analyse Customers Slide from Jim Austin
  33. 33. Software Sustainability Institute B: Build Community Building intellectual access ramps to support incremental engagement – building capacity and capability •Individual •Group •Consortium •W/ industry •Community •Active Teams change as project matures 35
  34. 34. Software Sustainability Institute C: Cultivate Contributors • Basics: Website, mailing list, code repository, issue resolution • Remove barriers to participation, increase efficiency • 1993: First public release; 2 devs • 1995: Code open sourced; 3 devs • 1996: r-testers list set up • 1997: lists split: r-announce, r-help, r-devel; public CVS; 11 devs • 2000: CRAN split and mirror • 2001: BioConductor • 2003: Namespaces • 2005: I8n, L8n • 2007: R-Forge • Today: BioConductor (33 core devs), R-Forge (532 projects, 1562 devs), CRAN (1400+ packages) 36
  35. 35. Software Sustainability Institute Reuse,Reuse, not reinventnot reinvent Innovate,Innovate, not replicatenot replicate
  36. 36. Software Sustainability Institute The Software Maturity Curve Softwareproliferation Time CustomisationInnovation Consolidation Portals Quantum chemistry Cloud Computing RDBMS Social Simulation Workflows Spatio- Temporal viz Molecular Dynamics Geospatial viz Digitised Doc Analysis Digital repositories
  37. 37. Software Sustainability Institute Enabling Innovation • Supporting emergent disciplines  Needs recognition of innovative software development as part of funding • Breaking down barriers  We cannot assume that the way people interact with resources will conform to expectations  e.g. researchers will use/store files outside of universities  Researchers will do whatever they can to get an edge – they will not always conform  We need to understand the relationship between e- Research and e-learning frameworks (VREs/VLEs?)
  38. 38. Software Sustainability Institute Supporting Consolidation • “e-Science is an organic, emergent process requiring ongoing, coordinated investment from multiple funders and coordinated action by multiple research and infrastructure communities. It is both an enabler of research and an object of research” – RCUK Review of e-Science • Bridging the expectation gaps between participants  Maintenance vs. research  Different timescales for “exciting” work • Well supported open platforms are the key in the age of the research mashup  Platforms to enable bottom-up innovation  Platforms to enable citizen participation  Competition/innovation built on top c.f. industry
  39. 39. Software Sustainability Institute Sustaining Customisation • “The time constants for real transformative impact and significant competitive advantage is decades” – RCUK Review of e-Science • Sustain software infrastructure in the long term  Differing models: through centres; within institutions; distributed • Need to change perceptions so that software is seen as valuable! (and not just invaluable) • Lower barriers to community growth and participation • Increase value of providing services • Virtually merge + map small amounts of effort / funding
  40. 40. Software Sustainability Institute Case Study: NeISS • Evaluate impact of traffic control measures over next 5/10/15 years • Access baseline demographic data about the city • Execute simulation of traffic system and population • Visualise simulation outputs • Augment with new forms of data • Run dynamic models to assess future patterns (congestion, health, social inequality)
  41. 41. Software Sustainability Institute Case Study: NeISS 44 Photo: NASA/GSFC,
  42. 42. Software Sustainability Institute Case Study: NeISS 45
  43. 43. Software Sustainability Institute One last questionOne last question How do we getHow do we get credit for sustainingcredit for sustaining software?software?
  44. 44. Software Sustainability Institute This is where YOU come in • Research credit is based on publication output • Data citations and credit for reuse are still not commonplace • Software credit is the next stage  Otherwise how can we persuade people to contribute back? • YOU are the researchers, developers, providers and funders… YOU can make it happen!
  45. 45. Software Sustainability Institute
  46. 46. Software Sustainability Institute In conclusion • Software Sustainability is not easy, but it is possible • Build it into the process and build the right teams • Three key things to remember:  ANALYSE CUSTOMERS  BUILD COMMUNITY  CULTIVATE CONTRIBUTORS • We need to give credit so people contribute back
  47. 47. Software Sustainability Institute A National Facility for Research Software Become our next collaborators! Email: Blog: Twitter: SlideShare: YouTube: Telephone: +44 (0) 131 650 5030 Pilot collaborators: • Fusion Energy • Climate Policy • Geospatial Linked Data • Crystal Structure • Brain Imaging • Scholarly Journals