Statistics from Greg WilsonAre academics software developers?Can research consortia manage production?Are timing constraints different?What is the role of the PI in software development management?Are the skills for software and research the same?
Drawing on pool of specialists to drive the continued improvement and impact of research software developed by and for researchers
JournalTOCS largest collection of TOCs from major publication
Economic, hydrology, earth systems, coastal models etc.
Update slide for surveymapper?
Update slide for surveymapper?
Update slide for surveymapper?
Update slide for surveymapper?
How does software sustainability fit within context of software engineering, community engagement, project management, fundingWhat are the external factors like change in effort, timelines and deadlines, licensing, step changes in product development
Allowing people to move makes it easier to bridge gaps as you have a chance of creating common communication structures
Become our next collaborator – email email@example.com
Software Sustainability Institute
Software Sustainabilitypreserving the future of research software<br />6 December 2010<br />University of Tsukuba – University of Edinburgh Joint Symposium<br />Neil ChueHong, Director<br />N.ChueHong@software.ac.uk<br />
Agenda<br />Why software sustainability is important for the UK<br />What the Software Sustainability Institute does, and its collaborative projects<br />The research challenges for software sustainability for the future<br />
3<br />UK investment in e-Science<br />Over 100 projects, worth more than £250 million<br />Facilities, services and community support<br />e-Science Institute<br />National and Regional Centres, NCeSS, Environmental e-Science Centres, …<br />NGS & Grid-PP, OMII-UK, Digital Curation Centre, …<br />Many ongoing activities and services unperturbed<br />European Bioinformatics Unit, HPCx & HECToR, …<br />Velocity magnitude flow field pertaining to a cerebral patient specific vascular tree, simulated with HemeLB, an example of urgent computing<br />Image courtesy Peter Coveney, UCL<br />Slide courtesy Malcolm Atkinson, UK e-Science Envoy<br />
Just the Nature of the problem?<br />No one starts writing software intending<br /> to abandon it completely and yet sustainability – developing and maintaining over a long period – is a problem for scientific software <br />Statistics courtesy of Greg Wilson, Software Carpentry, from Nature article<br />Published online 13 October 2010 | Nature 467, 775-777 (2010) <br />doi:10.1038/467775a<br />
What is needed<br />Effort, support and guidance to ensure that researchers can continue to use their chosen software as a cornerstone of their research<br /><ul><li>And beyond the lifetime of its original funding cycle</li></ul>The lifetime of a software code is now greater than that of hardware<br />Software is the capital investment, hardware is the consumable?<br />UK funding organisations want to ensure investment in software is not wasted<br />Similar efforts in the USA: NSF SI2 initiative and Australia<br />Situation in Japan?<br />
The Software Sustainability Institute<br />A national facility for research software <br />Providing services for research software users and developers<br />Developing research community interactions and capacity<br />Promoting research software best practice and capability<br />Sustaining software by helping to negotiate the stages of the software maturity cycle<br />
What the SSI brings<br />Provides specialist skills to drive the continued improvement and impact of research software<br /><ul><li>Drawn from a large and varied pool of expertise
PALs programme funds researcher champions</li></ul>Led by University of Edinburgh with Universities of Manchester and Southampton<br /><ul><li>Director: Neil Chue Hong
Funded by EPSRC for 5 years, 9.5 FTE, £4.2 million, !st June 2010 – 31st May 2015
Builds on existing collaborations and experience from OMII-UK and EPCC</li></li></ul><li>What the SSI does<br />Work in collaboration with research groups in many subjects within the UK to improve key research software <br /><ul><li>online materials (tutorials, guides)
consultative advice (software evaluation , development process, community engagement, dissemination, workshops+surgeries)
collaborative partnerships (usability, quality, maintainability)</li></ul>Engagement with international community, doctoral training centres and funding programmes to change policy towards software sustainability<br />
SSI Guides<br />Software development<br />Software development: general best practice <br />Developing maintainable software<br />Testing your software<br />Repositories<br />Choosing a repository for your software project <br />Migrating project resources: what to remember<br />Creating and managing SourceForge projects<br />Retrieving project resources from NeSCForge<br />Open source<br />Adopting an open-source licence<br />Supporting open-source software <br />Community building<br />Recruiting champions for your project<br />Recruiting student developers<br />
SSI Workshops<br />Collaborations Workshop<br />3-4 March 2011, Edinburgh<br />Software Sustainability Surgeries<br />Run on behalf of funders<br />Can also provide to universities<br />Software Sustainability Training<br />Potential Summer School for Doctoral Training Centres<br />Training on development practices for research teams<br />
SSI Pilot Projects<br />Pilot collaborators:<br /><ul><li>Fusion Energy
Scholarly Journals</li></li></ul><li>Case Study: TavernaWorkbench<br />Initially funded through e-Science myGrid project (2001-2005)<br />Directly funded through OMII-UK (2006-2010)<br />Plus marketing, outreach, legal and networking<br />Platform funding (2009-2014)<br />caBIG subcontract<br />Eli Lilly development<br />40,000+ downloads of Taverna 1.x<br />Take up in other domains,e.g. astronomy <br />
Case Study: NERC Data Grid Security<br />Provides single sign-on to federated data infrastructure <br />NDGS software now installed at major NERC data centres in the UK<br />Now used across multiple projects<br />Filter based approach and OpenID work used by US Earth System Grid for access to CMIP5 archive<br />NCAR deployed worldwide to provide interoperable access control mechanism for next IPCC report in 2013<br />METAFOR QUESTIONNAIRE<br />COWS/NCEO<br />Contributions back to Python community<br />ndg_saml, ndg_xacml, MyproxyClient<br />
Case Study: Fusion Research<br />GS2 used to study low-frequency turbulence in magnetized plasma<br />No common visualisation across different groups<br />Deliver mutually agreeable framework that can be extended easily and can be maintained by the small fusion community<br />Initially looking at Paraview<br />
Case Study: ECIAS<br />Enhanced Community Integrated Assessment System<br />Understand impact of climate policy<br />Bring together disparate numerical models and datasets from community into common framework<br />Provide public CLIMASCOPE portal<br />Figure: Modelled sea level rise from 1765 to 2100 using the simple models in the MAGICC package. <br />
Case Study: GeoTod-II<br />Transform legacy geospatial data sources to a ‘linked-data’ representation<br />Extend OGSA-DAI for transformation of both relational and file data sources to linked-data, and integrating geospatial data sources (e.g. ESRI ShapeFiles, Oracle Spatial)<br />Initial demonstration of exposing INSPIRE hydrography datasets as linked-data<br />Figure: Data from the National Hydrography Dataset (NHD) accessed by Gaia 3 geospatial viewer<br />
Case Study: NeISS<br />Evaluate impact of traffic control measures over next 5/10/15 years<br />Access baseline demographic data about the city<br />Execute simulation of traffic system and population<br />Visualise simulation outputs<br />Augment with new forms of data<br />Run dynamic models to assess future patterns (congestion, health, social inequality)<br />
Case Study:NanoCMOS<br />21<br />The simulation<br />Paradigm now<br />Vout2 [V]<br />A 22 nm MOSFET<br />In production 2008<br />Vout1 [V]<br />A 4.2 nm MOSFET<br />In production 2023<br />
Case Study: Crystal Energy Landscapes<br />Understanding polymorphism in drugs<br />E.g. Dosage profile<br />Chemists<br />Computational<br />Experimental<br />Developers<br />Domain<br />S/W Engineers<br />Integrators<br />Research Computing Services<br />Facilitator<br />http://www.youtube.com/watch?v=bkbRwOWmiwo<br />
Case Study: VIC + RAT<br />Media backbone tools for audio and video maintained by UCL since early 90s<br />Used as the basis for Access Grid, VRVS<br />OMII-UK funding when other sources cut<br />Allowed continued maintenance and bug fixes<br />Enabled projects from Australia, Korea to contribute<br />However difficulties in sustaining<br />Rapid changes in hardware / software<br />Too low profile<br />Other projects not contributing back <br />
Software Testing / Cloud Computing<br />Helping groups develop software tests appropriate to the situation<br />Looking at Cloud Computing / Virtualisation to provide infrastructure for UK groups<br />We would be very interested to collaborate on better frameworks for software testing, cloud computing and modelling of costs for CC <br />24<br />
Exploiting software for sustainability<br />Models<br />Grant Mosaic<br />Institutional support<br />Fully Costed Service<br />External Enterprise / Consultancy<br />Royalties and Fees<br />Donations<br />Advertising<br />T-shirt (spinoff merchandising)<br />Vehicles<br />University based<br />Spin out company<br />Consultancy and Customisation<br />Industrial knowledge transfer<br />Contracts<br />Licensing<br />Certification<br />Support services / training<br />Software as a Service<br />Software Foundation<br />Most common but what happens when PI retires?<br />
Software sustainability is part of the process<br />Comparable to risk management<br />No one right “solution” but many examples of best practice and process<br />Plan from before the start if possible<br />But must be reviewed regularly<br />No longer considering timescales bounded by a project, but considering the product<br />
Enabling Innovation<br />Supporting emergent disciplines<br />Needs recognition of innovative software development as part of funding <br />Breaking down barriers<br />We cannot assume that the way people interact with resources will conform to expectations<br />e.g. researchers will use/store files outside of universities<br />Researchers will do whatever they can to get an edge – they will not always conform<br />
Supporting Consolidation<br />“e-Science is an organic, emergent process requiring ongoing, coordinated investment from multiple funders and coordinated action by multiple research and infrastructure communities. It is both an enabler of research and an object of research” – RCUK Review of e-Science<br />Bridging the expectation gaps between participants<br />Maintenance vs. research<br />Different timescales for “exciting” work<br />Well supported open platforms are the key in the age of the research mashup<br />Platforms to enable bottom-up innovation<br />Platforms to enable citizen participation<br />Competition/innovation built on top c.f. industry <br />
Sustaining Customisation<br />“The time constants for real transformative impact and significant competitive advantage is decades” – RCUK Review of e-Science<br />Sustain software infrastructure in the long term<br />Differing models: through centres; within institutions; distributed<br />Need to change perceptions so that software is seen as valuable! (and not just invaluable)<br />Lower barriers to community growth and participation<br />Increase value of providing services<br />Virtually merge + map small amounts of effort / funding<br />
Invest in people<br />People are the most important investment<br />Adaptability, ability to recognise transferable skills, not strict career paths<br />Software developers come from many backgrounds <br />If e-Science is multi-disciplinary, multi-institution, multi-scale then make it easier to recognise peoples efforts as they move<br />University structures do not make it easy<br />These people are key to effective e-Science as they bridge the gap between other participants<br />
The credit question<br />How do we get credit for reusing, extending and sustaining software?<br />Research credit is based on publication output<br />Data citations and credit for reuse are still not commonplace<br />Software credit is the next stage<br />Otherwise how can we persuade people to contribute back?<br />This is a question which must be answered at an international level<br />
A National Facility for Research Software<br />Pilot collaborators:<br /><ul><li>Fusion Energy