Research Software Engineering
Inside and Outside the Library
Patrick McCann
Applications Developer (Research Computing)
RLUK Conference 2017
Research Software Engineering
Inside and Outside the Library
•Introduction of the Research Computing
team at St Andrews
•Working as RSEs from inside the Library
•Supporting RSEs outside the Library
Research Computing at
St Andrews
• Started over 10 years ago as a service providing
support for Arts and Humanities projects
• 2012 – renamed Research Computing and remit
widened to all researchers
• 2015 – moved from IT Services to the new Digital
Research division of the Library
Research Computing at
St Andrews
•3 Applications
Developers (2.5 FTE)
Research Computing at
St Andrews
•3 Applications
Developers Research
Software Engineers
(2.5 FTE)
•Reporting to Senior
Librarian (Digital
Humanities and
Research Computing)
Digital Research Division
•Open Access
•Research Data Management
•Research Systems
•Digital Humanities
•Research Computing
Research Computing
•Supports research in two principal ways
•As a development resource available to
researchers across the University
•By supporting research software engineering
across the University
Research Computing
•Supports research in two principal ways
•As a development resource available to
researchers across the University
•By supporting research software engineering
across the University
RSEs Inside the Library
•Assistance with funding applications
•Advice on appropriate technologies, working
within the St Andrews IT context
•Specifying software development to be carried
out during the project
• Team members costed into applications
RSEs Inside the Library
•Project Websites
•The most common
type of support
provided
•Own hosting
platform
RSEs Inside the Library
•Repositories
•Islandora
•Image Database
RSEs Inside the Library
•Custom applications
•Databases
•TEI encoding
•Document Indexes
•Web Applications
Publishing the Philosophical
Transactions
•PI: Dr. Aileen Fyfe, School of History
•AHRC project investigating scholarly
publishing over the 350 year history of the
Philosophical Transactions of The Royal
Society
Publishing the Philosophical
Transactions
• Virtual Registry of Papers
• Database populated initially
with details (inc. DOIs!) of all
published articles
• Researchers expanding on
these with details of publication
process from Royal Society
records
• Also adding records for
unpublished papers
• Key Fact Generator
• Dynamically generated
datasheet for specified year
• Overview
• Editorial and management details
• Print/circulation details
• Financial details
• Will be available via website
The Islamisation of Anatolia
• PI: Dr. Andrew Peacock, School of
History
• An ERC-funded examination of the
formation of Anatolian Islamic
society through extensive but
largely unstudied literary evidence
• Research Computing have helped
construct a searchable index of
that evidence
Arab Cultural Semantics in
Transition
• PI: Dr. Kirill Dmitriev, School of
Modern Languages
• An ERC-funded project to explore
the pivotal role of language
consciousness in the history of
Arab culture
• Research Computing are
developing an Analytical Database
of Arabic Poetry
RSEs Inside the Library
• This work has been the core function of the team since
before moving to the Library
• Enabling researchers to identify and address new questions
• Research data management
• Facilitating collaboration and dissemination of research
• The relationship with academics has developed
• Providing a service → working in partnership
RSEs Inside the Library
• Library Applications for
Digital Humanities
• Digital Collections
• Biographical Register
• Transcription Platform
Research Computing
•Supports research in two principal ways
•As a development resource available to
researchers across the University
•By supporting research software engineering
across the University
Research Computing
•Supports research in two principal ways
•As a development resource available to
researchers across the University
•By supporting research software engineering
across the University
RSEs Outside the Library
•We aren’t the only RSEs at the University
•Who are the others?
•What are they doing?
•How are they doing it?
•What support do they need?
RSEs Outside St Andrews
• Software Sustainability
Institute
• [Supporting] the UK’s
research software community
- a community that includes
the majority of UK’s
researchers.
• software.ac.uk
• @SoftwareSaved
http://software.ac.uk/attach/Flyer.pdf
RSEs Outside St Andrews
• Research Software
Engineers Association
• Influencing academia to
recognise the fundamental
role that software plays in
today’s research.
• rse.ac.uk
• @ResearchSoftEng
http://rse.ac.uk/wp-content/uploads/2017/01/cropped-UKRSE_website_logo.png
RSEs Outside St Andrews
•Software Carpentry
•Teaching basic lab
skills for research
computing
•software-carpentry.org
•@swcarpentry
https://zenodo.org/api/files/00000000-0000-0000-0000-000000000000/swcarpentry/logo.png
RSEs Outside the Library
• February 2016:
Blogpost: Do you write
software for research?
• ~300 views
• ~50 responses
• March 2016: Mailing list
• ~75 subscribers https://commons.wikimedia.org/wiki/File:Prog-languages.png
RSEs Outside the Library
• June 2016: Research
Computing Network launch
event
• ~30 people
• Issues raised:
• Availability of appropriate
version control systems
• Training, especially for PGRs
and early career researchers
RSEs Outside the Library
• In July 2016, the
Library conducted a
survey of research data
management practices
• Sent to 2000 research
staff and students
• 300 responses covering
every School
RSEs Outside the Library
•Do you write,
develop or
maintain
programs, scripts
or other code as
part of your
research?
Yes: 124
(41% of 299
respondents)
RSEs Outside the Library
•Do you use a code
repository or
version control
software (e.g. Git,
Mercurial,
Subversion) to
manage your
code?
Yes: 35
(28% of 124
developers)
RSEs Outside the Library
• Follow-up survey
•Nov/Dec 2016
•Sent to ~100 people
• From previous survey and mailing list
•29 responses
RSEs Outside the Library
• Version Control:
• GitHub.com is good enough for
most people most of the time…
• But there are some
requirements around access
control and data security which
could only be met by an
institutional system
• Working on project proposal to
put such a system in place
https://dwa5x7aod66zk.cloudfront.net/assets/labtocat-be5eee0434960a8f73e54910df8e87b8a5a3b2d651c0b301670c04a9cc26a70f.png
RSEs Outside the Library
• Training
• 45% of respondents were “self-
taught/learned on the job”
• All but 1 respondent interested in
one or more Software Carpentry
lessons for themselves or their
PGRs
• Working with CAPOD, our
Professional Development service
• 2 workshops this semester
• More to be offered on ongoing
basis
https://zenodo.org/api/files/00000000-0000-0000-0000-000000000000/swcarpentry/logo.png
RSEs Outside the Library
• Developing guidance on good practice for RSEs in order to
• Make software better
• Make research more reproducible
• Demonstrate impact
• Save effort
• Developing openly on GitHub:
• https://github.com/StAResComp/sta-rse-guidance
RSEs Outside the Library
• None of this was part of our service before moving to the
Library
• Fits on a continuum of Open Research support:
• OA → RDM → RSE
• Complements the support provided by the Digital Humanities
service
Research Software Engineering
Inside and Outside the Library
•RSEs in the Library working with academics
to develop the solutions they need
•The Library supporting RSEs across the
University so they can develop their own
solutions
Thank-you!
Patrick McCann
pgm5@st-andrews.ac.uk
research-computing@st-andrews.ac.uk

Research Software Engineering Inside and Outside the Library

  • 1.
    Research Software Engineering Insideand Outside the Library Patrick McCann Applications Developer (Research Computing) RLUK Conference 2017
  • 2.
    Research Software Engineering Insideand Outside the Library •Introduction of the Research Computing team at St Andrews •Working as RSEs from inside the Library •Supporting RSEs outside the Library
  • 3.
    Research Computing at StAndrews • Started over 10 years ago as a service providing support for Arts and Humanities projects • 2012 – renamed Research Computing and remit widened to all researchers • 2015 – moved from IT Services to the new Digital Research division of the Library
  • 4.
    Research Computing at StAndrews •3 Applications Developers (2.5 FTE)
  • 5.
    Research Computing at StAndrews •3 Applications Developers Research Software Engineers (2.5 FTE) •Reporting to Senior Librarian (Digital Humanities and Research Computing)
  • 6.
    Digital Research Division •OpenAccess •Research Data Management •Research Systems •Digital Humanities •Research Computing
  • 7.
    Research Computing •Supports researchin two principal ways •As a development resource available to researchers across the University •By supporting research software engineering across the University
  • 8.
    Research Computing •Supports researchin two principal ways •As a development resource available to researchers across the University •By supporting research software engineering across the University
  • 9.
    RSEs Inside theLibrary •Assistance with funding applications •Advice on appropriate technologies, working within the St Andrews IT context •Specifying software development to be carried out during the project • Team members costed into applications
  • 10.
    RSEs Inside theLibrary •Project Websites •The most common type of support provided •Own hosting platform
  • 11.
    RSEs Inside theLibrary •Repositories •Islandora •Image Database
  • 12.
    RSEs Inside theLibrary •Custom applications •Databases •TEI encoding •Document Indexes •Web Applications
  • 13.
    Publishing the Philosophical Transactions •PI:Dr. Aileen Fyfe, School of History •AHRC project investigating scholarly publishing over the 350 year history of the Philosophical Transactions of The Royal Society
  • 14.
    Publishing the Philosophical Transactions •Virtual Registry of Papers • Database populated initially with details (inc. DOIs!) of all published articles • Researchers expanding on these with details of publication process from Royal Society records • Also adding records for unpublished papers • Key Fact Generator • Dynamically generated datasheet for specified year • Overview • Editorial and management details • Print/circulation details • Financial details • Will be available via website
  • 15.
    The Islamisation ofAnatolia • PI: Dr. Andrew Peacock, School of History • An ERC-funded examination of the formation of Anatolian Islamic society through extensive but largely unstudied literary evidence • Research Computing have helped construct a searchable index of that evidence
  • 16.
    Arab Cultural Semanticsin Transition • PI: Dr. Kirill Dmitriev, School of Modern Languages • An ERC-funded project to explore the pivotal role of language consciousness in the history of Arab culture • Research Computing are developing an Analytical Database of Arabic Poetry
  • 17.
    RSEs Inside theLibrary • This work has been the core function of the team since before moving to the Library • Enabling researchers to identify and address new questions • Research data management • Facilitating collaboration and dissemination of research • The relationship with academics has developed • Providing a service → working in partnership
  • 18.
    RSEs Inside theLibrary • Library Applications for Digital Humanities • Digital Collections • Biographical Register • Transcription Platform
  • 19.
    Research Computing •Supports researchin two principal ways •As a development resource available to researchers across the University •By supporting research software engineering across the University
  • 20.
    Research Computing •Supports researchin two principal ways •As a development resource available to researchers across the University •By supporting research software engineering across the University
  • 21.
    RSEs Outside theLibrary •We aren’t the only RSEs at the University •Who are the others? •What are they doing? •How are they doing it? •What support do they need?
  • 22.
    RSEs Outside StAndrews • Software Sustainability Institute • [Supporting] the UK’s research software community - a community that includes the majority of UK’s researchers. • software.ac.uk • @SoftwareSaved http://software.ac.uk/attach/Flyer.pdf
  • 23.
    RSEs Outside StAndrews • Research Software Engineers Association • Influencing academia to recognise the fundamental role that software plays in today’s research. • rse.ac.uk • @ResearchSoftEng http://rse.ac.uk/wp-content/uploads/2017/01/cropped-UKRSE_website_logo.png
  • 24.
    RSEs Outside StAndrews •Software Carpentry •Teaching basic lab skills for research computing •software-carpentry.org •@swcarpentry https://zenodo.org/api/files/00000000-0000-0000-0000-000000000000/swcarpentry/logo.png
  • 25.
    RSEs Outside theLibrary • February 2016: Blogpost: Do you write software for research? • ~300 views • ~50 responses • March 2016: Mailing list • ~75 subscribers https://commons.wikimedia.org/wiki/File:Prog-languages.png
  • 26.
    RSEs Outside theLibrary • June 2016: Research Computing Network launch event • ~30 people • Issues raised: • Availability of appropriate version control systems • Training, especially for PGRs and early career researchers
  • 27.
    RSEs Outside theLibrary • In July 2016, the Library conducted a survey of research data management practices • Sent to 2000 research staff and students • 300 responses covering every School
  • 28.
    RSEs Outside theLibrary •Do you write, develop or maintain programs, scripts or other code as part of your research? Yes: 124 (41% of 299 respondents)
  • 29.
    RSEs Outside theLibrary •Do you use a code repository or version control software (e.g. Git, Mercurial, Subversion) to manage your code? Yes: 35 (28% of 124 developers)
  • 30.
    RSEs Outside theLibrary • Follow-up survey •Nov/Dec 2016 •Sent to ~100 people • From previous survey and mailing list •29 responses
  • 31.
    RSEs Outside theLibrary • Version Control: • GitHub.com is good enough for most people most of the time… • But there are some requirements around access control and data security which could only be met by an institutional system • Working on project proposal to put such a system in place https://dwa5x7aod66zk.cloudfront.net/assets/labtocat-be5eee0434960a8f73e54910df8e87b8a5a3b2d651c0b301670c04a9cc26a70f.png
  • 32.
    RSEs Outside theLibrary • Training • 45% of respondents were “self- taught/learned on the job” • All but 1 respondent interested in one or more Software Carpentry lessons for themselves or their PGRs • Working with CAPOD, our Professional Development service • 2 workshops this semester • More to be offered on ongoing basis https://zenodo.org/api/files/00000000-0000-0000-0000-000000000000/swcarpentry/logo.png
  • 33.
    RSEs Outside theLibrary • Developing guidance on good practice for RSEs in order to • Make software better • Make research more reproducible • Demonstrate impact • Save effort • Developing openly on GitHub: • https://github.com/StAResComp/sta-rse-guidance
  • 34.
    RSEs Outside theLibrary • None of this was part of our service before moving to the Library • Fits on a continuum of Open Research support: • OA → RDM → RSE • Complements the support provided by the Digital Humanities service
  • 35.
    Research Software Engineering Insideand Outside the Library •RSEs in the Library working with academics to develop the solutions they need •The Library supporting RSEs across the University so they can develop their own solutions
  • 36.

Editor's Notes

  • #4 I joined the team in 2014
  • #6 We are, at this point in time, officially "Applications Developers"; however, Research Software Engineer is emerging as the standard way to describe those who contribute to research by developing software, wherever they are located.
  • #12 The Islandora software stack is built around the Fedora repository software and the Drupal content management system; we maintain a multisite set-up allowing multiple sites to draw on a single Fedora repository. The Image Database system was developed in-house and has been running for over 10 years. Works and images are described using the Visual Resource Association metadata schema. The system is used in teaching and to support a number of research projects.
  • #13 We work with researchers on a number of projects to develop bespoke applications to meet their needs, using a wide range of technologies. I'll quickly run through some examples on the next few slides.
  • #14 2013-2017
  • #15 Research Computing developed and maintained the database and the interfaces for it. We also have helped to augment the data with information from other sources. 2013-2017
  • #16 Screenshots of search results and individual record 2012-2016
  • #17 Screenshots - select a root and you are shown lemmata; select one to see a sense, which you can then view in context. 2013-2017
  • #18 To me, these functions fit well within the Library Have always been seeking to work more in partnership. We maintain close links with IT Services.
  • #19 Another thing we've been doing since before the move is developing and supporting applications for Digital Humanities within the Library Digital Collections uses Islandora to make available material from Special Collections which is unique to or of particular relevance to St Andrews. The Biographical Register holds details of St Andrews alumni, officers and graduates between 1747 and 1897, drawn from a number of sources in Special Collections. The Transcription Platform has been set up to facilitate the transcription of material from Special Collections, potentially in a crowd-sourced manner.
  • #22 This isn’t a radical insight generally, but little awareness of these things centrally at St Andrews. First, lets look beyond the University and some of the external initiatives in this area.
  • #23 A national facility for cultivating and improving research software to support world-class research Based at the Universities of Edinburgh, Manchester, Oxford and Southampton, funded by EPSRC, ESRC, BBSRC and Jisc Leading international authority on software sustainability – important for reproducible research
  • #24  An association working to create a community and raise awareness of the UK’s Research Software Engineers
  • #25 Since 1998, Software Carpentry has been teaching researchers the computing skills they need to get more done in less time and with less pain. US-based non-profit.
  • #28 Biggest responses: Chemistry, Biology, Physics, Medicine, Mathematics
  • #30 Interesting in and of itself, but also a proxy for whether or not developers are working in a sustainable manner. Almost all developers from CS used version control, and half of those in Physics. Big drop-off elsewhere..
  • #32 If we want people to follow good practice, we need to facilitate it
  • #33 Alex Konovalov in Computer Science is a certified Software Carpentry Instructor and has been extremely helpful. Have also looked at Data Carpentry and Library Carpentry
  • #34 Also getting out and talking to researchers
  • #35 Some ad-hoc one-to-one support previously