Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scientific Software - what happens after the grant?


Published on

Presentation at Texas State

Published in: Science
  • Be the first to comment

  • Be the first to like this

Scientific Software - what happens after the grant?

  1. 1. Sustaining scientific infrastructures: transitioning from grants to peer production James Howison School of Information University of Texas at Austin 2 September 2016 @jameshowison (slides on slideshare, see twitter for link) This material is based upon work supported by the US National Science Foundation under Grant Nos. SMA- 1064209 (SciSIP), OCI-0943168 (VOSS) and ACI-145348 (CAREER).
  2. 2. Supporting Scientific software after grants run out • What happens when the grant ends? – It’s hard, hard work to keep the code from inevitable “bit-rot” @jameshowison
  3. 3. Extension needs up-to-date code @jameshowison
  4. 4. Just open source it! (How hard can it be???) @jameshowison
  5. 5. Open projects are not like grants 1. Governance 2. Collaboration infrastructures 3. Contribution processes 4. Service center vs. Base for community “open sourcing” means full-on sociotechnical change @jameshowison
  6. 6. A literature on transfer to open? • Copious literature on commercialization, “Technology Transfer” but not communities • Happily there are promising literatures – Studies of open source and online communities (Resnick, Crowston, Wiggins, Kittur, Kraut, Lampe, Ellison, …) – Studies of scientific practice (Palmer, Borgman, Vertesi, Edwards, Olsons, Finholt, Lee/Bietz, Østerlund, Sawyer, Tapia, Ludders, …) – Studies of infrastructural work (Bowker, Jackson, Vertesi, Ribes, …) @jameshowison
  7. 7. How can scientific software projects successfully transition from grant support to thriving peer production communities? Research Design: 1. Theoretically sampled case studies 1. Longitudinal panel study @jameshowison
  8. 8. Questions for each case: How did they succeed or fail in building peer production? – What actions were taken to change the project? – How did routines in the project change as a result? – What conditions are relevant to the success of those actions in causing change? @jameshowison
  9. 9. Sampling success and failure • Very hard to have people talk about failures – Records are often unavailable – Constant problem in studies of open source • Panel study offers help here – Enroll early, before outcome clear – Build trust, chart course, keep records – Selected the NSF SI2 funding program (program officer support) @jameshowison
  10. 10. Panel Study setup • SI2 program contributed to over 350 grants • Three step qualitative content analysis: 1. Did the grant intend to create software 2. What documents (URLs, Workshop reports, or Publications) are available? 3. Read these, apply coding scheme @jameshowison
  11. 11. Content analysis categories Code Description Project Presents Separate From Grant Does the grant support the project (e.g., pre-existing), Or is the project only there because of the grant inviteToContribute contributionProcess Is there an explicit invitation for outsiders to contribute? Is there a process for taking contributions? highlightsPublication e.g., Does the project have a “publications tab” creditsNonPI Contributors Are only the PIs credited “the PIs and their teams” or a wider group? associatedRepository CodeAvailable license Is code available? Is it openly hosted? Where? Under what license? Collaborative setup (wiki, bugtracker) Online meetings? What set of collaborative tools are they using? Offline meetings Does the project organize offline meetings, what kinds (user workshop, hackathon).
  12. 12. Build dataset over time • Training new graduate student on scheme – May involve additional students over time • Intend to code ~5 projects a weekday for two years – 300 projects, 250 weekdays in year, 5 projects a day, 2 coders, assume some missed days! – ~5-10 observations of each project a year • Also analyze repositories, where available. • Adding content analysis codes over time @jameshowison
  13. 13. Case Method: Sampling @jameshowison Use- context diversity Users Science project Generally unreachable area Domain Platform General purpose Unlikely region Individual low high Few Many ytENZO Eclipse PTP OODT/ Airavata
  14. 14. Case Method: analysis • Identify work episodes – Ground interviews in specific production work. – Source-code repositories help immensely – “Digital trace ethnography” (Ribes and Geiger) • Identify socio-technical changes that divide project into stages – Investigate actions that precipitated changes • Project narratives with illustrative vignettes @jameshowison
  15. 15. ENZO @jameshowison
  16. 16. ENZO pilot study Data: • 5 interviews, so far (thanks Eunyoung Moon!) • Publications, websites, workshop websites, source code repositories • Analysis: – Creation of timeline – Identification of episodes and 4 project phases (with their precipitating events) @jameshowison
  17. 17. @jameshowison • No central base to which changes are coming and going • Copy and pasting features across personal branches • Single lab
  18. 18. @jameshowison • ENZO lab reforms as “Service Center” (grant) • Mainline branch internally, releases externally • Little expectation of contributions coming back in • “Friendly user” labs internally functioning like “early days”
  19. 19. The “Week of Code” • Director of external lab (former post-doc) has new job at Stanford (with startup funds!) • Learns of various versions through conversations at conferences and reviewing(!) • Focus is on collaboration infrastructure, not governance. • Begin with the code of those not present @jameshowison
  20. 20. @jameshowison • Central branch to which both core and outsiders contribute • Development continues separately in external labs • Called “Wild West” by participants, autonomy concerns.
  21. 21. @jameshowison • Introduction of “code revision” (pull requests) • External lab members on similar footing to Core members • Review helps members not “step on each other’s work”
  22. 22. Change • What hasn’t changed: – Motivations (code is side-effect of scientific inquiry, papers first, code second), no commercial value • Challenges to change – Leadership’s emotional connection, difficulty of passing on leadership. – Giving up autonomy (being “blocked” in one’s work) @jameshowison
  23. 23. What worked • Always: collaboration technology before governance (contra “Collaboration Readiness” (Olson et al.) TORSC?). • Social proof: visible action in public • Inspiration from open source • Working alongside, rather than with. Superposition rather than Teamwork. @jameshowison
  24. 24. Additional CAREER elements • Teaching course on online communities – Incorporating more on managing software projects in science • Contributing modules to Software Carpentry – 2-3 day workshops with graduate students – Enough command line, python, SQL to get them working – I’m going to contribute module on contributing to and running software projects in science @jameshowison
  25. 25. Conclusions • Software engineering, but in a very specific context • Organization of software work but different to design and testing of methodologies • Can also link in resource and motivation situations • Learning from open source, building alternative paths alongside commercialization. @jameshowison