Downloaded 15 times
Uploaded byJames Howison
Scientific software sustainability and ecosystem complexity
This document discusses the sustainability of scientific software in complex ecosystems. It notes that while the ability to recombine digital information is beneficial for innovation, it also threatens sustainability by increasing the work needed to maintain software as usage diversifies. Three models for attracting resources to do this work - commercial markets, open source peer production, and grant funding - are examined. Grant funding is found to be weakest at gaining visibility into diverse usage contexts. Improving sustainability therefore requires either suppressing recombination or incentivizing greater visibility into how software is recombined in practice.
More Related Content
Similar to Scientific software sustainability and ecosystem complexity
Scientific software sustainability and ecosystem complexity
- 1. Sustainability in Scientific Software: Ecosystemcontext and science policy James Howison School of Information University of Texas at Austin (based on joint work with Jim Herbsleb at Carnegie Mellon) This material is based upon work supported by the US National Science Foundation under Grant Nos. SMA- 1064209 (SciSIP) and OCI-0943168 (VOSS). http://www.flickr.com/photos/demandaj/
- 2. How does workingon things made out of digital information change the way we collectively work? @jameshowison
- 3. Affordance 1: Reuse •Digital information can be copied – High design costs – Ultra-low instantiation costs – Cheap network distribution • Implications: – “Write once, run anywhere” – Think of software as an artifact – Everyone gets a car! @jameshowison
- 4. Affordance 2: Recombination •Digital information is very flexible – Patched – Wrapped – Extended – Recombined • Re-combinability is great for innovation – Lots of new ways to do things – But a sting in the tail? Schumpeter, Ethiraj and Levinthal, Baldwin and Clarke @jameshowison
- 5. Today’s questions 1. Howdoes the recombination affordance threaten sustainability? 2. What can be done about this? @jameshowison
- 6. Sustainability: the conditionthat results when the work needed to keep software scientifically useful is undertaken @jameshowison https://www.flickr.com/photos/kitch/5179039214
- 7. So what workneeds to be done? Develop Maintain ?@jameshowison
- 8. What drives theneed for work? 1. Difficulty of production 2. Difficulty of use 3. Changing scientific frontier 4. Changing technological capabilities 5. Ecosystem complexity @jameshowison
- 9. How do scientistsuse software? v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies Edwards, Batcheller, Deelman, Bietz and Lee, Segal, De Roure and Gobels, Ribes and Finholt, Howison and Herbsleb @jameshowison
- 10. Re-animating assemblages • Scientistspull an assemblage together, “get the plots” and often then leave it, often for months or years. • When they return they return to extend; to use the software assemblage for new purposes, for new science, not simply to replicate. @jameshowison
- 11. But the worldchanges … • Reanimation encounters change in the software ecosystem – Updated packages, New packages, New interfaces • And not just in the immediate components of a workflow, but in the dependencies. • This work is echoed at component producers, since components are themselves assemblages. @jameshowison
- 12.
- 13. What holds acomplex software ecosystem together (if anything)? • Sensing work – knowing how things “out there” are changing • Adjustment – making appropriate changes to account for changing surroundings • Synchronization – ensuring that changes in multiple components make sense together, avoiding cascades. @jameshowison
- 14. Number of users(reuse) v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies @jameshowison
- 15. Diversity of use(recombination) v 2.0.1 v 2.0.1 v 2.2.8 Workflow Software assemblage Dependecies @jameshowison
- 16. Ecosystem Position Diversity ofuse NumberofUsers Generally unreachable area Unlikely region low high Few Many @jameshowison
- 17. How do differentkinds of needed work scale with ecosystem position? • New feature development/Technology adaptation – At most linearly across both dimensions • User support – Linearly, perhaps even reducing at high numbers as users support each other • Sensing work – Linearly with diversity of use • Adjustment and synchronization – Exponentially with diversity of use (recombination) – Even assuming a constant rate of change of complements @jameshowison
- 18. Holding things togetheris hard work But you can’t unlock the potential of cyberinfrastructure without it @jameshowison
- 19. Strategies for reducingneeded work 1. Suppress the drivers 2. Increase the efficiency of work … but, ultimately, work is always needed … 1. Attract resources willing and able to do the work @jameshowison
- 20.
- 21.
- 22.
- 23.
- 24.
- 25. Questions of sustainability •How, and to what extent, does a project attract new resources? – Turn use and impact into more resources? • How does a resource attraction system handle sensing, adjustment, and synchronization work? @jameshowison
- 26. A commercial project Value-producing activity SoftwareUse ImpactResources $$$ @jameshowison
- 27. Open Source PeerProduction Howison and Crowston (2014) Collaboration through Open Superposition. MIS Quarterly Value-producing activity Software Use ImpactResources Devs Time @jameshowison
- 28. Grant-making Software Use ImpactResources Grant Peer Review Ribesand Finholt 2007; Howison and Herbsleb 2011 @jameshowison
- 29. How do resourceattraction systems cope with challenges of ecosystem complexity? @jameshowison
- 30. Markets and ecosystemcomplexity • Sales provide insight (sensing) at the same time as they generate resources • Emergence of Platform strategy: – Suppress complexity by disallowing recombination between customer apps. – Sensing undertaken by code review (via app store submission) – Adjustment and synchronization through new releases of the API @jameshowison
- 31. Photo credit: http://www.flickr.com/photos/smemon/5324223435 Theplatform strategy requires power and a willingness to use it to reduce recombination @jameshowison
- 32. Open source peerproduction • Sensing: – Contributions from the edge, driven by use value, provide direct insight into usage. • Adjustment: – Pushing upstream uses the cheap copies affordance of software to scale adjustment. • Synchronization: – Emergence of distributions which collate the adjustments and attempt synchronization. • Debian, Red Hat, Eclipse
- 33. Grant funding • Servicecenter argument places burden on core team • Little visibility into use context of software – Developers are not front-line scientists – No scalable way to see what is happening in use contexts • Success via focusing on low diversity of use/high user numbers – Works in specific locations (e.g., BLAST) – Often impossible, so focus on “power users” (Batcheller and Edwards; Bietz and Lee) @jameshowison
- 34. Attempting command andcontrol • Scientific software developers sometimes appeal for hierarchical control – “The funder should just make it mandatory” – “We should meet and plan a roadmap” – All focused on suppressing complexity • But digital flexibility inhibits this – Scientists are going to tinker, new technologies are going to suggest new technologies @jameshowison
- 35. What is tobe done? @jameshowison
- 36. Improving Synchronization • Fundsoftware distribution work and innovation in distribution – Distributions can manage cascades • Opportunities for research including simulating ecosystem impact of changes – If we knew how tools, data and questions were linked we could test possible changes. @jameshowison
- 37. Improving Adjustment • Acceptthat adjustment happens best at the edge. • Incentivize projects to be open to gathering and rationalize outside adjustments. • Overcome the “service center” framing • Inculcate stewardship orientation within grant funded projects. @jameshowison
- 38. Improving Sensing Fund, butmore importantly develop reputational rewards to, innovation in sensing the scientific software ecosystem: • Measure diversity of use contexts, understand how users recombine – Go beyond single tool user-studies. • Increase visibility of software in publications – See Citation session this afternoon. • Software that reports its own use? – See “Software Metrics” session this afternoon. • Overcome concerns about visibility and scientific competition – Yes, privacy matters, but users have responsibilities as well. @jameshowison
- 39. Takeaways • Recombination isa key affordance of software, but leads to diverse use contexts • The work needed to maintain usefulness in a complex ecosystem grows exponentially with use context diversity. • Two real options: – Suppress recombination – Gain visibility into recombination • Grant-making seems weaker than markets or open source at supporting visibility • Thus incentivizing visibility is key to a sustainable scientific software ecosystem. Paper at http://james.howison.name/publications.html#cyberinfrastructure @jameshowison
Editor's Notes
- #3 Going to explore two different affordances of software that are related to its nature as being made of information:
- #4 “Everyone gets a car!”
- #5 Archer points this out in his Chpt 5 – the progress of science is an exogenous (or structurationally linked) cause of change or pertubation in the system. He draws on Hughes “reverse salients”, points that hold back the progress of the system as a whole. The workflow idea is well understood in studies of science software (e.g., Dennis and Loft).
- #8 Show that the develop and maintain model relies only on the reuse affordance. Let’s look closer at Maintenance (Bietz and Lee).
- #10 Picture of assemblages. They pick up the software and adapt and improve it later.
- #12 It is far from uncommon to discover what appear to be cyclic dependencies, to require missing historical versions, or to require multiple, incompatible versions, requiring some level of jerry-rigging at points in the stack. This experience is common to those working with software, even outside science, and is described as a descent into "dependency hell." http://en.wikipedia.org/wiki/Dependency_hell
- #13 In aggregate, if we pull the camera back, we can envision the world of software workflows as a pond. Changes (from many different sources) spread out through their users and demand many small changes, each of which spreads on out further. Once the ripples start to overlap, or catch up with each other, true chaos emerges.
- #14 Exploding the category of maintenance work, from the perspective of a component producer.
- #15 A single component flows out to many places. Initially similar, but changing over time.
- #16 The component gets combined with a differing set of components.
- #17 You can place software along these dimensions. It’s hard to get to the top left, since as you get more users they will tend to combine your software in more ways. The bottom right is also pretty unlikely (thankfully) as useful recombinations tend to be useful to many people. Yet in some cases, especially in science, you could end up out there.
- #18 The more diverse the recombinations that your users are employing, the more points of origin for change, the more potential for cascading changes that undermine
- #20 How does the way we attract resources relate to the needed work (and, in particular, to the challenges of ecosystem complexity?)
- #31 Probably this splits into 2 slides
- #33 Probably also splits into two slides. Note that distributions handle synchronization, but only at the level of dependencies, not complements.
- #34 Echo Ribes and Finholt about the teams that prioritize research first, then software dev then maintenance. But maintenance is crucial.
- #36 Take these in reverse order, leaving the most actionable for last.