This document discusses architecting flow in software engineering. It argues that software engineering involves multi-person, multi-version development across different scales, from individual components to integrated systems. It emphasizes that information flow between technical and social tools can improve productivity and quality, especially with higher levels of socio-technical congruence. The document presents examples of information flow within and between systems, and how tool architectures that support information flow along value streams can help track important items like features, defects, risks, and debts across teams and tools.

1. Architecting Flow
in Software Engineering
G a i l C . M u r p h y
U n i v e r s i t y o f B r i t i s h C o l u m b i a
except for images where noted
@gail_murphy

2. Software engineering occurs at different scales
Image not copyable

3. INDIVIDUAL COMPONENT
E.G., BRAKE
CAR & ENVIRONMENT
E.G., LANE DEPARTURE
CAR-CAR INTERACTIONS
E.G., SMART TRAFFIC
Software engineering occurs at different scales
INTEGRATED COMPONENTS
E.G., BRAKE & CRUISE
Images not copyable

4. Software engineering involves
multi-person multi-version development
— Brian Randell
[Parnas 2011]

5. Software engineering involves
multi-person multi-version development
And many more…
Tools to manage technical artifacts

6. Software engineering involves
multi-person multi-version development
And many more…
Tools to support social interaction

7. Software engineering involves
multi-person multi-version development
Some integration between social and technical tools
But limited, more is possible

8. Architecting Flow in Software Engineering
Software engineering is a socio-technical endeavour
T
V
Y
I
1
S
S&T
T
S&T
T
Increasing focus on tool architectures and value streams
can increase quality and productivity
S = Social Tool
T = Technical Tool

9. 1 Socio-Technical
Congruence 2 Information Flow
3 Tool Architecture
4 Architecting Flow
“Why” “What”
“How”
“How”

10. 1 Socio-Technical
Congruence 2 Information Flow
3 Tool Architecture
4 Architecting Flow

11. Completely network collaborative
services via centric initiatives.
Completely network collaborative
services via centric initiatives.
Completely network collaborative
services via centric initiatives.
Socio-technical
congruence A
B
C
Technical dependencies between modules
Coordination needs between teams
Team
for A
Team
for C
Team
for B
Congruence leads to
improved productivity
Higher levels of congruence
associated with lower levels
of fault proneness
[Cataldo & Herbsleb, 2013]

12. Example 1: Within System
Cataldo & Herbsleb studied two developments in depth
- Project A - Distributed system for a data storage product
(114 developers, 5m lines of code, C & C++)
- Project B - Embedded system for automotive sector
(380 developers, 8 sites, 7m lines of code, C)
Less mature system
More mature system
finding that:
Higher-levels of socio-technical congruence are associated with better software quality. True for
both projects, but with more benefit for Project B (more mature).
Higher-levels of socio-technical congruence associated with improvements in productivity. True for
both projects, but with more benefit for Project A (less mature).
[Cataldo & Herbsleb, 2013]

13. Example 2: Integrated Systems
Integrator
(Boeing)
Supplier
(Crane/
GE)
https://www.reuters.com/article/us-airshow-boeing-787-idUSL1559730020080715

14. Example 3: Between Systems
What happens in ecosystems that are weakly inter-connected, like Java?
Guava
mcMMO
Vault
Netty
Assertj
Junit
Appsgate
JSONassert
0%
25%
50%
75%
100%
4 32 256 2048
Number of user repositories
R
s
:
Ratio
of
user
repositories
having
a
social
link
Red Dots are Java projects
on GitHub.
The farther right, the more
other technically dependent
Java systems.
[Palyart, Murphy & Masrani, 2017]

15. Example 3: Between Systems
What happens in ecosystems that are weakly inter-connected, like Java?
Guava
mcMMO
Vault
Netty
Assertj
Junit
Appsgate
JSONassert
0%
25%
50%
75%
100%
4 32 256 2048
Number of user repositories
R
s
:
Ratio
of
user
repositories
having
a
social
link
This is a testing library
(JUnit) with lots of
systems using it.
[Palyart, Murphy & Masrani, 2017]

16. Example 3: Between Systems
What happens in ecosystems that are weakly inter-connected, like Java?
Guava
mcMMO
Vault
Netty
Assertj
Junit
Appsgate
JSONassert
0%
25%
50%
75%
100%
4 32 256 2048
Number of user repositories
R
s
:
Ratio
of
user
repositories
having
a
social
link
The vertical axis provides an
indication of how many of
the using systems have a
social dependence.
Reasonable number of using
systems and high-level of social
dependencies from those
systems.
Lots of users, less social
dependencies from those
systems.
[Palyart, Murphy & Masrani, 2017]

17. Scale Why (examples) What How
Within system
Increased productivity
and quality
Integrated system Avoid integration
failures
Between systems Improve planning
Architecting Flow

18. Scale Why (examples) What How
Within system
Increased productivity
and quality
Integrated system Avoid integration
failures
Between systems Improve planning
Architecting Flow
Focus will be on “within system” and “between systems” for
remainder of talk

19. 1 Socio-Technical
Congruence 2 Information Flow
3 Tool Architecture
4 Architecting Flow

20. A
B
C
Technical dependencies between modules
Coordination needs between teams
Team
for A
Team
for C
Team
for B
Information flow
Information flow related to
technical dependence
Is related to software
architecture
Software architecture
specification, evolution,
recovery, practices, etc.
are well-established

21. A
B
C
Technical dependencies between modules
Coordination needs between teams
Team
for A
Team
for C
Team
for B
Information flow
Information flow to support
coordination needs is
less established
Focus in this talk will
be on information flow
related to coordination

22. 03
02
01
04
Features
Types of information
flowing through a
software development
Defects
Debts
Risks
Four types of Flow Items
— Mik Kersten, 2018

23. A “Within System”
Conceptual Example
An automotive company building a
hybrid or electric vehicle with
regenerative braking
Image not copyable

24. Requirement: Provide regenerative braking
Task: Design algorithm for regenerative braking
Task: Gain input from brake sensors
Task: Reverse engine
Task: Route electricity from motor to batteries
Task: Simulate algorithm for regenerative braking
Task: Implement algorithm for regenerative braking
Task: Test algorithm for regenerative braking
Task: Performance test algorithm for regenerative braking
Etc. etc. etc.
Information flow items
related to a Feature
Image not copyable

25. Requirement: Provide regenerative braking
Task: Design algorithm for regenerative braking
Task: Simulate algorithm for regenerative braking
Task: Implement algorithm for regenerative braking
Task: Test algorithm for regenerative braking
Task: Performance test algorithm for regenerative braking
Etc. etc. etc.
Information flow items
related to a Feature
Requirements Team
Design Team
Dev Team #1
Dev Team #2
Test Team
Image not copyable

26. Etc. etc. etc.
Information flow items
related to a Feature
Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team
Image not copyable

27. Etc. etc. etc.
Information flow items
related to a Defect
Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team
Defect Team
Defect: Problems with anti-lock braking
Etc. etc. etc.
Image not copyable

28. Etc. etc. etc.
Information flow items
related to a Feature
and Defect
Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team
Defect Team
Defect: Problems with anti-lock braking
Etc. etc. etc.
Technical: Algorithm Spec
Social: Discussion about Spec
Technical: Environment
conditions for defect
Social: Discussion to gather
more environmental info
Image not copyable

29. An “Integrated System”
Conceptual Example
An airline manufacturer
contracting the braking
system for the plane
Image not copyable

30. Information flow items
Feature:
Provide braking
component
Integrator Supplier
Image not copyable

31. Information flow items
Feature:
Provide braking
component
Integrator
Supplier
Requirements Team
Development Team
Quality Assurance Team
Technical: component
Social: interface discussion
More limited content on flow
Items between integrator and supplier
Image not copyable

32. Scale Why (examples) What How
Within system
Increased productivity
and quality
Features, Defects, Risks
and Debts
Integrated system Avoid integration
failures
Likely contractually
limited to features and
defects
Architecting Flow

33. 1 Socio-Technical
Congruence 2 Information Flow
3 Tool Architecture
4 Architecting Flow

34. Modularization of
software to reduce
team
interdependencies
degrades over time
Teams interchange
a variety of
technical and social
information during
development
Information flow is
constant, complex
and changing
Organization and
software structure
will not suffice

35. Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team
Tools can help track information
T
Requirements
V
Features
I
Tasks
Defects
I
Tasks
Defects
Y
Test

36. Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team

37. Requirements Team Design Team Dev Team #1 Dev Team #2 Test Team
Tool architectures with an
integration substrate can help
information flow
T
Requirements
V
Features
I
Tasks
I
Tasks
Defects Defects
Y
Test

38. An example tool architecture
With thanks to Tasktop Technologies Inc. for the diagram.
Diagram may not be copied.

39. Tool architectures vary
With thanks to Tasktop Technologies Inc. for the diagram.
Diagram may not be copied.

40. Tool architectures vary
With thanks to Tasktop Technologies Inc. for the diagram.
Diagram may not be copied.
An integration substrate (product) can enable flow
across the connectors between tools

41. What about integrated system development?
Integrator Supplier
A deep relationship between integrator
and supplier may see information flow
between tools
A less trusted relationship may
see the Integrator limited to sending
defects to the supplier

42. Scale Why (examples) What How
Within system
Increased productivity
and quality
Features, Defects, Risks
and Debts
Tool architecture
Integrated system Avoid integration
failures
Likely contractually
limited to features and
defects
Tool architecture
Architecting Flow

43. 1 Socio-Technical
Congruence 2 Information Flow
3 Tool Architecture
4 Architecting Flow

44. Tool architectures
supported by an
integration
substrate enable
information flow
But WHAT
information flows
should be
supported?
Information flow is
technical and
social information
and is about
features, defects,
risks and debts
Socio-technical
congruence can
increase quality
and improve
productivity
Where are we?
Images not copyable

45. Completely network collaborative
services via centric initiatives.
Information flow related to
value streams needs to be
supported
A value stream takes a product or service
from beginning through to a customer (who may
be internal or external)

46. Completely network collaborative
services via centric initiatives.
How does a value stream relate to software architecture?

47. 47
Tasktop Viz
© Tasktop Technologies, Inc.
2017-2018. All rights reserved.
Tracking Information Flow for Value Streams

48. Scale
Within system
Integrated system
Supporting software engineering at different scales
Tracking
Information Flow
for Value Steams
Commercial products
Commercial products
(depending on trust
relationship)

49. Scale Why (examples) What How
Within system
Increased productivity
and quality
Features, Defects, Risks
and Debts
Value streams and tool
architecture
Integrated system Avoid integration
failures
Likely contractually
limited to features and
defects
Value streams and tool
architecture
Architecting Flow

50. Many open questions
Conway’s law (adage)
1967
Modular software
architecture to enable
parallel work and
enable change
(1970s)
but…
In practice, changes crosscut software architecture, attention is being placed on (often
crosscutting) value streams and developers view their teams as fluid
so…
How should software be organized? How should teams be organized?
What metrics should be tracked?

54. T = Technical Tool
@gail_murphy

55. [Cataldo & Herbsleb, 2013] Cataldo, M. and Herbsleb, J. Coordination breakdowns and their
impact on development productivity and software failures. IEEE TSE, vol 39, no 3, 2013.
[Kersten, 2018] Kersten, M. Project to product: How to survive and thrive in the age of digital
disruption with the flow framework. IT Revolution Press, 2018.
[Palyart, Murphy & Masrani, 2017] Palyart, M., Murphy, G.C. and Masrani, V. A study of social interactions
in open source component use. IEEE TSE, vol 44, no 12, 2017.
[Parnas 2011] Parnas, D. Software engineering: Multi-person development of multi-version programs.
In Dependable and Historic Computing - Essays dedicated to Brian Randell on the Occasion of His 75th
Birthday. LNCS Vol. 6875, 2011.