1.
Software Modularity: Paradoxes, Principles, and Architectures Andrzej Olszak Jaroslav Tulach

2.
Andrzej Olszak• Ph.D. Research fellow @ SDU• Creator of Featureous tool• Fights cancer @ Dako

3.
Jaroslav Tulach• Founder and architect of NetBeans IDE and RCP• API designer• Speaker and author http://paradoxes.apidesign.org

4.
Agenda1. Dividing into modules (Andrzej): 1. Architectures 2. Principles2. Designing module APIs (Jaroslav): 1. Paradoxes 2. Principles

5.
MotivationGetting out of the monolithic cave

6.
The starting pointA monolithic system • No apparent logical parts – Everything changes together – Difficult to control complexity

7.
The goalA modular system • Decomposed into logical modules • Modules partial: – Comprehension – Change

8.
The toolA module system • Wraps logical modules into physical modules – a.k.a. advanced JARs – Enforce code isolation

9.
A possible result • Module system ⇏ Modular code – It all depends on how you design the modules

10.
Part IArchitectures

11.
A running example• Let’s assume that this is our system – 3 layers of code that provide 3 features to the users “UI” “Logic” “Domain Model”• This is the essence of your architecture if you use: – MVC, MVP, Onion arch, Hexagonal arch, …

12.
Modularizing layers• There are at lease three options: View Controller Model

13.
Modularizing layers• There are at lease three options: View Controller Model View feature2 feature1 feature3 App Controller Model

14.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: View feature2 feature1 feature3 Controller Model

15.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality View feature2 feature1 feature3 Controller Model

16.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality View feature2 feature1 feature3 Controller Model

17.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality View feature2 feature1 feature3 Controller Model

18.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality View feature2 feature1 feature3 Controller Model

19.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality View feature2 feature1 feature3 Controller Model

20.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality 2. Migrate UI to JavaFX View feature2 feature1 feature3 Controller Model

21.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality 2. Migrate UI to JavaFX View feature2 feature1 feature3 Controller Model

22.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality 2. Migrate UI to JavaFX View feature2 feature1 feature3 Controller Model

23.
Choosing ‘best’ modularization• Modularity is relative to change – Modularization quality depends on the future changes [Parn’72]• Let’s consider two change scenarios: 1. Modify user functionality 2. Migrate UI to JavaFX View feature2 feature1 No silver-bullet modularization feature3 Controller Model

24.
Anticipating future changes

25.
Anticipating future changes Feature-orientedevolutionary changes 75%Other evolutionary changes 25% Software costs in organizations [Moad’90, Nose’90, Erli’00]

26.
Anticipating future changes Feature-orientedevolutionary changes 75% Evolution & maintenance 60-90%Other evolutionary changes Initial development 25% 10-40% Software costs in organizations [Moad’90, Nose’90, Erli’00]

27.
Modularization in practice: the case of NDVis View• Neurological analysis tool by VisiTrend – Monolithic -> NetBeans MS Controller – Improve functional customizability Model – Improve reusability of core algorithms Feature-oriented restructuring• Starting point – 10 KLOC, 27 use cases feature2 feature3 feature1 – Unfamiliar source code – Complex and unfamiliar domain

28.
Modularizing features with Featureous tool

29.
Modularizing features with Featureous tool1. Feature location(a.k.a. traceability)

30.
Modularizing features with Featureous tool1. Feature location 2. Feature-oriented(a.k.a. traceability) analysis

31.
Modularizing features with Featureous tool1. Feature location 2. Feature-oriented(a.k.a. traceability) analysis 3. Iterative Restructuring 4. Module APIs

32.
Modularization @ 35 man-hours• Explicit and pluggable features• Reusable core

33.
Part IIPrinciples

34.
Separation of Concerns• “To study an aspect of a subject matter in isolation” [Dijk’74]• Software consists of “concerns” – Features – Persistence – Security – Caching ...• Refined by AspectJ [Kicz’96] and Hyper/J [Tarr’99] – Multiple dimensions of concern – one dominant – Scattering & Tangling

35.
Reducing Scattering & Tangling• Low Scattering – “a concern implemented by few modules“ – reduces change scope and delocalization [Leto’86] View Controller feature 1 scattering Model

36.
Scattering & Tangling• Low Scattering – “a concern implemented by few modules“ – reduces change scope and delocalization [Leto’86]• Low Tangling – "modules dedicated to a single concerns“ – reduces change propagation and interleaving [Ruga’95] tangling feature 2 View Controller feature 1 scattering Model

37.
A real-world metaphor

38.
A real-world metaphorhttp://xkcd.com/657/

39.
Mapping SoC to other principles SEPARATION OF CONCERNS

40.
Mapping SoC to other principles SEPARATION OF CONCERNS LOW SCATTERING LOW TANGLING

41.
Mapping SoC to other principles SEPARATION OF CONCERNS LOW SCATTERING LOW TANGLING Information hiding [Parn’72] Low coupling [Stev’74] DRY [Hunt’99]

42.
Mapping SoC to other principles SEPARATION OF CONCERNS LOW SCATTERING LOW TANGLING Information hiding Single responsibility [Parn’72] [Mart’02] Low coupling High cohesion [Stev’74] [Stev’74] DRY Common closure [Hunt’99] [Mart’02]

43.
Measuring SoC • Concern location + concern-oriented metrics – Trialed at Motorola [Simm’06] • Static analysis + Cohesion and Coupling – Issues: coupling vs. DRY, “uncohesive” java.util • Repository mining for change-sets:classes http://swerl.tudelft.nl/bin/view/Main/TestHistory

44.
Summary• Module system is a tool, not the goal• No “silver-bullet” modularization• Restructuring layers into features is viable• SoC – the root of all principles

45.
References[Parn’72] Parnas, D.L. (1972). On the criteria to be used in decomposing systems into modules. Communications of the ACM, 15(12).[Moad’90] Moad, J. (1990). "Maintaining the competitive edge". Datamation 61-62, 64, 66.[Erli’00] Erlikh, L. (2000). "Leveraging legacy system dollars for E-business". (IEEE) IT Pro, May/June 2000.[Nose’90] Nosek, J. & Palvia, P. (1990). "Software maintenance management: changes in the last decade". Journal of Software Maintenance: Research and Practice 2 (3).[Mart’11] Martin, R.C. (2011). http://blog.8thlight.com/uncle-bob/2011/09/30/Screaming-Architecture.html[Dijk’74 ] Dijkstra, E. W. (1974). On the role of scientific thought. Selected Writings on Computing: A Personal Perspective.[Kicz’96] Kiczales, G., Irwin, J., Lamping, J., Loingtier, J., M., Lopes, C., V., Maeda, C. and Mendhekar, A. (1996). Aspect- oriented programming. ACM Computing Surveys, vol. 28.[Tarr’99] Tarr, P., Ossher, H., Harrison, W. and Sutton, S. M. (1999). N degrees of separation: multi-dimensional separation of concerns. In ICSE’99: Proceedings of the 21st international conference on Software engineering.[Leto’86] Letovsky, S. and Soloway, E. (1986). Delocalized Plans and Program Comprehension. IEEE Software, 3(3).[Ruga’95] Rugaber, S., Stirewalt, K. and Wills, L. M. (1995). The interleaving problem in program understanding. In WCRE’95: Proceedings of the 2nd Working Conference on Reverse Engineering.[Simm’06] Simmons, S., Edwards, D., Wilde, N., Homan, J. and Groble, M. (2006). Industrial tools for the feature location problem: an exploratory study. Journal of Software Maintenance and Evolution Research and Practice, 18(6).[Mart’02] Martin, R. C. (2002). Agile Software Development, Principles, Patterns, and Practices. Prentice Hall.[Stev’74] Stevens, W. P., Myers, G. J. and Constantine, L. L. (1974). Structured Design. (E. Yourdon, Ed.)IBM Systems Journal, 13(2).[Hunt’99] Hunt, A., & Thomas, D. (1999). The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley Professional. (Many of these papers are available online through Google Scholar, Citeseerx and the authors’ websites)

46.
Feature tracer [backup]

47.
Part IIIParadoxes of API Design

48.
Modularity is relative to change• API are like stars (paradox 19) • Designing a universe• Distributed development (paradox 2) • Cant know all your users • Envisioning them via use-cases• Sustaining (paradox 3) • One try to get API right

49.
How to anticipate future changes?• Client vs. Provider APIs (paradox 9) • Open spaces • Fixed points• Stable API evolves • Backward compatibility (paradox 6)• Beware of API-less APIs (paradox 8) • Mediawiki experience • Loyal customer

50.
Logical vs. Physical Design• Design oriented on class relationship • UML, specifications• Packaging into JARs ignored (paradox 17) • Influences deployment (paradox 7) • Defines APIs• Good common ground (paradox 11) • Improves your design

51.
A „weight“ of a module• Environment for a module • Modules dont live in vacuum • Expressed by dependencies (paradox 17)• Weight • Number & scope of outgoing dependencies• Less is more (paradox 19) • Single responsibility principle

52.
Use vs. Re-use• Kirk Knoernchild • Monolithic API is easier to use • Modular API is easier to re-use• Blackbox pattern (paradox 18) • OSGi Capabilities• Good tooling • Wizards

53.
SOLID APIs• Single responsibility principle • Meaning of modifiers (paradox 15) • Client vs. provider APIs (paradox 9) • Lightweight API modules• Open/closed principle • OK for provider APIs • Disastrous for client APIs • Proliferation of instanceof in user code • Alternative behavior (paradox 16)

54.
SOLID APIs II• Liskov substitution principle • AWT Frame extends Component! • Dont expose deep hierarchies • Use delegation rather than inheritance • Client API should be in final classes • 1:N factory methods

55.
SOLID APIs III• Interface segregation principle • Lookup & discover • OSGi declarative services• Dependency inversion principle • Code against interfaces, not implementations • Does not imply classes are bad (paradox 9) • Dont fear (injectable) singletons (paradox 14)

56.
API in User Eyes• Clueless users (paradox 1) • Have always something else to do• Evaluation of an API (paradox 4) • Coolness • Time to market • Total cost of ownership

57.
Collaboration• Maintenance (paradox 10) • Rely on patches • Accepting unacceptable (paradox 13)• Beauty (paradox 5) • One writer and dozens of users • Sacrifice the writer

58.
Summary• http://paradoxes.apidesign.org