Calling All Modularity Solutions:A Comparative Study from eBayJavaOne 2011 Sangjin Lee, Tony Ng eBay Inc.
Agenda•  What is modularity?•  Why modularity?•  Evaluation criteria•  Scorecard•  Summary2
What is Modularity?Dictionary says...   modular: employing or involving a module or modules as the basis of design   or co...
Key modularity concepts for software•  Building blocks•  Re-use•  Granularity•  Dependencies•  Encapsulation•  Composition...
Challenges for large enterprises•  Some stats on the eBay code base    –    ~ 44 million of lines of code and growing    –...
Challenges for large enterprises•  Developer productivity/agility suffers as the knowledge goes down    –    Changes rippl...
Our goals with modularity efforts•  Tame complexity•  Organize our code base in loose coupling fashion    –  Coarse-graine...
Considerations on any modularity solutions•  Scalability: enterprise software tends to be large scale•  We need to conside...
Evaluation criteria•  Modularity concerns    –    Hide internals (“reduce the surface area”)    –    Enforce modularity   ...
Evaluation criteria•  End-to-end development lifecycle: IDE, command line build,   repository, server runtime, etc.     – ...
Evaluation criteria•  Migration concerns     –  Ease of conversion     –  Learning curve•  Adoption: user communities and ...
Evaluation candidates•  OSGi•  Maven•  Jigsaw•  JBoss modules Disclaimer: this is not a comprehensive discussion of these ...
OSGi•  Observations     –    “The only game in town” (?)     –    Strong modularity framework     –    Focuses more on run...
OSGiMETA-INF/MANIFEST.MF:	Bundle-ManifestVersion: 2	Bundle-SymbolicName: org.foo.bar	Bundle-Version: 1.2.1	Import-Package:...
OSGi•  Pros     –  Enforces modularity strongly: it will let you know if you violate it     –  Mature and comprehensive: c...
OSGi•  Cons     –  Can run two versions of the same class easily, and run into trouble     –  Some problems are nasty to t...
OSGi•  Hide internals: A     –  Declarative way to exclude module-private        packages     –  It’s strictly enforced: w...
OSGi•  Assemble application easily: A     –  Supported through subsystem provisioning (Karaf features, OSGi        applica...
OSGi•  Complete and mature tooling: B+     –  IDE: PDE, bndtools, sigil, ...     –  Command line build: maven + bundle plu...
OSGi•  Ease of migration: D     –  Binaries can be “wrapped” into OSGi bundles     –  All bad practices must be addressed ...
OSGi: scorecard     Hide internals                    A     Enforce modularity                A     Assemble application e...
Maven•  Observations     –  Normally not thought of as a “modularity solution”     –  But has some characteristics of modu...
Mavenpom.xml:	<?xml version=“1.0”?>	<project>	    <groupId>org.foo</groupId>	    <artifactId>bar</artifactId>	    <version...
Maven•  Pros     –    Most developers are already familiar with maven     –    Most third-party libraries are already avai...
Maven•  Cons: it’s not really meant as a modularity framework     –  It does not enforce modularity (either at build time ...
Maven•  Hide internals: F     –  Nothing there     –  Still the global classpath rules•  Enforce modularity: D-     –    M...
Maven•  Assemble application easily: B     –  Definitely possible     –  But it’s only as good as your POM dependencies•  ...
Maven•  Complete and mature tooling: A     –  Command line build, IDE (m2eclipse), and maven repo•  Integration of tools: ...
Maven: scorecard     Hide internals                    F     Enforce modularity                D-     Assemble application...
Project Jigsaw overview•  Part of JDK 8•  JSR 277, JSR 294•  Key Features     –    Static resolution     –    Versioning  ...
Jigsaw: module declaration (module-info)•  Source and binary formmodule a.b @ 1.0 {	    requires c.d @ [2.0,3.0);   //   d...
Jigsaw•  Pros     –  Native support in JDK (e.g. javac) and language     –  Covers build time, install-time, and runtime  ...
Jigsaw•  Hide internals: A•  Enforce modularity: A•  Easy app provisioning & creation: A•  Run two versions of the same cl...
Jigsaw•  Complete and mature tooling: F•  Integration of tools: F     –  Only OpenJDK tools support right now•  Migration ...
Jigsaw: scorecard     Hide internals                    A     Enforce modularity                A     Assemble application...
JBoss modules•  Basis for JBoss OSGi & app server•  No container required•  Features     –    Exact version match only    ...
JBoss modules: module declaration<module xmlns="urn:jboss:module:1.0" name="my.module">	     <main-class name="my.module.M...
JBoss modules•  Pros     –  Very simple     –  Fast static resolution     –  Lightweight (jboss-modules.jar ~240k)•  Cons ...
JBoss modules•  Hide internals: A     –  Can constrain exported packages easily•  Enforce modularity: A•  Easy app provisi...
JBoss modules•  Complete and mature tooling: F•  Integration of tools: F•  Migration concerns: D     –  Many of the OSGi b...
JBoss modules: scorecard     Hide internals                    A     Enforce modularity                A     Assemble appl...
Scorecard                                     OSGi Maven Jigsaw JBoss                                                     ...
Summary•  OSGi is the only true modularity solution that is “ready” now•  OSGi has challenges in migration and learning cu...
Thank you!Sangjin Lee: sangjin.lee@ebay.com, twitter @sjleeTony Ng: tonyng@ebay.com44
Upcoming SlideShare
Loading in...5
×

Calling All Modularity Solutions: A Comparative Study from eBay

752

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
752
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Calling All Modularity Solutions: A Comparative Study from eBay"

  1. 1. Calling All Modularity Solutions:A Comparative Study from eBayJavaOne 2011 Sangjin Lee, Tony Ng eBay Inc.
  2. 2. Agenda•  What is modularity?•  Why modularity?•  Evaluation criteria•  Scorecard•  Summary2
  3. 3. What is Modularity?Dictionary says... modular: employing or involving a module or modules as the basis of design or construction: modular housing units.Then what is a “module”? module: each of a set of standardized parts or independent units that can be used to construct a more complex structure, such as an item of furniture or a building.According to Wikipedia*: modular design: an approach that subdivides a system into smaller parts (modules) that can be independently created and then used in different systems to drive multiple functionalities. * http://en.wikipedia.org/wiki/Modular_design3
  4. 4. Key modularity concepts for software•  Building blocks•  Re-use•  Granularity•  Dependencies•  Encapsulation•  Composition•  Versioning Source: http://techdistrict.kirkk.com/2010/04/22/granularity-architectures-nemesis/ Author: Kirk Knoernschild4
  5. 5. Challenges for large enterprises•  Some stats on the eBay code base –  ~ 44 million of lines of code and growing –  Hundreds of thousands of classes –  Tens of thousands of packages –  ~ 4,000+ jars•  We have too many dependencies and tight coupling in our code –  Everyone sees everyone else –  Everyone affects everyone else5
  6. 6. Challenges for large enterprises•  Developer productivity/agility suffers as the knowledge goes down –  Changes ripple throughout the system –  Fallouts from changes/features are difficult to resolve –  Developers slow down and become risk averse –  Everyone affects everyone else: invites even more dependencies knowledge complexity code size6
  7. 7. Our goals with modularity efforts•  Tame complexity•  Organize our code base in loose coupling fashion –  Coarse-grained modules: number matters! –  Declarative coupling contract –  Ability to hide internals•  Establish clear code ownership, boundaries and dependencies•  Allow different components (and teams) evolve at different speeds•  Increase development agility7
  8. 8. Considerations on any modularity solutions•  Scalability: enterprise software tends to be large scale•  We need to consider a large group of developers with varying skill levels•  End-to-end development lifecycle is crucial•  Conversion/migration of existing code base is crucial –  We rarely start from vacuum –  We want to move over and modularize bulk of existing code –  It is imperative that we chart a realistic migration course that can be achieved within a reasonable amount of time –  We cannot afford disruption to business meanwhile: “change parts while the car is running”8
  9. 9. Evaluation criteria•  Modularity concerns –  Hide internals (“reduce the surface area”) –  Enforce modularity –  Provision/assemble application easily –  Isolate and run two versions of the same class9
  10. 10. Evaluation criteria•  End-to-end development lifecycle: IDE, command line build, repository, server runtime, etc. –  Complete and mature tooling –  Integration and fidelity of tools across phases pull/push SCM pull Command line IDE build (CI) consume publish/consume Deployment Server runtime Repository packaging deploy10
  11. 11. Evaluation criteria•  Migration concerns –  Ease of conversion –  Learning curve•  Adoption: user communities and knowledge11
  12. 12. Evaluation candidates•  OSGi•  Maven•  Jigsaw•  JBoss modules Disclaimer: this is not a comprehensive discussion of these technologies. We will focus only on modularity aspects and it will not be an exhaustive evaluation.12
  13. 13. OSGi•  Observations –  “The only game in town” (?) –  Strong modularity framework –  Focuses more on runtime than build time –  “All or nothing”: everything needs to be pretty much in an OSGi bundle –  Bundles and services are dynamic at runtime13
  14. 14. OSGiMETA-INF/MANIFEST.MF: Bundle-ManifestVersion: 2 Bundle-SymbolicName: org.foo.bar Bundle-Version: 1.2.1 Import-Package: org.foo.other;version=“[1.1,2.0)”, javax.xml.parsers Export-Package: org.foo.bar;version=“1.2.3”, org.foo.bar.sub;uses=“org.foo.bar”;version=“1.2.1” 14
  15. 15. OSGi•  Pros –  Enforces modularity strongly: it will let you know if you violate it –  Mature and comprehensive: covers pretty much all use cases regarding modularity –  Open standard –  Services: the ultimate decoupling force –  Can run two versions of the same class easily15
  16. 16. OSGi•  Cons –  Can run two versions of the same class easily, and run into trouble –  Some problems are nasty to troubleshoot (uses conflict anyone?) –  Still not many well-integrated tools across all phases: impedance mismatches –  Compared to strong runtime model, build side story is weak –  Migration can be quite painful –  Learning curve is still fairly steep16
  17. 17. OSGi•  Hide internals: A –  Declarative way to exclude module-private packages –  It’s strictly enforced: won’t resolve if it is violated•  Enforce modularity: A –  Requirements and capabilities declared in the manifest are strongly enforced: runtime will fail if violated17
  18. 18. OSGi•  Assemble application easily: A –  Supported through subsystem provisioning (Karaf features, OSGi application model, etc.) –  Works quite well because the underlying dependency metadata is high quality•  Run two versions of the same class: B+ –  Works great if consumers are well-separated –  Supports version range dependencies –  However, has a chance of introducing nasty problems if one is not careful: uses conflicts, accidental wiring, and ClassCastExceptions –  Be careful what you wish for18
  19. 19. OSGi•  Complete and mature tooling: B+ –  IDE: PDE, bndtools, sigil, ... –  Command line build: maven + bundle plug-in, ant, tycho, ... –  Repository: OBR, target platform, maven repository•  Integration of tools: D –  Tools at different phases do not integrate too well –  Left to deal with a lot of little impedance mismatches19
  20. 20. OSGi•  Ease of migration: D –  Binaries can be “wrapped” into OSGi bundles –  All bad practices must be addressed before they can be used, however: split packages, Class.forName, thread context classloader, etc. –  Not all third-party libraries are available as OSGi bundles –  Need to take a continuous refactoring approach•  Learning curve: C –  The learning curve is fairly steep mainly because modularity is not always easy to master –  Dynamic behavior of bundles/services introduces complexity•  Adoption: B- –  Fairly wide adoption, but less so in enterprise applications20
  21. 21. OSGi: scorecard Hide internals A Enforce modularity A Assemble application easily A Run two versions of the same class B+ Complete and mature tooling B+ Integration of tools D Ease of migration D Learning curve C Adoption B- Overall B-21
  22. 22. Maven•  Observations –  Normally not thought of as a “modularity solution” –  But has some characteristics of modularity through dependency management –  More of a build time solution than runtime22
  23. 23. Mavenpom.xml: <?xml version=“1.0”?> <project> <groupId>org.foo</groupId> <artifactId>bar</artifactId> <version>1.2.1</version> <packaging>jar</packaging> <dependencies> <groupId>org.foo</groupId> <artifactId>other</artifactId> <version>1.1.0</version> </dependencies> </project> 23
  24. 24. Maven•  Pros –  Most developers are already familiar with maven –  Most third-party libraries are already available as maven artifacts –  Comes with a fairly comprehensive and mature tooling ecosystem –  You can extend the behavior easily with your own plug-ins24
  25. 25. Maven•  Cons: it’s not really meant as a modularity framework –  It does not enforce modularity (either at build time or runtime) –  You cannot hide internals –  At runtime, the global classpath still rules25
  26. 26. Maven•  Hide internals: F –  Nothing there –  Still the global classpath rules•  Enforce modularity: D- –  More of a build time concern than runtime –  No enforcement of dependencies: transitive build classpath! –  Not even an option of strict dependency build –  There is a dependency plugin that analyzes your dependencies26
  27. 27. Maven•  Assemble application easily: B –  Definitely possible –  But it’s only as good as your POM dependencies•  Run two versions of the same class: F –  Nothing there –  Still the global classpath rules –  When there are version collisions, maven picks one for you J27
  28. 28. Maven•  Complete and mature tooling: A –  Command line build, IDE (m2eclipse), and maven repo•  Integration of tools: A•  Ease of migration: B+ –  Boils down to creating POMs for existing non-maven projects –  Third-party libs are pretty much maven artifacts these days•  Learning curve: B –  Most developers are already familiar with maven –  Maven has its own quirks•  Adoption: A28
  29. 29. Maven: scorecard Hide internals F Enforce modularity D- Assemble application easily B Run two versions of the same class F Complete and mature tooling A Integration of tools A Ease of migration B+ Learning curve B Adoption A Overall C+29
  30. 30. Project Jigsaw overview•  Part of JDK 8•  JSR 277, JSR 294•  Key Features –  Static resolution –  Versioning –  Optional modules –  Permits –  Virtual modules –  Native packaging –  Module-private accessibility at language level30
  31. 31. Jigsaw: module declaration (module-info)•  Source and binary formmodule a.b @ 1.0 { requires c.d @ [2.0,3.0); // dependencies export e.f.*; // package exported provide g.h @ 4.0; // virtual provider modules permit i.j; // module friendship class k.l; // main class } 31
  32. 32. Jigsaw•  Pros –  Native support in JDK (e.g. javac) and language –  Covers build time, install-time, and runtime –  Static resolution may provide optimization & simplicity•  Cons –  Not ready to be used yet –  Less mature, lesser known –  Primary focus is to modularize JDK (but open to others) –  Pace has been slow –  Static resolution means less flexibility at runtime32
  33. 33. Jigsaw•  Hide internals: A•  Enforce modularity: A•  Easy app provisioning & creation: A•  Run two versions of the same class: B33
  34. 34. Jigsaw•  Complete and mature tooling: F•  Integration of tools: F –  Only OpenJDK tools support right now•  Migration concerns: B- –  Extra effort would be given to minimize migration pain since the plan is to modularize JDK –  Some unknown on interoperability with OSGi•  Learning curve: B –  Not much info yet and still evolving (so far does not look too complicated)•  Adoption: F34
  35. 35. Jigsaw: scorecard Hide internals A Enforce modularity A Assemble application easily A Run two versions of the same class B Complete and mature tooling F Integration of tools F Ease of migration B- Learning curve B Adoption F Overall C35
  36. 36. JBoss modules•  Basis for JBoss OSGi & app server•  No container required•  Features –  Exact version match only –  Transitive and non-transitive dependencies –  Optional dependencies –  Import/export filters36
  37. 37. JBoss modules: module declaration<module xmlns="urn:jboss:module:1.0" name="my.module"> <main-class name="my.module.Main"/> <resources> <resource-root path="mymodule.jar"/> </resources> <dependencies> <module name="a.b"/> <module name="c.d" slot="1.3.0"/> <!-- Optional dependencies --> <module name="e.f" optional="true"/> </dependencies> </module> 37
  38. 38. JBoss modules•  Pros –  Very simple –  Fast static resolution –  Lightweight (jboss-modules.jar ~240k)•  Cons –  Lacks (public) tools: IDE, command line builds, ... –  Lacks advanced features –  No version ranges: exact match only –  Non-standards based38
  39. 39. JBoss modules•  Hide internals: A –  Can constrain exported packages easily•  Enforce modularity: A•  Easy app provisioning & creation: B –  It is possible by way of resolving and starting the first module –  It is unknown if there is a formal provisioning concept or framework/ library publicly available•  Run two versions of the same class: C –  No version ranges: exact match only –  Has potential to introduce multiple versions unintentionally39
  40. 40. JBoss modules•  Complete and mature tooling: F•  Integration of tools: F•  Migration concerns: D –  Many of the OSGi bad practices would be issues with JBoss modules too•  Learning curve: B –  Relatively simple to learn•  Adoption: D- –  Little adoption outside JBoss itself?40
  41. 41. JBoss modules: scorecard Hide internals A Enforce modularity A Assemble application easily B Run two versions of the same class C Complete and mature tooling F Integration of tools F Ease of migration D Learning curve B Adoption D- Overall C-41
  42. 42. Scorecard OSGi Maven Jigsaw JBoss modulesHide internals A F A AEnforce modularity A D- A AAssemble application easily A B A BRun two versions of the same class B+ F B CComplete and mature tooling B+ A F FIntegration of tools D A F FEase of migration D B+ B- DLearning curve C B B BAdoption B- A F D-Overall B- C+ C C- 42
  43. 43. Summary•  OSGi is the only true modularity solution that is “ready” now•  OSGi has challenges in migration and learning curve•  Dearth of integrated tooling is an issue with OSGi•  Maven does not offer much in the way of modularity: no enforcement•  Jigsaw will introduce language level modularity support•  Will there be interoperability between Jigsaw and OSGi?•  JBoss modules shares strong modularity traits with OSGi•  JBoss modules does not offer much tooling43
  44. 44. Thank you!Sangjin Lee: sangjin.lee@ebay.com, twitter @sjleeTony Ng: tonyng@ebay.com44

×