SlideShare a Scribd company logo
1 of 45
Download to read offline
Calling All Modularity Solutions:
A Comparative Study from eBay
JavaOne 2011




 Sangjin Lee, Tony Ng
 eBay Inc.
Agenda

•  What is modularity?
•  Why modularity?
•  Evaluation criteria
•  Scorecard
•  Summary




2
What is modularity?

Dictionary says...
   modular: employing or involving a module or modules as the basis of design
   or construction: modular housing units.
Then what is a “module”?
    module: each of a set of standardized parts or independent units that can be
    used to construct a more complex structure, such as an item of furniture or a
    building.




3
What is Modularity?

According to Wikipedia*:
  modular design: an approach that subdivides a system into smaller parts
  (modules) that can be independently created and then used in different
  systems to drive multiple functionalities.




    * http://en.wikipedia.org/wiki/Modular_design

4
Key modularity concepts for software

•  Building blocks
•  Re-use
•  Granularity
•  Dependencies
•  Encapsulation
•  Composition
•  Versioning




                      Source: http://techdistrict.kirkk.com/2010/04/22/granularity-architectures-nemesis/
                      Author: Kirk Knoernschild



5
Challenges for large enterprises

•  Some stats on the eBay code base
    –    ~ 44 million of lines of code and growing
    –    Hundreds of thousands of classes
    –    Tens of thousands of packages
    –    ~ 4,000+ jars

•  We have too many dependencies and tight coupling in our code
    –  Everyone sees everyone else
    –  Everyone affects everyone else




6
Challenges for large enterprises

•  Developer productivity/agility suffers as the knowledge goes down
    –    Changes ripple throughout the system
    –    Fallouts from changes/features are difficult to resolve
    –    Developers slow down and become risk averse
    –    Everyone affects everyone else: invites even more dependencies

                                            knowledge               complexity




                                                        code size
7
Our goals with modularity efforts

•  Tame complexity
•  Organize our code base in loose coupling fashion
    –  Coarse-grained modules: number matters!
    –  Declarative coupling contract
    –  Ability to hide internals

•  Establish clear code ownership, boundaries and dependencies
•  Allow different components (and teams) evolve at different speeds
•  Increase development agility




8
Considerations on any modularity solutions

•  Scalability: enterprise software tends to be large scale
•  We need to consider a large group of developers with varying skill
   levels
•  End-to-end development lifecycle is crucial
•  Conversion/migration of existing code base is crucial
    –  We rarely start from vacuum
    –  We want to move over and modularize bulk of existing code
    –  It is imperative that we chart a realistic migration course that can be
       achieved within a reasonable amount of time
    –  We cannot afford disruption to business meanwhile: “change parts
       while the car is running”




9
Evaluation criteria

•  Modularity concerns
     –    Hide internals (“reduce the surface area”)
     –    Enforce modularity
     –    Provision/assemble application easily
     –    Isolate and run two versions of the same class




10
Evaluation criteria

•  End-to-end development lifecycle: IDE, command line build,
   repository, server runtime, etc.
     –  Complete and mature tooling
     –  Integration and fidelity of tools across phases



                      pull/push           SCM          pull


                                                               Command line
              IDE
                                                                 build (CI)



                consume                             publish/consume



                                                              Deployment               Server runtime
                             Repository
                                                packaging                     deploy




11
Evaluation criteria

•  Migration concerns
     –  Ease of conversion
     –  Learning curve

•  Adoption: user communities and knowledge




12
Evaluation candidates

•  OSGi
•  Maven
•  Jigsaw
•  JBoss modules




 Disclaimer: this is not a comprehensive discussion of these technologies. We will
 focus only on modularity aspects and it will not be an exhaustive evaluation.


13
OSGi

•  Observations
     –    “The only game in town” (?)
     –    Strong modularity framework
     –    Focuses more on runtime than build time
     –    “All or nothing”: everything needs to be pretty much in an OSGi bundle
     –    Bundles and services are dynamic at runtime




14
OSGi

META-INF/MANIFEST.MF:
	
Bundle-ManifestVersion: 2	
Bundle-SymbolicName: org.foo.bar	
Bundle-Version: 1.2.1	
Import-Package: org.foo.other;version=“[1.1,2.0)”,	
  javax.xml.parsers	
Export-Package: org.foo.bar;version=“1.2.3”,	
  org.foo.bar.sub;uses=“org.foo.bar”;version=“1.2.1”	




15
OSGi

•  Pros
     –  Enforces modularity strongly: it will let you know if you violate it
     –  Mature and comprehensive: covers pretty much all use cases regarding
        modularity
     –  Open standard
     –  Services: the ultimate decoupling force
     –  Can run two versions of the same class easily




16
OSGi

•  Cons
     –  Can run two versions of the same class easily, and run into trouble
     –  Some problems are nasty to troubleshoot (uses conflict anyone?)
     –  Still not many well-integrated tools across all phases: impedance
        mismatches
     –  Compared to strong runtime model, build side story is weak
     –  Migration can be quite painful
     –  Learning curve is still fairly steep




17
OSGi

•  Hide internals: A
     –  Declarative way to exclude module-private
        packages
     –  It’s strictly enforced: won’t resolve if it is
        violated

•  Enforce modularity: A
     –  Requirements and capabilities declared in the
        manifest are strongly enforced: runtime will
        fail if violated




18
OSGi

•  Assemble application easily: A
     –  Supported through subsystem provisioning (Karaf features, OSGi
        application model, etc.)
     –  Works quite well because the underlying dependency metadata is high
        quality

•  Run two versions of the same class: B+
     –  Works great if consumers are well-separated
     –  Supports version range dependencies
     –  However, has a chance of introducing nasty problems if one is not
        careful: uses conflicts, accidental wiring, and ClassCastExceptions
     –  Be careful what you wish for




19
OSGi

•  Complete and mature tooling: B+
     –  IDE: PDE, bndtools, sigil, ...
     –  Command line build: maven + bundle plug-in, ant, tycho, ...
     –  Repository: OBR, target platform, maven repository

•  Integration of tools: D
     –  Tools at different phases do not integrate too well
     –  Left to deal with a lot of little impedance mismatches




20
OSGi

•  Ease of migration: D
     –  Binaries can be “wrapped” into OSGi bundles
     –  All bad practices must be addressed before they can be used, however:
        split packages, Class.forName, thread context classloader, etc.
     –  Not all third-party libraries are available as OSGi bundles
     –  Need to take a continuous refactoring approach

•  Learning curve: C
     –  The learning curve is fairly steep mainly because modularity is not
        always easy to master
     –  Dynamic behavior of bundles/services introduces complexity

•  Adoption: B-
     –  Fairly wide adoption, but less so in enterprise applications



21
OSGi: scorecard



     Hide internals                    A
     Enforce modularity                A
     Assemble application easily       A
     Run two versions of the same class B+
     Complete and mature tooling       B+
     Integration of tools              D
     Ease of migration                 D
     Learning curve                    C
     Adoption                          B-
     Overall                           B-


22
Maven

•  Observations
     –  Normally not thought of as a “modularity solution”
     –  But has some characteristics of modularity through dependency
        management
     –  More of a build time solution than runtime




23
Maven

pom.xml:
	
<?xml version=“1.0”?>	
<project>	
    <groupId>org.foo</groupId>	
    <artifactId>bar</artifactId>	
    <version>1.2.1</version>	
    <packaging>jar</packaging>	
    <dependencies>	
        <groupId>org.foo</groupId>	
        <artifactId>other</artifactId>	
        <version>1.1.0</version>	
    </dependencies>	
</project>	




24
Maven

•  Pros
     –    Most developers are already familiar with maven
     –    Most third-party libraries are already available as maven artifacts
     –    Comes with a fairly comprehensive and mature tooling ecosystem
     –    You can extend the behavior easily with your own plug-ins




25
Maven

•  Cons: it’s not really meant as a modularity framework
     –  It does not enforce modularity (either at build time or runtime)
     –  You cannot hide internals
     –  At runtime, the global classpath still rules




26
Maven

•  Hide internals: F
     –  Nothing there
     –  Still the global classpath rules

•  Enforce modularity: D-
     –    More of a build time concern than runtime
     –    No enforcement of dependencies: transitive build classpath!
     –    Not even an option of strict dependency build
     –    There is a dependency plugin that analyzes your dependencies




27
Maven

•  Assemble application easily: B
     –  Definitely possible
     –  But it’s only as good as your POM dependencies

•  Run two versions of the same class: F
     –  Nothing there
     –  Still the global classpath rules
     –  When there are version collisions, maven picks one for you J




28
Maven

•  Complete and mature tooling: A
     –  Command line build, IDE (m2eclipse), and maven repo

•  Integration of tools: A
•  Ease of migration: B+
     –  Boils down to creating POMs for existing non-maven projects
     –  Third-party libs are pretty much maven artifacts these days

•  Learning curve: B
     –  Most developers are already familiar with maven
     –  Maven has its own quirks

•  Adoption: A



29
Maven: scorecard



     Hide internals                    F
     Enforce modularity                D-
     Assemble application easily       B
     Run two versions of the same class F
     Complete and mature tooling       A
     Integration of tools              A
     Ease of migration                 B+
     Learning curve                    B
     Adoption                          A
     Overall                           C+


30
Project Jigsaw overview

•  Part of JDK 8
•  JSR 277, JSR 294
•  Key Features
     –    Static resolution
     –    Versioning
     –    Optional modules
     –    Permits
     –    Virtual modules
     –    Native packaging
     –    Module-private accessibility at language level




31
Jigsaw: module declaration (module-info)

•  Source and binary form

module a.b @ 1.0 {	
    requires c.d @ [2.0,3.0);   //   dependencies	
    export e.f.*;               //   package exported	
    provide g.h @ 4.0;          //   virtual provider modules	
    permit i.j;                 //   module friendship	
    class   k.l;                //   main class	
}	




32
Jigsaw

•  Pros
     –  Native support in JDK (e.g. javac) and language
     –  Covers build time, install-time, and runtime
     –  Static resolution may provide optimization & simplicity

•  Cons
     –    Not ready to be used yet
     –    Less mature, lesser known
     –    Primary focus is to modularize JDK (but open to others)
     –    Pace has been slow
     –    Static resolution means less flexibility at runtime




33
Jigsaw

•  Hide internals: A
•  Enforce modularity: A
•  Easy app provisioning & creation: A
•  Run two versions of the same class: B




34
Jigsaw

•  Complete and mature tooling: F
•  Integration of tools: F
     –  Only OpenJDK tools support right now

•  Migration concerns: B-
     –  Extra effort would be given to minimize migration pain since the plan
        is to modularize JDK
     –  Some unknown on interoperability with OSGi

•  Learning curve: B
     –  Not much info yet and still evolving (so far does not look too
        complicated)

•  Adoption: F


35
Jigsaw: scorecard



     Hide internals                    A
     Enforce modularity                A
     Assemble application easily       A
     Run two versions of the same class B
     Complete and mature tooling       F
     Integration of tools              F
     Ease of migration                 B-
     Learning curve                    B
     Adoption                          F
     Overall                           C


36
JBoss modules

•  Basis for JBoss OSGi & app server
•  No container required
•  Features
     –    Exact version match only
     –    Transitive and non-transitive dependencies
     –    Optional dependencies
     –    Import/export filters




37
JBoss modules: module declaration

<module xmlns="urn:jboss:module:1.0" name="my.module">	
     <main-class name="my.module.Main"/>	
     <resources>	
       <resource-root path="mymodule.jar"/>	
     </resources>	
     <dependencies>	
       <module name="a.b"/>	
       <module name="c.d" slot="1.3.0"/>	
       <!-- Optional dependencies -->	
       <module name="e.f" optional="true"/>	
     </dependencies>	
</module>	




38
JBoss modules

•  Pros
     –  Very simple
     –  Fast static resolution
     –  Lightweight (jboss-modules.jar ~240k)

•  Cons
     –    Lacks (public) tools: IDE, command line builds, ...
     –    Lacks advanced features
     –    No version ranges: exact match only
     –    Non-standards based




39
JBoss modules

•  Hide internals: A
     –  Can constrain exported packages easily

•  Enforce modularity: A
•  Easy app provisioning & creation: B
     –  It is possible by way of resolving and starting the first module
     –  It is unknown if there is a formal provisioning concept or framework/
        library publicly available

•  Run two versions of the same class: C
     –  No version ranges: exact match only
     –  Has potential to introduce multiple versions unintentionally




40
JBoss modules

•  Complete and mature tooling: F
•  Integration of tools: F
•  Migration concerns: D
     –  Many of the OSGi bad practices would be issues with JBoss modules too

•  Learning curve: B
     –  Relatively simple to learn

•  Adoption: D-
     –  Little adoption outside JBoss itself?




41
JBoss modules: scorecard



     Hide internals                    A
     Enforce modularity                A
     Assemble application easily       B
     Run two versions of the same class C
     Complete and mature tooling       F
     Integration of tools              F
     Ease of migration                 D
     Learning curve                    B
     Adoption                          D-
     Overall                           C-


42
Scorecard


                                     OSGi Maven Jigsaw JBoss
                                                       modules
Hide internals                       A    F     A      A
Enforce modularity                   A    D-    A      A
Assemble application easily          A    B     A      B
Run two versions of the same class   B+   F     B      C
Complete and mature tooling          B+   A     F      F
Integration of tools                 D    A     F      F
Ease of migration                    D    B+    B-     D
Learning curve                       C    B     B      B
Adoption                             B-   A     F      D-
Overall                              B-   C+    C      C-
 43
Summary

•  OSGi is the only true modularity solution that is “ready” now
•  OSGi has challenges in migration and learning curve
•  Dearth of integrated tooling is an issue with OSGi
•  Maven does not offer much in the way of modularity: no
   enforcement
•  Jigsaw will introduce language level modularity support
•  Will there be interoperability between Jigsaw and OSGi?
•  JBoss modules shares strong modularity traits with OSGi
•  JBoss modules does not offer much tooling



44
Thank you!




Sangjin Lee: sangjin.lee@ebay.com, twitter @sjlee
Tony Ng: tonyng@ebay.com




45

More Related Content

What's hot

Netbeans
NetbeansNetbeans
Netbeans
acosdt
 
Common Client Rich Client Platforms
Common Client   Rich Client PlatformsCommon Client   Rich Client Platforms
Common Client Rich Client Platforms
Geertjan Wielenga
 

What's hot (16)

Netbeans
NetbeansNetbeans
Netbeans
 
Ordina Accelerator program 2019 - Quality assurance
Ordina Accelerator program 2019 - Quality assuranceOrdina Accelerator program 2019 - Quality assurance
Ordina Accelerator program 2019 - Quality assurance
 
Virgo Project Creation Review
Virgo Project Creation ReviewVirgo Project Creation Review
Virgo Project Creation Review
 
Common Client Rich Client Platforms
Common Client   Rich Client PlatformsCommon Client   Rich Client Platforms
Common Client Rich Client Platforms
 
S/W Design and Modularity using Maven
S/W Design and Modularity using MavenS/W Design and Modularity using Maven
S/W Design and Modularity using Maven
 
Kitware: Qt and Scientific Computing
Kitware: Qt and Scientific ComputingKitware: Qt and Scientific Computing
Kitware: Qt and Scientific Computing
 
AD111 -- Harnessing the Power of Server-Side JavaScript and Other Advanced XP...
AD111 -- Harnessing the Power of Server-Side JavaScript and Other Advanced XP...AD111 -- Harnessing the Power of Server-Side JavaScript and Other Advanced XP...
AD111 -- Harnessing the Power of Server-Side JavaScript and Other Advanced XP...
 
OpenJDK Penrose Presentation (JavaOne 2012)
OpenJDK Penrose Presentation (JavaOne 2012)OpenJDK Penrose Presentation (JavaOne 2012)
OpenJDK Penrose Presentation (JavaOne 2012)
 
Object oriented programming-with_java
Object oriented programming-with_javaObject oriented programming-with_java
Object oriented programming-with_java
 
System Architecture using Maven Modularity
System Architecture using Maven ModularitySystem Architecture using Maven Modularity
System Architecture using Maven Modularity
 
Java modularization
Java modularizationJava modularization
Java modularization
 
Modules all the way down: OSGi and the Java Platform Module System
Modules all the way down: OSGi and the Java Platform Module SystemModules all the way down: OSGi and the Java Platform Module System
Modules all the way down: OSGi and the Java Platform Module System
 
Introduction to Java Part-2
Introduction to Java Part-2Introduction to Java Part-2
Introduction to Java Part-2
 
Jug Zurich Slides
Jug Zurich SlidesJug Zurich Slides
Jug Zurich Slides
 
High performance stream processing
High performance stream processingHigh performance stream processing
High performance stream processing
 
Os php-wiki1-pdf
Os php-wiki1-pdfOs php-wiki1-pdf
Os php-wiki1-pdf
 

Viewers also liked (6)

OSGi at eBay: JavaOne 2010
OSGi at eBay: JavaOne 2010OSGi at eBay: JavaOne 2010
OSGi at eBay: JavaOne 2010
 
Robust and Scalable Concurrent Programming: Lesson from the Trenches
Robust and Scalable Concurrent Programming: Lesson from the TrenchesRobust and Scalable Concurrent Programming: Lesson from the Trenches
Robust and Scalable Concurrent Programming: Lesson from the Trenches
 
Concurrency grab bag: JavaOne 2010
Concurrency grab bag: JavaOne 2010Concurrency grab bag: JavaOne 2010
Concurrency grab bag: JavaOne 2010
 
Timeline Service Next Gen (YARN-2928): YARN BOF @ Hadoop Summit 2015
Timeline Service Next Gen (YARN-2928): YARN BOF @ Hadoop Summit 2015Timeline Service Next Gen (YARN-2928): YARN BOF @ Hadoop Summit 2015
Timeline Service Next Gen (YARN-2928): YARN BOF @ Hadoop Summit 2015
 
Timeline Service v.2 (Hadoop Summit 2016)
Timeline Service v.2 (Hadoop Summit 2016)Timeline Service v.2 (Hadoop Summit 2016)
Timeline Service v.2 (Hadoop Summit 2016)
 
32 Ways a Digital Marketing Consultant Can Help Grow Your Business
32 Ways a Digital Marketing Consultant Can Help Grow Your Business32 Ways a Digital Marketing Consultant Can Help Grow Your Business
32 Ways a Digital Marketing Consultant Can Help Grow Your Business
 

Similar to Calling all modularity solutions

Jax london 2011
Jax london 2011Jax london 2011
Jax london 2011
njbartlett
 
DevHub 3 - Composer plus Magento
DevHub 3 - Composer plus MagentoDevHub 3 - Composer plus Magento
DevHub 3 - Composer plus Magento
Magento Dev
 
OSGi in Action Chapter 1 and 2
OSGi in Action Chapter 1 and 2OSGi in Action Chapter 1 and 2
OSGi in Action Chapter 1 and 2
pjhInovex
 
Enterprise OSGi at eBay
Enterprise OSGi at eBayEnterprise OSGi at eBay
Enterprise OSGi at eBay
Tony Ng
 

Similar to Calling all modularity solutions (20)

Clone Clone Make: a better way to build
Clone Clone Make: a better way to buildClone Clone Make: a better way to build
Clone Clone Make: a better way to build
 
Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett
Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil BartlettJava Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett
Java Core | Java 8 and OSGi Modularisation | Tim Ellison & Neil Bartlett
 
Jax london 2011
Jax london 2011Jax london 2011
Jax london 2011
 
How to Build Composite Applications with PRISM
How to Build Composite Applications with PRISMHow to Build Composite Applications with PRISM
How to Build Composite Applications with PRISM
 
Organinzing Your PHP Projects (2010 Memphis PHP)
Organinzing Your PHP Projects (2010 Memphis PHP)Organinzing Your PHP Projects (2010 Memphis PHP)
Organinzing Your PHP Projects (2010 Memphis PHP)
 
Organizing Your PHP Projects (2010 ConFoo)
Organizing Your PHP Projects (2010 ConFoo)Organizing Your PHP Projects (2010 ConFoo)
Organizing Your PHP Projects (2010 ConFoo)
 
DevHub 3 - Composer plus Magento
DevHub 3 - Composer plus MagentoDevHub 3 - Composer plus Magento
DevHub 3 - Composer plus Magento
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Architectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and ConsistentlyArchitectural Decisions: Smoothly and Consistently
Architectural Decisions: Smoothly and Consistently
 
Oracle ADF Architecture TV - Development - Version Control
Oracle ADF Architecture TV - Development - Version ControlOracle ADF Architecture TV - Development - Version Control
Oracle ADF Architecture TV - Development - Version Control
 
Использование maven для сборки больших модульных c++ проектов на примере Odin...
Использование maven для сборки больших модульных c++ проектов на примере Odin...Использование maven для сборки больших модульных c++ проектов на примере Odin...
Использование maven для сборки больших модульных c++ проектов на примере Odin...
 
Platform Engineering for the Modern Oracle World
Platform Engineering for the Modern Oracle WorldPlatform Engineering for the Modern Oracle World
Platform Engineering for the Modern Oracle World
 
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
10 clues showing that you are doing OSGi in the wrong manner - Jerome Moliere
 
OSGi in Action Chapter 1 and 2
OSGi in Action Chapter 1 and 2OSGi in Action Chapter 1 and 2
OSGi in Action Chapter 1 and 2
 
Enterprise OSGi at eBay
Enterprise OSGi at eBayEnterprise OSGi at eBay
Enterprise OSGi at eBay
 
June 2014 HUG - Continuuity Loom : Cluster Management
June 2014 HUG - Continuuity Loom : Cluster ManagementJune 2014 HUG - Continuuity Loom : Cluster Management
June 2014 HUG - Continuuity Loom : Cluster Management
 
Java Modularity with OSGi
Java Modularity with OSGiJava Modularity with OSGi
Java Modularity with OSGi
 
Alfresco DevCon 2018: SDK 3 Multi Module project using Nexus 3 for releases a...
Alfresco DevCon 2018: SDK 3 Multi Module project using Nexus 3 for releases a...Alfresco DevCon 2018: SDK 3 Multi Module project using Nexus 3 for releases a...
Alfresco DevCon 2018: SDK 3 Multi Module project using Nexus 3 for releases a...
 
Java 9 Module System Introduction
Java 9 Module System IntroductionJava 9 Module System Introduction
Java 9 Module System Introduction
 
A Journey through the JDKs (Java 9 to Java 11)
A Journey through the JDKs (Java 9 to Java 11)A Journey through the JDKs (Java 9 to Java 11)
A Journey through the JDKs (Java 9 to Java 11)
 

Recently uploaded

TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
ADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptxADP Passwordless Journey Case Study.pptx
ADP Passwordless Journey Case Study.pptx
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Navigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern EnterpriseNavigating Identity and Access Management in the Modern Enterprise
Navigating Identity and Access Management in the Modern Enterprise
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
Choreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software EngineeringChoreo: Empowering the Future of Enterprise Software Engineering
Choreo: Empowering the Future of Enterprise Software Engineering
 

Calling all modularity solutions

  • 1. Calling All Modularity Solutions: A Comparative Study from eBay JavaOne 2011 Sangjin Lee, Tony Ng eBay Inc.
  • 2. Agenda •  What is modularity? •  Why modularity? •  Evaluation criteria •  Scorecard •  Summary 2
  • 3. What is modularity? Dictionary says... modular: employing or involving a module or modules as the basis of design or construction: modular housing units. Then what is a “module”? module: each of a set of standardized parts or independent units that can be used to construct a more complex structure, such as an item of furniture or a building. 3
  • 4. What is Modularity? According to Wikipedia*: modular design: an approach that subdivides a system into smaller parts (modules) that can be independently created and then used in different systems to drive multiple functionalities. * http://en.wikipedia.org/wiki/Modular_design 4
  • 5. Key modularity concepts for software •  Building blocks •  Re-use •  Granularity •  Dependencies •  Encapsulation •  Composition •  Versioning Source: http://techdistrict.kirkk.com/2010/04/22/granularity-architectures-nemesis/ Author: Kirk Knoernschild 5
  • 6. Challenges for large enterprises •  Some stats on the eBay code base –  ~ 44 million of lines of code and growing –  Hundreds of thousands of classes –  Tens of thousands of packages –  ~ 4,000+ jars •  We have too many dependencies and tight coupling in our code –  Everyone sees everyone else –  Everyone affects everyone else 6
  • 7. Challenges for large enterprises •  Developer productivity/agility suffers as the knowledge goes down –  Changes ripple throughout the system –  Fallouts from changes/features are difficult to resolve –  Developers slow down and become risk averse –  Everyone affects everyone else: invites even more dependencies knowledge complexity code size 7
  • 8. Our goals with modularity efforts •  Tame complexity •  Organize our code base in loose coupling fashion –  Coarse-grained modules: number matters! –  Declarative coupling contract –  Ability to hide internals •  Establish clear code ownership, boundaries and dependencies •  Allow different components (and teams) evolve at different speeds •  Increase development agility 8
  • 9. Considerations on any modularity solutions •  Scalability: enterprise software tends to be large scale •  We need to consider a large group of developers with varying skill levels •  End-to-end development lifecycle is crucial •  Conversion/migration of existing code base is crucial –  We rarely start from vacuum –  We want to move over and modularize bulk of existing code –  It is imperative that we chart a realistic migration course that can be achieved within a reasonable amount of time –  We cannot afford disruption to business meanwhile: “change parts while the car is running” 9
  • 10. Evaluation criteria •  Modularity concerns –  Hide internals (“reduce the surface area”) –  Enforce modularity –  Provision/assemble application easily –  Isolate and run two versions of the same class 10
  • 11. Evaluation criteria •  End-to-end development lifecycle: IDE, command line build, repository, server runtime, etc. –  Complete and mature tooling –  Integration and fidelity of tools across phases pull/push SCM pull Command line IDE build (CI) consume publish/consume Deployment Server runtime Repository packaging deploy 11
  • 12. Evaluation criteria •  Migration concerns –  Ease of conversion –  Learning curve •  Adoption: user communities and knowledge 12
  • 13. Evaluation candidates •  OSGi •  Maven •  Jigsaw •  JBoss modules Disclaimer: this is not a comprehensive discussion of these technologies. We will focus only on modularity aspects and it will not be an exhaustive evaluation. 13
  • 14. OSGi •  Observations –  “The only game in town” (?) –  Strong modularity framework –  Focuses more on runtime than build time –  “All or nothing”: everything needs to be pretty much in an OSGi bundle –  Bundles and services are dynamic at runtime 14
  • 15. OSGi META-INF/MANIFEST.MF: Bundle-ManifestVersion: 2 Bundle-SymbolicName: org.foo.bar Bundle-Version: 1.2.1 Import-Package: org.foo.other;version=“[1.1,2.0)”, javax.xml.parsers Export-Package: org.foo.bar;version=“1.2.3”, org.foo.bar.sub;uses=“org.foo.bar”;version=“1.2.1” 15
  • 16. OSGi •  Pros –  Enforces modularity strongly: it will let you know if you violate it –  Mature and comprehensive: covers pretty much all use cases regarding modularity –  Open standard –  Services: the ultimate decoupling force –  Can run two versions of the same class easily 16
  • 17. OSGi •  Cons –  Can run two versions of the same class easily, and run into trouble –  Some problems are nasty to troubleshoot (uses conflict anyone?) –  Still not many well-integrated tools across all phases: impedance mismatches –  Compared to strong runtime model, build side story is weak –  Migration can be quite painful –  Learning curve is still fairly steep 17
  • 18. OSGi •  Hide internals: A –  Declarative way to exclude module-private packages –  It’s strictly enforced: won’t resolve if it is violated •  Enforce modularity: A –  Requirements and capabilities declared in the manifest are strongly enforced: runtime will fail if violated 18
  • 19. OSGi •  Assemble application easily: A –  Supported through subsystem provisioning (Karaf features, OSGi application model, etc.) –  Works quite well because the underlying dependency metadata is high quality •  Run two versions of the same class: B+ –  Works great if consumers are well-separated –  Supports version range dependencies –  However, has a chance of introducing nasty problems if one is not careful: uses conflicts, accidental wiring, and ClassCastExceptions –  Be careful what you wish for 19
  • 20. OSGi •  Complete and mature tooling: B+ –  IDE: PDE, bndtools, sigil, ... –  Command line build: maven + bundle plug-in, ant, tycho, ... –  Repository: OBR, target platform, maven repository •  Integration of tools: D –  Tools at different phases do not integrate too well –  Left to deal with a lot of little impedance mismatches 20
  • 21. OSGi •  Ease of migration: D –  Binaries can be “wrapped” into OSGi bundles –  All bad practices must be addressed before they can be used, however: split packages, Class.forName, thread context classloader, etc. –  Not all third-party libraries are available as OSGi bundles –  Need to take a continuous refactoring approach •  Learning curve: C –  The learning curve is fairly steep mainly because modularity is not always easy to master –  Dynamic behavior of bundles/services introduces complexity •  Adoption: B- –  Fairly wide adoption, but less so in enterprise applications 21
  • 22. OSGi: scorecard Hide internals A Enforce modularity A Assemble application easily A Run two versions of the same class B+ Complete and mature tooling B+ Integration of tools D Ease of migration D Learning curve C Adoption B- Overall B- 22
  • 23. Maven •  Observations –  Normally not thought of as a “modularity solution” –  But has some characteristics of modularity through dependency management –  More of a build time solution than runtime 23
  • 24. Maven pom.xml: <?xml version=“1.0”?> <project> <groupId>org.foo</groupId> <artifactId>bar</artifactId> <version>1.2.1</version> <packaging>jar</packaging> <dependencies> <groupId>org.foo</groupId> <artifactId>other</artifactId> <version>1.1.0</version> </dependencies> </project> 24
  • 25. Maven •  Pros –  Most developers are already familiar with maven –  Most third-party libraries are already available as maven artifacts –  Comes with a fairly comprehensive and mature tooling ecosystem –  You can extend the behavior easily with your own plug-ins 25
  • 26. Maven •  Cons: it’s not really meant as a modularity framework –  It does not enforce modularity (either at build time or runtime) –  You cannot hide internals –  At runtime, the global classpath still rules 26
  • 27. Maven •  Hide internals: F –  Nothing there –  Still the global classpath rules •  Enforce modularity: D- –  More of a build time concern than runtime –  No enforcement of dependencies: transitive build classpath! –  Not even an option of strict dependency build –  There is a dependency plugin that analyzes your dependencies 27
  • 28. Maven •  Assemble application easily: B –  Definitely possible –  But it’s only as good as your POM dependencies •  Run two versions of the same class: F –  Nothing there –  Still the global classpath rules –  When there are version collisions, maven picks one for you J 28
  • 29. Maven •  Complete and mature tooling: A –  Command line build, IDE (m2eclipse), and maven repo •  Integration of tools: A •  Ease of migration: B+ –  Boils down to creating POMs for existing non-maven projects –  Third-party libs are pretty much maven artifacts these days •  Learning curve: B –  Most developers are already familiar with maven –  Maven has its own quirks •  Adoption: A 29
  • 30. Maven: scorecard Hide internals F Enforce modularity D- Assemble application easily B Run two versions of the same class F Complete and mature tooling A Integration of tools A Ease of migration B+ Learning curve B Adoption A Overall C+ 30
  • 31. Project Jigsaw overview •  Part of JDK 8 •  JSR 277, JSR 294 •  Key Features –  Static resolution –  Versioning –  Optional modules –  Permits –  Virtual modules –  Native packaging –  Module-private accessibility at language level 31
  • 32. Jigsaw: module declaration (module-info) •  Source and binary form module a.b @ 1.0 { requires c.d @ [2.0,3.0); // dependencies export e.f.*; // package exported provide g.h @ 4.0; // virtual provider modules permit i.j; // module friendship class k.l; // main class } 32
  • 33. Jigsaw •  Pros –  Native support in JDK (e.g. javac) and language –  Covers build time, install-time, and runtime –  Static resolution may provide optimization & simplicity •  Cons –  Not ready to be used yet –  Less mature, lesser known –  Primary focus is to modularize JDK (but open to others) –  Pace has been slow –  Static resolution means less flexibility at runtime 33
  • 34. Jigsaw •  Hide internals: A •  Enforce modularity: A •  Easy app provisioning & creation: A •  Run two versions of the same class: B 34
  • 35. Jigsaw •  Complete and mature tooling: F •  Integration of tools: F –  Only OpenJDK tools support right now •  Migration concerns: B- –  Extra effort would be given to minimize migration pain since the plan is to modularize JDK –  Some unknown on interoperability with OSGi •  Learning curve: B –  Not much info yet and still evolving (so far does not look too complicated) •  Adoption: F 35
  • 36. Jigsaw: scorecard Hide internals A Enforce modularity A Assemble application easily A Run two versions of the same class B Complete and mature tooling F Integration of tools F Ease of migration B- Learning curve B Adoption F Overall C 36
  • 37. JBoss modules •  Basis for JBoss OSGi & app server •  No container required •  Features –  Exact version match only –  Transitive and non-transitive dependencies –  Optional dependencies –  Import/export filters 37
  • 38. JBoss modules: module declaration <module xmlns="urn:jboss:module:1.0" name="my.module"> <main-class name="my.module.Main"/> <resources> <resource-root path="mymodule.jar"/> </resources> <dependencies> <module name="a.b"/> <module name="c.d" slot="1.3.0"/> <!-- Optional dependencies --> <module name="e.f" optional="true"/> </dependencies> </module> 38
  • 39. JBoss modules •  Pros –  Very simple –  Fast static resolution –  Lightweight (jboss-modules.jar ~240k) •  Cons –  Lacks (public) tools: IDE, command line builds, ... –  Lacks advanced features –  No version ranges: exact match only –  Non-standards based 39
  • 40. JBoss modules •  Hide internals: A –  Can constrain exported packages easily •  Enforce modularity: A •  Easy app provisioning & creation: B –  It is possible by way of resolving and starting the first module –  It is unknown if there is a formal provisioning concept or framework/ library publicly available •  Run two versions of the same class: C –  No version ranges: exact match only –  Has potential to introduce multiple versions unintentionally 40
  • 41. JBoss modules •  Complete and mature tooling: F •  Integration of tools: F •  Migration concerns: D –  Many of the OSGi bad practices would be issues with JBoss modules too •  Learning curve: B –  Relatively simple to learn •  Adoption: D- –  Little adoption outside JBoss itself? 41
  • 42. JBoss modules: scorecard Hide internals A Enforce modularity A Assemble application easily B Run two versions of the same class C Complete and mature tooling F Integration of tools F Ease of migration D Learning curve B Adoption D- Overall C- 42
  • 43. Scorecard OSGi Maven Jigsaw JBoss modules Hide internals A F A A Enforce modularity A D- A A Assemble application easily A B A B Run two versions of the same class B+ F B C Complete and mature tooling B+ A F F Integration of tools D A F F Ease of migration D B+ B- D Learning curve C B B B Adoption B- A F D- Overall B- C+ C C- 43
  • 44. Summary •  OSGi is the only true modularity solution that is “ready” now •  OSGi has challenges in migration and learning curve •  Dearth of integrated tooling is an issue with OSGi •  Maven does not offer much in the way of modularity: no enforcement •  Jigsaw will introduce language level modularity support •  Will there be interoperability between Jigsaw and OSGi? •  JBoss modules shares strong modularity traits with OSGi •  JBoss modules does not offer much tooling 44
  • 45. Thank you! Sangjin Lee: sangjin.lee@ebay.com, twitter @sjlee Tony Ng: tonyng@ebay.com 45