Presentation given on OpenJDK project Penrose by Tim Ellison and David Bosschaert at Penrose BOF at JavaOne 2012

1. Penrose
@OpenJDK
Tim Ellison, IBM
David Bosschaert, Red Hat
September 2012

2. Agenda
• Level setting
o Introduction to modularity
o Modules and the JRE
o The Jigsaw project
o OSGi
• Introduction to Project Penrose
o Goal of the project
o Structure of the project
o Technical roadmap
• Technical work
o Achievements
o Observations
o To be done
• Advice and call to arms

3. What is a module?
Why do we need a module system?
• Modules facilitate software engineering & enhanced runtime capabilities
o Decomposition of complex systems
o Software assemblies of pre-built modules
o Provisioning / deployment, including module repositories
o Versioning and configuration management
• The module system is used to manage the modules, including installing, activating,
resolving module requirements and conflicts.
• Interacting with the module system provides a higher level abstraction than the
language provides directly.

4. Modules in Java
• The module as a unit of encapsulation...
• The Java language already has two kinds of module: classes and packages.
o classes encapsulate fields and methods
o packages encapsulate classes and resources
• Problem: neither of these is a deployable unit (too granular, too tightly coupled).
• Java's unit of deployment is the JAR file.
o No encapsulation, therefore not a module!
• Problem: need an entity that is a deployable unit of encapsulation.

5. Desirable characteristics of a module
• A module should be highly coherent
• A module should be loosely coupled to other modules.
• A module should be explicit about what it provides to other modules.
• A module should be explicit about what it depends on from other modules.

6. Why modules for the Java runtime (JRE)?
• Presently the JRE is a monolithic entity
o in general the runtime must assume your application will use anything and
everything
o class loaders provide runtime type definition and isolation
o download time and start-up time are directly impacted by the number of types
available in the runtime
• Application start-up time includes a linear search through the class path to find
system and application code
o e.g. Oracle 1.7 Windows bootclass path contains nearly 20k classes
o resources.jar rt.jar jsse.jar jce.jar charsets.jar
o rt.jar index alone is ~1Mb
• To the JRE, your application's jars’ indexes are disjoint & unsorted
o there are JRE implementation 'tricks' like look aside tables and shared classes
that can help
o class loading is a BIG time hog to amortize over the length of the run

7. Why modules for the Java runtime (JRE)? - continued
• Dependency management
o avoid “JAR hell” → trying satisfy competing requirements by simple path ordering
o type name space comprising (unversioned) package name is often insufficient
o e.g. my app depends upon foo-v2.jar and bar-v2.jar but foo-v2.jar depends
upon bar-v1.jar
- classpath foo-v2.jar; bar-v2.jar; bar-v1.jar –> my app “wins”
- classpath foo-v2.jar; bar-v1.jar; bar-v2.jar –> foo “wins”
• Module level visibility
o public, protected, private and package-private level visibility means some
implementation types (com.sun.) need to be marked public to be called by java.
APIs.
o convention asks people not to depend upon them...
• Version control
o ability to define the compatibility of a module based on numbering scheme
o different module versions for different target devices / resource goals

8. Introducing Project Jigsaw
• Project Jigsaw is an effort currently underway at OpenJDK
o defining a simple module system for Java
o modularizing the JRE itself
o originally intended as an implementation detail for the JRE / applications, but
being adopted as a Java standard (JSR)
• Working within constraints of backwards compatibility
o existing Java APIs “cannot” be changed
o Some packages contain wildly different functionality
e.g. java.util contains collection hierarchy and Locale infrastructure
• Introduces language changes to define modules
module com.example @ 1.0 {
src/com/example/myclass.java
requires jdk.base;
src/com/example/myclassimpl.java requires foo @ 2.0;
src/module-info.java requires bar @ 2.0;
}

9. Introducing OSGi
• Existing, mature, standards-based modularity solution for Java applications.
• Rich modularity model based on code-level (package) dependencies, module-level
dependencies, and services.
• First practical solution for effective component reuse
o well supported by multiple implementations, tooling, etc.
• Millions use applications built with OSGi
o most application servers are based on OSGi
• No language changes. Bundle-ManifestVersion: 2
Bundle-SymbolicName: com.ex.mybundle
o defines info in META-INF/MANIFEST.MF Bundle-Version: 1.0.0
Import-Package: com.example.bar
Export-Package: com.example.foo

10. Multiple dependency resolution
• The module system must resolve bundle dependencies, possibly in the face of multiple
valid alternatives
o e.g. my app depends upon (foo-v2.jar and (bar-v1.jar or bar-v2.jar)) and foo-
v2.jar depends upon bar-v1.jar
o if my app touches bar first, the module system may chose bar-v2.jar
o if my app touches foo first, the module system will chose bar-v1.jar
• OSGi's dynamic dependency resolution attempts to find the best fit for currently invoked
bundles at runtime
o modules are apparent at runtime
o runtime activation lifecycle: installed, resolved, starting, active, stopping, uninstalled
• Jigsaw resolves module dependencies during build, installation, or runtime
o gives opportunity for load time optimizations such as pre-verify, indexing, etc.
o Equinox OSGi implementation has similar optimizations.

11. Agenda
• Level setting
o Introduction to modularity
o Modules and the JRE
o The Jigsaw project
o OSGi
• Introduction to Project Penrose
o Goal of the project
o Structure of the project
o Technical roadmap
• Technical work
o Achievements
o Observations
o To be done
• Advice and call to arms

12. Project Penrose - the road to reality
• Both Jigsaw and OSGi are here to stay!
o Try to get the best of both worlds
o It need not be a zero-sum game
• OSGi is established and will continue to be well used
• Jigsaw is underway and a key component to the Java SE plans
• Project Penrose's goal is to demonstrate interoperability between Jigsaw and OSGi
OSGi support — It must be demonstrated by prototype to be feasible to modify an OSGi micro-
kernel such that OSGi bundles running in that kernel can depend upon Java modules. The
kernel must be able to load Java modules directly and resolve them using its own resolver,
except for core system modules. Core system modules can only be loaded using the module
system’s reification API. Java Module-System Requirements

13. Project Penrose - structure
• Penrose is structured to allow collaboration between OSGi and Jigsaw developers
o OSGi (Equinox) and Jigsaw are governed by different foundations, companies
and licenses
o Need to allow separation of developers
• Two Penrose code repositories
o Main code repository is downstream from Jigsaw
§ Provides opportunity to be experimental without disrupting Jigsaw
progress
§ Kept in sync with Jigsaw changes, easy to pass code upstream
§ “show me” working code, rather than arguments on Jigsaw lists
o Second code repository specifically for Penrose tools, demos, etc.
o These are licensed under GPLv2 + classpath exception
o Code developed in Penrose will be delivered through Jigsaw

14. Project Penrose - structure
• Two Penrose mailing lists
o Developer mailing list (penrose-dev)
§ Conversations around coding work on main Penrose repository
§ Incudes details of Jigsaw implementation and Penrose's modifications
§ This is licensed GPLv2
o Discuss mailing list (penrose-discuss)
§ Conversations around interoperability requirements, design and new APIs
§ No implementation code is exchanged directly on this list
§ This is dual licensed GPLv2 and BSD
• Java Specification Request (JSR)
o Looking forward to a JSR in which design discussions can take place

15. Project Penrose: technical roadmap
• Goal #0 : toleration
o ensure that OSGi frameworks continue to run unmodified on a Jigsaw enabled runtime
o creating modules / bundles that have both Jigsaw & OSGi metadata on the same JAR
• Goal #1 : interoperability of module info metadata
o ensure Jigsaw metadata can be enhanced with OSGi concepts
o teach OSGi to read Jigsaw module info
o mapping Jigsaw metadata into OSGi format for the framework to understand, e.g.
requires ⇒ Require-Bundle:
o resolve Jigsaw modules using the OSGi resolver
• Goal #2 : OSGi implementation exploit of Jigsaw modularity
o enhance OSGi to use Jigsaw publication repositories, API, etc
• Goal #3+ : Full interop
o a blend of OSGi and Jigsaw cross delegation on module phases

16. Agenda
• Level setting
o Introduction to modularity
o Modules and the JRE
o The Jigsaw project
o OSGi
• Introduction to Project Penrose
o Goal of the project
o Structure of the project
o Technical roadmap
• Technical work
o Achievements
o Observations
o To be done
• Advice and call to arms

17. Achievements in Level 0 : Toleration
• Achievement #1: pass the OSGi tests on a Jigsaw enabled runtime
o Equinox 3.7.2 – the OSGi reference implementation
o OSGi 4.3 Compliance Tests – with minor patch for test case error
o Running on Windows XP SP3 & Ubuntu 11.04
• Achievement #2: run a Java application as either OSGi or Jigsaw modules
o Took Java 2D demo
o Broke it into multiple functional units, and run the demo as OSGi bundles OR
Jigsaw modules

18. Observations: Jigsaw Module Metadata
• Jigsaw's module-info.java files compile into module-info.class files
o ultimate .jar module contains binary metadata
• Number of issues with these files
o cannot (or very hard) to extend
§ other module systems or tools may need to store additional module-
related data
o binary files are not easy to read
§ you need to use a tool
§ or create your own .class file reader
§ may make troubleshooting harder too
• Goal to create an readable, easily parsed metadata format
o extensible
o expressive enough for Jigsaw and OSGi

19. Achievements in Level 1: Module Metadata in
Penrose
• Penrose proposes using module-info.json
o Plain Text module definitions
o In JSON
• JSON offers
o Better extensibility, additions can be made inline
o Easier to read from modular JAR files
for humans
for tools
o For performance, implementations can still cache their own binary
representations

20. JSON Module Metadata
module-info.java (Jigsaw) module-info.json (Penrose)
"module" : {
module org.astro @ 1.2 { "name" : "org.astro",
"version" : "1.2",
exports org.astro;
"exports" :
}
[{
"name" : "org.astro",
"org.osgi.version" : "1.2.3"
In the Jar file: }]
}
binary module-info.class
Note the extension on the exports
In the Jar file:
plain text module-info.json

21. Demos
The current set of Penrose demos shows:
• Modules that use JSON metadata (no module-info.java/.class files)
• Modules that work both in JavaSE and in OSGi
o currently duplicate module information
o unified module information in the future

22. To be done Level 2/3: Deep integration with OSGi
This pilot needs to start, some areas to look at:
• Loading JavaSE modules in OSGi as bundles
o OSGi will most likely be extended to understand JavaSE Module Metadata.
• Define extensions to JavaSE module metadata to cover OSGi information
o Package Versions, Package Imports, etc...
• Wiring to existing JavaSE modules in OSGi package imports

23. Agenda
• Level setting
o Introduction to modularity
o Modules and the JRE
o The Jigsaw project
o OSGi
• Introduction to Project Penrose
o Goal of the project
o Structure of the project
o Technical roadmap
• Technical work
o Achievements
o Observations
o To be done
• Advice and call to arms

24. Final words of advice
• Stop using internal APIs and implementation classes
o You will be broken in a modular world
o Be modest in your API usage and we can likely help with a tailored JRE
• If you need a modularity story that works before Java 9 (expected summer 2015) then the
answer is OSGi
o Penrose will ensure OSGi continues to work beyond Java 9 too
• OSGi has greater expressiveness
o Experience of modularity has resulted in a rich set of behaviors
o While you may not need it now, do you want to change system when you do need it?
• We would like to see the JSR created right now so that a Java Community Expert Group can
get to work on designing the Java Module System.
• Come and participate in OpenJDK Jigsaw or Penrose projects!

25. Links
Penrose website:
http://openjdk.java.net/projects/penrose
Penrose mailing lists:
http://mail.openjdk.java.net/mailman/listinfo/penrose-dev
http://mail.openjdk.java.net/mailman/listinfo/penrose-discuss
Penrose forests:
http://hg.openjdk.java.net/penrose/jigsaw
http://hg.openjdk.java.net/penrose/penrose
JSON ModuleInfo documentation
http://hg.openjdk.java.net/penrose/penrose/raw-file/tip/doc/
PenroseModuleInfoJSON.html