Ian Robinson is an IBM Distinguished Engineer and WebSphere Foundation Chief Architect who has over 20 years of experience in transaction processing and distributed enterprise computing. The document discusses how WebSphere Application Server moved to an OSGi modular architecture to allow for higher density deployments, continuous delivery of new features without breaking compatibility, and reduced hardware costs through more efficient use of resources. It describes the stages of adopting OSGi, from initial modularization to dynamic runtime deployment and management of features. The challenges of OSGi adoption for both internal components and external applications are also examined.

1. Travelling Light for the Long Haul
Ian Robinson, IBM Distinguished Engineer
WebSphere Foundation Chief Architect

2. About Me
 IBM Distinguished Engineer
 WebSphere Foundation Chief Architect
 Over 20 years experience in
transaction processing and distributed
enterprise computing
 Product strategy & development and
enterprise Java standards
 Travels a lot, based in IBM’s Hursley
lab in the UK (near Southampton).
 Season ticket holder for 3rd most
successful club in English football:
Ian Robinson

3. Traditional
system
OSGi**
“Cognitive burden”
OSGi
Time
Based on http://www.tensegrity.hellblazer.com
Knowledge About System
Complexity
Carrying Baggage Can Be Expensive
OSGi
Traditional
system
Time
http://adaptevolve.blogspot.com

4. But Losing Baggage Can Be Worse
 “Baggage” is something your users wants you to keep. Forever.
– Baggage == Business Continuity
 WebSphere’s software support statement guarantees “N-2” for
application compatibility and platform support.
 Engineering challenge to deliver the new without breaking the
old. For a long time.
– Including the crazy experiments that got out.
 If I had a pound…

5. We Needed to Travel Lighter for the Long Haul
 WebSphere AppServer and OSGi were
both born in 1998.
 By 2006 WebSphere was 10 million lines
of Java code and growing.
 Global development team of 100s in many
development labs in different timezones
around the world.
 Tens of thousands of large customer
deployments in long-term support.
 Classic struggle to increase ratio of
innovation : support
– Variable “stickiness” of innovation but
uniform expectation of support.
 “Something had to change” (Part One)
1.0
1998
2.0
1999
3.0
2000
3.5
2001
4.0
2002
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

6. OSGi Maturity Model Summary
1.0
1998
2.0
1999
3.0
3.5
2000
2001
4.0
2002
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013
Level
Name
Summary
1
2
3
4
AdHoc
Modules
Modularity
Loose-Coupling
5
Dynamism
6
Devolution
Nothing
Formal identity, decoupled from artifact
Formal module contracts, decoupled from identity
Services, semantic versioning, decoupled from
implementation
Life-cycle awareness and independence, decoupled
from time
Modularity-aware repositories, collaboration,
governance, decoupled from ownership
Graham Charters, OSGi Community Event 2011: Towards a Maturity Model

7. Pre-OSGi Modules (Build View)
Level 2
Pre-dated Maven
 Componentized Build
 Components have identity and version
 Components produce a jar
interfaces
client
impl depends on
interfaces at build
time
impl
Factory in
interfaces uses
Class.forName to
load impl at runtime
1.0
1998
2.0
1999
3.0
2000
3.5
2001
4.0
2002
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

8. Pre-OSGi Modules (Runtime View)
 Java Bean Components
Level 2
init/start
– Implements a specific
interface
stop/destroy
A
 Init/start/stop/destroy phases
– Started in a specified order
B
 Makes use of:
– Class.getResource()
– Class.forName()
C
 Runtime couldn’t enforce build
modularity
 Expensive to maintain and
extend
C

9. WebSphere Gets OSGi (2006)
Level 3
 Internal re-engineering while simultaneously
adding external business capabilities
 Best-practice approach: start with small number
of large bundles and iterate over time
Approach 1
– Runtime modularity enforced
Jar A
Jar B
Jar C
Jar D
– Service maintenance and testing better targeted
– Runtime footprint no longer monolithic
 Challenge: Significant learning experience
across worldwide team
Jar A
Approach 2
Jar B
Content of Jars
A-D
Jar C
1.0
1998
2.0
1999
3.0
2000
Jar D
3.5
2001
4.0
2002
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

10. What’s Good for the Goose…
 We had rebuilt WAS V6.1 on
top of OSGi.
 This must surely make it
easier for developers to
benefit from OSGi in their
own applications. Right?

11. OSGi Moves
Up the Stack
logging f/w jar
persistence f/w f/w jar
logging jar
persistence f/w jar
persistence f/w jar
MVC f/w jar f/w jar
persistence
MVC f/w jar
MVC f/w jar
DI f/w jar f/w jar
MVC
DI f/w jar
DI f/w jar
DI f/w jar
Import-Package
OSGi Bundle Repository: Integrated
with WAS Admin
webA.war
webA.war
WEB-INF/classes/servletA.class
webA.war
WEB-INF/classes/servletA.class
webA.war
WEB-INF/web.xml
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/classes/servletA.class
WEB-INF/lib/…
WEB-INF/web.xml
WEB-INF/lib/…
WEB-INF/web.xml
WEB-INF/lib/… jar
logging f/w
WEB-INF/lib/… jar
logging f/w
webA.jar
webA.jar
WEB-INF/classes/servletA.class
webA.jar
WEB-INF/classes/servletA.class
webA.jar
WEB-INF/web.xml
WEB-INF/classes/servletA.class
WEB-INF/web.xml
WEB-INF/classes/servletA.class
META-INF/MANIFEST.MF
WEB-INF/web.xml
META-INF/MANIFEST.MF
WEB-INF/web.xml
META-INF/MANIFEST.MF
META-INF/MANIFEST.MF
Bundle Repository
• Manage multiple
versions of libraries
across an enterprise
• Isolate application
dependencies from the
server runtime
• Centralized location to
deliver critical fixes
• Flexibility to update to
new versions one app at
a time

12. Lessons We Learned
•
•
•
We did a better job for external apps
than we did for internal components
“Services” were a key part of the App
support
A Grade:
•
•
Cheaper to maintain, extend and test
Need do Better:
•
•
Insufficient dynamism - especially in fastchanging development environment
Components too tightly-coupled – didn’t
deliver on desired lightweight footprint.

13. The Omelette Challenge
Recipe:
 Create a lightweight profile of WebSphere
that starts in under 2 seconds
 Make it completely dynamic for all
changes to configuration
 Provide an unzip install <50 Meg in size
 Don’t break any eggs.
Provide complete backward compatibility

14. zosSecurity
Java EE
Full Profile
collectiveController
zosTransaction
mongodb
wsSecurity
wmqJmsClient
jmsMdb
wasJmsClient
collectiveMember
oauth
jaxrs
servlet
json
jpa
What we had
ssl
monitor
Feature Manager
beanvalidation
localConnector
wab
jsp
Runtime Services
&
Config Model
managedBeans
restConnector
blueprint
webCache
ldapRegistry
osgi.jpa
jsf
wasJmsSecurity
wasJmsServer
cdi
ejblite
Java EE
Web Profile
jaxws
jaxb
concurrent
clusterMember
appSecurity
sessionDatabase
jndi
HTTP Transport
jdbc
Application Manager
What we wanted
Multi-bundle feature
1.0
1998
2.0
1999
3.0
2000
3.5
2001
4.0
2002
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

15. Time Independence is Fundamental
 Recognized our error in not exploiting OSGi services when we
originally adopted OSGi.
 Services were the ONLY way we could achieve the size and
dynamism objective without massive and unnecessary re-invention
 A significant consideration for component design.
– Required us to replace existing factory patterns and configuration
management.
Complexity
– Modular implementation already suitable
package import
OSGi**
B v2
A v1
C v1.1
D v1
service reference
Time

16. A La Carte Alongside the Prix Fixe
Level 5
 2012: “Liberty Profile” of WebSphere supports arbitrary
combinations of OSGi and Java EE “features”.
– Remember the eggs: any app running on the Liberty
profile of WAS runs unchanged on the full profile of WAS.
X
 Same runtime bundles (mostly) but loaded and configured by
new OSGi subsystem-aware kernel as independent feature
subsystems (new in OSGi R5)
 Entirely self-contained metadata to describe bundle content,
services published, & configuration metatypes.
 We use features as
units of:
Config
Bundle A
Feature
Manifest
Metatype.xml
– Configuration
Bundle B
1998
2.0
1999
3.0
2000
3.5
2001
4.0
2002
5.0
5.1
2003
2004
– Extensibility
“Feature”
Bundle C
1.0
– Deployment
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

17. Keep the Engine Under the Bonnet/Hood/Kühlerhaube
OSGi details:
• are available to extenders
of the platform.
• stay on the inside for
users of the platform
Bundle A
Feature
Manifest
Application developers &
operators only see this
Config
Metatype.xml
Bundle B
Bundle C
jsp-2.2
3
4
<server description=“server1”>
Feature Manager
<featureManager>
<feature>jsp-2.2</feature>
<feature>jdbc-4.0</feature>
</featureManager>
2
1
Config Admin R4.2
felix scr1.1
equinox metatype 1.2.0
<application name="tradelite"
location="tradelite.war" />
</server>
server.xml configuration
equinox framework 3.8.2 (OSGi R5.0)
WebSphere Liberty Kernel

18. The Love That Dare Not Speak Its Name
Or Why We Stopped Bragging About OSGi
Dynamic Server Profile
Not static like Web Profile; configured
by app at a fine-grained level
“Developer First” Focus
Simplified, shareable XML server config. New
integrated messaging server, DynaCache support, new
prog. models, such as Web Services, JMS & EJB-Lite.
Start fast, run efficiently
Small Download
Starts in <3s; Mem footprint
<50MB; (TradeLite benchmark)
Integrated tools
Powerful tools in WDT Eclipse
feature. Enhanced for v8.5.5 prog
models, Maven integration, ++
Web Profile Certified
Create web apps for the Java
EE Web Profile standard.
Level 5
50MB for Web Profile features
WAS v8.5.5 Liberty
Profile &
WAS Developer
Tools for Eclipse
(WDT)
Unzip install and deploy
Liberty Extensions
IM or unzip to install. New option to
deploy “server package” of app +
config + required subset of server
runtime for highest density deploy
Add custom features and
integrate 3rd party
components via Liberty
extensions interface
Dynamically Extensible
Install new features from repository
(local or remote) with no svr restart
Lightweight cluster Mgmt
Liberty servers can join a
lightweight cluster for workload
balancing and high availability
Fidelity to full profile WAS
Same reliable containers & QOS.
Develop on Liberty profile and deploy
to Liberty or full-profile WAS

19. Runtime Package Management
Level 6
 OSGi standards here to help
– Subsystems, Resolver, Repository
Liberty Repository
 Flexible runtime assembly creates
opportunity for flexible runtime delivery
– Only install what you need
 Feature repositories for developers and
runtime provisioning
– Enterprise Subsystem Archives (.esa)
content e.g. Liberty Repository or
– Subsystem metadata that refer to externally
hosted bundles e.g. Apache Karaf
servlet
Feature
Manager
1.0
1998
2.0
1999
3.0
2000
>featureManager install jsp.esa
jpa
HTTP
Transport
3.5
2001
4.0
2002
Application
Manager
5.0
5.1
2003
2004
6.0
2005
6.1
2006
8.0
7.0
2007
2008
2009
2010
2011
8.5
2012
8.5.5
2013

20. Looking Ahead:
What Are the Challenges
for OSGi?

21.  Cooking is simplified by recipes.
– OSGi needs to do this.
Complexity
The Cognitive Burden of Cookery
Traditional
system OSGi**
OSGi
 OSGi standards provide high-quality
modular specifications
 Vendors choose which specifications to
incorporate into their solutions
 BUT no separation between application
and middleware.
– And not enough recognition of the
difference
 Customers want standard solutions
 The OSGi Application Framework
proposal for rich Web applications may
help…
Time

22. OSGi Application Framework
- A spring-board to cloud
What it could be:
 A profile of specifications available for application use.
– Apps can rely on vendor solutions including these
 Re-using existing technology where applicable
 Enabling first-class exploitation of OSGi
 Focussing on 80:20 rule
– Leave room for innovation to encourage vendor adoption
 Supporting flexible provisioning depending on application
need
 Recognizing the difference between applications and
containers. Embrace container management to simplify
app burden
https://github.com/osgi/design/blob/master/rfps/rfp-0160-ApplicationFramework.pdf

23. PaaS Buildpacks and OSGi in the Cloud
 PaaS is a good opportunity for OSGi
Application Framework in the cloud
 Heroku, Cloud Foundry and other
PaaS’ have extension points for
application stacks: “buildpacks”
 Provision and scale OSGi applications
using appropriate buildpack for OSGi
stack in the cloud.
 Ideal for simplifying the provisioning of
flexible “just what’s needed” app
instances for high density in the cloud
 OSGi dynamic services a natural fit for
Execution Agent
Execution Agent
Isolated
Execution Address Space
Agent
Isolated Address Space
Execution Agent
Application
Isolated Address Space
Execution Agent
ApplicationSpace
Isolated Middleware stack
Execution Address
Agent
ApplicationSpace
Isolated Middleware stack
Address
Execution Agent
Application
Isolated Middleware stack
Address Space
Application
Isolated App Instance
Isolated Middleware stack
App Instance
Application
Middleware stack
Application stack
Middleware
Middleware stack
cloud shared services
Application
-application
-server.xml
-resources
cf push app with buildpack
> cf push project [--buildpack
https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack]
http://www.ibmdw.net/wasdev/docs/deploying-an-osgi-app-to-liberty-in-the-cloud/
http://thoughtsoncloud.com/index.php/2013/10/possibilities-abound-with-osgi-running-on-cloud-foundry/
PaaS

24. Higher Density  Less Hardware  Less Cost
 Make it Small: Provision the smallest middleware stack needed
– For the Java stack, IBM is doing this using OSGi (WebSphere Liberty
buildpack) and IBM Java.
Feature 3
 Make it Smaller: IBM Multitenant
Feature 1
Feature 2
JVM*
– Isolated tenants
– Per-tenant Statics
– Control: threads, memory, cpu
Execution Agent
Isolated App Instance
Isolated App Instance
Application
Middleware stack
Application tenant 2
Application tenant 1
* http://www.ibm.com/developerworks/library/j-multitenant-java/

25. Are We Nearly There Yet?
Software Engineering view
of Line of Business
LoB view of
Software (over) Engineering
Carrying baggage is part of the business.
Strategies to reduce its cost are as important
now as they were 40 years ago.
Technologies like OSGi help.
They help best when you know how, when
and to whom to sell it to internally in your
organization.

26. Thank You
@ian__robinson
http://wasdev.net