Travelling light for the long haul

  • 3,279 views
Uploaded on

My keynote at Eclipsecon Europe 2013. …

My keynote at Eclipsecon Europe 2013.

One of the attractive qualities of OSGi is its role in enabling technologies that adopt it to manage the cost of their own success. Anything that gains adoption - in technology or elsewhere - picks up baggage as a result and needs to figure out how to deal with current installations while expanding in new directions. The WebSphere platform has been around for almost as long as Java and knows a thing or two about baggage but still manages to travel to many places with just a carry-on allowance. We adopted OSGi internally 8 years ago and have gradually increased our exploitation with each passing release, most recently and deeply with the lightweight WAS Liberty Profile. It hasn't all been plain sailing and we learned from a number of mistakes made along the way. When WebSphere Application Server first adopted OSGi it had over 10 million lines of code in a modest number of huge JARs. The engineering effort to modularize that into a “sensible” number of OSGi bundles was fairly significant. We had a global development team spread across a dozen labs and nearly as many timezones, all learning OSGi principles at the same time. What could possibly go wrong? We did not, for example, initially adopt the services part of the OSGi architecture but it’s how we can now start/stop individual technologies of the Java EE Web Profile on the WAS Liberty profile, in a 50MB install with a 2-second startup, while still supporting a massive deploy base of applications on older levels of Java EE.

One of the challenges OSGi continues to face is over when to be “front of office” and when to be “back”. As the industry accelerates towards cloud, OSGi is an internal part of IBM’s strategy for high-density virtualized Platform-as-a-Service through WebSphere Liberty. Today’s cloud provisioning strategies, for example the buildpacks used by Heroku and CloudFoundry, are designed to be technology-agnostic. As a programming model for the cloud, OSGi is in a position of strength with its heavily service-oriented architecture. But in the spirit of agnosticism, one of the next steps OSGi needs to take is simply greater availability of the core OSGi framework in some of the more popular cloud platforms. Once there are more OSGi services running in those environments then the value and simplicity of autowiring OSGi services as cloud services becomes more apparent. Expectations and vision has to be managed up and down any organization that invests in OSGi - from the executive leadership team responsible for the business's bottom line, through the distributed architecture/development teams building tomorrow's technology on top of today’s, to the marketing and sales organization who need to sell the result to both IT and line of business. The value proposition has to be tailored to the audience.

This is the story of how WebSphere has had outstanding success with the former four-letter acronym that IBM Marketing still wants to expand.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
3,279
On Slideshare
0
From Embeds
0
Number of Embeds
46

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Travelling Light for the Long Haul Ian Robinson, IBM Distinguished Engineer WebSphere Foundation Chief Architect
  • 2. About Me  IBM Distinguished Engineer  WebSphere Foundation Chief Architect  Over 20 years experience in transaction processing and distributed enterprise computing  Product strategy & development and enterprise Java standards  Travels a lot, based in IBM’s Hursley lab in the UK (near Southampton).  Season ticket holder for 3rd most successful club in English football: Ian Robinson
  • 3. Traditional system OSGi** “Cognitive burden” OSGi Time Based on http://www.tensegrity.hellblazer.com Knowledge About System Complexity Carrying Baggage Can Be Expensive OSGi Traditional system Time http://adaptevolve.blogspot.com
  • 4. But Losing Baggage Can Be Worse  “Baggage” is something your users wants you to keep. Forever. – Baggage == Business Continuity  WebSphere’s software support statement guarantees “N-2” for application compatibility and platform support.  Engineering challenge to deliver the new without breaking the old. For a long time. – Including the crazy experiments that got out.  If I had a pound…
  • 5. We Needed to Travel Lighter for the Long Haul  WebSphere AppServer and OSGi were both born in 1998.  By 2006 WebSphere was 10 million lines of Java code and growing.  Global development team of 100s in many development labs in different timezones around the world.  Tens of thousands of large customer deployments in long-term support.  Classic struggle to increase ratio of innovation : support – Variable “stickiness” of innovation but uniform expectation of support.  “Something had to change” (Part One) 1.0 1998 2.0 1999 3.0 2000 3.5 2001 4.0 2002 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 6. OSGi Maturity Model Summary 1.0 1998 2.0 1999 3.0 3.5 2000 2001 4.0 2002 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013 Level Name Summary 1 2 3 4 AdHoc Modules Modularity Loose-Coupling 5 Dynamism 6 Devolution Nothing Formal identity, decoupled from artifact Formal module contracts, decoupled from identity Services, semantic versioning, decoupled from implementation Life-cycle awareness and independence, decoupled from time Modularity-aware repositories, collaboration, governance, decoupled from ownership Graham Charters, OSGi Community Event 2011: Towards a Maturity Model
  • 7. Pre-OSGi Modules (Build View) Level 2 Pre-dated Maven  Componentized Build  Components have identity and version  Components produce a jar interfaces client impl depends on interfaces at build time impl Factory in interfaces uses Class.forName to load impl at runtime 1.0 1998 2.0 1999 3.0 2000 3.5 2001 4.0 2002 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 8. Pre-OSGi Modules (Runtime View)  Java Bean Components Level 2 init/start – Implements a specific interface stop/destroy A  Init/start/stop/destroy phases – Started in a specified order B  Makes use of: – Class.getResource() – Class.forName() C  Runtime couldn’t enforce build modularity  Expensive to maintain and extend C
  • 9. WebSphere Gets OSGi (2006) Level 3  Internal re-engineering while simultaneously adding external business capabilities  Best-practice approach: start with small number of large bundles and iterate over time Approach 1 – Runtime modularity enforced Jar A Jar B Jar C Jar D – Service maintenance and testing better targeted – Runtime footprint no longer monolithic  Challenge: Significant learning experience across worldwide team Jar A Approach 2 Jar B Content of Jars A-D Jar C 1.0 1998 2.0 1999 3.0 2000 Jar D 3.5 2001 4.0 2002 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 10. What’s Good for the Goose…  We had rebuilt WAS V6.1 on top of OSGi.  This must surely make it easier for developers to benefit from OSGi in their own applications. Right?
  • 11. OSGi Moves Up the Stack logging f/w jar persistence f/w f/w jar logging jar persistence f/w jar persistence f/w jar MVC f/w jar f/w jar persistence MVC f/w jar MVC f/w jar DI f/w jar f/w jar MVC DI f/w jar DI f/w jar DI f/w jar Import-Package OSGi Bundle Repository: Integrated with WAS Admin webA.war webA.war WEB-INF/classes/servletA.class webA.war WEB-INF/classes/servletA.class webA.war WEB-INF/web.xml WEB-INF/classes/servletA.class WEB-INF/web.xml WEB-INF/classes/servletA.class WEB-INF/lib/… WEB-INF/web.xml WEB-INF/lib/… WEB-INF/web.xml WEB-INF/lib/… jar logging f/w WEB-INF/lib/… jar logging f/w webA.jar webA.jar WEB-INF/classes/servletA.class webA.jar WEB-INF/classes/servletA.class webA.jar WEB-INF/web.xml WEB-INF/classes/servletA.class WEB-INF/web.xml WEB-INF/classes/servletA.class META-INF/MANIFEST.MF WEB-INF/web.xml META-INF/MANIFEST.MF WEB-INF/web.xml META-INF/MANIFEST.MF META-INF/MANIFEST.MF Bundle Repository • Manage multiple versions of libraries across an enterprise • Isolate application dependencies from the server runtime • Centralized location to deliver critical fixes • Flexibility to update to new versions one app at a time
  • 12. Lessons We Learned • • • We did a better job for external apps than we did for internal components “Services” were a key part of the App support A Grade: • • Cheaper to maintain, extend and test Need do Better: • • Insufficient dynamism - especially in fastchanging development environment Components too tightly-coupled – didn’t deliver on desired lightweight footprint.
  • 13. The Omelette Challenge Recipe:  Create a lightweight profile of WebSphere that starts in under 2 seconds  Make it completely dynamic for all changes to configuration  Provide an unzip install <50 Meg in size  Don’t break any eggs. Provide complete backward compatibility
  • 14. zosSecurity Java EE Full Profile collectiveController zosTransaction mongodb wsSecurity wmqJmsClient jmsMdb wasJmsClient collectiveMember oauth jaxrs servlet json jpa What we had ssl monitor Feature Manager beanvalidation localConnector wab jsp Runtime Services & Config Model managedBeans restConnector blueprint webCache ldapRegistry osgi.jpa jsf wasJmsSecurity wasJmsServer cdi ejblite Java EE Web Profile jaxws jaxb concurrent clusterMember appSecurity sessionDatabase jndi HTTP Transport jdbc Application Manager What we wanted Multi-bundle feature 1.0 1998 2.0 1999 3.0 2000 3.5 2001 4.0 2002 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 15. Time Independence is Fundamental  Recognized our error in not exploiting OSGi services when we originally adopted OSGi.  Services were the ONLY way we could achieve the size and dynamism objective without massive and unnecessary re-invention  A significant consideration for component design. – Required us to replace existing factory patterns and configuration management. Complexity – Modular implementation already suitable package import OSGi** B v2 A v1 C v1.1 D v1 service reference Time
  • 16. A La Carte Alongside the Prix Fixe Level 5  2012: “Liberty Profile” of WebSphere supports arbitrary combinations of OSGi and Java EE “features”. – Remember the eggs: any app running on the Liberty profile of WAS runs unchanged on the full profile of WAS. X  Same runtime bundles (mostly) but loaded and configured by new OSGi subsystem-aware kernel as independent feature subsystems (new in OSGi R5)  Entirely self-contained metadata to describe bundle content, services published, & configuration metatypes.  We use features as units of: Config Bundle A Feature Manifest Metatype.xml – Configuration Bundle B 1998 2.0 1999 3.0 2000 3.5 2001 4.0 2002 5.0 5.1 2003 2004 – Extensibility “Feature” Bundle C 1.0 – Deployment 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 17. Keep the Engine Under the Bonnet/Hood/Kühlerhaube OSGi details: • are available to extenders of the platform. • stay on the inside for users of the platform Bundle A Feature Manifest Application developers & operators only see this Config Metatype.xml Bundle B Bundle C jsp-2.2 3 4 <server description=“server1”> Feature Manager <featureManager> <feature>jsp-2.2</feature> <feature>jdbc-4.0</feature> </featureManager> 2 1 Config Admin R4.2 felix scr1.1 equinox metatype 1.2.0 <application name="tradelite" location="tradelite.war" /> </server> server.xml configuration equinox framework 3.8.2 (OSGi R5.0) WebSphere Liberty Kernel
  • 18. The Love That Dare Not Speak Its Name Or Why We Stopped Bragging About OSGi Dynamic Server Profile Not static like Web Profile; configured by app at a fine-grained level “Developer First” Focus Simplified, shareable XML server config. New integrated messaging server, DynaCache support, new prog. models, such as Web Services, JMS & EJB-Lite. Start fast, run efficiently Small Download Starts in <3s; Mem footprint <50MB; (TradeLite benchmark) Integrated tools Powerful tools in WDT Eclipse feature. Enhanced for v8.5.5 prog models, Maven integration, ++ Web Profile Certified Create web apps for the Java EE Web Profile standard. Level 5 50MB for Web Profile features WAS v8.5.5 Liberty Profile & WAS Developer Tools for Eclipse (WDT) Unzip install and deploy Liberty Extensions IM or unzip to install. New option to deploy “server package” of app + config + required subset of server runtime for highest density deploy Add custom features and integrate 3rd party components via Liberty extensions interface Dynamically Extensible Install new features from repository (local or remote) with no svr restart Lightweight cluster Mgmt Liberty servers can join a lightweight cluster for workload balancing and high availability Fidelity to full profile WAS Same reliable containers & QOS. Develop on Liberty profile and deploy to Liberty or full-profile WAS
  • 19. Runtime Package Management Level 6  OSGi standards here to help – Subsystems, Resolver, Repository Liberty Repository  Flexible runtime assembly creates opportunity for flexible runtime delivery – Only install what you need  Feature repositories for developers and runtime provisioning – Enterprise Subsystem Archives (.esa) content e.g. Liberty Repository or – Subsystem metadata that refer to externally hosted bundles e.g. Apache Karaf servlet Feature Manager 1.0 1998 2.0 1999 3.0 2000 >featureManager install jsp.esa jpa HTTP Transport 3.5 2001 4.0 2002 Application Manager 5.0 5.1 2003 2004 6.0 2005 6.1 2006 8.0 7.0 2007 2008 2009 2010 2011 8.5 2012 8.5.5 2013
  • 20. Looking Ahead: What Are the Challenges for OSGi?
  • 21.  Cooking is simplified by recipes. – OSGi needs to do this. Complexity The Cognitive Burden of Cookery Traditional system OSGi** OSGi  OSGi standards provide high-quality modular specifications  Vendors choose which specifications to incorporate into their solutions  BUT no separation between application and middleware. – And not enough recognition of the difference  Customers want standard solutions  The OSGi Application Framework proposal for rich Web applications may help… Time
  • 22. OSGi Application Framework - A spring-board to cloud What it could be:  A profile of specifications available for application use. – Apps can rely on vendor solutions including these  Re-using existing technology where applicable  Enabling first-class exploitation of OSGi  Focussing on 80:20 rule – Leave room for innovation to encourage vendor adoption  Supporting flexible provisioning depending on application need  Recognizing the difference between applications and containers. Embrace container management to simplify app burden https://github.com/osgi/design/blob/master/rfps/rfp-0160-ApplicationFramework.pdf
  • 23. PaaS Buildpacks and OSGi in the Cloud  PaaS is a good opportunity for OSGi Application Framework in the cloud  Heroku, Cloud Foundry and other PaaS’ have extension points for application stacks: “buildpacks”  Provision and scale OSGi applications using appropriate buildpack for OSGi stack in the cloud.  Ideal for simplifying the provisioning of flexible “just what’s needed” app instances for high density in the cloud  OSGi dynamic services a natural fit for Execution Agent Execution Agent Isolated Execution Address Space Agent Isolated Address Space Execution Agent Application Isolated Address Space Execution Agent ApplicationSpace Isolated Middleware stack Execution Address Agent ApplicationSpace Isolated Middleware stack Address Execution Agent Application Isolated Middleware stack Address Space Application Isolated App Instance Isolated Middleware stack App Instance Application Middleware stack Application stack Middleware Middleware stack cloud shared services Application -application -server.xml -resources cf push app with buildpack > cf push project [--buildpack https://github.com/cloudfoundry/ibm-websphere-liberty-buildpack] http://www.ibmdw.net/wasdev/docs/deploying-an-osgi-app-to-liberty-in-the-cloud/ http://thoughtsoncloud.com/index.php/2013/10/possibilities-abound-with-osgi-running-on-cloud-foundry/ PaaS
  • 24. Higher Density  Less Hardware  Less Cost  Make it Small: Provision the smallest middleware stack needed – For the Java stack, IBM is doing this using OSGi (WebSphere Liberty buildpack) and IBM Java. Feature 3  Make it Smaller: IBM Multitenant Feature 1 Feature 2 JVM* – Isolated tenants – Per-tenant Statics – Control: threads, memory, cpu Execution Agent Isolated App Instance Isolated App Instance Application Middleware stack Application tenant 2 Application tenant 1 * http://www.ibm.com/developerworks/library/j-multitenant-java/
  • 25. Are We Nearly There Yet? Software Engineering view of Line of Business LoB view of Software (over) Engineering Carrying baggage is part of the business. Strategies to reduce its cost are as important now as they were 40 years ago. Technologies like OSGi help. They help best when you know how, when and to whom to sell it to internally in your organization.
  • 26. Thank You @ian__robinson http://wasdev.net