Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managing Change

2,132 views

Published on

Presentation at the OSGi DevCon 2009 in Zurich.
Presenters:
Neil Bartlett
Mirko Jahn

Published in: Technology
  • Be the first to comment

Managing Change

  1. 1. Neil Bartlett Weigle Wilczek UK http://neilbartlett.name/blog Managing Change Mirko Jahn InterComponentWare AG, Germany http://osgi.mjahn.net
  2. 2. Agenda <ul><li>Where are we? </li></ul><ul><li>What we are working with </li></ul><ul><li>Versioning in practice </li></ul><ul><li>Tool support </li></ul><ul><li>Open challenges </li></ul>
  3. 3. Heraclitus Ephesius (* 535 BC; † 475 BC ) “ Nothing endures but change.“
  4. 4. Unknown Chinese Dude “ May you live in interesting times.”
  5. 5. Dealing with Change
  6. 6. How Microsoft does it <ul><li> + Unique version numbers </li></ul><ul><ul><li>+ Fixed dependencies </li></ul></ul><ul><ul><li>+ Assembly cache </li></ul></ul><ul><ul><li>Assembly qualified names </li></ul></ul><ul><ul><li>Versioning only on assemblies (not namespaces) </li></ul></ul><ul><ul><li>No flexible dependencies </li></ul></ul><ul><ul><li>No dynamism </li></ul></ul>
  7. 7. Doing it the Java way <ul><li>Jars as unit of reuse (packaging) </li></ul><ul><li>Assumed always backwards compatible </li></ul><ul><li>No enforced dependency model </li></ul><ul><li>No built-in multi version support </li></ul><ul><li>No runtime (except for Applets and JEE) </li></ul><ul><li>No dynamism anticipated, but possible with limitations </li></ul>
  8. 8. OSGi’s answer (in 2 slides) <ul><li>Thin layer on top of (plain) Java </li></ul><ul><li>Enriched meta data to thoroughly describe the unit of reuse called bundle aka jar with: </li></ul><ul><ul><li>Required dependencies </li></ul></ul><ul><ul><li>Exposed APIs </li></ul></ul><ul><ul><li>Runtime requirements </li></ul></ul><ul><ul><li>Strong version support </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>Sophisticated runtime to enforce modularity </li></ul>
  9. 9. OSGi’s answer (in 2 slides) <ul><li>In JVM services (Object references) to decouple API from implementation </li></ul><ul><li>Module dependencies designed on bundle and/ or package level </li></ul><ul><ul><li>Import-Package vs. Require-Bundle header </li></ul></ul><ul><ul><li>Exposed API with Export-Package Header </li></ul></ul><ul><li>Versioning theme: [[[[Major].Minor].Build].Revision] like: 1.3.2.iter1234 </li></ul><ul><li>Ranges to flexibly define dependencies </li></ul>
  10. 10. OSGi in Action a use case scenario
  11. 11. Evolving APIs <ul><li>Simple sample of a dictionary service </li></ul><ul><li>Taken from the Eclipse PDE sample </li></ul><ul><li>starting with 2 interfaces in package dictionary… </li></ul><ul><li>Implementation details: </li></ul><ul><ul><li>Separation in two bundles (impl and api) </li></ul></ul><ul><ul><li>Dependencies on package level </li></ul></ul><ul><ul><li>User/ Client will be a simple command extension to proof the point </li></ul></ul>V.1.0.0
  12. 12. Evolving APIs <ul><li>The implementation looks like this: </li></ul>dictionary.api 1.0.0 dictionary 1.0.0 dictionary.impl 1.0.0 [1.0.0,1.1.0) cl.client 1.0.0 [1.0.0,2.0.0) register s: dictionary.DictionaryService bind
  13. 13. Evolving APIs <ul><li>Small changes were introduced in the second version… </li></ul>V.1.1.0
  14. 14. Evolving APIs dictionary.api 1.0.0 dictionary 1.0.0 dictionary.impl 1.0.0 [1.0.0,1.1.0) cl.client 1.0.0 [1.0.0,2.0.0) register bind dictionary.api 1.1.0 dictionary 1.1.0 dictionary.impl 1.1.0 [1.0.0,1.2.0) or register cl.client 1.1.0 [1.1.0,2.0.0) may bind [1.0.0,2.0.0) or or
  15. 15. And in “real world” <ul><li>Few people are using OSGi in such a highly dynamic fashion (yet) </li></ul><ul><li>Mostly the “is this bundle compatible with me” feature of versions is used and bundles get statically deployed as a product. </li></ul><ul><li>Getting versions right is tough!!! Currently we only version exports. Imports only if we really know what we are doing and require this feature </li></ul><ul><li>Typical problems you’re facing on a daily basis: </li></ul><ul><ul><li>What dependency is it? API, Impl or Client? </li></ul></ul><ul><ul><li>Syntactic vs. Semantic incompatibilities </li></ul></ul><ul><ul><li>Method/ property shadowing in forward compatibility </li></ul></ul><ul><ul><li>Wrong/ incompatible versioned artifacts in official repositories </li></ul></ul>
  16. 16. OSGi Issues and Questions
  17. 17. Gotcha Number 1 <ul><li>Any qualifier is “higher” than no qualifier. </li></ul><ul><li>Example: ROME (RSS/Atom library), an OSGi bundle developed by Sun Microsystems. </li></ul><ul><li>Release Candidate: Bundle-Version: 1.0.0.RC1 </li></ul><ul><li>Release: Bundle-Version: 1.0.0 </li></ul><ul><li>This default is surprising for new OSGi users. </li></ul><ul><li>Other systems (e.g. Maven, Jigsaw) use the opposite default. </li></ul>
  18. 18. Related Gotcha: Remember your A,B,Cs <ul><li>Release Candidate: Bundle-Version: 1.0.0.RC1 </li></ul><ul><li>Release: Bundle-Version: 1.0.0.Final </li></ul><ul><li>Using “RC” doesn't leave enough space. </li></ul><ul><li>Recommendation: use Alpha, Beta, Candidate and Final. </li></ul><ul><li>May clash with existing corporate processes and policies for naming releases. </li></ul><ul><li>Or, just use a date/time stamp on each build. This is the Eclipse approach, but makes it hard to identify release versions. </li></ul>
  19. 19. Gotcha Number 2 <ul><li>Beta 1: Bundle-Version: 1.0.0.Beta1 </li></ul><ul><li>Beta 2: Bundle-Version: 1.0.0.Beta2 </li></ul><ul><li>… </li></ul><ul><li>Beta 10: Bundle-Version 1.0.0.Beta10 </li></ul><ul><li>Oops, 10 < 2! </li></ul><ul><li>Recommendation: zero-pad numbers in the qualifier, e.g. Beta01 . </li></ul><ul><li>Are 99 betas enough...? </li></ul><ul><li>.Net does not allow alphanumeric characters in versions at all. </li></ul>
  20. 20. Gotcha Number 3 <ul><li>Import-Package: org.foo; version=”1.0.0” </li></ul><ul><li>Accepts 1.0.0 to Infinity! </li></ul><ul><li>Do we really want to accept ALL future versions of org.foo? </li></ul><ul><li>Is this an appropriate default? </li></ul>
  21. 21. Choosing an Import Range (1) <ul><li>Import-Package: org.apache.log4j;version=”[1.2,1.3)” </li></ul><ul><li>Informally 1.2.*. Will match all future versions up to but not including 1.3. </li></ul><ul><li>The most common choice. But this is a forward-looking statement . </li></ul><ul><li>But what if 1.2.32 (released in 2014 after the acquisition of Switzerland by OraGoogSunBay) breaks compatibility? </li></ul><ul><li>Cannot retrospectively change the released bundle. Must release a new version of our bundle with different import ranges. </li></ul>
  22. 22. Choosing an Import Range (2) <ul><li>Play it safe? Versions 1.2.13, 1.2.14 and 1.2.15 have been tested and are known to work, so: </li></ul><ul><li>Import-Package: org.apache.log4j;version=”[1.2.13,1.2.15]” </li></ul><ul><li>What if 1.2.16 is released tomorrow and is absolutely fine? </li></ul><ul><li>What if another bundle requires version 1.2.12? </li></ul><ul><li>Tight version ranges cause fragmentation. </li></ul>
  23. 23. No Disjoint Ranges <ul><li>Cannot say Import-Package: org.foo;version=”[1.0.0,2.0.0)” excluding 1.2 because it is buggy </li></ul><ul><li>Should we allow definition of disjoint ranges? </li></ul><ul><li>E.g. “[1.0,1.2) && [1.3,2.0)” </li></ul><ul><li>Makes a complex area even more complex. </li></ul><ul><li>Again cannot retrospectively add the exclusion. </li></ul>
  24. 24. Lack of Semantics <ul><li>The semantics of a version change are arbitrary. </li></ul><ul><li>Is version 1.3 of Log4J compatible with version 1.2? </li></ul><ul><li>There are suggested semantics but every library must be checked for compliance (and may stop complying at any time). </li></ul><ul><li>Only the authors of a library truly know whether 1.3 is backwards compatible. </li></ul>
  25. 25. Idea: Reverse the Ranges? <ul><li>Idea originally suggested by Peter Kriens to CPEG </li></ul><ul><li>Make exporters specify a range, not importers </li></ul><ul><li>E.g. Export-Package: “[1.0.0, 1.4.0]” </li></ul><ul><li>Meaning: “this is version 1.4 and it is compatible back to 1.0”. </li></ul><ul><li>Should importers still import a range, where any intersection in ranges will match? </li></ul>
  26. 26. Non-Standard Semantics on Versions <ul><li>Usually higher means better, but not always </li></ul><ul><li>E.g. Linux kernel, 1.1 and 1.3 are unstable, 1.0 and 1.2 are stable. </li></ul><ul><li>In production we should favour 1.2.18 over 1.3.1. </li></ul><ul><li>But in an lab/development environment we may favour 1.3.1 </li></ul><ul><li>Do we need system policies defined by a deployer or administrator? </li></ul><ul><li>How on Earth could we manage such a policy if we have 1000s of packages? </li></ul><ul><li>A rule-based system may be necessary. </li></ul><ul><li>.Net allows limited overriding using a “publisher policy file”. </li></ul>
  27. 27. Bundle Repositories
  28. 28. Getting Bundles <ul><li>Most 3 rd party libraries are STILL not OSGi bundles </li></ul><ul><li>Even if they are, they may have poor-quality metadata (e.g. ROME) </li></ul><ul><li>OSGifying libraries ourselves is a major drag </li></ul>
  29. 29. Bundle Repositories <ul><li>Some vendors have created bundle repositories with many open source libraries (e.g. SpringSource, Sonatype) </li></ul><ul><li>But how reliable are they? </li></ul><ul><li>If a bundle says Import-Package: org.foo;version=”[1.0,2.0)” , did the bundle creator really test every version in that range? </li></ul><ul><li>If not, where does this range come from? Did they just guess? </li></ul><ul><li>Who can I sue if it breaks?? </li></ul>
  30. 30. Verified Repositories <ul><li>There is scope for a vendor to provide a truly verified repository backed by testing. </li></ul><ul><li>Probably needs to be a paid service – somebody who can be sued </li></ul><ul><li>What does it mean to test a library? </li></ul><ul><li>Need to support revocation when bundles are found to have problems </li></ul>
  31. 31. Enhanced Querying <ul><li>Existing repositories answer queries based on bundle name, package name, etc. </li></ul><ul><li>Can we partially offload resolving to the repository? </li></ul>
  32. 32. Tooling support
  33. 33. BND <ul><li>Created by Peter Kriens, now under the Apache Felix umbrella (ASF 2.0 license) </li></ul><ul><li>Swiss Army Knife for bundle generation </li></ul><ul><li>Can be used as: </li></ul><ul><ul><li>Command line tool </li></ul></ul><ul><ul><li>Eclipse plug-in </li></ul></ul><ul><ul><li>Ant task </li></ul></ul><ul><ul><li>Maven goal </li></ul></ul><ul><li>Templating mechanism for version ranges </li></ul><ul><li>Source code analysis for imports </li></ul><ul><li>DS support </li></ul><ul><li>Macros </li></ul>
  34. 34. BND <ul><li>A sample configuration for the Maven plug-in (shortened): </li></ul><ul><li>... </li></ul><ul><li>< plugin > </li></ul><ul><li>< groupId > org . apache . felix </ groupId > </li></ul><ul><li>< artifactId > maven -bundle- plugin </ artifactId > </li></ul><ul><li>< extensions > true </ extensions > </li></ul><ul><li>< configuration > </li></ul><ul><li>< manifestLocation > META-INF </ manifestLocation > </li></ul><ul><li>< instructions > </li></ul><ul><li><!-- enable simple spring xml file analysis --> </li></ul><ul><li>< _plugin > aQute. lib .spring.SpringComponent </ _plugin > </li></ul><ul><li>< Export-Package > ${ osgi .export.package} </ Export-Package > </li></ul><ul><li>< Private-Package > ${ osgi .private.package} </ Private-Package > </li></ul><ul><li></ instructions > </li></ul><ul><li></ configuration > </li></ul><ul><li></ plugin > </li></ul><ul><li>... </li></ul>
  35. 35. Bundlor <ul><li>SpringSource’s tool to create Manifest files only </li></ul><ul><li>Mainly a template mechanism </li></ul><ul><li>The tool to use when contributing artifacts to their repo </li></ul><ul><li>Can be used as: </li></ul><ul><ul><li>Command line tool </li></ul></ul><ul><ul><li>Eclipse plug-in </li></ul></ul><ul><ul><li>Ant task </li></ul></ul><ul><ul><li>Maven goal </li></ul></ul><ul><li>Templating mechanism for version ranges </li></ul><ul><li>Source code analysis for imports </li></ul><ul><li>OSGi Profiles </li></ul>
  36. 36. Bundlor <ul><li>A sample configuration for the Maven plug-in (taken from SpringSource): </li></ul><ul><li>Manifest-Version: 1.0 </li></ul><ul><li>Bundle-ManifestVersion: 2 </li></ul><ul><li>Bundle-Name: GreenPages Service </li></ul><ul><li>Bundle-SymbolicName: greenpages </li></ul><ul><li>Bundle-Vendor: SpringSource Inc. </li></ul><ul><li>Bundle-Version: 1.0 </li></ul><ul><li>Import-Template:org.springframework.*;version=&quot;[2.5.6.A,3.0) </li></ul><ul><li>Excluded-Exports: greenpages.internal </li></ul>
  37. 37. Eclipse PDE <ul><li>Manifest centric development of bundles </li></ul><ul><li>Many wizards and custom views to get you started </li></ul><ul><li>Live validation of your configuration based on the target platform definition </li></ul><ul><li>Integrated runtime to test you bundles without packaging them </li></ul><ul><li>Issues with fragments during development </li></ul><ul><li>Headless built possible </li></ul><ul><li>API tooling support… </li></ul>
  38. 38. Eclipse PDE – API analysis support <ul><li>Baseline centric API comparison mechanisms </li></ul><ul><li>Enriched with annotations </li></ul><ul><li>Eclipse and Ant support </li></ul><ul><li>Automated version increase according to the Eclipse versioning scheme possible </li></ul><ul><li>Identity leakage of non-API types into API </li></ul><ul><li>Many features are currently developed and will be included in Eclipse 3.5 </li></ul><ul><li>Based on source code and generated meta data </li></ul><ul><li>OSGi aware </li></ul>
  39. 39. Eclipse PDE – API analysis support
  40. 40. jDiff <ul><li>Source code based analysis </li></ul><ul><li>Can be used as Ant task, Maven goal or Command Line tool </li></ul><ul><li>Different types of reports like JavaDoc and XML </li></ul><ul><li>No OSGi awareness </li></ul>
  41. 41. jDiff
  42. 42. Remaining Questions <ul><li>Versioning is not entirely explored </li></ul><ul><li>There is no suitable version scheme officially defined (no standard) </li></ul><ul><ul><li>What is binary compatible </li></ul></ul><ul><ul><li>What is back-/ upwards compatible </li></ul></ul><ul><ul><li>In the context of OSGi how do dependency affect compatibility? </li></ul></ul><ul><li>How to ensure the correctness of the applied versions? </li></ul><ul><li>Dynamic applications need new strategies for testing. How could one ensure the compatibility with the future versions and rule out incompatibilities for an almost infinite set of bundle combinations </li></ul><ul><li>How to ensure a bundle satisfies its version definition? (Certification?) </li></ul>
  43. 43. State of the Art <ul><li>NO existing tool can manage: </li></ul><ul><ul><li>1000s of packages </li></ul></ul><ul><ul><li>Multiple versions of each </li></ul></ul><ul><ul><li>Sensible import ranges </li></ul></ul><ul><ul><li>Different import range policies for different providers </li></ul></ul>
  44. 44. Conclusion <ul><li>OSGi opens new opportunities, not imaginable before </li></ul><ul><li>Sophisticated versioning support available </li></ul><ul><li>A runtime that leverages using different versions, while guaranteeing compatibility as far as possible (if done right) </li></ul><ul><li>Still, the field is not entirely explored and needs further research </li></ul><ul><ul><li>What implies a version is not satisfyingly defined </li></ul></ul><ul><ul><li>How to ensure software quality in a dynamic world </li></ul></ul><ul><li>Tools are not there yet to remove the burden from the developer </li></ul>
  45. 45. Questions & Answers Neil Bartlett Weigle Wilczek UK http://neilbartlett.name/blog Mirko Jahn InterComponentWare AG, Germany http://osgi.mjahn.net ?

×