p2, modular provisioning for OSGi Pascal Rapicault IBM Rational TM , p2 lead

OSGi proves the power of componentization Componentization naturally spreads More components -> more management Management is hard It’s all about the Contract Defining Instantiating Executing Maintaining OSGi is good for you!

OSGi proves the power of componentization

Componentization naturally spreads

More components -> more management

Management is hard

It’s all about the Contract

Defining

Instantiating

Executing

Maintaining

How does p2 help? Manages the contract Dependencies Code Settings (VM args, start level, etc) Integrations Non-OSGi parts (e.g. native launcher) Extensible GUI and Headless One consistent model from installation to servicing

Manages the contract

Dependencies

Code

Settings (VM args, start level, etc)

Integrations

Non-OSGi parts (e.g. native launcher)

Extensible

GUI and Headless

One consistent model from installation to servicing

A provisioning solution for OSGi™ systems Managing non-running instance (only on Equinox for now) Start level, framework extension Fine-grained dependency management An extensible provisioning platform p2 is about installing!

A provisioning solution for OSGi™ systems

Managing non-running instance (only on Equinox for now)

Start level, framework extension

Fine-grained dependency management

An extensible provisioning platform

Concepts Architecture Tooling

Concepts

Architecture

Tooling

One construct to rule them all Decouple decision making from the actual content Everything is an IU Everything is installable It is also referred to as “metadata” IU (id, ver)

Anatomy of an IU Provided Capabilities Required Capabilities Properties Artifact reference Actions IU (id, ver)

Anatomy of an IU, requirements / capabilities Capabilities and requirements are the mechanism by which an IU express what it provides and what needs. A capability is composed of a: Namespace (string), name (string) and version A requirement is composed of a: Namespace, name and version range Namespace, name and version are open ended. The requirements and capabilities expressed by IUs can be arbitrary and don’t have to all be in the same namespace. Resources – RPMs, .exes, docs, … Virtual – if you can define a capability, I can depend on it

Capabilities and requirements are the mechanism by which an IU express what it provides and what needs.

A capability is composed of a:

Namespace (string), name (string) and version

A requirement is composed of a:

Namespace, name and version range

Namespace, name and version are open ended.

The requirements and capabilities expressed by IUs can be arbitrary and don’t have to all be in the same namespace.

Resources – RPMs, .exes, docs, …

Virtual – if you can define a capability, I can depend on it

Separation of concerns Installable Unit Exists independently of the repository Metadata Repository Store only Installable Units. API, no specified serialization format Artifact Repository Store only Artifacts API, no specified serialization or layout format Metadata Artifacts

Installable Unit

Exists independently of the repository

Metadata Repository

Store only Installable Units.

API, no specified serialization format

Artifact Repository

Store only Artifacts

API, no specified serialization or layout format

Artifacts Bytes/content to be installed Any form JARs (e.g., bundles, features, …)‏ Binary executables RPM, MSI, … Defined, maintained, loaded and used separately from the metadata

Bytes/content to be installed

Any form

JARs (e.g., bundles, features, …)‏

Binary executables

RPM, MSI, …

Defined, maintained, loaded and used separately from the metadata

Concepts Architecture Tooling

Concepts

Architecture

Tooling

p2 Architecture Metadata Artifacts Runtimes Profiles OS Eclipse Engine Eclipse Classic Eclipse for C++ Other Director Repositories Touchpoints

Terminology p2 / Agent The provisioning infrastructure on client machines Installable Unit (IU) Metadata that describes things that can be installed/configured Artifact The actual content being installed/configured(e.g., bundle JARs) Repository A store of metadata or artifacts Profile The target of install/management operations Planner The decision-making entity in the provisioning system Engine The mechanism for executing provisioning requests Touchpoint The part of the engine responsible for integrating the provisioning system to a particular runtime or management system

p2 / Agent

The provisioning infrastructure on client machines

Installable Unit (IU)

Metadata that describes things that can be installed/configured

Artifact

The actual content being installed/configured(e.g., bundle JARs)

Repository

A store of metadata or artifacts

Profile

The target of install/management operations

Planner

The decision-making entity in the provisioning system

Engine

The mechanism for executing provisioning requests

Touchpoint

The part of the engine responsible for integrating the provisioning system to a particular runtime or management system

p2 in Action – internal management - milli Transports Http/Https File system Volume Director Profiles Runtimes Provisioning operation requested Metadata fetched and constraints analyzed IU install, uninstall, update operations Artifact availability and mirroring Mirroring Data transfer IUs configured into runtimes Profile updated Repositories p2 Update Site Engine OSGi Native/OS

External management - nano Managed application Agent Director OSGi fwk The agent writes out file to control the application. The “provisioning” presence in the managed application is small. Engine OSGi Native/OS Repositories p2 Update Site Transports Http/Https File system Volume

External management - micro Director OSGi fwk The decision making on what to install is done on the agent and communicated to the managed application (no remoting provided in the open source). The agent could also carry a copy of the profile registry. Demo - http://eclipsesource.com/blogs/2009/05/05/remote-provisioning-with-p2 / Managed application Agent Engine OSGi Native/OS Repositories p2 Update Site Transports Http/Https File system Volume Repositories p2 Update Site

Concepts Architecture Tooling

Concepts

Architecture

Tooling

Build, when the metadata comes to be Metadata matters The p2.publisher is responsible for metadata / artifact generation Produces p2 metadata from bundles, Eclipse features and products Can be used in any build system PDE Build is the richest (from source to repo) Maven / tycho integration Not all metadata can be inferred Actions to be executed on a given phase Some tweaking is necessary Author metadata advice

Metadata matters

The p2.publisher is responsible for metadata / artifact generation

Produces p2 metadata from bundles, Eclipse features and products

Can be used in any build system

PDE Build is the richest (from source to repo)

Maven / tycho integration

Not all metadata can be inferred

Actions to be executed on a given phase

Some tweaking is necessary

Author metadata advice

Repository management Everybody creates repos, they need to be managed Problems You want to promote one build over another one Composite repositories You build more than you want to make available Slicing You want to replicate builds from one repository to another Mirroring applications Repository validation Repository validation that everything is installable For every IU, each artifact is available IU comparison tools, to ensure that the metadata is not changed Artifact comparison tool to ensure that one artifact has not changed [1] Repository diff’ing tool [1] - Talk on versioning and provisioning - http://www.eclipsecon.org/2009/sessions?id=585

Everybody creates repos, they need to be managed

Problems

You want to promote one build over another one

Composite repositories

You build more than you want to make available

Slicing

You want to replicate builds from one repository to another

Mirroring applications

Repository validation

Repository validation that everything is installable

For every IU, each artifact is available

IU comparison tools, to ensure that the metadata is not changed

Artifact comparison tool to ensure that one artifact has not changed [1]

Repository diff’ing tool

What is p2? A provisioning solution for OSGi™ systems An extensible provisioning platform A complete offering from Build time to Runtime A community and an ecosystem

A provisioning solution for OSGi™ systems

An extensible provisioning platform

A complete offering from Build time to Runtime

A community and an ecosystem

References p2 landing page http://wiki.eclipse.org/Equinox_p2 All p2 articles http://wiki.eclipse.org/Category:Equinox_p2 Getting the code http://wiki.eclipse.org/Equinox_p2_Getting_Started_for_Developers Contacting us Equinox newsgroup Mailing list: p2-dev@eclipse.org

p2 landing page http://wiki.eclipse.org/Equinox_p2

All p2 articles http://wiki.eclipse.org/Category:Equinox_p2

Getting the code http://wiki.eclipse.org/Equinox_p2_Getting_Started_for_Developers

Contacting us

Equinox newsgroup

Mailing list: p2-dev@eclipse.org

Appendix

Appendix

Profile A profile is the complete description in terms of IU of what is installed A profile contains Properties defining the “environment” such as os, ws, arch, install location, bundle pool location The list of IUs Properties associated with IUs Class: Org.eclipse.equinox.internal.provisional.p2.engine.IProfile

A profile is the complete description in terms of IU of what is installed

A profile contains

Properties defining the “environment” such as os, ws, arch, install location, bundle pool location

The list of IUs

Properties associated with IUs

Class: Org.eclipse.equinox.internal.provisional.p2.engine.IProfile

Planning a profile change Because of inter IU dependencies, modification against a profile should be planned ProfileChangeRequest Capture the changes you want to make to the profile (e.g. Install, Uninstall). The request is processed by the Planner An update is a removal and an addition Planner The entity responsible for the evaluation of the change request, computing the transitive closure and checking the dependencies If a solution exists it will find it ProvisioningPlan The planner returns a provisioning plan. The planning succeed pass or fail. Upon failure explanations are provided. Upon success the plan returns a set of operands to go from the initial state of the profile to the desired state.

Because of inter IU dependencies,

modification against a profile should be planned

ProfileChangeRequest

Capture the changes you want to make to the profile (e.g. Install, Uninstall).

The request is processed by the Planner

An update is a removal and an addition

Planner

The entity responsible for the evaluation of the change request, computing the transitive closure and checking the dependencies

If a solution exists it will find it

ProvisioningPlan

The planner returns a provisioning plan. The planning succeed pass or fail. Upon failure explanations are provided. Upon success the plan returns a set of operands to go from the initial state of the profile to the desired state.

Engine The mechanism by which the profile is actually changed The engine ensures consistency of the modification by performing a “transaction”. Runs over a set of operands (usually resulting from a planning operation) and executes a given set of phases on them. No profile consistency validation is done Engine.perform(Operand[] ops, PhaseSet phases, IProgressMonitor pm) For each phase, the engine interprets from the touchpoint data of the IU, the action specific to the phase. The engine looks up the action to execute. Currently 8 phases are defined, and the typical set of phases (and their order) is defined in DefaultPhaseSet . The design allows for new phases could be added. The engine emit events on the IProvisoningEventBus to describe what is happening. Events on the bus are post events and can not be vetoed.

The mechanism by which the profile is actually changed

The engine ensures consistency of the modification by performing a “transaction”.

Runs over a set of operands (usually resulting from a planning operation) and executes a given set of phases on them. No profile consistency validation is done

Engine.perform(Operand[] ops, PhaseSet phases, IProgressMonitor pm)

For each phase, the engine interprets from the touchpoint data of the IU, the action specific to the phase. The engine looks up the action to execute.

Currently 8 phases are defined, and the typical set of phases (and their order) is defined in DefaultPhaseSet . The design allows for new phases could be added.

The engine emit events on the IProvisoningEventBus to describe what is happening. Events on the bus are post events and can not be vetoed.

Installable Unit Fragments An installable unit fragment is an entity that attaches to an installable unit. Much like OSGi fragments, IU fragments are used to complement an existing installable unit and appear as one with the IU they attached to. They are typically used to deliver action to an installable unit (e.g. start level) because IUs should stay as context agnostic as possible IU fragments can be attached to several IU at the same time. See for example the tooling.osgi.bundle.default IU that applies to all bundles. Note that IU fragments are not how OSGi fragments are delivered. OSGi fragments are delivered as regular IU with requirements on their host.

An installable unit fragment is an entity that attaches to an installable unit.

Much like OSGi fragments, IU fragments are used to complement an existing installable unit and appear as one with the IU they attached to.

They are typically used to deliver action to an installable unit (e.g. start level) because IUs should stay as context agnostic as possible

IU fragments can be attached to several IU at the same time. See for example the tooling.osgi.bundle.default IU that applies to all bundles.

Note that IU fragments are not how OSGi fragments are delivered. OSGi fragments are delivered as regular IU with requirements on their host.

Installable Unit Patches An installable unit patch has the ability to “modify” the requirements of any other installable unit. Deal with three concerns: The IUs to which the patch apply The lifecycle of patch The changes applied Note: In 3.5 the feature patch editor only exposes some of those capabilities .

An installable unit patch has the ability to “modify” the requirements of any other installable unit.

Deal with three concerns:

The IUs to which the patch apply

The lifecycle of patch

The changes applied