An analysis of the process and developer impact of converting the Eclipse Papyrus source projects to Maven projects using the M2Eclipse Maven Nature.

1. DIA Agency, Inc.
Maven Nature in Papyrus
Source projects
Prepared for: The Eclipse Papyrus Project
Committer Team
Prepared by: Christian W. Damus
18 February, 2015

2. DIA Agency, Inc.
Background
Motivation
It has been proposed to convert the source projects (primarily plug-in projects and
feature projects) in the Papyrus git repository to “m2e projects”, more formally adding
the Maven Nature to these projects and enabling the Maven Project Builder. Reasons
include but may not be limited to:
•Alignment with the mechanisms of the Hudson build from which the Papyrus
integration and release builds are published, which in the Mars release are based
on the Tycho plug-ins for Maven
•Integration of new build processes/steps, such as generation of code from EMF
generator models , which have already been implemented as Maven plug-ins and1
would otherwise need to be re-implemented as Eclipse builders (perhaps tied to
additional new project natures)
Concerns
As it constitutes a significant change to the management and behaviour of the Papyrus
source projects in every developer’s workbench, there are naturally concerns raised
about potential impact on developer workflows and productivity. Also, the historical
experience of individual developers with the technologies involved (Maven, Eclipse
m2e) raises questions. Most concerns relate to performance of Eclipse and scalability of
the implementation in workspaces that import most or all of the Papyrus source
projects:
See, for example, the Gerrit change implementing EMF model generation: https://git.eclipse.org/r/#/c/41167/1
Maven Nature in Papyrus Source Projects of1 13

3. DIA Agency, Inc.
•Off-line work: does a workspace build with Maven projects need to be on-line to
check remote repositories and download artifacts?
•The Papyrus builds currently perform very poorly in the project dependencies
resolution phase. Will workspace builds manifest similar performance issues?
•Do Maven projects actually run maven builds to accomplish a workspace build?
•Does this scale to the several hundred projects that comprise the complete Papyrus
source base?
Analysis
To answer the questions and concerns raised above, it is prudent to actually try an
experiment to prove one way or another the performance and usability of a large
Papyrus workspace when all of the source projects have the Maven Nature enabled.
This section records observations from just such an experiment, exercising various
scenarios of a typical Papyrus developer’s activities on a modern computer system: a
mid 2012 Retina MacBook Pro with a 750GB SSD, quad-core Intel Core i7, 16 MB RAM
running OS X 10.10.2.
Initial Import
The experiment started with a fresh workspace into which all of the Papyrus “main”
and “main tests” projects were imported using the Oomph Setup model and built in the
normal way, from the git commit ff639eed9c4371bfa80f3bc987229a9efe7deb3a (18 Feb
2015 09:15:23 EST) on the master branch. Next, certain Maven preferences were
configured to ensure that the experiment was well isolated:
Maven Nature in Papyrus Source Projects of2 13

4. DIA Agency, Inc.
Ensuring that Maven does not update anything from the Internet unnecessarily
Using a dedicated local Maven repository stored in a mounted disk image for ease of clean-up
Next, all projects in the Project Explorer were selected and converted to m2e projects
using the Configure → Convert to Maven Project context menu action. This initiated a
Maven Nature in Papyrus Source Projects of3 13

5. DIA Agency, Inc.
lot of automatic work, pegging the CPU for more than 20 minutes (the timing is
unknown due to absence from the office) but it did complete normally, with the Eclipse
workbench settling down to idleness without any error dialogs. However, the result
was not ready to use, as none of the projects yet understood the Tycho plug-in required
for successful build:
Project build configuration errors caused by missing lifecycle mappings for Tycho plug-ins
This is remedied by installing the Tycho connector for m2e and restarting Eclipse:
The m2e connector discovery automatically finds the Tycho connector
and then proceeding with the installation:
Maven Nature in Papyrus Source Projects of4 13

6. DIA Agency, Inc.
Installing the Tycho connector for m2e
This connector should be added to the Papyrus Setup model before any conversion of
the Papyrus projects to Maven Nature is implemented, so that developers do not have
to go through the connector discovery process themselves.
On restarting Eclipse after completing this installation, all of the source projects could
now be built. The resulting impact on the git workspace is considerable:
Converting to Maven projects results in 881 changes in git: new, changed, and deleted metadata files
Maven Nature in Papyrus Source Projects of5 13

7. DIA Agency, Inc.
But already this exercise has been illuminating because it highlights some potentially
significant problems, including:
Some projects that should be in the Papyrus release clearly aren’t being built by Hudson
Other projects still have inconsistent build metadata despite recent efforts to clean that up
Maven Nature in Papyrus Source Projects of6 13

8. DIA Agency, Inc.
Developer Workflows
The initial import and configuration of projects discussed above is a one-time cost that
will be absorbed by whichever committer does the Maven conversion. It should not be
something that any other Papyrus developer deals with. This section looks at typical
development scenarios that Papyrus developers do see every day.
Workspace Re-build
From time to time, Eclipse needs to re-build a significant subset of the projects in the
workspace (or even all of them) when some build configuration changes. That may be
default compiler settings or the dependencies declared in a bundle manifest. In the test
workspace, out of 441 projects, most depend on the org.eclipse.papyrus.infra.core.emf
plug-in either directly or indirectly. Changing the dependencies in this plug-in’s
manifest triggers a rebuild (because ‘build automatically’ is on) of this project and all of
its dependents. This re-build:
•took 2 minutes and 10 seconds the first time. Subsequent re-builds of a similar
nature would typically only take about a minute
•this compares to about 45 seconds currently in the regular PDE projects
•resulted in 1800+ new compile errors in the Class Diagram plug-in (that first time)
•conversion to Maven project updated the plug-in classpath, exposing a
disagreement between the required execution environment (BREE) and its
compiler settings. These are the compile errors pictured above. This is a good
result and easily fixed
•resulted in two errors on MANIFEST.MF in a test fragment because the Maven
conversion removed source path entries from the .classpath file (probably because
the build.properties file omitted the necessary source path specification)
•on fixing this, a dependency on the wrong Java SE version was revealed in the
same fragment
So, in fact, the performance of re-build is comparable to the status quo and the Maven
project nature highlights further build inconsistencies that were not revealed by the
Maven Nature in Papyrus Source Projects of7 13

9. DIA Agency, Inc.
Hudson build set-up. Notably, re-builds triggered by changes in API signatures that are
used by other projects are practically instantaneous. These are far more common than
the clean re-builds required by classpath changes.
Xtend Projects
Sometimes the Java sources generated from Xtend drift a bit out of sync and we force a
re-build by cleaning the project via the Project → Clean… menu action. In the
Mavenized org.eclipse.papyrus.uml.profile.assistants.generator projects this takes on
the order of a few seconds, comparable to the status quo. It is worth noting that this is
an Xtend project in which the generated Java sources are not committed to the git
repository.
Run-time Workbenches
A lot of a developer’s testing is done in a run-time workbench launched from the
development instance. This shows no problems, working as usual. In particular, the
plug-in configuration of the run-time instance is assembled from the PDE target and the
projects in the workspace. Likewise JUnit plug-in test launches.
Maven Projects Under the Hood
It is evident that m2e actually does not perform a maven build at all when building
most projects. Its primary function is to configure the project’s classpath based on the
dependencies specified in the pom.xml for the benefit of the regular JDT build. For the
most part, all build lifecycle phases are mapped to the internal builders provided by
Eclipse such as the JDT/PDE builders, Xtext builder, etc.:
•if one deletes the Java and Xtext builders from an Xtend project, then invoking
Project → Clean… on the project has no effect at all. Nothing cleaned and nothing
built
•however, one can run an external Maven build, if necessary, from the context menu
Run As → Maven Xyz action. But there would almost never be a reason to
•the main function that m2e provides is to give JDT a classpath container that
resolves the classpath from the POM dependencies, but the Tycho connector turns
Maven Nature in Papyrus Source Projects of8 13

10. DIA Agency, Inc.
that around and implies the POM dependencies from the MANIFEST.MF just as
PDE does for JDT
However, for build plug-ins that do not map to internal Eclipse builders, m2e uses an
embedded maven3 engine to run maven plug-ins as required. For example, to do a
clean build of the org.eclipse.papyrus.infra.constraints[.edit[or]] plug-ins as
reconfigured in Gerrit review 41167 :2
•the Maven builder provided by m2e generates the sources from the EMF genmodel
into the src-gen/ folder
•if one deletes the generated sources, it takes two clean re-builds to return to a
correctly compiled project: the first one generates the sources and the second one
re-compiles custom code that depends on the generated code
•this is easily fixed: it is only needed to move the Maven Project Builder ahead
of the other builders in the .project file
The Maven lifecycle mappings for a typical Xtend plug-in project
See https://git.eclipse.org/r/#/c/41167/2
Maven Nature in Papyrus Source Projects of9 13

11. DIA Agency, Inc.
Running an external Maven build, which is never necessary
The lifecycle mappings for custom Maven plug-ins that the Maven builder runs during project build
Maven Nature in Papyrus Source Projects of10 13

12. DIA Agency, Inc.
Additional Build Scenarios
Hopefully, most of the time, re-building the Eclipse workspace is an incremental
process. Only projects and/or files particularly affected by some specific change need
to be rebuilt. JDT manages this well enough, and as we have seen, in Maven projects
JDT is still in charge. However, there is still the need occasionally for a clean re-build of
all projects in the workspace (for example after pulling changes from a remote git
repository), which even without Maven tends to the tedious.
Full Clean Build
A clean re-build of the entire workspace, after first deleting the entire contents of the
local Maven repository:
•takes about 65 seconds for the clean phase, including downloading core Maven
plug-ins, Tycho plug-ins, Xtext plug-ins required for cleaning and their
dependencies
•takes about 3 minutes and 55 seconds for the entire build (including the
aforementioned cleaning phase). The build
•uses the PDE target: EMF plug-ins are only downloaded that are needed to
support the Papyrus EMF model generator plug-in
•includes downloading the rest of the maven plug-ins required for build (but,
again, not the target platform, which is entirely provided by PDE)
•this compares to 1 minute 30 seconds for a full clean build in the pre-Maven
workspace (in which the clean step takes a mere 10 seconds)
The point about the PDE target, above, implies that we need make no changes to the
Papyrus Oomph Setup model, because we still need it to provision the PDE target,
except that we want to add the Tycho m2e connector. In particular, there is (under
normal circumstances) no duplication of target dependencies between the PDE Target
as configured by Oomph and the local maven repository, unless the developer also uses
the same repository to perform external (that is, outside of Eclipse) maven builds of the
Papyrus source projects.
Maven Nature in Papyrus Source Projects of11 13

13. DIA Agency, Inc.
Initial Build
An earlier section detailed the initial build experience of the committer that implements
the Maven nature in all of the Papyrus source projects. Every other Papyrus developer
will not have that same experience, but will still have an initial build when either
•pulling the commit(s) that implement(s) the Mavenization of Papyrus, or
•cloning the Papyrus git repository to set up a new development workspace
The experiment focuses on the latter scenario, emulated by checking in all changes
described previously to implement the Maven nature on a new branch, deleting all
projects from the Eclipse workspace, and doing a
$ git clean -d -x -f
to clean all derived and otherwise untracked files from the git workspace. Subsequently
importing all of the “main” and “main test” projects back into the Eclipse workspace:
•Oomph import (without any p2 or targlet tasks, just the project import and
working-set tasks) takes about 50 seconds
•total time to import, including the Oomph import and initial build ('build
automatically' is on): 4 minutes 15 seconds. The build finishes with
•many projects having build errors. The vast majority of these are of the "project
cannot be built until its prerequisite xyz is built" kind, with a only a handful of
projects being the roots of these dependency graphs
•most of these trace to an erroneous bin/ library entry in the classpath of the
org.eclipse.papyrus.view.properties project. Fixing that kicked off a workspace
re-build that took 15 seconds to clear up almost all problems. Fixing the same
misconfiguration in a few other projects (notably Model Explorer related and
the properties customization plug-ins) fixes all remaining build problems. So,
implementing the Maven nature should also make these fixes to ensure a clean
first build experience for developers
Maven Nature in Papyrus Source Projects of12 13

14. DIA Agency, Inc.
•is comparable to the roughly 3 minutes and 30 seconds time to import the same
projects without the Maven nature (as currently)
Conclusions
This initial experiment seems to demonstrate clearly that converting the Papyrus source
projects to the Maven nature will not adversely affect the performance of Eclipse for the
Papyrus developer community. Moreover, it will have the benefit of unifying the
integration of EMF model generation and possibly other custom build steps in both the
developer and Hudson environments. Other benefits remain unclear (the process of
converting projects uncovers additional project configuration problems, but this may be
a one-time thing).
Most importantly, the majority of developer workflows are not perturbed:
•Oomph setup and incremental updates, for example to pull in a new nightly
Papyrus build under a minimalist source workspace
•incremental workspace builds; bundle manifest and other metadata changes;
source code editing
•testing with run-time workbench and JUnit launch configurations
That said, it is perhaps rather late in the Mars release cycle to introduce such a
significant change in the configuration of the Papyrus codebase to the developer
community. Doing this conversion in the first milestone post Mars for the next annual
release (on the master branch, not the Mars maintenance branch) may be advisable.
Maven Nature in Papyrus Source Projects of13 13