Download to read offline
![Updating Other-Developer ODPs
#!/usr/bin/env bash
set -e
REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/.."
find $REPO_DIR/osgi-plugin/nsfs/ -name pom.xml
-not -path */nsf-dashboard/pom.xml
-not -path */nsf-plans/pom.xml
-print0 | xargs -0 -n 1 sh -c 'mvn org.openntf.maven:nsfodp-maven-
plugin:3.5.0:generate-odp -X -f $0 || exit 255'
Update ODPs for projects other than those properly done in Git:](https://image.slidesharecdn.com/2021-10-collabsphere-xpagesworkflow-211021141539/75/CollabSphere-2021-DEV114-The-Nuts-and-Bolts-of-CI-CD-With-a-Large-XPages-And-REST-App-24-2048.jpg)
![XPages Runtime - Tweaks
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
NotesThread.sinitThread();
Set<Long> handles = new HashSet<>();
if(!Factory.isInitialized()) {
Factory.initThread(new Factory.ThreadConfig(new Fixes[0], AutoMime.WRAP_32K, true));
}
if(sessions.get() == null) {
sessions.set(createSession((HttpServletRequest)request, handles));
sessionsAsSigner.set(createNativeSession());
sessionsFull.set(createNativeSession());
if(StringUtil.isEmpty(expectedDbPath)) {
throw new IllegalStateException(ENV_DBPATH + " environment variable should be set to the Domino API path to the context database");
}
try {
databases.set(sessions.get().getDatabase(expectedDbPath));
} catch(Throwable e) {
}
if(!checkAccess(databases.get())) {
databases.set(null);
}
}
// ...
}
Use a @WebFilter to provide context sessions/database
(There's a lot more to this in practice)](https://image.slidesharecdn.com/2021-10-collabsphere-xpagesworkflow-211021141539/75/CollabSphere-2021-DEV114-The-Nuts-and-Bolts-of-CI-CD-With-a-Large-XPages-And-REST-App-35-2048.jpg)
![Testcontainers - Example Code
driver.get(getContainerRootUrl() + "index.xsp");
assertEquals("Expected Page Title", driver.getTitle());
// Test all CSS and JS to make sure there's no 404s
URI rootUri = URI.create(getRootUrl());
driver.findElements(By.xpath("//link[@rel='stylesheet']"))
.stream()
.map(link -> link.getAttribute("href"))
.map(href -> rootUri.resolve(href))
.map(this::decontainerize) // etc.
.forEach(uri -> checkUrlWorks(uri, client));
driver.findElements(By.xpath("//script[@href]"))
// etc.](https://image.slidesharecdn.com/2021-10-collabsphere-xpagesworkflow-211021141539/75/CollabSphere-2021-DEV114-The-Nuts-and-Bolts-of-CI-CD-With-a-Large-XPages-And-REST-App-45-2048.jpg)
Uploaded byJesse Gallagher
CollabSphere 2021 - DEV114 - The Nuts and Bolts of CI/CD With a Large XPages And REST App
The document discusses the implementation of continuous integration and continuous deployment (CI/CD) for a large application built with XPages and REST services. It details the use of various tools and technologies, primarily Maven and Docker, highlighting techniques for building, testing, and deploying the application in multiple runtime environments. Additionally, it outlines the complexities involved in managing dependencies and adapting the code for compatibility across different platforms.
More Related Content
Similar to CollabSphere 2021 - DEV114 - The Nuts and Bolts of CI/CD With a Large XPages And REST App
![Vibe Coding vs. Spec-Driven Development [Free Meetup]](https://cdn.slidesharecdn.com/ss_thumbnails/vibecodingvsspecdrivendevelopment-251209105622-43f455e7-thumbnail.jpg?width=640&height=640&fit=bounds)
CollabSphere 2021 - DEV114 - The Nuts and Bolts of CI/CD With a Large XPages And REST App
- 1. THE NUTS ANDBOLTS OF CI/CD WITH A LARGE XPAGES AND REST APP IT'S... TECHNICALLY DOABLE
- 2. JESSE GALLAGHER CTO - IKNOW SOME GUYS IP MANAGER - OPENNTF HTTPS://FROSTILLIC.US @GIDGERBY
- 4. The Moving Parts Coreset of OSGi plugins XPages Library JAX-RS–based REST services A handful of compile-time tests NPM-build–based JavaScript applications Some delivered from OSGi One delivered via NSF A set of NSFs Some XPages apps Most Notes client apps Most not properly run with source control A set of JEE webapps The main XPages app One of the JS apps Testcontainers-based tests
- 5. The Toolchain Maven maven-bundle-plugin (usedto use Tycho) frontend-maven-plugin NSF ODP Tooling Eclipse (Usually) NPM (via frontend-maven- plugin) XPages Jakarta EE Support Open Liberty XPages Runtime project Testcontainers (Docker)
- 6. Opening Caveat Every explanationin this presentation comes with an implicit "...but it gets more complicated in practice"
- 7.
- 8. Brief Aside: Maven It’snot (inherently) scary! Just look how cute that owl is Maven is a tool used for build automation It’s primarily used for Java projects, but is adaptable to many things via plugins A “pom.xml” file is a clear giveaway of Maven’s use We use it here because it allows for cleanly- reproducible builds across systems, without environment-specific details
- 9. Tycho vs. MBP Historically,we used Tycho to manage the OSGi build It presents an OSGi environment very similar to Domino It can strongly enforce expectations and dependencies Good support for OSGi-based test suites with Notes environments I switched to maven-bundle-plugin for added flexibility It makes it much easier to use normal Maven dependencies ...but it requires a lot more knowledge of the "gotchas" of the Domino stack I still hand-tweak OSGi options, and the compile step doesn't validate this There are other options (bnd-maven-plugin), but I don't know them enough yet https://frostillic.us/blog/posts/2019/8/22/converting-tycho-projects-to-maven-bundle-plugin-initial-phase
- 10. OpenAPI Generation <plugin> <groupId>io.openapitools.swagger</groupId> <artifactId>swagger-maven-plugin</artifactId> <version>2.1.5</version> <executions> <execution> <id>generate-openapi</id> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> We useswagger-maven-plugin and swagger-annotations to generate an openapi.yaml file during the build
- 11.
- 12. Runtime Independence The coreparts of the app target three main environments: XPages on Domino (OSGi) Servlets on Domino (OSGi) .war-based Jakarta EE (not OSGi) Can't make a lot of assumptions!
- 13. RuntimeEnvironment Idiom We usea RuntimeEnvironment type to handle differences Contains methods like getSession(), getServletRequest(), etc. Each target runtime has an implementation that does the dirty work Core code calls RuntimeEnvironment.getSession() and works anywhere It gets fiddly! Error handling, detecting the right runtime, etc.
- 14. Balancing Dependencies maven-bundle-plugin makesthird-party dependencies easier ...but Domino still makes it hard Some dependencies need to be re-packaged for Domino's OSGi environment Some are safest to avoid due to Domino's polluted classpath (Guava)
- 15.
- 16. NPM-BASED BUILDS INMAVEN
- 17. frontend-maven-plugin frontend-maven-plugin allows youto run various frontend-dev tools (npm, yarn, webpack, etc.) based on configuration in a Maven pom We have the JS apps in a separate directory away from Java The build config specifies node+npm versions and the target directory One project copies the results into an OSGi plugin, another copies them into an NSF during build
- 18. Example Use Install LocalNode + NPM `npm install` `npm run build` <execution> <id>install node and npm</id> <goals> <goal>install-node-and-npm</goal> </goals> <phase>generate-resources</phase> </execution> <execution> <id>actionItems install</id> <goals> <goal>npm</goal> </goals> <phase>generate-resources</phase> <configuration> <workingDirectory>${buildDir}</workingDir ectory> </configuration> </execution> <execution> <id>actionItems build</id> <goals> <goal>npm</goal> </goals> <phase>generate-resources</phase> <configuration> <workingDirectory>${buildDir}</workingDir ectory> <arguments>run build</arguments> </configuration> </execution>
- 19.
- 20.
- 21. NSF ODP Tooling Theglue for working with NSFs: Compiles ODPs to NSF Updates ODPs for server-side changes outside Git Provides DXL and XSP autocomplete in Eclipse
- 22. Compilation Compiles ODPs intoNSFs Uses fresh build of the OSGi plugins to ensure compatibility Runs in a local Equinox environment similar to Domino Sets $TemplateBuild fields
- 23. Updating Other-Developer ODPs <build> <plugins> <plugin> <groupId>org.openntf.maven</groupId> <artifactId>nsfodp-maven-plugin</artifactId> <configuration> <databasePath>Some/Server!!db/app.nsf</databasePath> </configuration> </plugin> </plugins> </build> ConfigureODP projects for the canonical source:
- 24. Updating Other-Developer ODPs #!/usr/bin/envbash set -e REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )/.." find $REPO_DIR/osgi-plugin/nsfs/ -name pom.xml -not -path */nsf-dashboard/pom.xml -not -path */nsf-plans/pom.xml -print0 | xargs -0 -n 1 sh -c 'mvn org.openntf.maven:nsfodp-maven- plugin:3.5.0:generate-odp -X -f $0 || exit 255' Update ODPs for projects other than those properly done in Git:
- 25. XML Autocomplete Provides "goodenough" XML autocomplete for XSP Covers core and custom controls, but not third-party library controls No preview pane, but eh... DXL autocomplete uses the provided- with-Domino schema files
- 26. NSF ODP Links https://frostillic.us/blog/posts/2018/3/5/301B92A0DF1F2CAF85258247007 BD28F https://frostillic.us/blog/posts/2020/8/27/nsf-odp-tooling-setting-up-jenkins- builds https://www.openntf.org/main.nsf/project.xsp?r=project/NSF%20ODP%20 Tooling https://github.com/OpenNTF/org.openntf.nsfodp Lastyear's session: https://youtu.be/7MeCMz0F-vM
- 27. XPAGES JAKARTA EESUPPORT
- 28. XPages Jakarta EESupport Acts as a "platform update" for the app Brings in, in particular, EL 3 and JAX-RS 2.1, much newer than the built-in variants Also brings in CDI, which is used fully in the webapps and REST services Like managed beans, but much better Not required for the rest of this, but it sure is nice
- 29. Example JAX-RS Resource @Path("/serverInfo") @PermitAll publicclass ServerInfoResource { @Inject private ServerInfo serverInfo; @GET @Produces(MediaType.APPLICATION_JSON) @Tag(name = TAG_EXTERNALPORTAL) @Operation(summary = "Get information about server-specific resources", description = "...") public ServerInfo get() { return serverInfo; } }
- 30. XPages Jakarta EELinks https://frostillic.us/blog/posts/2018/6/3/6AC99D0B866A92AA852582A 1006C2FA6 https://www.openntf.org/main.nsf/project.xsp?r=project/XPages%20Ja karta%20EE%20Support https://github.com/OpenNTF/org.openntf.xsp.jakartaee
- 31.
- 32. XPages Runtime Open-Source projectthat allows running XPages in a normal Servlet container I've only run it in Liberty, but others should work Turns out all the stuff from pre-OSGi is still in there Requires some tweaks with third-party libraries, and code has to not assume it's in OSGi This enables CI deployment servers and full automated tests
- 33. XPages Runtime Some shimcode handles bootstrapping ODA, etc. JAX-RS services use Liberty's built-in support The webapp projects use symlinks to point to the ODP for XPages, Java, etc.
- 34. XPages Runtime -Tweaks Make no assumptions about OSGi The RuntimeEnvironment idiom goes a long way The IBM Commons ExtensionManager accounts for this Duplicate any plugin.xml extensions in META-INF/services With a JEE server, no JAX-RS runtime (Wink, RestEasy, etc.) is needed On Liberty, don't enable JSF or JSP features (conflict with XPages)
- 35. XPages Runtime -Tweaks @Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { NotesThread.sinitThread(); Set<Long> handles = new HashSet<>(); if(!Factory.isInitialized()) { Factory.initThread(new Factory.ThreadConfig(new Fixes[0], AutoMime.WRAP_32K, true)); } if(sessions.get() == null) { sessions.set(createSession((HttpServletRequest)request, handles)); sessionsAsSigner.set(createNativeSession()); sessionsFull.set(createNativeSession()); if(StringUtil.isEmpty(expectedDbPath)) { throw new IllegalStateException(ENV_DBPATH + " environment variable should be set to the Domino API path to the context database"); } try { databases.set(sessions.get().getDatabase(expectedDbPath)); } catch(Throwable e) { } if(!checkAccess(databases.get())) { databases.set(null); } } // ... } Use a @WebFilter to provide context sessions/database (There's a lot more to this in practice)
- 36. XPages Runtime -Tweaks @Override public Object resolveVariable(FacesContext facesContext, String varName) throws EvaluationException { switch(StringUtil.toString(varName)) { case "session": case "sessionAsSigner": case "sessionAsSignerFullAccess": return ODAFilter.sessions.get(); case "database": return ODAFilter.databases.get(); } // ... } Use a VariableResolver to find custom session/database
- 37.
- 38.
- 39. Dockerized XPages Apps Westart with the open-liberty:full-java8-openj9 image Then we bring in the Notes runtime parts of the official HCL Domino Docker images Then we bring in the built application WAR It's already been adapted to read the context server/DB from the environment Et voilà ! This is how I deploy CI servers with Jenkins nowadays
- 40. Dockerfile # Configure theruntime image FROM --platform=linux/amd64 open-liberty:full-java8-openj9 # Bring in the Liberty app and configuration COPY --chown=default:users webapp.war /apps/ COPY config/* /config/ # Bring in the Domino runtime COPY --from=domino-docker:V1101_03212020prod /opt/hcl/domino/notes/11000100/linux /opt/hcl/domino/notes/latest/linux COPY --from=domino-docker:V1101_03212020prod /local/notesdata /local/notesdata # Bring in our Domino config and assign the data directory to the default user USER root COPY --chown=default:users notesdata/* /local/notesdata/ RUN mkdir -p /local/notesdata/IBM_TECHNICAL_SUPPORT RUN chown -R default:users /local/notesdata USER default ENV LD_LIBRARY_PATH "/opt/hcl/domino/notes/latest/linux" ENV NotesINI "/local/notesdata/notes.ini" ENV Notes_ExecDirectory "/opt/hcl/domino/notes/latest/linux" ENV Directory "/local/notesdata" ENV PATH="${PATH}:/opt/hcl/domino/notes/latest/linux:/opt/hcl/domino/notes/latest/linux/res/C" ENV InitDominoRuntime="1" # DB context env var is set when launching the container and read by the app EXPOSE 8080 8443
- 41. Containerized XPages AppsLinks https://frostillic.us/blog/posts/2020/6/28/weekend-domino-apps-in- docker-experimentation https://frostillic.us/blog/posts/2020/8/13/executing-a-complicated-osgi- nsf-surefire-npm-build-with-docker https://github.com/jesse-gallagher/domino-docker-war-example
- 42.
- 43. Testcontainers Testcontainers lets usrun the full webapp (XPages + REST) with true headless browsers Requires Docker, a containerized app, and patience I've only gone so far with it - with work, this could run full automated UI tests of XPages and JavaScript apps
- 44. Testcontainers - MavenProcess Use maven-resources-plugin to copy a Dockerfile, built WAR, and Notes runtime files to scratch space Use dockerfile-maven-plugin to build the app image `<pullNewerImage>false</pullNewerImage>` to avoid trying to resolve official Domino images from Docker Hub Use maven-failsafe-plugin to run all `it.*` classes Set environment `TESTCONTAINERS_RYUK_DISABLED=true` to avoid trouble I hit when running on Jenkins and instead manage lifecycles manually
- 45. Testcontainers - ExampleCode driver.get(getContainerRootUrl() + "index.xsp"); assertEquals("Expected Page Title", driver.getTitle()); // Test all CSS and JS to make sure there's no 404s URI rootUri = URI.create(getRootUrl()); driver.findElements(By.xpath("//link[@rel='stylesheet']")) .stream() .map(link -> link.getAttribute("href")) .map(href -> rootUri.resolve(href)) .map(this::decontainerize) // etc. .forEach(uri -> checkUrlWorks(uri, client)); driver.findElements(By.xpath("//script[@href]")) // etc.
- 46.
- 47.