In this session, it is demonstrated a development and deployment workflow used at Alkacon to maintain various OpenCms sites, currently with version 9.5 and in the future with OpenCms 10. Steps in this workflow include working directly in the OpenCms VFS through a network share, exporting modules automatically and committing the changed modules in a Git repository. The modules from the Git repository are then used in an automated setup with Jenkins and Docker to instantly deploy the site on a different server. There are also shown enhancements in OpenCms 10 that will make such module deployment workflows easier. These include the option to install / update / export and delete modules directly through a network share. OpenCms 10 also improves the export format to reduce merge conflicts in revision control systems such as Git. Moreover, it will be possible to provide a local change set file for the configuration, which allows to distribute global XML configuration that is adjusted locally.

1. Daniel Seidel – Alkacon Software GmbH
OpenCms development
workflow using Jenkins and
Docker
28.09.2015

2. 2
What can you expect?
Introduction to the Alkacon
development infrastructure
Presentation of the new
OpenCms Dev-Features

3. Development Scenario: Core
3
localhost
ci-autobuild
ci-10-0-x
ci-release

4. Development Scenario: Core
4
localhost
ci-autobuild
ci-10-0-x
ci-release
Why does it work good?
● Much Java Programming
● Each developer with his own task
● Always default configuration

5. What is different from the core scenario?
● Less Java Code, more Formatters etc.
● Developers may interact closer
● Special configurations
● Far more module import/export
● Less restarts necessary (jar update)
5
Development Scenario: Website
Is the core infrastructure good as well?

6. ci-autobuild
ci-10-0-x
ci-release
Development Scenario: Website
6
localhost
Why doesn’t it work good?
● Painful module import/export/check-in
● Configurations may diverge
● Merge conflicts for XML contents
● Many projects, many servers
● Problems getting from development
to production

7. 7
Website: Better environment
localhost
website
docu
haproxy
ci-docker
...
online.server.com
HTTP and SMB
HTTP and SMB

8. 8
Website: Accomplishments (I)
localhost
online.server.com
HTTP and SMB
HTTP and SMB
website
docu
haproxy
ci-docker
...
Docker intro (very short)
● Container like minimal VM
● Build from scripts
● Communication facilities
● Very portable
Docker benefits
● One (simple) server
● Automatic setup for containers
● Shared base images
● Easy switch
development  production

9. 9
Website: Accomplishments (II)
localhost
online.server.com
HTTP and SMB
HTTP and SMB
website
docu
haproxy
ci-docker
...
Flexible workflow
● Work on
● Server only
● Mount VFS
● Locally only
● Check-in from
● OpenCms
● Server/Docker
● Localhost
● Even the production server

10. ● Live Demo
Git & Jenkins & Docker
Demo
Demo Demo
Demo
デモ
Development infrastructure

11. ● SMB module management:
● Import, export or delete modules via network share
● Git support for OpenCms
● Check in to Git from the OpenCms workplace
● XSLT transformations for configuration files
● Manipulate the configuration on start up
● Reduced export format
● Reduce manifest changes
● Avoid merge conflicts
11
We use: New OpenCms features

12. ● New Wrapper for CmsJLanRepository
● Configure in opencms-importexport.xml
12
SMB module management (I)
<opencms>
<!-- other configuration options -->
<repositories>
<repository name="OPENCMS"
class="org.opencms.jlan.CmsJLanRepository>
<params>
<!-- other parameters -->
<param name="wrapper">
org.opencms.file.wrapper.CmsResourceWrapperModules
</param>
</params>
</repository>
</repositories>
</opencms>

13. ● When you mount the network share:
● A special folder “/modules” is present
● With the empty subfolder “import”
● And the subfolder “export” listing all installed modules
● With the folders you perform module operations
13
SMB module management (II)
Operation Performed via
Import module (add or replace) Drop module zip into “/modules/import/”
Export module Copy module zip from “/modules/export/”
Delete module Delete module zip in “/modules/export/”

14. How to determine the import site?
● New option in the module management
● Edit module: “Import site”
● Specifies the import site
● Default
● Root site for the SMB module management
● Current site in the administration view
14
SMB module management (III)

15. ● Live Demo
Add, update, export, delete a module
Demo
Demo Demo
Demo
デモ
SMB module management

16. How to check in a module?
16
Git support for OpenCms (I)
Can we do this automatically?

17. ● Live Demo
From OpenCms to Git in one step
Demo
Demo Demo
Demo
デモ
Git support

18. How does the support work?
● Workplace app + shell script + config file
● WEB-INF/git-scripts/module-checkin.sh (executable)
● WEB-INF/git-scripts/module-checkin.conf (config)
● Also: Stand-alone as shell script + config file
● Useful with SMB share’s modules feature
● One-click update of (local) repository
18
Git support for OpenCms (II)
config
executable
Read defaults
Read defaults
Export modules and
Call with params
update

19. How to enable Git support?
● Make the file
● WEB-INF/git-scripts/module-checkin.sh
● executable.
● Place a suitable configuration under
● WEB-INF/git-scripts/module-checkin.conf
● (you may copy the module-checkin.conf.demo)
19
Git support for OpenCms (III)

20. Usage for website development
● Git support
● Direct check-in via the workplace app
● Backup configured modules on Docker restart via
script
● (local check-in via script)
● SMB module management
● Backup configured modules on Docker restart
● (local check-in via script)
● (one-click module updates)
20
Short summary – feature usage

21. The problem
● Different config for different dockers
● Different config development  production
21
Config adjustment via XSLT (I)
Typically replaced items
● Site configuration (hostname)
● Email server configuration
● ...
Possible solutions
● Different config files
● Script-based config file manipulation during setup
● XSLT transformation on config load

22. Implementation details
● One XSLT file for all OpenCms config files
● XSLT 1.0 + special macro + parameters
● Transformations performed on start-up
How to apply transformations?
● Just place a file opencms-configuration.xslt
under WEB-INF/config/
22
Config adjustment via XSLT (II)

23. 23
A typical XSLT file
<?xml version="1.0" encoding="UTF-8" ?>
<xsl:stylesheet xmlns:xsl="..." version="1.0">
<!-- Parameter for current file name. Set by OpenCms. -->
<xsl:param name="file" />
<!-- Mandatory. The @dtd@ string is a special macro. -->
<xsl:output doctype-system="@dtd@" indent="yes" />
<!-- Copy everything that isn't matched by the other rules. -->
<xsl:template match="@* | node()">
<xsl:copy><xsl:apply-templates select="@* | node()"/></xsl:copy>
</xsl:template>
<!-- Node that really does something -->
<xsl:template match="/opencms/system/sites">
<sites>
<workplace-server>http://days-demo</workplace-server><!-- ... -->
</sites>
</xsl:template>
</xsl:stylesheet>

24. What happens if You alter configurations
inside OpenCms?
● Question for you
Where to look for details?
● CmsConfigurationManager
● transformConfiguration(...)
Good for “Playing”
● The adjusted configuration is written in the log
at debug level
24
Just for the records

25. ● Live Demo
How we use for website development
Demo
Demo Demo
Demo
デモ
XSLT transformations

26. What’s the problem?
● Module manifest is very verbose
● In particular information like
● Export date
● Modification dates
● Authors
● (UUIDs)
● Information of “non-module” parent folders
● May cause merge conflicts
● Will cause many unnecessary and confusing
changes in the Git repository
26
Reduced export mode (I)

27. Possible solutions
● Do not write “critical” data to the manifest at all
● Replace “critical” data with macros, e.g.,
● FILE_TIME
● IMPORT_TIME
Possible problems
● Information loss
● What are the best dates?
● Time shifts?
● Renewing exported JSPs needs newer timestamp
27
Reduced export mode (II)

28. Current solution
● Two export modes
● “Default”: with all information, optional macros for
modification dates
● “Reduced”: with “critical” information omitted
● Mode set on a module basis
28
Reduced export mode (III)
Discuss possible problems or suggest
improvements!

29. ● Live Demo
Reduced vs. Default
Demo
Demo Demo
Demo
デモ
Comparison of export formats

30. ● The shown setup
● Is very flexible in how you work
● Simplifies module import/export significantly
● Uses various additional software
● Jenkins
● Docker
● Git
● Eases deployment
● Takes advantage of various new OpenCms
features
30
Summary – Development setup

31. ● SMB module management
● Good for automatization
● Eases module import / export
● Auto-install via auto-copy on build (we use gradle)
● Be aware: restarts for code changes remain
● Git support
● Wide variety of usage scenarios
● App at server or localhost
● Script only and SMB-share
● Fine-grained control on module check-in steps
● Consider: precise commit  maximal automation
● At the moment: only for Linux (shell script)
31
Summary - Features

32. ● XSLT-Transformations
● Easy config adjustment for different servers
● Transformations applied on startup
● One XSLT file for all files included by opencms.xml
● Good for updates?
● Reduced export format
● Purpose: Avoid merge conflicts
● Think about information loss
● Alternative:
timestamp macros (FILE_TIME, IMPORT_TIME)
32
Summary - Features

33. ● Git-support:
● User-specific settings
● (cross-platform solution via jGit)
● Module export format
● Enable reduced export also for database exports
● (Separate manifest in various files)
● One per resource
● One for the module
● (Combine reduced format and timestamp macros)
33
Planned improvements
Any suggestions? Please let us know!

34. Daniel Seidel
Alkacon Software GmbH
http://www.alkacon.com
http://www.opencms.org
Thank you very much for your attention!