At Maersk Line, not only the world's biggest container ships, the 'Triple-E' class vessels were built. Continuous Integration and Delivery on a similar scale using Hudson, Maven and tools like Kokki (similar to Puppet or Chef, but written in Python) are also practiced there. This session is going to give a brief overview of Multi-Configuration (Matrix) job types used in most projects at Maersk around the globe. Things are being built and deployed in a heterogenous environment, otherwise probably found only at very large vendors of Public Cloud services like Google or Amazon. Provisioning of various OS is automated through Vagrant. Management and Planning of all tasks and 'Sprints' is following Agile principles, especially DevOps style Kanban. Where possible planned and controlled by Eclipse-based tools such as Mylyn Connectors accessing planning tools like TeamConcert, Xplanner or Mantis. While feature projects use Eclipse for Java or Scala/Play!, the DevOps teams use PyDev for Jython/WSTL or Python development.

1. Triple-E’ class DevOps
with Hudson, Maven, Kokki
Multiconf & PyDev
Werner Keil
PyUGAT
15th October 2014

2. Images © 2 © 2012-2013 Creative Arts & Technologies Maersk Line and Others

3. Overview
• Introduction
• Kokki
• Configuration
• Multiconf
• PyDev
• Development
• Maven
• Build
• Hudson
• Automation
• Demo
•Q&A
3 © 2012-2013 Creative Arts & Technologies

4. Who am I?
Werner Keil
• Consultant – Coach
• Creative Cosmopolitan
• Open Source Evangelist
• Software Architect
• Java Godfather
• UOMo Project Lead
• Mærsk DevOps Guy
…
Email werner@catmedia.us
4 © 2012-2013 Creative Arts & Technologies
Twitter @wernerkeil

5. Kokki
• Infrastructure Automation → Configuration
Management
• Part of Provisioning
• Library and simple command line tool
• Currently no client/server component
• Can use GitHub in place of client/server
• Alternative to Puppet/Chef/Cfengine
• Implemented in Python
5 © 2012 Creative Arts & Technologies

6. Kokki | Terms
• Kokki is a configuration management framework.
• It can be used standalone or as a part of a more complex system.
• It means “cook” in Finnish
• Environment: Execution environment
• Resource: Describes a file, service, package, etc..
• Provider: Knows how to execute a resource
• Kitchen: Container for cookbooks
• Cookbook: Container for recipes and libraries
• Recipe: group of resource definitions
• Library: utility methods, resources, and providers
6 © 2012 Creative Arts & Technologies

7. Kokki | Quick Example
from kokki import *
with Environment() as env:
Package("vim", action="upgrade")
File("/etc/hosts",
owner = "root",
group = "root",
content = "127.0.0.1 localhostn"
"255.255.255.255 broadcasthostn"
"::1 localhostn"
"fe80::1%lo0 localhostn")
env.run()
7 © 2012 Creative Arts & Technologies

8. Kokki++ | Multiconf
•Multiconf is a framework for describing complex
environments using Python.
•Why
• It started from a simple need of continuous deployment of Java EE projects.
• Using a bunch of Python scripts, and a bunch of plain text property files.
• When number of property files increased number of scripts increased, too…
• Out of this dilemma Multiconf was born.
8 © 2012 Creative Arts & Technologies

9. Multiconf
•How
• Imagine a project, using multiple servers like Tomcat, Jetty, JBoss, Glassfish
or Weblogic.
• You are going to use database connections and four environments:
• Local (for developer's local machine)
• Dev
• Test
• Prod
• Project configuration will be similar on different environments, but something
will be different (databases and ports, for example).
9 © 2012 Creative Arts & Technologies

10. Multiconf (2)
•What Multiconf is not
• Multiconf itself doesn't know how to create environments.
• Multiconf doesn't know how to create any of the environment's components
• Multiconf has nothing to execute
• Perfect match for a system such as Kokki
• Same language
• Pluggable recipes and Libraries make Kokki open to such Extensions
•What Multiconf is
• Multiconf allows you to define your software stack and get early warning that
something within your definition is wrong. Other tools use YAML or JSON to
define settings of the components, but then you need something to validate
those settings. Multiconf is both - definition and validation.
10 © 2012 Creative Arts & Technologies

11. Multiconf | JSF Example
from javax_faces import ProjectStage
from valid_envs import local, dev, test, pp, prod
def jsf_stages():
with ProjectStage() as ps:
ps.name(local=”Development”,
dev=”Development”,
test=”UnitTest”,
pp=”SystemTest”,
prod=”Production”)
11 © 2012 Creative Arts & Technologies

12. Multiconf | Inspired...
Apache Tamaya
• Started by
• Anatole Tresch
(Crédit Suisse)
• Mit etwas Hilfe
u.a. durch den
„Java Godfather“
•Proposal for
• An Apache Incubator Project
• See.: https://github.com/java-config
© 2012-2014 Creative Arts & Technologies

13. Multiconf | ”at Apache”
Apache Tamaya
•Scope
• Provide type-safe enterprise-ready configuration standard for SE and EE
(optionally also ME?)
• Ensure it is a simple but extensible
• Ready for complex scenarios like Java EE, Multi-Tenancy etc.
• Ensure interoperability with existing solutions
• Align Configuration Aspects with EE8 and the Java Platform
• Other languages (as Apache is rather polyglot)
© 2012-2014 Creative Arts & Technologies

14. Multiconf | ”at Apache”
Apache Tamaya
• State
• Already running code base
• Large support in User Groups around the world
• Supported by leading industry experts
© 2012-2014 Creative Arts & Technologies

15. PyDev
•PyDev is a product of Appcelerator, a platform
and services company that is enabling Web
developers to build cross-platform / native
applications for Mobile, Tablet and Desktop
platforms
Winner of 2011 Eclipse Award
for Best Developer Tool
Appcelerator also makes Aptana Studio and
Titanium Studio, Eclipse-based IDEs targeting
Web and Mobile platforms.
15 © 2012 Creative Arts & Technologies

16. PyDev Now
•PyDev ist now Open Source Project,
lead by Fabio Zadrozny in Florianópolis, Brasil:
www.pydev.org
© 2012-2014 Creative Arts & Technologies Bilder © FIFA

17. PyDev | Code Completion
17 © 2012 Creative Arts & Technologies

18. PyDev | Code Navigation
18 © 2012 Creative Arts & Technologies

19. PyDev | Code Coverage
19 © 2012 Creative Arts & Technologies

20. PyDev | Debugging
20 © 2012 Creative Arts & Technologies

21. PyDev | Unit Tests
21 © 2012 Creative Arts & Technologies

22. PyDev
DEMO

23. Maven
The evolution of build systems
23 © 2012 Creative Arts & Technologies

24. Maven
“Lean” Usage
• Using Maven for what it’s Best at
• Dependency Management
• Nexus
•Managed Environment
• Generated / Called by Kokki
• Hudson Plugins
24 © 2012 Creative Arts & Technologies

25. Hudson
•Hudson is a powerful and widely used open source
continuous integration server providing
development teams with a reliable way to monitor
changes in source control and trigger a variety of
builds.
(The Hudson Book)
25 © 2012 Creative Arts & Technologies

26. Hudson | Downloads
26 © 2012 Creative Arts & Technologies

27. Hudson | Commits
27 © 2012 Creative Arts & Technologies

28. Hudson | Job Coordination
28 © 2012 Creative Arts & Technologies

29. Hudson | Job Coordination Triple-E size
29 © 2012 Creative Arts & Technologies

30. Hudson | Promoted Builds
30 © 2012 Creative Arts & Technologies

31. The Price of Continuous Integration
32 © 2012 Creative Arts & Technologies

32. Let‘s talk
Q & A

33. Links
Kokki
https://github.com/samuel/kokki
Multiconf
https://github.com/lhupfeldt/multiconf
PyDev
http://www.pydev.org

34. Links (2)
Eclipse – Hudson
http://www.eclipse.org/hudson/
Apache Maven
http://www.apache.org/maven/

35. Twitter @wernerkeil
Thank you Email werner@catmedia.us
36 © 2012 Creative Arts & Technologies Images © Maersk Line