This document provides a summary of best practices for developing and deploying Java applications with Docker. It begins with an introduction and overview of Docker terminology. It then demonstrates how to build a simple Java web application as a Docker image and run it as a container. The document also covers deploying applications to clusters as services and stacks, and techniques for application management, configuration, monitoring, troubleshooting and logging in Docker environments.

1. Best Practices for Developing &
Deploying Java Applications with
Docker
Eric Smalling - Solution Architect, Docker Inc.
@ericsmalling
JavaOne 2017 | CON7957

2. 2
Who Am I?
● Eric Smalling
○ Solution Architect
Docker Customer Success Team
● ~25 years in software development,
architecture, version control admin, etc…
● ~10 years in build & test automation
● Docker user since pre-1.0 days
● Java developer since 1.1.x days

3. Agenda
● Docker 101
● Running a simple Java web application in Docker
● Services, stacks & deploying to clusters
● Application management & troubleshooting
● Application Configuration
● Q & A

4. Docker 101
A quick overview of Docker

5. Some Docker vocabulary
Docker Image
The basis of a Docker container. Represents a full application
Docker Container
The standard unit in which the application service resides and executes
Docker Engine
Creates, ships and runs Docker containers deployable on a physical or virtual, host
locally, in a datacenter or cloud service provider
Registry Service (Docker Hub or Docker Trusted Registry)
Cloud or server based storage and distribution service for your images

6. Docker File System
Images, Layers & Containers
● Logical file system by grouping different file system primitives into branches (directories,
file systems, subvolumes, snapshots)
● Each branch represents a layer in a Docker image
● Allows images to be constructed / deconstructed as needed vs. a huge monolithic image
(ala traditional virtual machines)
● When a container is started a writeable layer is added to the “top” of the file system

7. Docker File System
Containers & Copy on Write
● Super efficient:
Sub second instantiation times for containers
New container can take <1 Mb of space
● Containers appears to be a copy of the original image
But, it is really just a link to the original shared image
● If someone writes a change to the file system, a copy of the affected file/directory is
“copied up”

8. Docker File System
What about data persistence?
● Volumes allow you to specify a directory in the container that exists outside of the docker
file system structure
● Can be used to share (and persist) data between containers
● Directory persists after the container is deleted
Unless you explicitly delete it
● Can be created in a Dockerfile or via CLI

9. Dockerfile - Linux + Java Example: Initial state

10. Image Layers
Kernel
Ubuntu Linux 16:04
Update apt catalogs
Install JDK and curl
Download Tomcat
Install Tomcat
Copy webapp
Start tomcat
Initial State

11. Building the image
The docker client command
“build” = build an image
“-t” = apply a name and optional build
Image name and optional tag
Path to build context and Dockerfile

12. Running the image in a container
The docker client command
“run” = start a container
“--rm” = delete container when it exits
“-t” = run with a tty (for console i/o)
“-i” = run in interactive mode
These often are used in combination like this
Image name and optional tag

13. Demo
Build and run demonstration

14. Dockerfile - Linux + Java Example: Optimization step 1

15. Image Layers
Optimization Step 1
Kernel
Ubuntu Linux 16:04
Update apt catalogs, install JDK and curl, clean up
Download Tomcat
Install Tomcat
Copy webapp
Start tomcat

16. Dockerfile - Linux + Java Example: Optimization step 2

17. Image Layers
Optimization Step 2
Kernel
OpenJDK:8-alpine
Update apk catalogs, install curl
Download Tomcat
Install Tomcat
Copy webapp
Start tomcat

18. Dockerfile - Linux + Java Example: Fully Optimized

19. Image Layers
Fully optimized
Kernel
tomcat:8.5-alpine
Copy webapp

20. Deploying to Clusters
Services, Stacks and Swarms

21. More terminology
● Swarm
○ A group of docker hosts, connected and running as a cluster
○ 1-n managers
○ 1-n workers
● Service
○ An application (or part of an application) that provides a specific function
(catalog lookup, web front end, payment processing)
● Stack
○ A way of representing multi-service applications
○ Made up of 1-n services

22. Stack deploy demo
Simple J2EE application deployment with 2 containers:
● React based front end
● Java based back end

23. Application Management
Monitoring & Troubleshooting

24. Health Checks
Helping Docker help you
● HEALTHCHECK instruction in DockerFile
● Tells Docker how to test a container to check that it is still working
● New status added to container lists
● Adds “(healthy)” to Status column in a “docker ps response”

25. Health Checks
Helping Docker help you
● Examples:
○ HEALTHCHECK CMD curl --fail http://localhost || exit 1
○ HEALTHCHECK --interval=12s --timeout=12s --start-period=30s
CMD node /healthcheck.js
● References:
○ Documentation: https://docs.docker.com/engine/reference/builder/#healthcheck
○ Elton Stoneman blog about not using curl/iwr: https://t.co/Zgdd1lyzhk

26. JVM Memory
Tips and tricks
● Always explicitly specify JVM heap size with “-Xmx” arguments
○ By default, J2SE 5.0+ will use up to 25% of the host machine’s RAM or 1GB (whichever is smaller)
○ Container memory limits (enforced via cgroups) are ignored* (*cgroup awareness is planned for Java 9)
○ It’s just a good practice to specify it anyway
● Do use Docker cpu and memory reservations and limits to avoid over-subscribing your host machines
○ --memory
○ --memory-reservation
○ --cpus
○ etc…
● If limiting cpu, be sure to update GC Thread limiter in JVM
○ -XX:ParallelGCThreads

27. Logging
Dealing with application logs
● Docker EE Reference Architecture document about this: http://dockr.ly/logging
● Do not output logs into the container’s RW layer
○ slow
○ have to exec or cp out of the container to see them
● Option 1: send logs to stdout (see logging drivers below)
○ Visible via “docker logs” command
○ Visible via Docker UCP web console
● Option 2: send logs to volume
○ Many use a centralized NAS/SAN volume for this
● Option 3: Docker logging drivers

28. Docker Log Drivers
Log drivers available (as of 9/4/17)
Latest always available at: https://docs.docker.com/engine/admin/logging/overview/#supported-logging-drivers

29. Application Log Drivers
Consider the following when selecting application log drivers:
● syslog and splunk:
○ Good options if log data is highly sensitive since they can be configured to use TLS for
transporting logs.
● journald:
○ great for retaining the usage of docker logs as well as logging Docker daemon logs
○ allows for easier troubleshooting and log portability at the same time
○ logs write first locally, so that there is less reliance on logging infrastructure.
● awslogs or gcplogs:
○ Only if cluster exist solely on a single cloud provider

30. Application Log Drivers (continued)
Consider the following when selecting application log drivers:
● gelf and fluentd:
○ good choice if there's a NoSQL database somewhere in the environment where the logs can
be stored.
Again, see http://dockr.ly/logging for much more detail on logging.

31. Troubleshooting
How to use Java tools with container based JVMs
● JVM command line tools via docker exec
○ GC Stats: jstat --gcutil
○ Heap dumps/histograms: jmap
● Expose JMX ports for jconsole or other utilities
● Intelligent health checks
○ More than just “port 8080 is listening”
● Check third party monitoring tools for updated to be “container aware”
○ i.e. Licensing issues with older monitoring tools because each container appears as a new
host
● Also, docker specific commands/tools:
○ docker stats
○ ctop

32. Application Configuration
Managing multi-environment config’s

33. Application Configuration
Deploying to disparate environments with identical images
● Build artifacts are your Docker images, not .war files or similar
● Build images in CI, store in registry, deploy same images everywhere
● Patterns and tools to deal with configuration differences
○ Separate Stack yaml files
○ Docker secrets
○ Application configuration via volume mounts
○ Third party configuration tools such as Consul and/or Vault
■ consul-template
■ Joyent Containerpilot
■ Roll-your-own

34. Environment specific Stacks
● Different environment variable values
● Services that mock production endpoints
○ db
○ web service
prod.yml
dev.yml

35. Docker Secrets
● Stored encrypted in swam
● Exposed only to nodes that run services that need them
● Presented in container via RAM only tmpfs files
○ never persisted to disk in encrypted format
○ when container stops, secret is no longer present
● All communications between swam nodes via TLS, so secret never in the clear on the wire either
● Different secret values per environment using tags
● UCP can manage who/where secrets are available

36. Application configuration in volume mounts
● Use volumes that are only available in physical environment they apply to
● Contain environment-specific application configuration properties
● DO NOT store secrets in these (use Docker Secrets or other secure mechanism)
● You can bind mount files (doesn’t have to be full directory structures)

37. Resources
So much to talk about, so little time to do so!

38. Resources
So much to talk about, so little time to do so!
● Docker Resources: https://www.docker.com/products/resources
○ Logging Reference Architecture: http://dockr.ly/logging
○ Training: https://training.docker.com
■ Instructor led
■ Self paced with “Play With Docker”
○ Containerizing legacy applications?
■ https://docker.com/MTA
● SquareSpace Blog: Understanding Linux Container Scheduling (with JVMs)
https://engineering.squarespace.com/blog/2017/understanding-linux-container-scheduling

39. THANK YOU :)
@ericsmalling