Downloaded 85 times
Uploaded byDavid Currie
IBM WebSphere Liberty and Docker Deep Dive
The document provides a comprehensive overview of deploying IBM WebSphere Liberty using Docker, detailing aspects like image creation, container provisioning, logging, and security best practices. It also highlights IBM's private container registry and associated services but includes important legal disclaimers regarding accuracy and liability of the information presented. Furthermore, it encourages feedback from attendees of the Interconnect 2016 conference.
More Related Content
Similar to IBM WebSphere Liberty and Docker Deep Dive
IBM WebSphere Liberty and Docker Deep Dive
- 1. IBM WebSphere Libertyand Docker Deep Dive David Currie | david_currie@uk.ibm.com | @dcurrie Tom Banks | tom.banks@uk.ibm.com | @tomwillbanks
- 2. Governance Advisory Board Maintainers/ Contributors Open Container Initiative Cloud Native Computing Foundation Strategic Partnership Product Enablement
- 3. Developer experience Docker Toolbox/ Repositories Docker Hub Docker Tooling for Eclipse WebSphere Developer Tools
- 4. Building an image Ourwebsphere-liberty image or yours? What base OS? Customized vs Reusable? Follow best practices Build pipeline https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
- 5. Sharing images Docker Hub IBMContainers registry docker save/load ‘registry’ image Docker Trusted Registry
- 6. Docker host provisioning Architecture:x86, ppc or z Bare metal or VMs OS Provisioning
- 7. Docker engine options Filesystem drivers: aufs/devicemapper/btrfs/overlay Remote API access
- 8.
- 9. Container run options cpuset/shares/quota Memory/kernelmemory limits Read-only Restart policy
- 10.
- 11. Networking Out of thebox: bridge/none/host/container/overlay Weave Calico Kuryr
- 12. Logging Logging drivers: json-file/syslog/journald/ gelf/fluentd/awslogs/splunk Export /logsas volume logstash/logmet feature
- 13. Security Docker Engine runningas root Root user in container https://docs.docker.com/engine/security/security/ https://blog.docker.com/2015/05/understanding-docker-security-and-best-practices/
- 14. IBM Containers Private registrywith Vulnerability Advisor Available across multiple regions Per-minute billing Scaling groups with load balancing Integrated logging and monitoring Private VLAN and persistent volumes
- 16. Notices and Disclaimers 16 Copyright© 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law
- 17. Notices and DisclaimersCon’t. 17 Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.
- 18. Thank You Your Feedbackis Important! Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.
Editor's Notes
- #3 Before we get in to the detail, I’d just like to emphasise that IBM has bought in to Docker big time. As with other open technologies that IBM invests in such as OpenStack and Node.js, IBM has sought an open governance process for Docker and is a founding member for the Governance Advisory Board. We don’t believe in setting the direction without helping deliver on that direction and IBM employs full-time maintainers and contributors on the core Docker projects. IBM is also a founding member of the Open Container Initiative looking to standardise the image format and container runtime. It was quickly recognised that there would be benefit in standardising at a high level of abstraction which is the purpose of the Cloud Native Computing Foundation. IBM also has a strategic partnership with Docker, acting as a reseller of the Docker Trusted Registry and providing L1 and L2 support. Lastly, and the subject of this presentation, IBM has been enabling its products with support for Docker.
- #4 It’s not all or nothing. Docker may be used for deployment and not development or vice-versa. Developers may, for example, still create server packages and then the middleware team build that in to a Docker image. There are many benefits to Developers using Docker though. The developer experience starts with installing Docker, via Docker Toolbox on Mac/Windows or via a repository for their Linux distribution. A typical starting point is then to use images from Docker Hub has a starting point. Beware of what you run though. Favour official images and those with automated builds where you can see what an image contains. Don’t run untrusted containers in privileged mode! The Docker Tooling for Eclipse provides an easy to use UI for managing images and containers. In the February beta of the WebSphere Developer Tools we now have support for working with Liberty in Docker containers. It is possible to remotely attach to a running Liberty container, deploy applications to it, and switch it to and from debug mode.
- #5 Our websphere-liberty image is for development use-only but you can upgrade the licnese files to make it a production image. Alternatively we provide Dockerfiles and instructions on how to build the same image using production binaries. This still uses the community Ubuntu image as a basis. Consider how much of that Ubuntu distribution is actually being used though? You are free to take the Dockerfiles and modify them to build from a different base image e.g. RHEL. If we provided support for the base OS, would that make a difference? Should the version of Java and Liberty matter or should you always go with latest? Liberty’s zero migration policy and feature set tags help here. We provide tags for common feature sets e.g. webProfile7 and javaee7 but you can also use the kernel image and installUtility to build an image that contains just the features required by the application. Doing so results in a smaller image but also limits reusability of the image. If you do build your own images, try and follow the best practices for Dockerfile. Linters are now becoming available – our websphere-liberty Dockerfiles come up clean. Lastly, you should use a build pipeline to create the images, either directly after the application is built or trigger off deployment of new application to artefact repository. Jenkins is one option, Urban Code Build is another and for a cloud based solution, look at IBM DevOps Services in Bluemix.
- #6 At the end of the build pipeline you will end up with an image that you wish to share. One option is to push the image to Docker Hub, alternatively you could push it to a private registry in IBM Containers on Bluemix. We will return to this option later in the presentation. For an on-premise solution, the simplest option is to save images as a tar file and then load them in again. In doing so, you lose out on the benefit of caching layers which are unchanged. Another option is the registry image from Docker Hub that implements the v2 API. This comes with no support for authentication and you will need to provide a signed certificate for the registry to be trusted by Docker engines. The final option is to purchase the Docker Trusted Registry from Docker or via IBM as a reseller. This provides authentication and management capabilities.
- #7 With an image built, we now need somewhere to run it. The first decision is platform: x86, ppc or z, then whether to run on bare metal or VMs? A bare metal deployment allows you to get the full performance benefit of the lack of a hypervisor but you may wish to partition physical servers with VMs and to provide a further level of isolation. The next choice is what OS to use. This might be your default choice in the enterprise or you might opt for one of the more streamlined options such as Core OS or Red Hat Atomic. You then need to provision the Docker engine. This could be with your favourite configuration management tooling, whether that be Salt, Puppet, Ansible, Chef of Urban Code Deploy. If you go down the VM route, you could use a pre-configured VM. Docker machine provides one route for provisioning and managing these across many cloud providers.
- #8 When configuring the Docker engine, one key option is the choice to use for the union file system. Long the preferred option for Docker deployments, aufs is generally fast but can slow down when accessing large files or large numbers of layers. It has also never made it in to the mainline kernel. Device mapper is the default on Red Hat and does block level thin provisioning. Ensure that you use real rather than sparse devices to prevent allocation delays. Btrfis uses chunked storage. It gives poor performance for large numbers of small files. It also does not allow page cache sharing which can lead to large memory usage. Overlay FS is the latest option and is now merged in to the 3.18 kernel. It does allow page cache sharing but, as with aufs, can suffer from poor performance with large files. You also need to consider whether to enable remote access to the Docker API. This certainly simplifies management of multiple servers but ensure that it is secured.
- #9 With one or more Docker engines deployed and running, we now need a mechanism to launch containers. For simple deployments with small numbers of containers, don’t overlook a simple scripted approach. The next step up would be to apply more automation tooling, for example using UrbanCode Deploy or another configuration management tool. If you are looking to run larger numbers of containers, then you might consider using Docker Swarm which provides for scheduling of containers across multiple Docker hosts. Swarm itself deals in individual containers. You may also want to look at Docker Compose as a mechanism for deploying multiple containers that form an application deployment. Note that Swarm is only just starting to provide capabilities around rescheduling in the event of node failure. Google Kubernetes is another option. In addition to providing an application abstraction in the form of pods, Kubernetes provides other capabilities such as load balancing via service proxies. Now in beta, there is also the possibility to use the Liberty collective controller as a means to deploy images across multiple hosts, in the same way that packaged servers may be deployed today. This will open the gateway to other intelligent management capabilities such as auto-scaling and health management.
- #10 When running a container, there may be additional options that we wish to apply. Some of these are around applying CPU constraints. cpuset can be used to control the cores that a container may run on. If multiple containers are running on a core, then shares can be used to control how CPU time is shared between them. Lastly, quota allows a hard limit to be placed on the amount of CPU time that a containers has. You may also want to restrict the amount of memory that a container is given. One cgroup constraint that didn’t make Docker 1.10 is for the number of PIDs – this is required to prevent fork bombs. You can also specify that the container should be read-only. Parts of the container that need write access should be mounted as volumes (e.g. /logs) or using the new --tmpfs option in Docker 1.10. Lastly, you may wish to configure a restart policy when a container exits other than via a ‘docker stop’. This can be particularly useful for containers that should be started when the Docker daemon starts.
- #11 Typically you will also want to provide environment specific configuration to a container. Environment variables are undoubtedly the simplest way to achieve this, particularly as the ‘run’ command allows a file of environment variables to be specified. This doesn’t support more complex constructs and should not be used for passwords or keys as environment variables are exposed in a number of places e.g. logs and ‘docker inspect’. For these cases, a better option is typically to mount a file in to the container at runtime. This file may be static or it could be generated dynamically e.g. via Consul and confd. The file may be consumed directly (e.g. via Liberty configDropins) or by an entrypoint script running in the container.
- #12 Networking is another area where there are many options. Out-of-the-Box options include: Bridge – the default, a virtual Ethernet bridge Docker0 with a pair of peer interfaces: one is containers eth0 and other is in host’s namespace;. A private IP range assigned to bridge. None – this could be used for a container performing batch operations Host – disables network isolation; faster (no routing) Container – reuse networking namespace of another container – as used by Kubernetes pods Overlay – SDN from SocketPlane; peer-to-peer; pluggable key-store (etcd/Consul/ZooKeeper) to distribute cluster state Weave – virtual network; traverse firewalls and allows for partially connected networks Calico – standard IP routing; layer 3 solution; organization has control over physical network fabric Kuryr – brings OpenStack Neutron networking to Docker
- #13 For logging, the starting point should be the built-in logging driver support giving stdout/stderr for each container. There are a wide selection of logging drivers available out-of-the-box. Not all log information may go to stdout/stderr e.g. FFDC or HTTP access logs in the Liberty case. Here, export the /logs directory as a volume and then attach a log collector either on the host or in another container. A new option, available in the February beta is a new Liberty feature that right directly to a logstash server or the ElasticSearch based logmet service in Bluemix.
- #14 Lastly, it’s worth saying a few words about security although the links referenced here are worth reading and you should apply the benchmark to your environments. We’ll address two particular concerns here though. The first is the fact that the Docker Enginer is running as root and giving unrestricted access to the Docker engine API is equivalent to giving root access to the Docker host. DON’T DO IT! Docker is starting to address this with the introduction in Docker 1.10 of authorization plugin that can be used to restrict access e.g. to prevent use of privileged mode or mounting of volumes. The second point is the use of the root user with the websphere-liberty image on Docker Hub. This is on advice from Docker in that doing so makes it easier for users to bind mount files on the host at runtime. Fortunately Docker 1.10 comes to the rescue again with the introduction of user namespace support so that root within the container can be mapped to a non-root user on the host.
- #15 Lastly, if all of this seems very complied, consider a hosted solution such as IBM Containers in Bluemix. You get a private registry with a Vulnerability Advisor that can advise/prevent deployment of images containing vulnerable packages. The service is available across multiple geographic regions and supports per-minute billing. In addition to supporting the deployment of individual containers, it supports deploying a container as a scalable group with load balancing in front. There is also integrated logging and monitoring capabilities. Containers run on a private VLAN by default with the option to assign public IPs and there is support for persistent volumes.
- #16 Any questions?
- #19 Please provide feedback for session 2340!