Docker and configuration management
•Download as PPTX, PDF•
1 like•1,679 views
The document discusses using Docker and configuration management tools like Ansible and Chef together. It provides examples of building Docker images from code using tools like SBT and Maven. It also describes using Ansible playbooks and Chef recipes to automate pulling, running and managing Docker containers. Configuration files and credentials can be handled securely. Overall, the document shows how Docker and configuration management can be integrated to provide an automated and flexible deployment pipeline.
More Related Content
What's hot
What's hot (19)
Viewers also liked
Viewers also liked (16)
Similar to Docker and configuration management
Similar to Docker and configuration management (20)
More from Mukta Aphale
Recently uploaded
Recently uploaded (20)
Docker and configuration management
- 1. DOCKER AND CONFIGURATION MANAGEMENT PUNE DOCKER MEETUP, 25 APRIL 2015 MUKTA APHALE
- 2. AGENDA • Use Case: Considering Docker in Production • Configuration Management and Docker • Docker and Ansible • Docker and Chef @muktaa
- 3. GOAL @muktaa •git push •Triggers build Code Generate Docker Image Build Process Save Image Docker Image Unique Tag Docker Registry •docker pull •docker stop •docker run Deploy
- 4. EXAMPLE • git push to https://github.com/muktaa/HelloScala • Triggers a build on your CI server • sbt docker • docker push muktaa/hello-scala • Deploy • Build tools offer docker integration • Eg: Maven has docker-maven-plugin • https://github.com/spotify/docker-maven-plugin • mvn clean package docker:build -DpushImage
- 5. ~/github/HelloScala > sbt docker [info] Loading project definition from /Users/muktaaphale/github/HelloScala/project [info] Set current project to hello-scala (in build file:/Users/muktaaphale/github/HelloScala/) [info] Creating docker image with name: 'muktaa/hello-scala' : [info] Sending build context to Docker daemon [info] Step 0 : FROM dockerfile/java [info] ---> 1126c85d8a06 [info] Step 1 : ADD /app/hello-scala_2.11-1.4-one-jar.jar /app/hello-scala_2.11- 1.4-one-jar.jar [info] ---> Using cache [info] ---> 61871958f108 [info] Step 2 : ENTRYPOINT java -jar /app/hello-scala_2.11-1.4-one-jar.jar [info] ---> Using cache [info] ---> a8005b32ddc4 [info] Successfully built a8005b32ddc4 [info] Successfully built Docker image: muktaa/hello-scala [success] Total time: 1 s, completed Mar 3, 2015 2:10:04 PM ~/github/HelloScala > docker images | grep hello-scala muktaa/hello-scala latest a8005b32ddc4 12 hours ago 715 MB ~/github/HelloScala > docker run muktaa/hello-scala Hello, world! #1 Hello, world! #2 Hello, world! #3
- 6. DOCKER REGISTRY Docker Hub Link: https://registry.hub.docker.com/u/muktaa/hello- scala Automated Build in Docker: https://registry.hub.docker.com/u/muktaa/helloscala- automated-build/
- 7. CHALLENGES • Docker based CI server (Travis-CI) • Tight coupling with build tool • Base Image • Monitoring • Debugging • Handling configuration for various environments • Secure Credential Management @muktaa
- 8. LESSONS LEARNT • Running apps in containers is easy • Debugging apps in containers is difficult • You can very well run multiple services inside a docker container • Ah the woes of Docker networking! • Sequential Progression @muktaa
- 9. CONFIGURATION MANAGEMENT VS DOCKER • Control the environment Vs System Image / Runtime image • Tradeoff between flexibility and convenience • CM is the vein of DevOps • Shell scripts -> Chef • Immutable Infrastructure @muktaa
- 11. ANSIBLE • Ansible Tower • Playbooks • Ansible Tower API @muktaa •git push •Triggers build Code Generate Docker Image Build Process Save Image Docker Image Unique Tag Docker Registry •docker pull •docker stop •docker run Deploy
- 12. ANSIBLE TOWER API • http://www.ansible.com/tower • Dashboard • Job handling and scheduling ease • Rest API • Tower CLI @muktaa
- 13. ANSIBLE TOWER CLI • https://github.com/ansible/tower-cli • Configure • $ tower-cli config host tower.example.com • $ tower-cli config username mukta • $ tower-cli config password password • Launch a job. • $ tower-cli job launch --job-template=144 @muktaa
- 14. ANSIBLE PLAYBOOK - name: pull latest repo command: docker pull muktaa/hello-scala - name: Stop existing container shell: docker ps | grep 8585 | awk -F" " '{print $1}' register: result ignore_errors: true @muktaa
- 15. ANSIBLE PLAYBOOK - name: Run new docker image command: docker run -d -i -t --privileged -p 8585:8585 -h={{ ansible_hostname }} muktaa/hello-scala -DCLUSTER_IP={{ ansible_hostname }} - Dcom.sun.management.jmxremote.port=9999 - Dcom.sun.management.jmxremote.authenticate=false - Dcom.sun.management.jmxremote.ssl=false -DSEED_HOST={{ ansible_seed }} register: result ignore_errors: true @muktaa
- 16. ANSIBLE DOCKER MODULE - name: akka seed container docker: name: seed image: muktaa/hello-scala state: started expose: - 8585 volumes_from: - mydata env: SECRET_KEY: xyz @muktaa
- 18. DOCKER COOKBOOK • Available in Supermarket: https://supermarket.chef.io/cookbooks/docker • Install docker • Build docker image • Pull image and run container • Push docker image to registry • LWRPs • Docker_container • Docker_image • Docker_registry • https://github.com/bflad/chef-docker/blob/master/README.md
- 19. CREDENTIAL MANAGEMENT secret = Chef::EncryptedDataBagItem.load_secret @docker_cred = Chef::EncryptedDataBagItem.load( node['docker']['creds']['databag'], node['docker']['user'], secret ) docker_registry ‘https://registry.hub.docker.com/u/muktaa/he llo-scala/’ do email docker_cred['email'] username docker_cred['username'] password docker_cred['password'] end
- 20. DOCKER_IMAGE # Build a docker image using docker_image resource docker_image node['docker']['image'] do tag node['docker']['image']['tag'] source '/var/docker' action :build end # Push the image to docker registery docker_image node['docker']['image'] do action :push end # Delete the image from the machine docker_image node['docker']['image'] do action :remove end
- 21. DOCKER_CONTAINER # Run Container docker_container ‘muktaa/hello-scala’ detach true port ‘8081:8081’, ‘8085:8085’ env ‘ENVIRONMENT=pre-prod’ volume ‘/mnt/docker/docker-storage’ action :run end
- 22. GENERATE DOCKERFILE # Generate a docker file using template. template "#{node['docker']['directory']}/Dockerfile" do source 'dockerfile.erb' variables image: node['docker']['base']['image']['name'], maintainer: @docker_cred['maintainer'], email: docker_cred['email'], build_cmd: node['docker']['build']['commands'], entry_point: node['docker']['build']['entry_point'] action :create end
- 23. WORKFLOW Build Application • Save the Artifact to a Repository Manager Build Docker Image • Docker cookbook would build and save the docker image Deploy • Docker cookbook runs the container on the nodes
- 24. CHEF CONTAINERS • Package • Provides Configuration Management for containers
- 26. WHY CHEF CONTAINERS? • Bootstrap chef-client without SSH connection • Manage multiple services inside your container • Manage running state of your container • Consistency across Architectures • Mixed Architecture Applications
- 27. BEST SUITED FOR • Transitioning traditional architecture to containers • Handling last mile configuration when container boots • Getting the best of two worlds without complexity
- 28. KNIFE CONTAINER DOCKER INIT • Gem install knife-container • knife container docker init NAMESPACE/IMAGE_NAME [options] • -f base docker image (default is ubuntu 12.04) - chef container should be already installed on it • -r runlist • -z chef client local mode • -b use berkshelf
- 29. EXAMPLE $ sudo knife container docker init muktaa/hello-scala-cc Compiling Cookbooks... Recipe: knife_container::docker_init * directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc] action create * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Dockerfile] action create - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/Dockerfile from none to 943017 - * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/.dockerignore] action create - create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/.dockerignore - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/.dockerignore from none to e3b0c4 * directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef] action create - create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/client.rb] action create - create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/client.rb - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/client.rb from none to 7de61f * file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first-boot.json] action create - create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/first- boot.json - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/first-boot.json from none to 5269ef * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/.node_name] action create - create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/.node_name - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/.node_name from none to 4764d2 * template[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/Berksfile] action create (skipped due to only_if) * directory[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc/chef/secure] action create - create new directory /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/secure * file[/home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/secure/validation.pem] action create - create new file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/secure/validation.pem - update content in file /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala- cc/chef/secure/validation.pem from none to ec1f3e - change mode from '' to '0600' Downloading base image: chef/ubuntu-12.04:latest. This process may take awhile... Tagging base image chef/ubuntu-12.04 as muktaa/hello-scala-cc Context Created: /home/ubuntu/chef-repo/dockerfiles/muktaa/hello-scala-cc
- 30. KNIFE CONTAINER DOCKER BUILD • run command docker images • knife container docker build • resolve docker dependencies • build docker image • cleanup chef artifacts
- 31. EXAMPLE $ sudo knife container docker build muktaa/hello-scala-cc Sending build context to Docker daemon 9.728 kB Sending build context to Docker daemon Step 0 : FROM muktaa/hello-scala-cc ---> 50d3c5c9e133 Step 1 : ADD chef/ /etc/chef/ ---> 4933cc9e13e0 Removing intermediate container da0a08413a91 Step 2 : RUN chef-init --bootstrap ---> Running in add27db609cc [2015-03-31T21:44:44+00:00] INFO: Starting Supervisor... [2015-03-31T21:44:44+00:00] INFO: Supervisor pid: 9 [2015-03-31T21:44:49+00:00] INFO: Starting chef-client run... [2015-03-31T21:44:50+00:00] INFO: Forking chef instance to converge... [2015-03-31T21:44:50+00:00] INFO: *** Chef 11.16.2 *** [2015-03-31T21:44:50+00:00] INFO: Chef-client pid: 16 [2015-03-31T21:44:53+00:00] INFO: Client key /etc/chef/secure/client.pem is not present - registering [2015-03-31T21:44:53+00:00] INFO: HTTP Request Returned 404 Object Not Found: error [2015-03-31T21:44:54+00:00] INFO: Setting the run_list to [] from CLI options [2015-03-31T21:44:54+00:00] INFO: Run List is [] [2015-03-31T21:44:54+00:00] INFO: Run List expands to [] [2015-03-31T21:44:54+00:00] INFO: Starting Chef Run for muktaa-hello-scala-cc-build [2015-03-31T21:44:54+00:00] INFO: Running start handlers [2015-03-31T21:44:54+00:00] INFO: Start handlers complete. [2015-03-31T21:44:55+00:00] INFO: Loading cookbooks [] [2015-03-31T21:44:55+00:00] WARN: Node muktaa-hello-scala-cc-build has an empty run list. [2015-03-31T21:44:55+00:00] INFO: Chef Run complete in 1.121705004 seconds [2015-03-31T21:44:55+00:00] INFO: Running report handlers [2015-03-31T21:44:55+00:00] INFO: Report handlers complete [2015-03-31T21:44:55+00:00] INFO: Sending resource update report (run-id: 6f637baf-18cc-4620-b3e2- 9afc90e8cd6b) ---> 2c2ec6fab1ef Removing intermediate container add27db609cc Step 3 : RUN rm -rf /etc/chef/secure/* ---> Running in 30a3611b083f ---> cab28d6eed90 Removing intermediate container 30a3611b083f Step 4 : ENTRYPOINT ["chef-init"] ---> Running in 0a9f4e96bbf7 ---> a8577b66b103 Removing intermediate container 0a9f4e96bbf7 Step 5 : CMD ["--onboot"] ---> Running in f9a444817229 ---> 21b3800bc9b3 Removing intermediate container f9a444817229 Successfully built 21b3800bc9b3
- 32. DOCKER IMAGES $ sudo docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE muktaa/hello-scala-cc latest 21b3800bc9b3 2 hours ago 311.9 MB <none> <none> b343c8301cc8 2 hours ago 311.9 MB chef/ubuntu-12.04 latest 50d3c5c9e133 6 months ago 311.9 MB $ sudo docker push muktaa/hello-scala-cc $ sudo docker –d run muktaa/hello-scala-cc
Editor's Notes
- Here are the steps in detail, of how this can be done. A sample demo project, HelloScala triggers a Travis-CI build. SBT is “scala build tool”. Sbt docker command creates a docker image. Muktaa/hello-scala is a repo on the docker registry where we would save the docker images. Other build toolsl ike maven offer docker integration too. Run the command mvn docker:build to create docker image That’s it! Did I hear using Chef is complex? Esp in this particular case?
- That’s how the command execution looks…
- We talked about docker registry which woud save docker images. Docker hub provides a hosted docker registry. The links above show 2 different repos on the docker registry. We can setup automated builds in that registry. With git push, this build can be triggered, or the build can be triggered from your CI server using the API call. However this feature is in a crude phase with very less flexibility, it takes a long time for the docker image to build. You can setup the docker registry in your data centre too. It works very similar to the docker hub.
- The docker cookbook is available in chef supermarket Using it, you can install docker, build docker images, commit & push to docker registry, pull image and run container. For the docker image management and deploy, the 3 LWRPs are useful.
- Using lwrp docker_image to build and push the image to docker registry
- Elegant and working solution
- Package that provides config managemet for your containers
- chef-client Latest chef-client that runs within the container. runit - RUNit - lightweight cross-platform init scheme to ensure all child processes are properly managed chef-init - root process which can launch and manage multiple processes inside a container . It is custom built by chef. Each OS has an init. docker replaces the init of the OS. chef-init runs as PID1 and delegates managing child processes to runit.
- Bootstrap the chef-client without an SSH connection Manage multiple services inside your container Use the knife container plugin to work with Linux containers; use the docker build and docker init arguments to manage Docker image contexts Use chef-client resources the same way in a container as on any UNIX- or Linux-based platform Consistency across architecgures – you don’t need different CM tools for Physical, virtual, or machines on cloud. If you have invested in Chef already which runs for you on a physical machine you leverage the same config on chef container Mixed architecture. You might be using docker for development or test environments. But not in production. If you use chef containers for your dev & test env but physical machines on production, then it is lot easier to manage the config on each, as the config is idempotent if you use Chef for CM.
- transitioning trad architecture to containers - all you need is the chef run list, it can run on bare metal, VMs or containers. handling last mile config when container boots - registering an agent you need for monitoring maybe, or some env specific changes. e.g.: in our DEV env we read some reference data which is a truncated version of the real data while in the test env its a full blown test data. and in prod it is the real data & not the synthetic test data. Using chef containers is not complex. You can define container configuration using chef recipies instead of long bash scripts Chef container can be used to manage docker images. You can set certain services to launch when the container launches using the enable action of the chef resource. When chef-init starts, it can launch chef-client and you can configure which service you want to start using start action of the chef resource.
- 1. install knife-container gem knife container is the only command needed to manage the docker lifecycle 2 Knife container docker init creates the docker context which comprises of docker components and chef components To initialize the Docker context, use the init command. The knife container uses a folder called dockerfiles to organize all the Docker contexts that you manage. By default, the dockerfiles folder is created in your chef repo. To initialize the Docker context, type the following command: Pass in your image name (in this example,demo/apache2), a run list, a –zand a –b. The –z is for local mode and the –b says to generate a Berksfile.
- { "run_list": [ "recipe[apache2]" ], "container_service": { "apache2": { "command": "/usr/sbin/apache2 -k start" } } } Resolves chef dependencies Build docker image Cleansup chef artifacts
- Note step 2: chef-init —bootstrap (this runs the chef-client) step 1: adds the chef dir to /etc/chef step 0: & last tag: only 1 copy of the image, see docker images so we created an image from ubuntu 14.04 and created image, the new image will have same tag but diff image id.