Intro to Docker and clustering with Rancher from scratch

194 views

Published on

John will fully explain what Docker is, why it is useful, how it works and most importantly discuss the pros and cons of it from real world production experience. We will then build out (from scratch) a clustered Docker environment using Rancher (machine setup done with Ansible) across multiple virtual machines in the cloud using Terraform learning tips, tricks and what is happening under the covers along the way.


Expect to leave the talk feeling confident about if Docker might make sense for your next project (or might not!) and how to get started easily if it looks like it is the right tool for the job for you.

Published in: Software
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
194
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
46
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • (already done) docker pull mongo

    docker images

    docker run --name=myfirstmongo mongo
    INTERACTIVE SHELL WHOOPS!

    docker run --detached --name=myfirstmongo mongo
    CONTAINER ALREADY EXISTS WHOOPS!

    docker ps
    docker ps -a
    docker rm myfirstmongo

    docker run --detached --name=myfirstmongo mongo
    docker run --detached --name=mysecondmongo mongo

    docker ps





  • docker build -t first-image .
    docker run --name first-container first-image
    docker ps --a
    docker run -it first-container sh
    UNCOMMENT COPY SECOND FILE
    docker build -t second-image .
    docker run --name second-container second-image
  • Intro to Docker and clustering with Rancher from scratch

    1. 1. john culviner github: github.com/johnculviner blog: johnculviner.com twitter: @johnculviner email: john@johnculviner.com intro to with a side of
    2. 2. About Me  Free range, sometimes organic Full-stack Independent Consultant @ Veritas in Roseville  Backend  DevOps (Docker, Ansible, Linux etc)  NoSql (ElasticSearch, MongoDB)  Distributed systems (RabbitMQ, Kafka etc.)  Node.js  Groovy/Spring/Java  C#  Front End  Angular.js, React.js, Knockout.js, Durandal.js, jQuery, CSS/SASS etc.  SPA development  Open Source “Street Cred”  AngularAgility  jQuery File Download  FluentKnockoutHelpers
    3. 3. Overview  Docker  How does it work  Why would I use it  Rancher  What does it give me  Building a Clustered Docker + Rancher environment from scratch  Terraform (DigitalOcean)  Ansible  Node.js Microservice  Objective: To leave feeling confident about if Docker might make sense for your next project (or might not!) and how to get started easily if it looks like it is the right tool for the job for you.
    4. 4. What is ?  It’s all about the containers!
    5. 5. Images Internal Docker Registry hostname: MY_REG:5000 myapp:1.0 myapp:1.1 yourapp:1.0 yourapp:1.1 … Public Docker Registry AKA: hub.docker.com elasticsearch:5.0.0 elasticsearch:5.0.1 rabbitmq:3.6.4 rabbitmq:3.6.5 … Any machine running Docker MY_REG:5000/myapp:1.1 elasticsearch:5.0.1 may equal when :tag not specified defaults to
    6. 6. Confused? Container vs Image  A container is an “instance” of an “immutable” image  Could be running or stopped My machine running Docker for Mac Loaded Images mongo:latest Running Containers Image Name mongo:latest Container Name myfirstmongo Image Name mongo:latest Container Name mysecondmongo …
    7. 7. Moderate Mongo Mess  mongo:latest isn’t terribly useful to know what the version really is  There is no external/port level access to the containers  There are no volume mounts for persistent data (very bad for perf on with high I/O applications)  If the container dies it’s not coming up again without me restarting it  Fortunately? there is: docker run --name=myfirstmongo --detach --publish="27017:27017" -- restart=always --volume="/some/local/path:/data/db" mongo
    8. 8. A better way: docker-compose  Tearse & readily source controlled YAML definition  docker-compose.yml Idempotence (to an extent)
    9. 9. docker-compose for CI/CD!  Run isolated integration testing CI/CD of your whole app stack from anywhere! (local, Jenkins etc.) Builds a local Dockerfile Define DNS aliases of references only available from my_app stdout/err comes out to pass/fail Jenkins build Test command: stdout/err comes out of container to pass/fail the build Mongo only addressable to my_app at DNS “mongodb” No stdout/err Real live chrome/selenium server in a container using xvfb
    10. 10. Benefits of Images & Containers  Better Isolation & Consistency with Images  Docker Repository vs. Artifactory, NPM, Nuget etc.  Debug a production image on my local machine  EX: Run 10 different YOUR_FAV_LANG apps using 10 different versions the runtime all on port 8080 on same box*  *with a SDN (software defined network)  Security*  *When you don’t run as root, use SELinux, sandbox volumes among other things +Docker
    11. 11. Building images with layers  Done with a Dockerfile, lets do it!  See layers with “docker inspect IMAGE_NAME”  What we did: image layer: alpine:latest image layer: first_file added image layer: second_file added container: second-container container: first-container
    12. 12. Layer re-creation/sharing  Docker will re-use existing layers when it can:  When a layer changes subsequent layers are invalidated otherwise they are re-used  This effects: Proportion of Image Size Changes every build (probably) npm install only runs if package.json (a dependency/package manifest) changes! pull/push HTTP traffic server filesystem usage repository storage space BUILD TIMES!
    13. 13. Docker Observations  Set up development environment quickly with a docker-compose for a project  E2E Integration testing easily with a docker-compose  Image consistency to production  stdout/stderr aggregation QA servers myapp:1.2.3 PROD servers myapp:1.2.3 DEV servers myapp:1.2.3 - Commit - Build - Test deploy server-a server-b server-c server-d ElasticSearch + Kibana stdout/err from all containers
    14. 14. Well that was cool for DEV but…  How do I run containers on multiple machines and orchestrate them?  How do I ensure HA (high availability)  How do I load balance HTTP/S applications  How do I schedule based on load Does Docker actually make sense to run real applications in PROD? *well I have at least with less work and less downtime than other approaches I’ve encountered… so far
    15. 15. Partial lay of the land* *as I see it: grain of salt please +
    16. 16. What is ?  A really slick UI that illustrates what is going on in a very clear manner  Actually helps you learn real Docker (full API surface almost!) visually and then helps you script things after you have “pointed and clicked your way to success”  Easily runs in Docker container(s)  Container orchestration/clustering support for a variety of different platforms:
    17. 17. What is Cattle?  A relatively simple container orchestration framework that is natively supported by Rancher  Pros  Built in layer 5 (haproxy based) load balancer that supports scaling, rolling upgrades, rollback changes etc.  Slick SDN (Software Defined Network) does DNS based round- robin inter-container network resolution  Simpler & quicker to get going than anything else  “3AM Googleability” is very high / vibrant community  Works with Docker rather than against it  Realistically free!  I’ve battle tested it and has worked well so far  Cons  Scheduler is rather simple / no automatic container creation support
    18. 18. + +  Setup entire stack from scratch in a repeatable (idempotent), clear & source controllable manner  *Some of the Rancher stuff we will “point and click our way to success” for brevity and to show you the UI but I’ve done it before with 100% Ansible + docker/rancher- compose files.  Requirements  POSIX shell  DigitalOcean account with API key env variable  SSH  ~/.ssh/id_rsa + ~/.ssh/id_rsa.pub setup  Ansible (get with Python, PIP)  Terraform  Web browser
    19. 19. The Goal docker0 docker2docker1 docker3 rancher/server rancher/agent rancher/agent rancher/agent rancher/agent Idempotent Cloud VM creation tool Cloud VM Provider Ubuntu 16.04 Cloud VMs w/ Containers Idempotent Server Provisioning Tool johnculviner/ nodejs-echo-hostname johnculviner/ nodejs-echo-hostname johnculviner/ nodejs-echo-hostname johnculviner/ nodejs-echo-hostname johnculviner/ nodejs-echo-hostname johnculviner/ nodejs-echo-hostname …… … rancher haproxy load balancer HTTP Traffic + few SSH commands The code https://github.com/johnculviner/doc ker-rancher-presentation
    20. 20. E2E IRL Ideas A Jenkins pipeline build
    21. 21. questions/comments? john culviner github: github.com/johnculviner blog: johnculviner.com twitter: @johnculviner email: john@johnculviner.com

    ×