Docker allows applications to be packaged with all their dependencies and run consistently across computing environments. It provides isolation, security and portability for applications. This document discusses setting up an Eh Avatar application to run in Docker containers for Postgres, Redis and the application itself. It covers bringing up the dependency containers, building a custom Docker image for the application, and using Docker Compose to define and run the multi-container application. While this provides an introduction, there is still more to learn about optimizing Docker usage and avoiding common pitfalls.

1. 101
Engineering Team
Minh Nguyen & Luong Vo

2. - Let’s look into Eh Avatar, an application to generate a
beautiful avatar for you <3
- https://github.com/Thinkei/eh-avatar
- Let’s setup and run the project
Before we start

3. Application Flow
Web server
1) POST /avatars
--data name=”Minh”
Postgres
Sidekiq server
Redis
2) Save into DB
3) Send to
sidekiq via Redis
4) Background job to
generate avatar
RMagick
5) Get /avatars/1

4. Section 1: What is docker?

5. What is Docker
First, let’s talk about Linux containers….

6. Linux container technology
- Linux containers contain applications in a way that keep
them isolated from the host system that they run on.
- Allow a developer to package up an application with all of
the parts it needs, such as libraries and other
dependencies
- Make it easier to provide a consistent experience between
development and production environment
- Easy to deploy and replicate deployments

7. But why
should we
care ?

8. The challenge

9. The challenge…. but simpler

10. The real-life solution

11. The solution that we won’t talk about

12. But...

13. Why Containers but not VM?

14. Why Containers but not VM?
- We don’t need an entire operating system, just some
components to make everything up and run.
- Container use host kernel to ensure the isolation and
resource control, instead of full layer of visualization
- Therefore, containers are
- Fast in both starting up and operation
- Lower memory footprint
- Lightweight

15. Linux container technology is not new
- First form of container was Chroot, 1982
- FreeBSD Jails, 2000
- Solaris Zones, 2004
- Linux OpenVZ, 2005
- LXC, 2008
- Docker, 2013
- Built on LXC
- Moved to libcontainer (March 2014)
- Moved to runC (July 2015)

16. What is Docker?
- Container solution that provides full packages:
- Image management
- Resource Isolation
- File System Isolation
- Network Isolation
- Change Management
- Sharing
- Process Management
- Service Discovery (DNS since 1.10)

17. Section 2: Working with Docker

18. Docker concepts
Source code
Dockerfile
Build
Docker image
Push
Image
registry
Host
Pull
Docker
container 1
Docker
container 2
Start

19. Image vs Container?
- Images are read-only.
- An instance of an image is called a
container.
- You can have many running containers
of the same image.
- Images are created with the build
command, and they'll produce a
container when started with run.

20. Docker concepts
- Each image has a name and usually attached tags
- All docker images with the same name are grouped
- The most popular docker image registry is Docker Hub
- https://hub.docker.com/r/library/postgres/tags/
- There are other public / private image registries from
AWS, Google, etc.

21. Practice 1: Bring up dependencies
- Checkout branch docker_1 of the repository
- Or view directly on this link:
- https://github.com/Thinkei/eh-avatar/blob/docker_1/DOC
KER_1.md

22. Practice 1: Bring up dependencies
- Install Docker on your local machine.
https://docs.docker.com/install/
- Pull image of redis
docker pull redis:latest
- Pull image of postgres
docker pull postgres:latest

23. Practice 1: Bring up dependencies
- Start redis
docker run --name my-redis redis
- Test the redis server by command
redis-cli -h localhost
- And it fails :troll:

24. Practice 1: Bring up dependencies
- Why couldn’t we connect to the redis server in postgres?
+ All containers are network isolated
+ It means that containers could not access others’
network and the host could not access containers’
network

25. Docker network diagram (simplified)
Host’s network interface (no port)
Network interface A (open 6379)
Container A: Redis (port 6379) Network interface B
Container B
Process Sidekiq
Process redis-cli

26. Practice 1: Bring up dependencies
- Forward port 6379 of container to outside with port 6378
docker run --name my-redis -p 6378:6379
redis
- Test the redis server by command
redis-cli -h localhost -p 6378
- And it works

27. Docker network diagram (simplified)
Host’s network interface
(open 6378)
Network interface A (open 6379)
Container A: Redis (port 6379) Network interface B
Container B
Process Sidekiq
Process redis-cli

28. Practice 1: Bring up dependencies
- Bring up postgres
docker run
--name my-postgres
-p 5433:5432
-e POSTGRES_PASSWORD=password
-e POSTGRES_USER=username
-e POSTGRES_DB=ehavatar
postgres

29. Practice 1: Bring up dependencies
- Setup schema again
DATABASE_URL=postgres://username:password@localhost:5433/
ehavatar
REDIS_URL=redis://localhost:6378
bundle exec ruby setup_schema.rb

30. Practice 1: Bring up dependencies
- Start web server on host machine
DATABASE_URL=postgres://username:password@localhost:5433/
ehavatar
REDIS_URL=redis://localhost:6378
bundle exec puma

31. Practice 1: Bring up dependencies
- Start sidekiq on host machine
DATABASE_URL=postgres://username:password@localhost:5433/
ehavatar
REDIS_URL=redis://localhost:6378
bundle exec bundle exec sidekiq -r
./config/environment.rb

32. Practice 1: Bring up dependencies
- Let’s turn off the postgres docker container and restart
- Oops, all data is gone :’(

33. Practice 1: Bring up dependencies
- Let’s turn off the postgres docker container and restart
- Oops, all data is gone :’(
- What did happen?
+ All containers are file system isolated
+ All containers data are not mounted (linked) to the
host. When it starts again, no data is retained

34. Docker File system diagram (simplified)
Container A’s Virtual FIle system
Container A: Postgres
Host’s real file system

35. Practice 1: Bring up dependencies
- Try again with this command
docker run
--name my-postgres
-p 5433:5432
-e POSTGRES_PASSWORD=password
-e POSTGRES_USER=username
-e POSTGRES_DB=ehavatar
-e PGDATA=/var/data
-v $(pwd)/tmp/data:/var/data
postgres

36. Docker File system diagram (simplified)
Container A’s Virtual FIle system
Container A: Postgres
Host’s real file system
/var/data
/Users/ahihi/tmp/data

37. Docker concepts
- Dockerfile is a text document that contains all the commands a user could
call on the command line to assemble an image.
- It defines:
- Is current image based on other image? If yes, what is it?
- Dependencies installation commands
- How to start the container of this image?
- What environments are allowed to be passed in?
- Which ports the container will expose?
- etc.

38. Practice 2: Build your own docker image
- Checkout branch docker_2
- Or view online at
https://github.com/Thinkei/eh-avatar/blob/docker_1/DOC
KER_1.md

39. Practice 2: Build your own docker image
- Start Postgres and Redis container again we don't need to expose ports
- Create a new file Dockerfile with the following content
FROM ruby:2.4.0
RUN mkdir -p /app
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN gem install bundler && bundle install --jobs 20 --retry 5 --without test
COPY . ./
EXPOSE 9292
CMD ["bundle", "exec", "puma"]

40. Notable concept that you won’t know
- Union File Systems - because sharing is caring

42. Practice 2: Build your own docker image
- Build the docker image with command
docker build -t ehavatar .
- The building process fails. We fail to build RMgick gem. Need to add
dependencies. Add this line to Dockerifle
RUN apt-get update -qq --fix-missing && apt-get install
-y libmagickwand-dev

43. Practice 2: Build your own docker image
- Build again. And it is sure to be successful.
- Start web server and sidekiq
docker run -p 9292:9292 eh-avatar
docker run -p 9292:9292 eh-avatar bundle exec sidekiq -r
./config/environment.rb
- And it fails again. We fail to connect to redis and postgres

44. Practice 2: Build your own docker image
- Using `link` to link the application containers to dependencies
docker run
--link my-redis
--link my-postgres
-e DATABASE_URL=postgres://username:password@my-postgres:5432/ehavatar
-e REDIS_URL=redis://my-redis:6379
-v $(pwd)/tmp:/app/tmp
-p 9292:9292
eh-avatar

45. Practice 2: Build your own docker image
- Using `link` to link the application containers to dependencies
docker run
--link my-redis
--link my-postgres
-e DATABASE_URL=postgres://username:password@my-postgres:5432/ehavatar
-e REDIS_URL=redis://my-redis:6379
-v $(pwd)/tmp:/app/tmp
eh-avatar bundle exec sidekiq -r ./config/environment.rb

46. Section 3: Docker Compose

47. Docker Compose

48. Docker Compose
- A tool for defining and running multi-container Docker applications.
- Run multiple isolated environments on a single host with a single command.
- Better Development environments.
- Easier build / run / scale.

49. Practice 3: Use docker-compose

50. Best practice and pitfalls
1. Containers should be ephemeral
2. Specify a build context
3. Use a .dockerignore file
4. Use multi-stage builds
5. Avoid installing unnecessary packages
6. Minimize the number of layers
7. ….

51. After this, do you become a master of Docker?
- This is just a beginning :troll:
- There are a lot of things haven’t been mentioned yet

52. The end

53. Q&A