This document discusses how Docker can be integrated into DevOps workflows to improve the development and testing process. Docker allows developers to build, ship, and run applications inside lightweight Linux containers, enabling consistent environments for development, testing, and production. By using Docker images and containers, developers can achieve faster iteration and more reliable automated testing across different environments like development, QA, and production. The document also addresses how Docker can be used on non-Linux platforms like Windows and MacOS using tools like Boot2Docker and Vagrant.

1. Fits Docker into Devops
Evans Ye
2014.10.6

2. Let’s discuss about Devops first
#TrendInsight

3. PRE-Devops
• Developer builds and tests on dev servers
• Devloper hands off code to operation
• likely deployed manually
• Sometimes it broken in Production
• Need to be fixed, Now

4. 4

5. Devops
• Getting changes to users quickly, reliably, and
securely
• Quick release cycle from code to Production
• More confidence due to automated testing
• reproducibility
– bugs are not always 100% reproducible

6. 6
Fast Iteration
backed by automated testing

7. Google Devops (2013)
• 75 million test cases are run daily
"Automated tests transform fear into boredom."
-- Eran Messeri, Google

8. Continuous Integration
• Same environment in Dev, QA and Production
• Auto deploy
– deploy reliability and consistency
• Automated tests
– unit test
– smoke test (integration test)

9. How do you test hadoop features?
• Manually test on dev cluster
– Hey folks, don’t move, I’m testing my own features.
• Pig unit
– adhoc-query? dev environment only
• HBase mini-cluster
– does not support Windows?
• Still not exactly the same environment as Production
• what if written in python, ruby, php?

10. What if we can do Dev and Test
in the same way, efficiently?
10

12. Docker
• lightweight, portable Linux containers (like VMs)
– fast creation
– system isolated
– resource constrained
• Build, ship, run containers everywhere
– containers as dev and test environment

13. Containers offer faster automation

14. Introduction to Docker
#TrendInsight

15. The Docker Engine
runs containers
15

16. Architecture
VM
Docker

17. 17
The Docker Hub
stores and ships
images

18. Docker image
• $ docker runs <IMAGE>
to create a container
• base image stores /bin,
/etc, /usr (Linux distro.)
• add you app(diff) on top
of base image
• commit containers
• push, pull images

19. Docker offers you
• Deploy efficiently
• Quick start
• Lightweight

20. Limitation of Docker
• Any Linux with kernel > 3.8.x
• Linux only (Linux container, after all)
Linux

21. 21
What about
Windows and
Macbook…?

22. Vagrant
• A open source command line VM(or IaaS)
provisioning tool
Providers
Vagrantfile
VM

23. Solution for OS X / Windows
Container Container
boot2docker
(2XMB, tiny core Linux)
OS X / Windows
with vagrant and virtualbox
installed
 docker
containers
 docker platform
auto provisioned by
vagrant
 $ vagrant
up
DONE!

25. Hadoop Integration
25

26. Develop and test hadoop related features
10/22/2014
link
service
(hadoop-client)
data
(hadoop-client)
hadoop fs –put …

27. Q & A
#TrendInsight