Presented by Laura Frank, Engineer, Codeship Testing software is necessary, no matter the size or status of your company. Introducing Docker to your development workflow can help you write and run your testing frameworks more efficiently, so that you can always deliver your best product to your customers and there are no excuses for not writing tests anymore. You’ll walk away from this talk with practical advice for using Docker to run your test frameworks more efficiently, as well as some solid knowledge of software testing principles.

1. Stop Being Lazy and Test Your Software!
Laura Frank
Software Engineer, Codeship

2. 0.7.0
Codeship
ImageLayers
Panamax
Berlin

3. Agenda
Continuous Delivery
Why Do We Test?
Testing with Docker Compose
Faster Testing with Docker

4. Why Do We Test?
Motivations and frustrations

5. Testing software has been a
practice since the very
first machines were used
for computing.

7. The women who
programmed the ENIAC to
calculate trajectories
periodically checked the
results with a correct,
hand-computed table.

8. working software
in production
testing
code reviews
version control
pair programming
high-availability architecture

9. Motivators
Update application without breaking things!
Validate functionality of updates
Be able to trust deployment checks (in CI/CD workflow)
Confirm that refactoring didn’t break existing functionality

10. Why do you skip tests?
45%
15%
5%
35% Too slow to run suite 35%
Annoying extra step 15%
Don’t see the point 5%
Slows down deployment 45%
10

11. Frustrators
It takes me an hour to write a single test
My new tests just duplicate existing coverages so there’s no point
(integration overlapping with unit)
It takes my test suite over 20 minutes to run, so I don’t run it locally
Testing distracts me from my normal development workflow
Setting up a testing environment is a huge pain
It takes too long to deploy when I have a huge test suite

12. Frustrators
It takes me an hour to write a single test
My new tests just duplicate existing coverages so there’s no point
(integration overlapping with unit)
It takes my test suite over 20 minutes to run, so I don’t run it locally
Testing distracts me from my normal development workflow
Setting up a testing environment is a huge pain
It takes too long to deploy when I have a huge test suite

13. Using Docker can alleviate
some frustrations
associated with testing.

14. Testing with Docker
Compose
It’s easy, I promise

15. Docker and testing are a great pair.
❌ ✔

16. With Docker Compose, it is incredibly
easy to create consistent,
reproducible environments.
❌ ✔

17. Many of us already use
Docker Compose to set
up dev environments.

18. …But we stop there.

19. Use your Docker Compose
environment for testing
as well.

20. app:
build: .
command: rails server -p 3000 -b '0.0.0.0' volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres:
image: postgres:9.4
ports:
- '5432'
A simple Rails app

21. ?!
git clone && docker-compose up
hackity hack
How do I make tests happen?
Development workflow

22. app:
build: .
command: rails server -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres:
image: postgres:9.4
ports:
- '5432'
A simple Rails app

23. app:
build: .
command: rspec spec
volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres:
image: postgres:9.4
ports:
- '5432'
A simple Rails app

25. In most cases, we need to
do a bit of setup
before running tests.

26. You can run one-off
commands against a service
with Docker Compose.

27. docker-compose run -e "RAILS_ENV=test"
app rake db:setup
docker-compose run -e "RAILS_ENV=test"
app rspec path/spec.rb
docker-compose up #-d if you wish
Usual Docker Compose development
environment
Run rake task to set up db
Then run tests against Docker services

29. Docker delivers a
predictable, reproducible
testing environment. Period.

30. Continuous Integration,
Testing, and Docker
Fail fast and not in production!

31. Continuous integration and
testing go hand in hand.
👯

32. Relying on CI/CD without
adequate test coverage
is not a great idea.

33. Would you rather…
Find out your code is broken by
seeing a failed run of your CI/CD
system?
Find out your code is broken by
seeing 500s on 500s on 500s in
production?
✔
💩

34. What about running tests
like this…
…inside a container?

35. A word of warning!

37. —Jérôme Petazzoni
“Docker-in-Docker is not 100%
made of sparkles,
ponies, and unicorns.
37

38. Docker in Docker

39. CAUTION!
YMMV
See https://jpetazzo.github.io
- Mixing file systems == bad time
- Build cache == bad time

40. Shared daemon – what we really want

41. Parallel Testing with
Docker
Break it up.

42. Why do you skip tests?
45%
15%
5%
35% Too slow to run suite 35%
Annoying extra step 15%
Don’t see the point 5%
Slows down deployment 45%
42

43. When developing, it’s easy
to think of a container as
a small VM that runs a
specific workload.

44. ✔

45. But if we change our thinking
to treat containers as just
processes, then we can do
some pretty cool stuff.

46. ✔

47. A syntax similar to Docker Compose for
service definition…
47
web:
build:
image: my_app
dockerfile_path: Dockerfile
links:
- redis
- postgres
redis:
image: redis
postgres:
image: postgres

48. And use it to also define testing steps…
48
- type: parallel
steps:
- service: demo
command: ruby check.rb
- service: demo
command: rake teaspoon suite=jasmine
- service: demo
command: rake test

49. - type: parallel
steps:
- service: demo
command: ruby check.rb
- service: demo
command: rake teaspoon suite-jasmine
- service: demo
command: rspec spec
Each step is run independently in
a container
49

50. - service: demo
command: ruby check.rb
- service: demo
command: rake teaspoon suite-jasmine
- service: demo
command: rspec spec
And each container may be located in a range
of availability zones
50

52. Docker makes this possible by
providing the means to create
a predictable, reproducible
testing environment.

53. Resources
#keepshipping

54. —Edsger Dijkstra
“Testing can be a very effective
way to show the presence of
bugs, but is hopelessly
inadequate for showing their
absence.”
54

55. Resources
Highly recommended talks about development, testing, and lots of
interesting stuff: https://speakerdeck.com/searls
Ruby gem for parallel tests: grosser/parallel_tests
Parallel Docker testing: Jet (from Codeship)
CI/CD with Docker: http://pages.codeship.com/docker
Running commands with Docker Compose:
http://docs.docker.com/compose
The perils of Docker-In-Docker: https://jpetazzo.github.io
This talk: slideshare.net/rheinwein

56. Testing does not
guarantee that
our software works.

57. We test to know
when our software is
definitely broken.

58. Thank you!
Laura Frank
@rhein_wein
laura@codeship.com