Ansible playbooks and roles are code and as any other code it should be tested automatically before it is applied in production environments. But unit tests are not as usefull for Ansible code. We need integrations tests with a fresh linux system everytime. We test with the triple-A concept: • Arrange: boot one or more fresh linux VMs (Vagrant, Virtualbox) • Act: apply Ansible code to test • Assert: test state of the VMs with another Ansible playbook More topics: • Why testing the Ansible code too? • How to test on every git commit? • Troubles to create a testing environment

1. Automated Tests of Ansible code
Klaus.Franken@ing.de
with GitLab, Vagrant, VirtualBox and Ansible
OpenSourceCampAnsible, Berlin, 2019-05-16

2. Ansible playbooks and roles are code and as any other code it should be tested
automatically before it is applied in production environments. But unit tests are not as usefull
for Ansible code. We need integrations tests with a fresh linux system everytime. We test with
the triple-A concept:
Arrange: boot one or more fresh linux VMs (Vagrant, Virtualbox)
Act: apply Ansible code to test
Assert: test state of the VMs with another Ansible playbook
More topics:
Why testing the Ansible code too?
How to test on every git commit?
Troubles to create a testing environment
Topics from Agenda
2

3. • ING Germany
• aka “ING–DiBa“
• more an IT company than a bank
• Ansible is used for lot of different jobs
• Jobs in Frankfurt/Nürnberg:
https://www.ing.jobs/Deutschland/Stellenangebote.htm?keyword=ansible
• Klaus Franken
• Deployment-Pipelines for Java-Apps (Tomcat, JBoss) with Ansible (Tower)
• Ansible since 2014
About
3

4. Ansible code is CODE!
Every code:
• is buggy
• must be adjusted to environment changes from time to time
We want reusable code (i.e. Ansible shared roles)
⇒ Ansible Code must be tested (automatically) just as application code.
But Ansible code cannot be tested in the same way.
Why testing?
4
because we can ...

5. The preferred and first step to test application code is using unit tests.
Ansible code is changing infrastructure. So we have to test changing infrastructure. Mocking
is not possible/reasonable.
We need integration tests with real (virtual) infrastructure.
Why testing different?
5

6. • Ansible code is developped and tried out on existing systems (i.e. VMs).
• Some tools, packages, files and directories are already there – but you don‘t no why. The
dependencies are not obviously.
• When using a role in a new/fresh system you have a lot of „surprises“ (failures).
Test always a reproducible origin
6

7. • Arrange
• create new VMs with Vagrant and Virtualbox
• Act
• run your Ansible code you want to test
• Assert
• check the result, i.e. with Ansible too
• Automated
• by every commit and/or scheduled
• use gitlab-ci
Idea
7
Triple-A + Automation

8. • create a gitlab-runner
• install a VM with needed tools for testing:
• ansible
• vagrant
• virtualbox
• install and connect gitlab-runner (see README)
• enable this runner for your project(s) (FIXME: is there a default?)
Automation 1/2
8
with gitlab-ci

9. https://gitlab.strukturpunkt.de/kfr/ansible_integration_test
Live Demo
9

10. • create a „.gitlab-ci.yml“ within you repo:
integration_test:
tags:
- vbox
script:
- cd integration_test && ./arrange.sh
Automation 2/2
10
with gitlab-ci

11. #!/bin/bash
# arrange
ansible-galaxy install -r ../roles/requirements.yml
vagrant up
vagrant ssh-config > ssh_config
# act
ansible-playbook act.yml
# assume
ansible-playbook assume.yml
# cleaning
vagrant destroy -f
Arrange
11
integration_test/arrange.sh

12. Vagrant.configure("2") do |config|
ENV['LC_ALL']="en_US.UTF-8"
config.vm.define "web" do |web|
web.vm.box = "centos/7"
web.vm.hostname = "web.example.de"
web.vm.provision "ansible" do |ansible|
ansible.playbook = "ping.yml"
ansible.host_key_checking = false
ansible.compatibility_mode = "2.0"
end
web.vm.network "forwarded_port", guest: 80, host: 8042
end
end
Vagrant
12
integration_test/Vagrantfile

13. • when creating the VM: start an ansible provisioner
• => vagrant create an ansible inventory with port and ssh-key
• let ansible create an ssh-config: vagrant ssh-config > ssh_config
• => regular ssh login with ssh –F ssh_config $server
I just ping the VM and use normal ansible environment for act and assum, because it‘s
easier to develop and debug.
Arrange: vagrant and ssh
13
vagrant know how to ssh into VM, we need this too

14. • call your ansible playbook and/or your ansible roles
- import_playbook: "../website.yml"
Act
14
integration_test/act.yml

15. - hosts: web
vars:
ansible_become: false
tasks:
- name: "check index.html"
delegate_to: localhost
uri:
url: http://localhost:8042/
return_content: yes
register: webcontent
failed_when: "'Hello' not in webcontent.content"
- debug:
var: webcontent
Assume
15
integration_test/assume.yml

16. • create .gitlab-ci.yml
• create integration-test/ from skeleton
• configure:
• Vagrantfile # ports, ressources, more than one VM
• act.yml # call your code to test
• assume.yml # things to test
Developer HowTo
16
steps to be done in git-repo

17. by every commit
Pros:
• you know exactly wich commit broke the code
• the causer get a note immediatly
Cons:
• commit frequence could be shorter than test duration
• queuing needed
• ressourcen needed
• don‘t find environment changes
Manually:
Cons:
• will be forgotten
Automation: by commit or scheduled or manually?
17

18. • Vagrantfile can do this
Test a environment with more VMs
18

19. we need a loop in arrange.sh: arrange, act & assume
Test a role more than one with different parameters
19

20. Alternatives
20
Assume
• https://serverspec.org/
• https://www.inspec.io/
• InfraSpec
Ansible Testing
• Molecule
• iX 4.2019

21. • the act-playbooks are a good documentation how to use a role
• the assert-playbooks are a promise, good documentation what you can rely on as a
consumer of a role
performance description for free.
more benefits when testing
21

22. • integration testing is possible
• integration testing isn‘t too hard
• testing against a reproducible origin
Questions ?
Repo&Slides: https://gitlab.strukturpunkt.de/kfr/ansible_integration_test
klaus.Franken@ing.de
Conclusion
22