Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Anth Courtney, Solution Architect, anth.courtney@cmdsolutions.com.au
Adam Durbin, Director of Strategy and Architecture, a...
Overview
Why should we test our Ansible artifacts?
What can we test?
What should we test?
How should we test?
But first, a contrived service!
Testing Our Playbook
We can immediately start to test our playbook, without
the need to write large amounts of additional ...
Syntax Checking Playbooks and Roles
What –syntax-check does do
 Validates the syntax of a playbook
 Validates the syntax...
"Ask yourself,
what would a build
agent do?"
Rob Shand, General DevOps Legend
Typically, we want to:
 Create an environme...
Testing Idempotency
An operation is idempotent if the result of performing it once is exactly the same as the
result of pe...
Non-Idempotent Playbooks and Roles
Non-idempotent playbooks and roles can be indicative of:
 A resource being changed mul...
Use Case: Using Idempotency, Check-Mode
and Diffs for Security
Testing Jinja2 Templates
For copy and template tasks, Ansible provides the ability to perform validation
before copying in...
Testing Jinja2 Templates
For copy and template tasks, Ansible provides the ability to perform validation before
copying in...
Batteries Included!
Ansible includes a number of modules which are test-orientated:
 assert
 fail
 stat
 wait_for
 ur...
Built-in Test Module: assert
- name: Check ansible version is suitable for this playbook
assert:
that: ansible_version.ful...
Built-in Module: uri
- name: Verify that the nginx status page is reachable
uri:
url: "http://{{ inventory_hostname }}/sta...
Using 'assert' With Multiple Conditions
- name: Make authenticated request
uri:
url: "http://localhost/api/users/anth"
met...
Built-in Test Module: stat
- name: Determine if ssh private key exists
stat:
path: /home/ec2-user/.ssh/id_rsa
register: ss...
Built-in Test Module: fail
 Used to ‘bail out’ of the execution of a playbook when a certain condition is met.
 Uses whe...
Built-in Module: wait_for
 Can be used to wait for a port to become available
 Can be used to wait for a file state to c...
Built-in Parameter: failed_when
 Provides the ability to define ‘what constitutes failure?’ for a task.
- name: Run comma...
How Do People Test Ansible Playbooks?
 From experience, we’ve typically seen a ‘whole box and dice’ approach.
 Take a ‘t...
Test the outcome, not the
implementation.
Michael De Haan, Creator of Ansible
The Golden Rule of Testing in Ansible
 Ansible’s modules are designed to “achieve state” and “fail fast”.
 It is not nec...
A Suggested Playbook Testing Pattern
For a playbook, have the ability to apply the playbook (and run the embedded
function...
A Suggested Role Testing Pattern
The objectives of testing a role should be similar to that of a playbook:
 Is the YAML s...
I Like Unit Tests! I Do! I Like Them, Sam I Am
If you want to unit test your Ansible playbooks or roles, try to keep these...
Is Your Role Worthy Of A Gold Star?
The maintainers of Ansible Galaxy are discussing automated manners to give a gold
star...
Playbook and Role Testing Using Tox
Meet Tox: https://tox.readthedocs.io/en/latest/
$ pip install tox
“tox aims to automat...
Role Testing Using Molecule
Meet Molecule: http://molecule.readthedocs.io/
$ pip install molecule
“Molecule is designed to...
Molecule is CI Friendly!
Molecule provides a number of commands which are ‘continuous integration’ friendly.
$ molecule cr...
Miscellaneous Tips and Tricks
 Use ‘blocks’ to execute all tests while delaying assertions until the end.
 Use an ‘MVP’ ...
Summary
 Having one test for a playbook or role is better than zero tests.
 Ask “what would a build agent do?” and model...
Testing Ansible
Upcoming SlideShare
Loading in …5
×

Testing Ansible

A helicopter ride over the Ansible testing landscape.

  • Login to see the comments

Testing Ansible

  1. 1. Anth Courtney, Solution Architect, anth.courtney@cmdsolutions.com.au Adam Durbin, Director of Strategy and Architecture, adam.durbin@cmdsolutions.com.au A helicopter ride over the Ansible testing landscape
  2. 2. Overview Why should we test our Ansible artifacts? What can we test? What should we test? How should we test?
  3. 3. But first, a contrived service!
  4. 4. Testing Our Playbook We can immediately start to test our playbook, without the need to write large amounts of additional code. Test the syntax of our playbook and roles Evaluate the repeatability of our playbooks and roles. Validate configuration files generated by jinja2 templates.
  5. 5. Syntax Checking Playbooks and Roles What –syntax-check does do  Validates the syntax of a playbook  Validates the syntax of any included roles.  Validates presence of any other includes. i.e. other playbooks, vars_files.  Validates that tasks/modules specified within a playbook or role are valid. What –syntax-check doesn't do  Doesn’t execute a playbook.  Doesn’t test the validity of parameters to tasks.  Doesn’t test for the existence of undefined variables.  Doesn’t test the ability to sudo/become as defined.  Doesn’t test the validity of any in- line jinja2. $ ansible-playbook –syntax-check <playbook>
  6. 6. "Ask yourself, what would a build agent do?" Rob Shand, General DevOps Legend Typically, we want to:  Create an environment or workspace for execution of our playbook.  Prepare the environment or workspace for execution of the playbook.  Apply the playbook against the target environment.  Verify the playbook execution.  Destroy the environment or workspace.
  7. 7. Testing Idempotency An operation is idempotent if the result of performing it once is exactly the same as the result of performing it repeatedly without any intervening actions. First run of playbook: PLAY RECAP ********************************************************************* default : ok=7 changed=10 unreachable=0 failed=0 Nth run of playbook: PLAY RECAP ********************************************************************* default : ok=17 changed=0 unreachable=0 failed=0
  8. 8. Non-Idempotent Playbooks and Roles Non-idempotent playbooks and roles can be indicative of:  A resource being changed multiple times within the same playbook run.  A conditional that isn’t being evaluated correctly.  Tasks not being properly configured for idempotence. - name: Install custom application - non-idempotent approach command: /tmp/application-1.0.0/bin/install --dest /opt/application - name: Install custom application - idempotent approach command: /tmp/application-1.0.0/bin/install --dest /opt/application args: creates: /opt/application
  9. 9. Use Case: Using Idempotency, Check-Mode and Diffs for Security
  10. 10. Testing Jinja2 Templates For copy and template tasks, Ansible provides the ability to perform validation before copying into place. - name: Create nginx configuration template: src: "etc/nginx/nginx.conf.j2" dest: "/etc/nginx/nginx.conf" validate: "nginx -t -c %s" notify: Reload nginx
  11. 11. Testing Jinja2 Templates For copy and template tasks, Ansible provides the ability to perform validation before copying into place. - name: Copy nginx modular configuration template: src: "etc/nginx/conf.d/modules.conf.j2" dest: "/etc/nginx/conf.d/modules.conf" validate: "echo 'http { include '%s'; }' > /tmp/nginx.conf && nginx -t -c /tmp/nginx.conf" notify: Reload nginx
  12. 12. Batteries Included! Ansible includes a number of modules which are test-orientated:  assert  fail  stat  wait_for  uri  command
  13. 13. Built-in Test Module: assert - name: Check ansible version is suitable for this playbook assert: that: ansible_version.full | version_compare('2.1', '>=') msg: "Ansible version 2.1 or greater is required for this playbook"
  14. 14. Built-in Module: uri - name: Verify that the nginx status page is reachable uri: url: "http://{{ inventory_hostname }}/status" register: result until: result.status == 200 retries: 60 delay: 1
  15. 15. Using 'assert' With Multiple Conditions - name: Make authenticated request uri: url: "http://localhost/api/users/anth" method: GET force_basic_auth: yes user: "admin" password: "admin123" register: api_request - name: Validate that api request response contains required information assert: that: - api_request.json.name is defined - api_request.json.username == 'anth' - api_request.json.email | search("@")
  16. 16. Built-in Test Module: stat - name: Determine if ssh private key exists stat: path: /home/ec2-user/.ssh/id_rsa register: ssh_private_key - name: Validate that ssh private key exists and has correct mode assert: that: - ssh_private_key.stat.exists - ssh_private_key.stat.mode == "0640" msg: "ssh private key does not exist or has wrong mode"
  17. 17. Built-in Test Module: fail  Used to ‘bail out’ of the execution of a playbook when a certain condition is met.  Uses when to provide the condition for termination. - name: Check ansible version is suitable for this playbook fail: msg: "Ansible version 2.1 or greater is required for this playbook" when: ansible_version.full | version_compare('2.1', '<')
  18. 18. Built-in Module: wait_for  Can be used to wait for a port to become available  Can be used to wait for a file state to change or for a string/regex to appear  Can also be used to wait for active connections to be closed before continuing - name: Verify that cron ran configured @reboot jobs wait_for: path: "/var/log/cron" search_regex: "Running @reboot jobs"
  19. 19. Built-in Parameter: failed_when  Provides the ability to define ‘what constitutes failure?’ for a task. - name: Run command to test status of application command: /opt/cmd/bin/test_application.sh register: command_result failed_when: "'FAILED'" in command_result.stdout
  20. 20. How Do People Test Ansible Playbooks?  From experience, we’ve typically seen a ‘whole box and dice’ approach.  Take a ‘test everything’ approach, including unit and functional testing.  Use tools like ServerSpec, Goss, Ansible, etc.  Increased cost due to duplication of effort and maintenance of tests.
  21. 21. Test the outcome, not the implementation. Michael De Haan, Creator of Ansible
  22. 22. The Golden Rule of Testing in Ansible  Ansible’s modules are designed to “achieve state” and “fail fast”.  It is not necessary to test that services are started, packages are installed, etc.  Try to keep it simple. “Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius—and a lot of courage to move in the opposite direction.” - E. F. Schumacher in “Small Is Beautiful”.
  23. 23. A Suggested Playbook Testing Pattern For a playbook, have the ability to apply the playbook (and run the embedded functional tests) against a virtual environment. $ make USE_VM=true clean setup <--- Applies playbook to virtual environment $ make USE_VM=true test <--- Runs functional test against virtual environment For a playbook, have embedded functional tests which can be toggled on and off by feature flags or tags. $ make clean setup <--- Applies playbook to environment $ make test <--- Runs functional tests against environment
  24. 24. A Suggested Role Testing Pattern The objectives of testing a role should be similar to that of a playbook:  Is the YAML syntax of my role correct?  Is the role built in an idempotent way?  Does the role run through all tests without failing? For a role, have the ability to apply the role and run functional tests against a virtual environment. $ make clean setup test <--- Applies role to a VM and runs functional tests
  25. 25. I Like Unit Tests! I Do! I Like Them, Sam I Am If you want to unit test your Ansible playbooks or roles, try to keep these principles in mind:  The unit tests should only test a single associated task.  The unit tests should be tightly associated with the related task that they are testing.  The unit tests are read-only in nature.  The use of tags allows selected execution of the unit tests.
  26. 26. Is Your Role Worthy Of A Gold Star? The maintainers of Ansible Galaxy are discussing automated manners to give a gold star or other indicator to a role that meets certain criteria which are an indicator of quality.  Documentation should exist  Documentation should include a definition of all default variables  travis.yml should include asserts and be more than just a syntax check  Check that the role does not call deprecated modules  Check that the role does not use deprecated syntax  Check that all variables have a default or are included in vars/main.yml  Role repo should not include issues that have been open for a long time
  27. 27. Playbook and Role Testing Using Tox Meet Tox: https://tox.readthedocs.io/en/latest/ $ pip install tox “tox aims to automate and standardize testing in Python.”  Allows for testing playbooks/roles against different versions of ansible.  Allows for testing playbooks/roles against different versions of python.  Works on Windows and Linux.
  28. 28. Role Testing Using Molecule Meet Molecule: http://molecule.readthedocs.io/ $ pip install molecule “Molecule is designed to aid in the development and testing of Ansible roles..."  Supports multiple drivers, including Docker, Vagrant and OpenStack.  Supports multiple providers, including Virtualbox, Libvrt, Parallels, and VMware Fusion.  Supports multiple verifiers, including Testinfra, Serverspec, Goss  Also includes syntax validation and ansible-lint checking.
  29. 29. Molecule is CI Friendly! Molecule provides a number of commands which are ‘continuous integration’ friendly. $ molecule create <--- Creates instance(s) $ molecule converge <--- Apply ansible playbook to instance(s) $ molecule idempotence <--- Validates idempotence of role $ molecule verify <--- Runs tests against converged instance(s) $ molecule destroy <--- Destroys instance(s) By default, all of these commands are executed when running: $ molecule test.
  30. 30. Miscellaneous Tips and Tricks  Use ‘blocks’ to execute all tests while delaying assertions until the end.  Use an ‘MVP’ playbook pattern to test ‘is this how you do it?’ type things.  Use the ping module to check the readiness of hosts.
  31. 31. Summary  Having one test for a playbook or role is better than zero tests.  Ask “what would a build agent do?” and model your approach accordingly.  Test the outcome, not the implementation.

×