Downloaded 63 times
![TESTING PROGRAMMABLE INFRASTRUCTURE
UNDER THE CUCUMBER, PLAIN RUBY
Then(/^I should be able to echo "([^"]*)"$/) do |text|
cmd = "echo #{text}"
output = @session.exec!(cmd)
close_ssh(@session)
expect(output.to_s.strip).to eql(text)
end](https://image.slidesharecdn.com/stablefoundations3-170217101103/85/London-Hashicorp-Meetup-8-Testing-Programmable-Infrastructure-By-Matt-Long-60-320.jpg)
![TESTING PROGRAMMABLE INFRASTRUCTURE
TestCreate Config Destroy
]TEST KITCHEN MANAGES YOUR TEST LIFECYCLE](https://image.slidesharecdn.com/stablefoundations3-170217101103/85/London-Hashicorp-Meetup-8-Testing-Programmable-Infrastructure-By-Matt-Long-84-320.jpg)
Uploaded byOpenCredo
London Hashicorp Meetup #8 - Testing Programmable Infrastructure By Matt Long
This document discusses testing programmable infrastructure. It describes testing different aspects of infrastructure including a cloud broker and Kubernetes cluster. It recommends applying software development testing practices like unit testing, integration testing, and acceptance testing to infrastructure code and configuration. Various tools are presented for different types of infrastructure testing including Serverspec, Bats, Goss, Terraform_validate, and using Test Kitchen with Inspec. The document emphasizes that testing is important for infrastructure but often overlooked, and that testers and operations teams should work together to address new challenges in testing programmable systems.
More Related Content
Similar to London Hashicorp Meetup #8 - Testing Programmable Infrastructure By Matt Long
London Hashicorp Meetup #8 - Testing Programmable Infrastructure By Matt Long
- 1.
- 2. PROGRAMMABLE INFRASTRUCTURE IS GREAT,BUT WE'RE MISSING SOMETHING. TESTING.
- 3. I'M A TESTER HELLO,I'M MATT
- 4.
- 5. I AM NOTA SYSADMIN
- 6.
- 7. TESTING PROGRAMMABLE INFRASTRUCTURE THEAPPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS.. THE INTERNET
- 8. TESTING PROGRAMMABLE INFRASTRUCTURE EXAMPLESOF PROGRAMMABLE INFRASTRUCTURE ▸ Automated provisioning & configuration ▸ Configuration as code ▸ Version / source controlled
- 9.
- 10. PROGRAMMABLE INFRASTRUCTURE IS AWESOME! Credit: VaultBoy, Bethesda Softworks
- 11.
- 12.
- 13.
- 14.
- 15. TESTING IS USEDTO MITIGATE COMPLEXITY & RISK
- 16. BUT TESTING ISRARE Credit: Gunshow, KC Green
- 17. TESTING PROGRAMMABLE INFRASTRUCTURE WHATI'M GOING TO TALK ABOUT ▸PART 1: Testing a cloud broker ▸PART 2: Building a Kubernetes cluster ▸CONCLUSIONS
- 18. TESTING A CLOUD BROKER ANINFRASTRUCTURE HEAVY PRODUCT
- 19.
- 20. TESTING PROGRAMMABLE INFRASTRUCTURE WEWANT TO MOVE TO THE CLOUD... BUT WE'RE WARY OF LOCK IN Large organisation
- 21. TESTING PROGRAMMABLE INFRASTRUCTURE USERS USEMULTIPLE CLOUD PROVIDERS
- 22. TESTING PROGRAMMABLE INFRASTRUCTURE PROBLEMS ▸Different interfaces, feature sets & lingo ▸ Can't switch easily ▸ Spending difficult to track ▸ Temptation to fall back on most popular
- 23.
- 24. TESTING PROGRAMMABLE INFRASTRUCTURE BENEFITS ▸Quick, easy provisioning ▸ one team previously took 3 months ▸ Common interface to cloud features ▸ Templates for common dev environments ▸ Built in best practice: monitoring, security ▸ Track spending
- 25. THIS IS AREALLY COMPLICATED APPLICATION
- 26.
- 27. TESTING PROGRAMMABLE INFRASTRUCTURE WORKFLOW ▸Log into Web UI ▸ Fill in information about environment ▸ Broker creates and bootstraps resources ▸ SSH into resources
- 28. TESTING PROGRAMMABLE INFRASTRUCTURE WEBTESTING ▸ Log into Web UI ▸ Fill in information about environment
- 29. TESTING PROGRAMMABLE INFRASTRUCTURE ??? ▸Broker creates and bootstraps resources ▸ SSH into resources
- 30. HOW DO YOUTEST INFRASTRUCTURE?
- 31. TESTING PROGRAMMABLE INFRASTRUCTURE WHATTO TEST? Do our deployment scripts work? Does the VPN server work? Can instances access one another? Are services running? Can I SSH into a server?
- 32.
- 33. TESTING PROGRAMMABLE INFRASTRUCTURE Doesthe VPN box work? Can I SSH into a server? Do our deployment scripts work? Are services running? ANOTHER TESTING PYRAMID? credit: Ubuntu dev quality guide https://developer.ubuntu.com/en/phone/platform/quality/ Can instances access one another?
- 34.
- 35. TESTING PROGRAMMABLE INFRASTRUCTURE TOOLSAVAILABLE ▸ Bats ▸ ShUnit2 ▸ Goss ▸ ServerSpec / Inspec / TestInfra ▸ Test Kitchen
- 36.
- 37. TESTING PROGRAMMABLE INFRASTRUCTURE BATS ▸"Bash Automated Testing System" ▸ Unit testing for bash ▸ Like JUnit
- 38. TESTING PROGRAMMABLE INFRASTRUCTURE SHUNIT 2 ▸ Shell unit testing framework ▸ Runs on all Bourne shells ▸ sh, BASH, DASH, ksh, zsh ▸ No activity or support?
- 39. INTEGRATION TESTING OR: SERVERVALIDATION
- 40. TESTING PROGRAMMABLE INFRASTRUCTURE GOSS ▸Go based ▸ Specs in YAML ▸ Minimal, fast, and simple ▸ Some neat features ▸ .. have to run on the server ▸ .. no Windows support
- 41. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPEC ▸Server based assertions ▸ Ruby/RSpec based ▸ Probably the most famous ▸ Can SSH into instances
- 42. TESTING PROGRAMMABLE INFRASTRUCTURE INSPEC ▸Written & maintained by Chef ▸ Very similar to ServerSpec ▸ Different feature set ▸ More focused on compliance
- 43. TESTING PROGRAMMABLE INFRASTRUCTURE TESTINFRA ▸ServerSpec, but in Python
- 44.
- 45. TESTING PROGRAMMABLE INFRASTRUCTURE TESTKITCHEN ▸ Orchestrates setup, test, teardown ▸ Runs BATS, shUnit2, RSpec, Serverspec ▸ Popular in the Chef community ▸ Not suitable for our cloud broker
- 46.
- 47.
- 48. TESTING PROGRAMMABLE INFRASTRUCTURE USERS WEBTEST FRAMEWORK
- 49. TESTING PROGRAMMABLE INFRASTRUCTURE USERS INFRASTRUCTURETEST FRAMEWORK
- 50. TESTING PROGRAMMABLE INFRASTRUCTURE USERS WEBTESTS https://github.com/opencredo/test-automation-quickstart
- 51. TESTING PROGRAMMABLE INFRASTRUCTURE INFRASTRUCTURETESTS Serverspec
- 52. TESTING PROGRAMMABLE INFRASTRUCTURE INFRASTRUCTURETESTING STACK / Serverspec ???
- 53. TESTING PROGRAMMABLE INFRASTRUCTURE WHYRUBY? ▸ Fantastic testing community ▸ More suitable for SSHing into boxes ▸ "Win RM" gem ▸ Ops already familiar with it ▸ Reduces tech stack
- 54. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPECSMOKE TESTS ▸ Run before everything else ▸ Really quick ▸ Catches obvious errors ▸ Not complex tasks
- 55. TESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPECEXAMPLE describe package('jenkins') do it { should be_installed } end describe service('jenkins') do it { should be_enabled } it { should be_running } end describe port(8080) do it { should be_listening } end
- 56. TESTING PROGRAMMABLE INFRASTRUCTURE Background: Givenenvironment has been created And the following user details: | user_alias | username | public_key | | userA | envoy | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world" CUCUMBER FOR ACCEPTANCE TESTING
- 57. TESTING PROGRAMMABLE INFRASTRUCTURE Background: Givenenvironment has been created And the following user details: | user_alias | username | public_key | | userA | envoy | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world" CUCUMBER FOR ACCEPTANCE TESTING Cloud broker APIs
- 58. TESTING PROGRAMMABLE INFRASTRUCTURE Background: Givenenvironment has been created And the following user details: | user_alias | username | public_key | | userA | envoy | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world" CUCUMBER FOR ACCEPTANCE TESTING Standard Ruby
- 59. TESTING PROGRAMMABLE INFRASTRUCTURE Background: Givenenvironment has been created And the following user details: | user_alias | username | public_key | | userA | envoy | test | Scenario: IPA - Login via SSH Key authentication succeeds Given user "userA" is authorised to access environment vms When user "userA" starts ssh session in host "env" Then I should be able to echo "hello world" CUCUMBER FOR ACCEPTANCE TESTING RSpec assertions
- 60. TESTING PROGRAMMABLE INFRASTRUCTURE UNDERTHE CUCUMBER, PLAIN RUBY Then(/^I should be able to echo "([^"]*)"$/) do |text| cmd = "echo #{text}" output = @session.exec!(cmd) close_ssh(@session) expect(output.to_s.strip).to eql(text) end
- 61.
- 62. TESTING PROGRAMMABLE INFRASTRUCTURE THEGOOD ▸ Specialised tests for each layer ▸ Really quick, expressive ServerSpec tests ▸ Power of a full programming language for user tests
- 63. TESTING PROGRAMMABLE INFRASTRUCTURE THEBAD ▸ Over reliance on acceptance tests ▸ Awkward switching between two suites ▸ Out of my comfort zone
- 64. TESTING PROGRAMMABLE INFRASTRUCTURE THEUGLY ▸ Starting infrastructure is SLOW. ▸ It's expensive...
- 65. IT WAS WORTHIT DESPITE ALL THAT
- 66. BUILDING A KUBERNETES CLUSTER APPLYINGTDD TO INFRASTRUCTURE
- 67.
- 68. I LEARNED ALOT! Credit: The Simpsons, Fox
- 69. TESTING PROGRAMMABLE INFRASTRUCTURE BUILDTHIS: WITH THESE:
- 70. TESTING PROGRAMMABLE INFRASTRUCTURE NOTA STRAIGHTFORWARD TASK
- 71. TESTING PROGRAMMABLE INFRASTRUCTURE BUTHOW TO TEST IT? ▸ This is a dev activity ▸ Want fast feedback ▸ Complexity is mitigated by testing!
- 72. TESTING PROGRAMMABLE INFRASTRUCTURE ▸Provisions cloud infrastructure ▸ Declarative files ▸ Some support for variables TERRAFORM
- 73. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORMCOMMANDS ▸ terraform plan ▸ Tells you what will change ▸ terraform apply ▸ Applies changes ▸ terraform validate ▸ Lints terraform syntax
- 74. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORMFILE EXAMPLE resource "aws_instance" "etcd-node" { count = 3 ami = "ami-7abd0209" # centos availability_zone = "eu-west-1a" # ireland instance_type = "t2.micro" subnet_id = .... private_ip = .... key_name = "${aws_key_pair.my-key.key_name}" }
- 75. TESTING PROGRAMMABLE INFRASTRUCTURE LINTWITH 'TERRAFORM VALIDATE' COMMAND Omitting a variable:
- 76. TESTING PROGRAMMABLE INFRASTRUCTURE BUTIT DOESN'T CATCH ALL PROBLEMS Duplicate subnet CIDRS:
- 77. TESTING PROGRAMMABLE INFRASTRUCTURE LINTINGISN'T ENOUGH ▸ Devs don’t just rely on compilers ▸ We need something more powerful Credit: Nick Cave, "Soundsuit"
- 78.
- 79. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM_VALIDATE ▸Python based unit testing ▸ NOT to be confused with 'validate' command ▸ Builds map of resources & properties ▸ Totally offline ▸ New and incomplete https://github.com/elmundio87/terraform_validate
- 80. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM_VALIDATEFORK OC has forked the terraform validate repo https://github.com/opencredo/terraform_validate
- 81.
- 82. TESTING PROGRAMMABLE INFRASTRUCTURE GOSS ▸Easy to get up and running ▸ Doesn’t support remote # example usage: ./goss-test.sh 34.248.91.167 TARGET='centos@'$1 SSH_KEY_PATH=~/.ssh/aws ssh -t -t -i $SSH_KEY_PATH $TARGET 'curl -fsSL https://goss.rocks/install | sudo sh' scp ./goss.json $TARGET:~/goss.yaml ssh -t -t -i $SSH_KEY_PATH $TARGET 'goss validate' https://gist.github.com/burythehammer/081d6ee11cc33c2f4c4729ae67622f5b
- 83. TESTING PROGRAMMABLE INFRASTRUCTURE ▸Terraform compatibility ▸ Already a talk about this ▸ “Untangling Infrastructure Code” by Nell Shamrell-Harrington TEST KITCHEN + INSPEC
- 84. TESTING PROGRAMMABLE INFRASTRUCTURE TestCreateConfig Destroy ]TEST KITCHEN MANAGES YOUR TEST LIFECYCLE
- 85. TESTING PROGRAMMABLE INFRASTRUCTURE TestCreateConfig Destroy TEST KITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS
- 86. TESTING PROGRAMMABLE INFRASTRUCTURE TESTKITCHEN DOESN'T SUPPORT MULTIPLE PROVISIONERS https://github.com/test-kitchen/test-kitchen/issues/329
- 87. TESTING PROGRAMMABLE INFRASTRUCTURE TERRAFORM'NULL RESOURCE' resource "null_resource" "ansible" { triggers { instance_ids = "${join(",", aws_instance.etcd-node.*.id)}" } provisioner "local-exec" { command = "sleep 20 && cd ../ansible/ && ansible-playbook etcd.yaml" } }
- 88.
- 89.
- 90. TESTING PROGRAMMABLE INFRASTRUCTURE THEGOOD ▸ Tooling exists! ▸ You can totally get a test suite working Credit: Overwatch, Blizzard Entertainment
- 91. TESTING PROGRAMMABLE INFRASTRUCTURE THEBAD ▸ Unit testing extremely immature ▸ Tools immature in general Credit: Overwatch, Blizzard Entertainment
- 92. TESTING PROGRAMMABLE INFRASTRUCTURE THEHACKY ▸ Be prepared to hack ▸ It might not even be possible Credit: Overwatch, Blizzard Entertainment
- 93. THIS IS BRAND NEWGROUND REMEMBER:
- 94. TESTING TOOLS DEPEND ONYOUR STACK
- 95.
- 96.
- 97. TESTERS AND OPS SHOULDWORK TOGETHER WE NEED TO GET OUT OF OUR COMFORT ZONES
- 98. TOOLS EXIST BUT BEPREPARED TO HACK
- 99.
- 100. TESTING PROGRAMMABLE INFRASTRUCTURE THEAPPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS..
- 101. TESTING IS ASOFTWARE DEVELOPMENT METHOD WE SHOULD APPLY IT TO INFRASTRUCTURE
- 102.