What does testing provide? What should you test? When do tests go bad?

1. Testing
expect(subject).to
be_awesome

2. Why Test?
To prevent regression
Refactoring safety net
Posterity of intention
Doubles as documentation
To expose sub-optimal code design
“Proving your work”
Fixes are more expensive the later it is detected*

3. Cost of Defects
Stage Cost
Requirements Time to rewrite requirement
Coding Addtl. Hours for engineer
QA Addtl. Hours for engineer + PM + QA
User Acceptance Testing Addtl. Hours eng + PM + QA + customer
Production Addtl. Hours eng + support + PM +
customer + QA. Reputational loss. Fixing
production data.
10x more expensive if bug found in Production than in Requirements

4. The Delivery Pipeline Dream
Branch
merged
CI runs
tests
Tests pass
Deploys to
production
JIRA
updated
“Trust your test suite with your life”
In 2012 Github deployed 12,602 times. Busiest day saw 175 deploys.

5. What Makes a Good Test?
Testing the interface
Good enough coverage, but not burdensome
Considers edge cases
Clear, concise, comprehensive, correct
Verify that you’ve met the stated requirements

6. When Tests Go Bad
Brittleness (mocking too much, testing implementation, etc)
Unreliable (Full stack tests reliant on timing guesswork)
Sluggishness
Incomprehensible
Size of test suite*

7. The Testing Pyramid

8. What Should I Test?

9. What Should I Test?
Good rule of thumb: If you are manually setting up data (Rails console, other
REPL, etc) and testing a change you are making, this should be your test
Same requirements as what the US considers patentable: “clever or non-obvious”
Aim for 80% test coverage - Complete coverage is asymptotic
Regressions (“Fool me once...”)
Code to be refactored

10. Test Types
Unit (component)
Integration
Full stack (e.g. “browser”)
Functional tests (e.g. “controller” aka “black box”)
Manual Testing (QA)
Performance/Stress/Load
A/B
End to End

11. Unit Testing Demo
Lead.mobile - Context, it, Factory pattern, expectation matcher, running tests
Lead.determine_business_unit_id - code branches, build vs create, running context
Lead.import_addresses_from_identifier_field - decomposing, stubbing

12. Controller / Functional Testing Demo
LeadsController.new - before/after hooks, missing records, traits, ivar, type matchers,
multiple assertions

13. Full Stack (Browser) Test Demo
AuthenticationSpec - feature, Capybara definitions

14. Summary
Testing is not an action taken “after the fact” that is meant to check a box
Proper testing helps guide code structure
The extreme of this is TDD (Test Driven Development) where tests are written
(and fail) before the code is written that allows the tests to pass
Lower defect count
Generally results in smaller, focused methods
Being a better tester will make you a better developer

15. Thank You
Ben Simpson
thehoagie@gmail.com
@mrfrosti