Presented at CukeUp! NYC 2014 in Brooklyn, NY. Session description: Acceptance tests have become a common part of building software for an agile process. However, I've observed a trend of growing build times - with teams saying "Our build is 15 minutes... well, actually it's 45 minutes because our Cucumbers took so long, so we parallelized the cukes." One reason for this may be that teams treat their acceptance tests as "Write-Once": create for a new story, get them to pass, and leave them be. In this talk we'll explore the technical and social forces that cause an acceptance test suite to grow, and discuss some ways to refactor and improve these tests. We'll touch on hexagonal architecture, declarative vs. imperative style, and alternate Cucumber step implementations.

1. Living with!
Acceptance Tests:
Beyond Write-Once
Daniel Wellman
Twitter: @wellman!
E-mail: dan@danielwellman.com
www.intentmedia.com

6. Activity

7. Goal!
Build the least helpful,
most frustrating
acceptance test suite

9. Revise Your Tests
With the Same Care as
Your Production Code

11. Check for Duplication

12. Refactor with
New Insights

13. Monitor for Problems

14. Problem:!
Slow and Flaky Tests

15. Server
End-to-End Tests
Test Database Queue

16. New Project
End-to-End!
Acceptance Test

17. New Project
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
End-to-End!
Acceptance Test
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test
End-to-End!
Acceptance Test

18. How did it get this
way?

19. Run Acceptance
Tests in Parallel

20. Faster Build
Same Risk

21. Failures are
Hard to Diagnose

22. Problem:
Flaky Tests

23. Don’t make every
acceptance test run
end-to-end

24. Ports and Adapters!
a.k.a. Hexagonal Architecture

25. Example: Fraud Protection
for an Amazon-like store

26. Example: Fraud Protection
for an Amazon-like store
Example from Gojko Adzic, “Specification by Example”

27. Example: Fraud Protection
for an Amazon-like store
Given a user with no previous transaction history,
And the user’s account registration country is the UK,
When the user places an order with delivery country U.S.,
Then the transaction is marked as suspicious,
But the user sees order status as “Pending.”
Example from Gojko Adzic, “Specification by Example”

28. Domain
Ports
Adapters

29. Domain
Ports
Adapters
GUI
DB

30. Domain
Ports
Adapters
GUI
DB
Memory

31. Domain
Ports
Adapters
GUI
DB
Memory
End-to-End Test

32. Domain
Ports
Adapters
GUI
DB
Memory
End-to-End Test
Test Using!
Domain

34. Before: End-to-End
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
!
!
!
!
!
!
!
!
!
!
end

35. Before: End-to-End
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
!
!
!
!
!
!
!
!
!
!
end
# ...
fill_in 'Address', with: ...
select country_name, from: 'Country Selector’
click_button 'Ship To This Address'
expect(page).to have_content
'Your order has been placed’
# ...

36. After: Using Domain Model
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
!
!
!
!
!
!
!
!
!
end

37. After: Using Domain Model
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
# ...
country = Country.new(country_name)
order = Order.new( ...,
!
!
!
!
!
!
!
!
!
end
ship_to : Address.new(
street: ...,
country: country))
sales_clerk.accept(order)
# ...

38. UI
Service
Unit
The Testing Pyramid

39. UI
Service
Unit
The Testing Pyramid

40. Create a Separate
System Test Suite

41. Alternate Step
Implementations

42. Alternate Steps:
Use Conditionals
use_browser = ENV['USE_BROWSER_FOR_TESTS']

43. Alternate Steps:
Use Conditionals
use_browser = ENV['USE_BROWSER_FOR_TESTS']
!
!
When(/^the user places an order with
delivery country (.*),$/) do |country_name|
if use_browser
# drive a browser using Capybara; slower
else
# use domain objects; faster
end
end

46. Write Tests
Without Specifying
Implementation Details

47. Imperative!
vs.!
Declarative!
Style

48. Imperative Style

49. Imperative Style
Given I am on the registration page,
And I enter "UK" as my billing country,
And I press the "Register" button,
And I add an item to my cart,
When I checkout
And I enter "U.S." as my shipping country,
And ...

50. Declarative Style

51. Declarative Style
Given a user with no previous transaction history,
And the user’s account registration country is the UK,
When the user places an order with delivery country U.S.,
Then the transaction is marked as suspicious,
But the user sees order status as “Pending.”
Example by Gojko Adzic, “Specification by Example”

52. Declarative Style
You can’t tell how these are implemented!
Given a user with no previous transaction history,
And the user’s account registration country is the UK,
When the user places an order with delivery country U.S.,
Then the transaction is marked as suspicious,
But the user sees order status as “Pending.”
Example by Gojko Adzic, “Specification by Example”

53. Use Declarative Style
to Help Keep
Implementation
Details Out
of Tests

54. Problem: I’m not sure
what is actually
tested…

55. How did it get this
way?

56. Periodically
Evaluate
Your
Test Suite

57. Have you introduced a
new domain concept?

58. Are you in a new stage
of your product’s life?

60. Suggestions
• Maintain your acceptance tests just like
production code
• Write tests which describe what, not how, so their
implementations can easily evolve later
• Decide when you can start moving acceptance
tests from end-to-end to using the domain directly
• Re-evaluate your product’s tests and testing
strategy as your system grows

61. Resources

62. Photo Credits
“HighLine_061409_2779” by Jessica Sheridan is licensed under CC BY 2.0
“Highline New York” by Rebecca Krebs is licensed under CC BY 2.0
“NYC: The Highline" by Wally Gobetz is licensed under CC BY-NC-ND 2.0
“Rot”, “Yellow Cart”, and “Many Hands Make Light Work” by Oliver Rich are licensed
under CC BY-NC-ND 2.0
“highline-1010384.jpg” by keroism is licensed under CC BY-SA 2.0
“highline” by Elijah Porter is licensed under CC BY-NC-SA 2.0
“Highline_030” by Timothy Vogel is licensed under CC BY-NC 2.0
“201106073451_DSC_0014_1.JPG” by Leonel Ponce is licensed under CC BY-NC 2.0
“193” by verndogs is licensed under CC BY-NC 2.0

63. Living with!
Acceptance Tests:
Beyond Write-Once
Daniel Wellman
Twitter: @wellman!
E-mail: dan@danielwellman.com
www.intentmedia.com