The Evolution of
Development Testing
Overview
•

Automating

•

Testing Categories + Scope

•

Development Testing

•

Principles, Practices + Tool Support
Overview
•

Automating
Automating
•

We automate “stuff” for a living

•

Developer/Programmer/Engineer/Automator

•

When we’re doing things man...
Automating
•

The stuff we automate might be
•

validation of a web form

•

reporting of a message parsing error

•

calc...
Automating
•

That automation might look like
•

choose validation rules, implement, enforce, report

•

log parsing error...
Automating
•

Manual solutions might look like
•

review user errors later in workflow e.g.“back office” manual
activity

•
...
Automating
•

We automate “stuff” for a living

•

Developer/Programmer/Engineer/Automator

•

When we’re doing things man...
Overview
•

Automating

•

Testing Categories + Scope
Test Categorisation
•

Useful concept - up to a point

•

Blurred lines + Overloaded terminology

•

Difficulties conversin...
Test Categorisation
•

Unit

•

Build Acceptance

•

Component

•

System

•

API

•

End-to-End

•

Acceptance

•

User A...
Test Categorisation
Test Categorisation
Test Categorisation
•

Useful concept - up to a point

•

Blurred lines + Overloaded terminology

•

Difficulties conversin...
Test Categorisation
Test Categorisation
Overview
•

Automating

•

Testing Categories + Scope

•

Development Testing
Development Testing
•

What it is

•

What it should not be

•

How it has evolved
What it is
•

A Development practice first and foremost

•

White box / Black box / Grey box

•

Has Quality aims

•

Has E...
JUnit
•

Test automation framework of choice for most Java
developers

•

Synonymous with Unit testing

•

Widely used for...
JUnit
“Never in the field of software development have so
many owed so much to so few lines of code”
Martin Fowler
(Develo...
What it is
•

A Development practice first and foremost

•

White box / Black box / Grey box

•

Has Quality aims

•

Has E...
What it is
•

Can take a technical perspective on scenarios

•

Can take a user functionality perspective

•

Exercises pr...
What it is
•

Often used to discover and refine solution design

•

Often leaves fine-grained regression coverage safety net...
What it is not
•

Automated full System tests a.k.a E2E, System Integration

•

Zero control over the state of the system ...
Its Evolution
•

OOP - slow evolution

•

Automated Developer Testing around a lot less time

•

A lot less mainstream tha...
Overview
•

Automating

•

Testing Categories + Scope

•

Development Testing

•

Principles, Practices + Tool Support
Principles, Practices + Tool Support
•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec

•

...
Test structure
•

Arrange, Act, Assert

•

Specify Inputs

•

Given, When, Then

•

Specify the state of the System

•

Sp...
Test structure
•

Naming conventions + structure
@Test	
public
//
	
//
	
//
}	

void methodUnderTest_GivenABC_ThenExpectXY...
Test structure
•

IDE assistance for consistency + evolve conventions
Principles, Practices + Tool Support
•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec
“Any fool can write code
that a computer can understand”
!

Martin Fowler
Test Qualities
•

Readability
•

What is it? Why does it matter?
Test Qualities
•

Readability == …?

•

Code that you can understand with ease

•

Language affects Thought

•

Thought af...
Test Qualities
•

Tools can directly address Readability qualities

•

Domain Specific Languages = a fancy term
•

•

JUnit...
“Test automation is a first class software
engineering problem”
!

Brian Marick?
Test Qualities
•

Maintainability
•

What is it? Why does it matter?
Test Qualities
•

Maintainability == …?

•

To change, evolve with ease

•

Keeping the cost of change down
Test Qualities
•

They can be context sensitive. Many are not.

•

You may opt to sacrifice:
•
•

Readability for Maintaina...
Principles, Practices + Tool Support
•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec

•

...
Specifying Expectations
•

Stating Expectations / Asking Questions

•

From Computer dialect ——> Human dialect
// then	
as...
Specifying Expectations
•

Stating Expectations / Asking Questions

•

From Computer dialect ——> Human dialect
// Using Ba...
Specifying Expectations
•

Easier + Faster to write expressive statements

•

Faster to read + review

•

Easier to mainta...
Specifying Inputs
•

Trivial example. Test with very narrow scope
•

e.g.Input = Single Object
@Test	
public void getPermi...
Specifying Inputs
•

Non-trivial example. Test with broader scope
•

e.g. Input = Object Graph
•

i.e. objects linked to o...
@Test public void 	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() {	
// give...
Specifying Inputs
•

Consider many non-trivial tests, with “raw” setup
•

Impact of changing one setter or constructor

•
...
Specifying Inputs - Techniques
•

Helper methods

•

Object Mother Pattern

•

Test Data Builder Pattern
@Test public void 	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() {	
// give...
@Test public void // Using helpers	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders...
@Test public void // Using helpers + hiding irrelevant details	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIs...
@Test public void 	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() {	
// give...
@Test public void // Using helpers	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders...
@Test public void // Using helpers + hiding irrelevant details	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIs...
Specifying Inputs - Techniques
•

Object Mother Pattern
public class ObjectMother {	

!
public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, ...
Specifying Inputs - Techniques
•

Test Data Builder Pattern
@Test public void // Using Test Data Builders	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMat...
public class TraderBuilder {	
public static String DEFAULT_USERNAME = "trader1";	
public static UserDetail DEFAULT_USERDET...
@Test public void // Using Test Data Builders	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMat...
@Test public void // Using helpers + hiding irrelevant details	
getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIs...
@Test public void // Using Test Data Builders to create input data variations	
getOrders_GivenTraderWithVariousPermissions...
Specifying Inputs - Techniques
•

Test Data Builder Pattern - with tool support
•

Make-It-Easy - addresses Builder class ...
Specifying Inputs - Techniques
•

Test Data Builder Pattern - Advantages
•

Your test / spec documents only the inputs tha...
Specifying Inputs - Techniques
•

Test Data Builder Pattern - Disadvantages
•

Requires much new test-supporting code

•

...
Specifying the System State
•

State of the SUT / the Test Context

•

Communicating what the context, limitations and
bou...
Specifying the System State
•

Problems + solutions for Specifying Inputs normally apply

•

Unique to System State is you...
Specifying the System State - Techniques
•

Hand-rolling Test Doubles

•

Using Test Isolation frameworks to provide Test ...
Specifying the System State - Techniques
•

Stub being used in a state-based test
@Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details	
getOrders_GivenTraderWit...
@Test public void // Stubbing system state + stubbing inputs (Mockito)	
getOrders_GivenTraderWithAccountPerms_SearchByAcco...
@Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs)	
getOrders_GivenTraderWithAccountPerms...
Specifying the System State - Techniques
•

A Mock being used in a interaction-based test
@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito)	
getOrders_GivenTraderWithAccount...
@Test public void // Using a Mock to test interactions + Stubbing system state (Mockito)	
getOrders_GivenTraderWithAccount...
Specifying the System State - Techniques
•

Often a test will have
•

Zero to many Stubs. Zero or one Mock

•

Mocks help ...
Specifying the Event
•

Often trivial

•

Might be a sequence of events that we want to fire

•

Sequence might actually be...
Specifying it All Together
•

Spring MVC Test support / DSL

@Test 	
public void helloWorld() throws Exception {	
mockMvc....
Principles, Practices + Tool Support
•

Structural Principles / Characteristics

•

Qualities of a “good” test / spec

•

...
Continuous Testing
•

Continuous Integration - development practice

•

CI tools - run automated tests frequently

•

Cont...
Overview
•

Automating

•

Testing Categories + Scope

•

Development Testing

•

Principles, Practices + Tool Support
Summary
•

Tools/APIs just help with heavy lifting

•

Sound coding principles are paramount

•

Creating unreadable and u...
Summary
•

The power of marginal improvements

•

for … each, <> operator, closure support, Guava

•

Continuous iterative...
Resources
•

Books
•

GOOS - Growing Object Oriented Software, Guided By
Tests

•

Effective Unit Testing

•

Working Effe...
Resources
•

Podcasts
•
•

Hanselminutes - Roy Asherove - The Art of Unit Testing

•
•

Sustainable Test Driven Developmen...
The Evolution of Development Testing
Upcoming SlideShare
Loading in...5
×

The Evolution of Development Testing

233
-1

Published on

A look at concepts, practices and tools related to Java test code readability and maintainability

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
233
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Evolution of Development Testing

  1. 1. The Evolution of Development Testing
  2. 2. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  3. 3. Overview • Automating
  4. 4. Automating • We automate “stuff” for a living • Developer/Programmer/Engineer/Automator • When we’re doing things manually we’re usually missing an opportunity to automate
  5. 5. Automating • The stuff we automate might be • validation of a web form • reporting of a message parsing error • calculation of an average execution price • execution + status reporting of tests for all of the above
  6. 6. Automating • That automation might look like • choose validation rules, implement, enforce, report • log parsing error, automate monitoring/reporting of error • define calculation rules, implement, provide result • define inputs, context + expectations, execute scenario, determine + report status
  7. 7. Automating • Manual solutions might look like • review user errors later in workflow e.g.“back office” manual activity • message loss caused an incident, manually track it down • export raw data to csv, load into Excel, apply formula • run UI functional scenarios that use/execute the code in mind • fire up debugger, step through mis-behaving code • add detailed logging, monitor logs for hints
  8. 8. Automating • We automate “stuff” for a living • Developer/Programmer/Engineer/Automator • When we’re doing things manually we’re usually missing an opportunity to automate
  9. 9. Overview • Automating • Testing Categories + Scope
  10. 10. Test Categorisation • Useful concept - up to a point • Blurred lines + Overloaded terminology • Difficulties conversing about testing
  11. 11. Test Categorisation • Unit • Build Acceptance • Component • System • API • End-to-End • Acceptance • User Acceptance • Integration • Non-functional • Functional
  12. 12. Test Categorisation
  13. 13. Test Categorisation
  14. 14. Test Categorisation • Useful concept - up to a point • Blurred lines + Overloaded terminology • Difficulties conversing about testing
  15. 15. Test Categorisation
  16. 16. Test Categorisation
  17. 17. Overview • Automating • Testing Categories + Scope • Development Testing
  18. 18. Development Testing • What it is • What it should not be • How it has evolved
  19. 19. What it is • A Development practice first and foremost • White box / Black box / Grey box • Has Quality aims • Has Efficiency aims
  20. 20. JUnit • Test automation framework of choice for most Java developers • Synonymous with Unit testing • Widely used for testing beyond the Unit scope • Ported to many languages - generically known as xUnit
  21. 21. JUnit “Never in the field of software development have so many owed so much to so few lines of code” Martin Fowler (Development big-wig)
  22. 22. What it is • A Development practice first and foremost • White box / Black box / Grey box • Has Quality aims • Has Efficiency aims
  23. 23. What it is • Can take a technical perspective on scenarios • Can take a user functionality perspective • Exercises precise control over the test context • Used + extended by Dev during initial and future coding
  24. 24. What it is • Often used to discover and refine solution design • Often leaves fine-grained regression coverage safety net • Enables instant feedback on failures due to future enhancements and refactorings
  25. 25. What it is not • Automated full System tests a.k.a E2E, System Integration • Zero control over the state of the system (the test context) • Test scope coupled to other external systems, reference data quality, env availability • Heavily reliant on driving a UI to execute many finegrained scenarios
  26. 26. Its Evolution • OOP - slow evolution • Automated Developer Testing around a lot less time • A lot less mainstream than OOP • Growing body of knowledge / experience available
  27. 27. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  28. 28. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  29. 29. Test structure • Arrange, Act, Assert • Specify Inputs • Given, When, Then • Specify the state of the System • Specify the Event • Specify the Expectations
  30. 30. Test structure • Naming conventions + structure @Test public // // // } void methodUnderTest_GivenABC_ThenExpectXYZ() { given when then @Test public void methodUnderTest_GivenInputsABC_AndSystemStateDEF_ThenExpectXYZ() { // given - inputs // given - system state ! // when // then }
  31. 31. Test structure • IDE assistance for consistency + evolve conventions
  32. 32. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec
  33. 33. “Any fool can write code that a computer can understand” ! Martin Fowler
  34. 34. Test Qualities • Readability • What is it? Why does it matter?
  35. 35. Test Qualities • Readability == …? • Code that you can understand with ease • Language affects Thought • Thought affects Action
  36. 36. Test Qualities • Tools can directly address Readability qualities • Domain Specific Languages = a fancy term • • JUnit, Hamcrest, FEST, JMock, Mockito, Gherkin Language -> Thought -> Action
  37. 37. “Test automation is a first class software engineering problem” ! Brian Marick?
  38. 38. Test Qualities • Maintainability • What is it? Why does it matter?
  39. 39. Test Qualities • Maintainability == …? • To change, evolve with ease • Keeping the cost of change down
  40. 40. Test Qualities • They can be context sensitive. Many are not. • You may opt to sacrifice: • • Readability for Maintainability • • DRY Principle for Readability Maintainability for Obviousness Expect debate
  41. 41. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  42. 42. Specifying Expectations • Stating Expectations / Asking Questions • From Computer dialect ——> Human dialect // then assertTrue(trader.getPermissions().isEmpty()); assertThat(trader.getPermissions().isEmpty(), is(true)); assertThat(trader.getPermissions()).isEmpty(); assertThat(trader).hasNoPermissions(); // // // // // // // Computer dialect . . . . . Human dialect
  43. 43. Specifying Expectations • Stating Expectations / Asking Questions • From Computer dialect ——> Human dialect // Using Basic JUnit assertEquals(orders.size(), 2); Iterator<Order> itr = orders.iterator(); assertEquals(itr.next(), order3); assertEquals(itr.next(), order4); // Using Hamcrest assertThat(orders, hasItems(order3, order4)); assertThat(orders.size(), is(2)); // Using FEST Assertions assertThat(orders).containsOnly(order3, order4); // Using FEST Assertions (chained assertions) assertThat(orders).containsOnly(order3, order4) .containsSequence(order3, order4);
  44. 44. Specifying Expectations • Easier + Faster to write expressive statements • Faster to read + review • Easier to maintain • Encourages other better practices
  45. 45. Specifying Inputs • Trivial example. Test with very narrow scope • e.g.Input = Single Object @Test public void getPermissions_GivenNewUser_ThenReturnsNoPermissions() { // given Trader trader = new Trader(); // when + then assertThat(trader.getPermissions()).isEmpty(); }
  46. 46. Specifying Inputs • Non-trivial example. Test with broader scope • e.g. Input = Object Graph • i.e. objects linked to objects
  47. 47. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  48. 48. Specifying Inputs • Consider many non-trivial tests, with “raw” setup • Impact of changing one setter or constructor • Quantity of code there is to read, understand, change
  49. 49. Specifying Inputs - Techniques • Helper methods • Object Mother Pattern • Test Data Builder Pattern
  50. 50. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  51. 51. @Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  52. 52. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  53. 53. @Test public void getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v1() { // given - inputs Trader trader = new Trader(); TradingAccount tradingAcct = new TradingAccount(new TradingFirm("TraderzRUs", "TRDRZ"), new ClearingFirm("TooBig2Fail", "2BIG2F"), ACTIVE); trader.setPermissions(Arrays.asList(new Permission(tradingAcct))); String goldIsinCode = "COF24680"; // given - system state Future oil3MnthFuture = new Future("OIL.3MNTH", new ISIN("OIL3M0123")); Future oil6MnthFuture = new Future("OIL.6MNTH", new ISIN("OIL6M0456")); Future goldFuture = new Future("GLD.3MNTH", new ISIN(goldIsinCode)); String orderId = "ordId"; Order order1 = new Order(orderId + 1, tradingAcct, oil3MnthFuture, 1000, 2500); Order order2 = new Order(orderId + 2, tradingAcct, oil6MnthFuture, 1500, 2600); Order order3 = new Order(orderId + 3, tradingAcct, goldFuture, 200, 4300); Order order4 = new Order(orderId + 4, tradingAcct, goldFuture, 150, 4300); OrdersDAO orderDAO = new OrdersDAOInMemory(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); // when List<Order> orders = orderSearchService.getOrders(trader, tradingAcct, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  54. 54. @Test public void // Using helpers getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String accountCode = "TRDRZ"; String goldIsinCode = "GLD24680"; TradingAccount tradingAcct = createTradingAccount(accountCode); Trader trader = createTraderWithPermissionsFor(tradingAcct); // given - system state Future oil3MnthFuture = createFuture("OIL.3MNTH", "OIL3M0123"); Future oil6MnthFuture = createFuture("OIL.6MNTH", "OIL6M0456"); Future goldFuture = createFuture("GLD.3MNTH", goldIsinCode); Order order1 = createOrder(accountCode, oil3MnthFuture, qty(1000), price(2500)); Order order2 = createOrder(accountCode, oil6MnthFuture, qty(1500), price(2600)); Order order3 = createOrder(accountCode, goldFuture, qty(200), price(4300)); Order order4 = createOrder(accountCode, goldFuture, qty(150), price(4300)); OrderSearchService orderSearchService = createOrderService( createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = orderSearchService.getOrders(trader, accountCode, goldIsinCode); // then assertThat(orders).containsOnly(order3, order4); }
  55. 55. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  56. 56. Specifying Inputs - Techniques • Object Mother Pattern
  57. 57. public class ObjectMother { ! public static OrdersDAOInMemory createOrdersDAO(Order order1, Order order2, Order order3, Order order4) { return new OrdersDAOInMemory(order1, order2, order3, order4); } ! public static OrderSearchService createOrderService(OrdersDAO ordersDAO) { OrderSearchService orderSearchService = new OrderSearchServiceImpl(ordersDAO); return orderSearchService; } private OrderSearchService prepareOrderService(Order order1, Order order2, Order order3, Order order4) { OrdersDAO orderDAO = createOrdersDAO(order1, order2, order3, order4); OrderSearchService orderSearchService = new OrderSearchServiceImpl(orderDAO); return orderSearchService; } ! public static Order createOrder(String accountCode, Future dummyFuture) { return createOrder(accountCode, dummyFuture, qty(0), price(0)); } ! public static Order createOrder(String accountCode, Future future, int qty, int price) { return new Order(nextOrderId(), accountCode, future, qty, price); } private Order createOrder(String orderId, TradingAccount tradingAcct, Future future, int qty, int price) { return new Order(orderId, tradingAcct.getTradingFirm().getCode(), future, qty, price); } ! public static Future createFuture(String desc, String isinCode) { return new Future(desc, new ISIN(isinCode)); } public static Future createFuture(String isinCode) { return createFuture(DEFAULT_FUTURE_DESC, isinCode); } ! public static Future dummyFuture() { return createFuture(DEFAULT_FUTURE_DESC, DEFAULT_ISIN_CODE); } !
  58. 58. Specifying Inputs - Techniques • Test Data Builder Pattern
  59. 59. @Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  60. 60. public class TraderBuilder { public static String DEFAULT_USERNAME = "trader1"; public static UserDetail DEFAULT_USERDETAIL = new UserDetail(); public static List<Permission> NO_PERMISSIONS = Collections.emptyList(); public static List<Restriction> NO_RESTRICTIONS = Collections.emptyList(); ! private String userName = DEFAULT_USERNAME; private UserDetail userDetail = DEFAULT_USERDETAIL; private List<Permission> permissions = NO_PERMISSIONS; private List<Restriction> restrictions = NO_RESTRICTIONS; private TraderBuilder() { } public static TraderBuilder aTrader() { return new TraderBuilder(); } public Trader build() { Trader trader = new Trader(); trader.setUserName(userName); trader.setUserDetail(userDetail); trader.setPermissions(permissions); trader.setRestrictions(restrictions); return trader; } public TraderBuilder withUserName(String userName) { this.userName = userName; return this; } ! public TraderBuilder with(UserDetail userDetail) { this.userDetail = userDetail; return this; } public TraderBuilder withPermissions(List<Permission> permissions) { this.permissions = new ArrayList<>(permissions); return this; } public TraderBuilder this.permissions return this; } public TraderBuilder this.permissions return this; } with(PermissionBuilder permission) { = new ArrayList<>(Arrays.asList(permission.build())); withNoPermissions() { = Collections.emptyList(); ! public TraderBuilder with(RestrictionBuilder restriction) { this.restrictions = new ArrayList<>(Arrays.asList(restriction.build())); return this; } public TraderBuilder withRestrictions(List<Restriction> restrictions) { this.restrictions = new ArrayList<>(restrictions); return this; } public TraderBuilder but() { return new TraderBuilder() .withUserName(userName) .with(userDetail) .withPermissions(permissions) .withRestrictions(restrictions); } }
  61. 61. @Test public void // Using Test Data Builders getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = aTrader().with(aPermission() .with(aTradingAccount() .with(aTradingFirm().withCode("TRDRZ")))).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  62. 62. @Test public void // Using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", dummyFuture()), order2 = createOrder("TRDRZ", dummyFuture()), order3 = createOrder("TRDRZ", createFuture("GLD24680")), order4 = createOrder("TRDRZ", createFuture("GLD24680")); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order3, order4); }
  63. 63. @Test public void // Using Test Data Builders to create input data variations getOrders_GivenTraderWithVariousPermissions_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v5() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; TradingAccountBuilder tradingAccount = aTradingAccount().with(aTradingFirm().withCode("TRDRZ")); Trader traderWithNoPerms = aTrader().withNoPermissions().build(); Trader traderWithPerms = aTrader().with(aPermission().with(tradingAccount)).build(); Trader traderWithInactivePerms = aTrader().with(aPermission().with( tradingAccount.but().isInActive())).build(); // given - system state OrderBuilder matchingOrder = anOrder().withAccountCode("TRDRZ") .with(aFuture().with(anISIN().withIsinCode("GLD24680")).build()); Order order1 = anOrder().withAccountCode("TRDRZ").build(), order2 = anOrder().withAccountCode("TRDRZ").build(), order3 = matchingOrder.build(), order4 = matchingOrder.build(); OrderSearchService service = createOrderService(createOrdersDAO(order1, order2, order3, order4)); // when List<Order> orders1 = service.getOrders(traderWithNoPerms, searchAccount, searchIsin); List<Order> orders2 = service.getOrders(traderWithPerms, searchAccount, searchIsin); List<Order> orders3 = service.getOrders(traderWithInactivePerms, searchAccount, searchIsin); // then assertThat(orders1).isEmpty(); assertThat(orders2).containsOnly(order3, order4); assertThat(orders3).isEmpty(); }
  64. 64. Specifying Inputs - Techniques • Test Data Builder Pattern - with tool support • Make-It-Easy - addresses Builder class boiler-plate • Model Citizen - annotation-based “Blueprints”
  65. 65. Specifying Inputs - Techniques • Test Data Builder Pattern - Advantages • Your test / spec documents only the inputs that matter • Decouples your test from being impacted by a wide range of changes • Supports “declarative” code style + useful templates • user = EXPIRED_USER.but( with ( aBalance, 100));
  66. 66. Specifying Inputs - Techniques • Test Data Builder Pattern - Disadvantages • Requires much new test-supporting code • Chained methods take getting used to
  67. 67. Specifying the System State • State of the SUT / the Test Context • Communicating what the context, limitations and boundaries are • Isolating your test from certain dependencies / interactions • Use of Test Doubles • Hand-rolled Test Double • common misnomer Mocking Frameworks
  68. 68. Specifying the System State • Problems + solutions for Specifying Inputs normally apply • Unique to System State is your Test Isolation requirements
  69. 69. Specifying the System State - Techniques • Hand-rolling Test Doubles • Using Test Isolation frameworks to provide Test Doubles • “Mocking” + “Mocks Aren’t Stubs”
 http://martinfowler.com/articles/mocksArentStubs.html • Dummy / Fake / Stubs / Mocks / Spies
  70. 70. Specifying the System State - Techniques • Stub being used in a state-based test
  71. 71. @Test public void // Stubbing system state (Mockito) + using helpers + hiding irrelevant details getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  72. 72. @Test public void // Stubbing system state + stubbing inputs (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); TradingAccount tradingAccountStub = mock(TradingAccount.class); given(permissionStub.getTradingAccount()).willReturn(tradingAccountStub); given(tradingAccountStub.isActive()).willReturn(Boolean.TRUE); given(tradingAccountStub.getTradingFirm()).willReturn(new TradingFirm("", searchAccount)); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  73. 73. @Test public void // Stubbing system state + stubbing input (Mockito w/ Deep Stubs) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v3_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; // Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); Trader traderStub = mock(Trader.class); Permission permissionStub = mock(Permission.class, RETURNS_DEEP_STUBS); given(traderStub.getPermissions()).willReturn(Arrays.asList(permissionStub)); given(permissionStub.getTradingAccount().isActive()).willReturn(Boolean.TRUE); given(permissionStub.getTradingAccount().getTradingFirm().getCode()).willReturn(searchAccount); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); OrderSearchService service = createOrderService(ordersDAOStub); // when List<Order> orders = service.getOrders(traderStub, searchAccount, searchIsin); // then assertThat(orders).containsOnly(order1, order2); }
  74. 74. Specifying the System State - Techniques • A Mock being used in a interaction-based test
  75. 75. @Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_1() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); }
  76. 76. @Test public void // Using a Mock to test interactions + Stubbing system state (Mockito) getOrders_GivenTraderWithAccountPerms_SearchByAccountAndIsin_ThenReturnsMatchingOrders_v4_2() { // given - inputs String searchAccount = "TRDRZ", searchIsin = "GLD24680"; Trader trader = createTraderWithPermissionsFor(createTradingAccount("TRDRZ")); // given - system state Order order1 = createOrder("TRDRZ", createFuture("GLD24680")), order2 = createOrder("TRDRZ", createFuture("GLD24680")); OrdersDAO ordersDAOStub = mock(OrdersDAO.class); given(ordersDAOStub.findBy(searchAccount, searchIsin)).willReturn(Arrays.asList(order1, order2)); PermissionService permissionServiceMock = mock(PermissionService.class); given(permissionServiceMock.hasViewPermissions( any(Trader.class), any(Order.class))).willReturn(Boolean.TRUE); OrderSearchService service = createOrderServiceV2(ordersDAOStub, permissionServiceMock); // when List<Order> orders = service.getOrders(trader, searchAccount, searchIsin); // then verify(permissionServiceMock).hasViewPermissions(trader, order1); verify(permissionServiceMock).hasViewPermissions(trader, order2); assertThat(orders).containsOnly(order1, order2); }
  77. 77. Specifying the System State - Techniques • Often a test will have • Zero to many Stubs. Zero or one Mock • Mocks help you answer questions about what happened • When a Mock is present, you ask it something at the end • A Stub should not be asserted against • There to help create + control the System State for the test
  78. 78. Specifying the Event • Often trivial • Might be a sequence of events that we want to fire • Sequence might actually be context / SUT setup • Practices? Tools?
  79. 79. Specifying it All Together • Spring MVC Test support / DSL @Test public void helloWorld() throws Exception { mockMvc.perform(get("/hello").accept(MediaType.TEXT_PLAIN)) .andDo(print()) // print the request/response in the console .andExpect(status().isOk()) .andExpect(content().contentType(MediaType.TEXT_PLAIN)) .andExpect(content().string("Hello World!")); }
  80. 80. Principles, Practices + Tool Support • Structural Principles / Characteristics • Qualities of a “good” test / spec • Practices + Tools for improving test qualities
  81. 81. Continuous Testing • Continuous Integration - development practice • CI tools - run automated tests frequently • Continuous Testing - development practice • CT tools - run automated tests locally, frequently • e.g. Infinitest, JUnitMax
  82. 82. Overview • Automating • Testing Categories + Scope • Development Testing • Principles, Practices + Tool Support
  83. 83. Summary • Tools/APIs just help with heavy lifting • Sound coding principles are paramount • Creating unreadable and unmaintainable test, even with “the right” tools • Unreadable + high maintenance tests will hinder efficiency
  84. 84. Summary • The power of marginal improvements • for … each, <> operator, closure support, Guava • Continuous iterative improvements nudge us forward • Continuous iterative improvements push down cost of reading, writing, evolving tests • Drives the cost down to “mass market” level • Beyond Java
  85. 85. Resources • Books • GOOS - Growing Object Oriented Software, Guided By Tests • Effective Unit Testing • Working Effectively with Legacy Code • xUnit Test Patterns - Refactoring Test Code
  86. 86. Resources • Podcasts • • Hanselminutes - Roy Asherove - The Art of Unit Testing • • Sustainable Test Driven Development (Series) Hanselminutes - Quetzal Bradley - Beyond Unit Testing All Code Examples are available at • http://github.com/cathalking/devtestevolution
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×