Thi presentation was given at Selenium Camp on 21/02/2020 and show how we can divide better our API tests following the Test Pyramid concept and break it down on the service layer, using a pipeline to clarify and execute the tests in the order of importance.

1. API Testing following the
Test Pyramid
Elias Nogueira
@eliasnogueira

2. Elias Nogueira
Brazilian guy living in the Netherlands.
I (try to) help people and companies to
deliver a high-quality software.
@eliasnogueira
github.com/eliasnogueira

3. Disclaimer
The Test Pyramid focus will be on the API part,
not related to unit and web tests

4. Concept

5. Original Test Pyramid
UI
Tests
Service Tests
Unit Tests
more
isolation
more
integration
faster
slower

6. Ideal Test Pyramid

7. Individual APIs
API
API
API
API
API
API
API
API

8. Individual APIs
API
API
API
API
API
API
API
API
We’ll assume that
each API is
already tested
Unit Test must
be done

9. Individual APIs
API
API
API
API
API
API
API
API
Functional testing
in each API

10. Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing

11. Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing
Guarantee that
they are
communicating
without problem

12. Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing
Assure that they
can be used
together (e2e)

13. Recall

14. Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}

15. Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services.
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
import libraries

16. Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
request pre-condition

17. Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
action (request)

18. Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
assert the response body

19. SUT – System Under Test

20. SUT – System Under Test | Front-end
We should inform a CPF
* Its a Brazilian social security number
If a restriction is found,
show a message

21. SUT – System Under Test | Front-end
Fill in loan information

22. SUT – System Under Test | Front-end
CRUD operations

23. Pipeline

24. API
Pipeline
Health Check
Contract
Functional
Acceptance
Verify if the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

25. Health Check
Pipeline (pyramid view)
Contract
Acceptance
Functional

26. Pipeline
GitLab CI
https://gitlab.com/elias.nogueira/test-combined-credit-api/pipelines

27. Enabling the Pipeline
Create a way to, easily, filter the tests by their focus/function/level…
@Test(groups = "functional")
public void testNgGroup() {
// test goes here
}
@Test
@Tag("functional")
void junitTag() {
// test goes here
}
XML file Suite Class Suite

28. [how to] Enabling the Pipeline
In the example
1. Create a strategy to filter your tests
○ Example of using groups in a test [1]
○ Example of a test suite [2]
2. Create a strategy to enable the test be executed by command line
○ pom.xml showing show to run a suite [3]
3. Create a pipeline as a code
○ .gitlab-ci.yml [4]
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsFunctionalTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/resources/suites/e2e.xml
[3] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/pom.xml#L101
[4] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/.gitlab-ci.yml

29. API
health-check
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

30. heath-check
Verify if the API is available
If there’s no way to verify by a monitoring strategy we can make a request and
validate the status code
@Test(groups = "health")
public void healthCheckViaActuator() {
basePath = "/actuator";
when().
get("/health").
then().
statusCode(200).
body("status", is("UP"));
}
via monitoring
@Test(groups = "health")
public void healthCheckViaAPI() {
given().
pathParam("cpf", "81016654049").
when().
get("/restrictions/{cpf}").
then().
statusCode(200);
}
via API

31. [how to] health-check
1. Identify if you have a health check endpoint
○ If true find out, beyond the endpoint, any return status
○ If false, make a request to the easiest endpoint (GET?)

32. [how to] health-check
In the example
● In the local project hit http://localhost:8088/actuator/health
○ You’ll see the health status
● See the implemented tests on:
○ CreditHealthCheckTest [1]
● Items to pay attention:
○ It’s getting the health context from the file because it does not have the
/api/v1 to hit the actuator endpoint
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/general/CreditHealthCheckTest.java

33. API
contract
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

34. ● It’s the name given to the pact between producer and
consumer
● Ensures that API changes do not invalidate consumption:
● path
● parameters
● sending data (request)
● return data (response body)
● json-schema is a contract that defines the expected data,
types and formats of each field in the response
contract

35. {
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
json-schema

36. {
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
json-schema has the attribute
name and data type
json-schema

37. {
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
”age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
both attributes must be present
json-schema

38. {
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
”name",
”age"
],
"additionalProperties": false
}
json-schema
no other attributes are
allowed

39. [how to] contract
1. Create schema files
○ We can use online tools to create if it’s not available
2. Create a test making a request and validating the json schema
○ Add json-schema-validator library
○ Statically import the matcher
○ Use the matcher against json schema file

40. [how to] contract
In the example
● See the implemented tests on:
○ RestrictionsContractTest.java [1]
○ SimulationsContractTest.java [2]
● Items to pay attention:
○ The validation is done by the matcher matchesJsonSchemaInClasspath,
checking the response body with the json schema file
○ If you want to see different API enable the test contractOnV2 on
RestrictionsContractTests
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsContractTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/simulations/SimulationsContractTest.java

41. API
functional
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

42. functional
Validate positive and negative scenarios (happy and sad path)
@Test(groups = {"functional"})
public void existentSocialSecirityNumber() {
given().
pathParam("cpf", "66414919004").
when().
get("/simulations/{cpf}").
then().
statusCode(200).
body(
"id", equalTo(1),
"name", equalTo("Oleksander"),
"cpf", equalTo("66414919004"),
"email", equalTo("oleksander@gmail.com"),
"amount", equalTo(11000f),
"installments", equalTo(3),
"insurance", equalTo(true)
);
}
data validation

43. [how to] functional
1. Create the tests avoiding repetitions
○ Use Test Data Factories and Data Driver in your advantage
2. Apply the strategy you like
○ All tests in the same class
○ One class per test

44. [how to] functional
In the example
● See the implemented tests on:
○ RestrictionsFunctionalTest.java [1]
○ SimulationsFunctionalTest.java [2]
● Items to pay attention:
○ The tests are using a soft assertion for the body validation
○ MessageFormat class is used to properly concatenate String
○ SimulationsFunctionalTests is using Test Data Factory, Builders for the
model object and Data Driven
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsFunctionalTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/simulations/SimulationsFunctionalTest.java

45. Assert all the criteria from the requirement +
happy/sad paths
API
acceptance
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert that the most important user
scenarios still works

46. Testing from the user's perspective
● Access the page and make the restriction search
by the CPF
● Insert a credit simulation

47. acceptance (e2e)
@Test(groups = "e2e")
public void completeSimulation() {
new RestrictionsClient().queryRestrictionAndReturnNotFound();
new SimulationsClient().submitSuccessfulSimulation();
}

48. [how to] acceptance (e2e)
1. Create the tests avoiding repetitions
○ Use Request and Response Specifications
○ Try to break out in reusable methods common actions
2. Do the User Journey
○ The few most important user journeys must run before the
functional tests
https://martinfowler.com/articles/practical-test-pyramid.html#RestApiEnd-to-endTest

49. [how to] functional
In the example
● See the implemented tests on:
○ FullSimulationE2ETest.java [1]
● Items to pay attention:
○ There’s a utility class with the suffix Client in order to have an easy way
to make request and response calls
○ The request and response calls are Specification Re-use [2] created to
reuse these actions
○ See also the packages client [3] and specs [4]
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/e2e/FullSimulationE2ETest.java
[2] https://github.com/rest-assured/rest-assured/wiki/usage#specification-re-use
[3] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/tree/master/src/main/java/com/eliasnogueira/credit/client
[4] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/tree/master/src/main/java/com/eliasnogueira/credit/specs

50. For free
Work in progress in 2 languages
Access Leanpub and subscribe

51. Thank you!
@eliasnogueira
github.com/eliasnogueira