API Testing following the
Test Pyramid
Elias Nogueira
@eliasnogueira

Elias Nogueira
Brazilian guy living in the Netherlands.
I (try to) help people and companies to
deliver a high-quality software.
@eliasnogueira
github.com/eliasnogueira

Disclaimer
The Test Pyramid focus will be on the API part,
not related to unit and web tests

Concept

Original Test Pyramid
UI
Tests
Service Tests
Unit Tests
more
isolation
more
integration
faster
slower

Ideal Test Pyramid

Individual APIs
API
API
API
API
API
API
API
API

Individual APIs
API
API
API
API
API
API
API
API
We’ll assume that
each API is
already tested
Unit Test must
be done

Individual APIs
API
API
API
API
API
API
API
API
Functional testing
in each API

Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing

Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing
Guarantee that
they are
communicating
without problem

Contract and E2E testing
API
API
API
API
API
API
API
API
Contract
&
E2E Testing
Assure that they
can be used
together (e2e)

Recall

Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}

Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services.
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
import libraries

Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
request pre-condition

Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
action (request)

Rest-Assured
http://rest-assured.io
Java DSL for simplifying testing of REST based services
import static io.restassured.RestAssured.*;
import static org.hamcrest.Matchers.*;
public class RestAssuredExampleTest {
@Test
public void welcome() {
given().
param("name", "Elias").
when().
post("/register").
then().
body("message", is("Hello Elias"));
}
}
assert the response body

SUT – System Under Test

SUT – System Under Test | Front-end
We should inform a CPF
* Its a Brazilian social security number
If a restriction is found,
show a message

SUT – System Under Test | Front-end
Fill in loan information

SUT – System Under Test | Front-end
CRUD operations

Pipeline

API
Pipeline
Health Check
Contract
Functional
Acceptance
Verify if the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

Health Check
Pipeline (pyramid view)
Contract
Acceptance
Functional

Pipeline
GitLab CI
https://gitlab.com/elias.nogueira/test-combined-credit-api/pipelines

Enabling the Pipeline
Create a way to, easily, filter the tests by their focus/function/level…
@Test(groups = "functional")
public void testNgGroup() {
// test goes here
}
@Test
@Tag("functional")
void junitTag() {
// test goes here
}
XML file Suite Class Suite

[how to] Enabling the Pipeline
In the example
1. Create a strategy to filter your tests
○ Example of using groups in a test [1]
○ Example of a test suite [2]
2. Create a strategy to enable the test be executed by command line
○ pom.xml showing show to run a suite [3]
3. Create a pipeline as a code
○ .gitlab-ci.yml [4]
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsFunctionalTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/resources/suites/e2e.xml
[3] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/pom.xml#L101
[4] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/.gitlab-ci.yml

API
health-check
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

heath-check
Verify if the API is available
If there’s no way to verify by a monitoring strategy we can make a request and
validate the status code
@Test(groups = "health")
public void healthCheckViaActuator() {
basePath = "/actuator";
when().
get("/health").
then().
statusCode(200).
body("status", is("UP"));
}
via monitoring
@Test(groups = "health")
public void healthCheckViaAPI() {
given().
pathParam("cpf", "81016654049").
when().
get("/restrictions/{cpf}").
then().
statusCode(200);
}
via API

[how to] health-check
1. Identify if you have a health check endpoint
○ If true find out, beyond the endpoint, any return status
○ If false, make a request to the easiest endpoint (GET?)

[how to] health-check
In the example
● In the local project hit http://localhost:8088/actuator/health
○ You’ll see the health status
● See the implemented tests on:
○ CreditHealthCheckTest [1]
● Items to pay attention:
○ It’s getting the health context from the file because it does not have the
/api/v1 to hit the actuator endpoint
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/general/CreditHealthCheckTest.java

API
contract
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

● It’s the name given to the pact between producer and
consumer
● Ensures that API changes do not invalidate consumption:
● path
● parameters
● sending data (request)
● return data (response body)
● json-schema is a contract that defines the expected data,
types and formats of each field in the response
contract

{
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
json-schema

{
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
json-schema has the attribute
name and data type
json-schema

{
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
”age": {
"type": "integer"
}
},
"required": [
"name",
"age"
],
"additionalProperties": false
}
both attributes must be present
json-schema

{
"name": "Elias",
"age": 37
}
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"type": "string"
},
"age": {
"type": "integer"
}
},
"required": [
”name",
”age"
],
"additionalProperties": false
}
json-schema
no other attributes are
allowed

[how to] contract
1. Create schema files
○ We can use online tools to create if it’s not available
2. Create a test making a request and validating the json schema
○ Add json-schema-validator library
○ Statically import the matcher
○ Use the matcher against json schema file

[how to] contract
In the example
● See the implemented tests on:
○ RestrictionsContractTest.java [1]
○ SimulationsContractTest.java [2]
● Items to pay attention:
○ The validation is done by the matcher matchesJsonSchemaInClasspath,
checking the response body with the json schema file
○ If you want to see different API enable the test contractOnV2 on
RestrictionsContractTests
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsContractTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/simulations/SimulationsContractTest.java

API
functional
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert all the criteria from the requirement +
happy/sad paths
Assert that the most important user
scenarios still works

functional
Validate positive and negative scenarios (happy and sad path)
@Test(groups = {"functional"})
public void existentSocialSecirityNumber() {
given().
pathParam("cpf", "66414919004").
when().
get("/simulations/{cpf}").
then().
statusCode(200).
body(
"id", equalTo(1),
"name", equalTo("Oleksander"),
"cpf", equalTo("66414919004"),
"email", equalTo("oleksander@gmail.com"),
"amount", equalTo(11000f),
"installments", equalTo(3),
"insurance", equalTo(true)
);
}
data validation

[how to] functional
1. Create the tests avoiding repetitions
○ Use Test Data Factories and Data Driver in your advantage
2. Apply the strategy you like
○ All tests in the same class
○ One class per test

[how to] functional
In the example
● See the implemented tests on:
○ RestrictionsFunctionalTest.java [1]
○ SimulationsFunctionalTest.java [2]
● Items to pay attention:
○ The tests are using a soft assertion for the body validation
○ MessageFormat class is used to properly concatenate String
○ SimulationsFunctionalTests is using Test Data Factory, Builders for the
model object and Data Driven
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/restrictions/RestrictionsFunctionalTest.java
[2] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/simulations/SimulationsFunctionalTest.java

Assert all the criteria from the requirement +
happy/sad paths
API
acceptance
Health Check
Contract
Functional
Acceptance
Verify it the endpoint is alive
Assert that the specs haven’t changed
Assert that the most important user
scenarios still works

Testing from the user's perspective
● Access the page and make the restriction search
by the CPF
● Insert a credit simulation

acceptance (e2e)
@Test(groups = "e2e")
public void completeSimulation() {
new RestrictionsClient().queryRestrictionAndReturnNotFound();
new SimulationsClient().submitSuccessfulSimulation();
}

[how to] acceptance (e2e)
1. Create the tests avoiding repetitions
○ Use Request and Response Specifications
○ Try to break out in reusable methods common actions
2. Do the User Journey
○ The few most important user journeys must run before the
functional tests
https://martinfowler.com/articles/practical-test-pyramid.html#RestApiEnd-to-endTest

[how to] functional
In the example
● See the implemented tests on:
○ FullSimulationE2ETest.java [1]
● Items to pay attention:
○ There’s a utility class with the suffix Client in order to have an easy way
to make request and response calls
○ The request and response calls are Specification Re-use [2] created to
reuse these actions
○ See also the packages client [3] and specs [4]
[1] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/blob/master/src/test/java/com/eliasnogueira/credit/e2e/FullSimulationE2ETest.java
[2] https://github.com/rest-assured/rest-assured/wiki/usage#specification-re-use
[3] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/tree/master/src/main/java/com/eliasnogueira/credit/client
[4] https://gitlab.com/elias.nogueira/test-combined-credit-api/-/tree/master/src/main/java/com/eliasnogueira/credit/specs

For free
Work in progress in 2 languages
Access Leanpub and subscribe

Thank you!
@eliasnogueira
github.com/eliasnogueira