More Related Content

Similar to API Testing with Open Source Code and Cucumber(20)


More from SmartBear(20)


API Testing with Open Source Code and Cucumber

  1. API testing with Open Source Code libraries and Cucumber Ole Lensmar – SmartBear
  2. 22 Short about “Shift-Left” • Enterprise testing and QA is moving from late to early phases in the development life-cycle – QA -> Dev -> Design • Tests are being expressed as code instead of using visual tools (by both QA and Dev) – Re-use of existing Test assets / scripts for different types of tests • Code-driven approach not just for functional testing – Load/performance testing – Security testing – Mocking / Virtualization
  3. 33 Testing at API layer vs Unit tests • API-layer testing = integration testing of implementing classes/components • Unit-testing frameworks often used to ”orchestrate” – Junit / TestNG / Failsafe etc. • Tests are managed together with implementing code - using traditional SCM tools • Tests are executed as part of existing builds – Test running instance of API vs implementation
  4. 44 Code driven approach to API Testing • HTTP clients – Apache HTTPClient, OkHttp, Super-agent, etc. • Generated clients – Swagger Codegen, WSDL2Java, etc. • Special purpose libraries – REST-Assured (Java/Groovy) – Chakram, (JS) – ReadyApi4j (Java) • Cucumber / BDD - many platforms/languages
  5. 55 Approach for comparison • Simple scenario with single and chained REST requests testing the SwaggerHub integration API • Framework specific highlights • All available on GitHub: • All run with maven (“mvn test”) except javascript sample
  6. 66 Apache HTTP-Client • Widely used java client library for HTTP • Exposes fluent API for building / executing requests and reading responses • Lacks built in support for JSON / XML parsing or assertions – Need to do this manually using Jackson/DOM/etc.
  7. 77 Generated clients • Use code-generation tools to generate client from API description • Write tests using generated client – Automate client generation as part of build • Pros – API changes can break tests – Isolates from network layer – tests at functional level • Cons – Relies on correctness of client-generator – Isolates from network layer – tests at functional level
  8. 88 REST-Assured • Fluent Java API for testing REST APIs • Support both JSON and XML content assertions – Uses standalone JSONPath and Xpath implementations • Built in support for multiple authentication types, XSRF, etc. • Very customizable at the network/http/marshalling layers •
  9. 99 ReadyAPI4j • Fluent Java API – Leverages the SoapUI Testing engine – REST, SOAP, Assertions, orchestration, etc. • Internally compiles tests into “Test Recipes” (json) – Can be reused for performance/security testing in SoapUI • Execution either locally or remotely – Local execution for ease of use (open source) – Remote execution for access control, performance, reporting, etc. (commercial)
  10. 1010 Maven/Java project ReadyApi4j execution model API Test /src/test/java ReadyApi4j API Local Engine ReadyAPI Testserver API(s) under test use use call API calls
  11. 1111 Challenges with code-driven API testing • Running against different environments – Temporary environments (Docker can help) – Actual environments • Testing either with or without network – Network is more realistic – more work to automate • External dependencies may need to be mocked – 3rd party APIs, databases, etc. – For both functional and non-functional testing • Simulate errors, response times, invalid responses, etc.
  12. Cucumber for API testing
  13. 1313 Cucumber – “Executable Specifications” • Heavily associated with Behavior Driven Development • “Plain language” verbalization of requirements – POs, Business Analysts, Developers, Testers, etc. • For example Given a user has logged in with valid credentials When a deposit of 100 USD is made Then the account balance increases by 100 USD And the user gets a confirmation message
  14. 1414 Scenarios/Features written in “Gherkin” • Given X • When Y • Then Z • More keywords: And, Feature, Scenario, Background, etc. • Packaged into “feature files” • Cucumber reads feature files and executes corresponding “StepDefs” or “GlueCode”
  15. 1515 Automated Build / Test executions Cucumber “execution pipeline” Cucumber Runner StepDefs Component / System under test 1. Parse 2. Invoke .feature files Ant/Maven/Gra dle/CLI/etc [@tag]
  16. 1616 Cucumber StepDefs • Code that is executed by the cucumber framework for each statement • Must be implemented manually to support the desired vocabulary • Cucumber Java example -> code!
  17. 1717 Cucumber “styles” • Declarative style – Domain-specific vocabulary for describing the business requirements – Appeals to non-technical users – High overhead in creation and maintenance • Imperative style – Generic vocabulary for describing requirements – Appeals more to technical users – Lower maintenance – but “Misses the point” • Either requires implementation of StepDefs
  18. 1818 Declarative API Testing Example Feature: SwaggerHub REST API Scenario: Default API Listing When a request to the API listing is made Then a list of APIs should be returned within 500ms Scenario: TestServer API Retrieval Given an owner named smartbear And an api named ready-api-testserver And a version named 1.0.0 When a request to the API listing is made Then an API definition should be returned within 500ms
  19. 1919 Imperative API Testing Example Feature: Petstore API Scenario: Find pet by status Given the API running at When a GET request to /pet/findByStatus is made And the status parameter is test And the Accepts header is application/json Then a 200 response is returned within 50ms
  20. 2020 Imperative API Testing Example with Swagger Feature: Petstore API operationId gives path and method operation parameter type not neededa defined response
  21. 2121 Managing Feature files and StepDefs • Keep feature files together with source code and/or in separate repository • Use tagging feature of Cucumber to selectively (re)run tests • Use datasets for parameterization – Inline or external • Bundle StepDefs in separate library – Automate execution
  22. 2222 ReadyApi4j - Cucumber for APIs • Out-of-the box generic vocabulary for REST API Testing - integrated Swagger support • Extensible – built on Cucumber for Java – Open-source on GitHub – • Packaged as library or Docker image – • Uses ReadyApi4j as execution engine (either local or remote)
  23. 2323 That’s it! – Questions? • @olensmar • • • • • • • • •