Peter Thomas presents Karate, an open source tool for writing web service API acceptance tests. Built on Cucumber's BDD syntax, Karate provides a domain specific language focused on HTTP testing. It allows testing REST and SOAP APIs by making HTTP calls and validating JSON and XML payloads in a concise yet readable manner. Karate tests are faster and easier to write compared to traditional approaches like using Java with libraries like REST Assured.

1. Peter Thomas | Architect | Intuit
@ptrthomas | @KarateDSL
Karate
Web Service API Testing Made Simple

2. Intuit Open Source
Released 09-Feb-2017
A DSL for writing web-service acceptance tests
Built on Cucumber | BDD syntax
Karate github.com/intuit/karate

3. 3
Karate in the top 5 OSS API Testing Tools (TechBeacon)
Tool Inception Years
REST-assured 2010 7
Postman 2012 5
SoapUI 2005 12
JMeter 1998 19
Karate 2017 0.5

4. 4
• Easier to Write
• Readable and Concise
• Faster Execution
• Better Assertions
• Integrates into existing CI / CD
API-tests with Karate
Developer
Productivity
Maintain-
ability
Quality

5. 5
Many teams use Java for writing HTTP-API regression tests.
Java is not the best language to manipulate JSON / XML.
Consequences:
• Over-dependence on Java objects for building and validating
data-payloads
• Object-equality checks done by hand, effort-intensive
• So tests end-up taking “short-cuts”, and don’t validate the full
response, coverage & quality suffers
• Proliferation of helper methods, code bloat & in-house layers
• Result: poor Dev Productivity & Maintainability
Current State of Web-API Testing
• DSL laser-focused on HTTP
• JSON and XML are “first-class”
citizens of the syntax
• Compare deeply-nested payloads in
a single step
• 3:1 Reduction in Lines of Code
• Tests are super-readable and
concise
• Testing is Easier & actually Fun
Karate’s Solve

6. 6
Scenario: create and retrieve a cat
Given url 'http://myhost.com/v1/cats'
And request { name: 'Billie' }
When method post
Then status 201
And match response == { id: '#notnull', name: 'Billie' }
Given path response.id
When method get
Then status 200
Hello World
Intuitive DSL
for HTTP
Payload
assertion in one
line
Second HTTP
call using
response data
JSON is ‘native’
to the syntax

7. 7
Java
vs
JSON

8. 8
Validation examples: JsonPath & RegEx
Type Directive
Ignore / Exists #ignore | #null | #notnull
Primitive Type #boolean | #number | #string
Complex Type #array | #object
Special #uuid
RegEx #regex <regex string>
Predicate #? <expression>

9. 9
JSON ‘templating’
Type Directive
Replace #(<expression>)

10. 10
XML and XPath

11. 11
Standard form In-line form
* match foo[0] == exact * match foo[0] == '#(exact)'
* match foo[0] contains partial * match foo[0] == '#(^partial)'
* match foo[0] !contains nope * match foo[0] == '#(!^nope)'
* match each foo == exact * match foo == '#[] exact'
* match each foo contains partial * match foo == '#[] ^partial'
* match each foo !contains nope * match foo == '#[] !^nope'
* match foo contains only reversed * match foo == '#(^^reversed)'
* match foo contains first * match foo == '#(^first)'
* match foo !contains others * match foo == '#(!^others)'
* assert foo.length == 2 * match foo == '#[2]'
Schema Validations

12. 12
Comparison with REST-assured
http://tinyurl.com/karateraREST-assured Karate
Plain Text ❌ (needs compilation) ✅
Parallel Execution ? (partial) ✅
Data Driven Testing ❌ (needs TestNG etc.) ✅ (built-in)
Environment Switching ❌ ✅ (built-in)
Match full payload in one step ❌ ✅
Update JSON payload / object ❌ ✅
@Test public void
lotto_resource_returns_200_with_expected_id_and_winners() {
when().
get("/lotto/{id}", 5).
then().
statusCode(200).
body("lotto.lottoId", equalTo(5),
"lotto.winners.winnerId", containsOnly(23, 54));
Scenario: lotto resource returns 200 with expected id and winners
Given path ‘lotto’, 5
When method get
Then status 200
And match $.lotto.lottoId == 5
And match $.lotto.winners[*].winnerId contains only [23, 54]
given().
param("key1", "value1").
param("key2", "value2").
when().
get("/somewhere").
then().
body(containsString("OK"));
Given param key1 = ‘value1’
And param key2 = ‘value2’
And path ‘somewhere’
When method get
Then response contains ‘OK’
Detailed Comparison

13. 13

14. 14
Metrics – port of an open-source example
Reduction of lines of code from 431  67
Link: https://github.com/mwinteringham/api-framework/pull/3

15. 15
Components
Karate DSL interpreter
(Cucumber “Step Definitions”)
HTTP Client Abstraction
JSON XML
JS
Engine
(Nashorn)
Karate ScriptJUnit / Test
NG
(Runner)
Java8JRE
HTTP/S calls
Cucumber-
JVM
Data / Assertions
match, get, set
Reports
Apache Jersey
Mock
Servlet

16. 16
The “missing” Cucumber Features that Karate adds
Capability Cucumber Karate
Step Definitions built-in, no Java code needed ❌ ✅
Re-Use Feature files from other Features ❌ ✅
Dynamic Data-Driven Testing ❌ ✅
Parallel Test Execution and Reporting ❌ ✅
Option to run routines only once per Feature ❌ ✅

17. 17
Example Report

18. 18
• Scripts are plain-text, no compilation or IDE required
• REST as well as SOAP support
• Assert, manipulate, compare and re-use JSON / XML
• User defined functions (Java or JavaScript)
• Configuration switching (e.g. dev | e2e | pre-prod)
• Simple ‘plug and play’ HTTP Header & Auth management
• Comprehensive HTTP support: SSL, Proxy, File-Upload
• JUnit XML reports compatible with all CI tools
• Speed - Parallel Execution of Tests
• Mock Servlet – test without starting a container
Karate Features
BDD
Full HTTP
CI / CD
Ready

19. Demos

20. Q & A
https://github.com/intuit/karate
@ptrthomas | @KarateDSL
Thank You !

21. Extra Slides

22. 22
Karate UI (Alpha)

23. 23
CI Report
(Jenkins)