Peter Thomas | Distinguished Engineer | Intuit
@ptrthomas | @KarateDSL
Karate
Test Automation Made Simple
Intuit Open Source
Released Feb-2017
A DSL for Test-Automation
Inspired by Cucumber | BDD syntax
Karate github.com/intuit/karate
API Testing
Test Doubles
Perf Testing
UI Automation
3
@ptrthomas | @KarateDSL
ThoughtWorks Tech Radar
April 2019: “Assess” May 2020: “Trial”
4
@ptrthomas | @KarateDSL
Evolution of Web User Experiences
4
5
@ptrthomas | @KarateDSL
AJAX ! https://thehistoryoftheweb.com/what-does-ajax-even-stand-for/
6
@ptrthomas | @KarateDSL
Server-Side vs Client-Side
7
@ptrthomas | @KarateDSL
Karate for API Testing
8
@ptrthomas | @KarateDSL
Scenario: create and retrieve a cat
Given url 'http://myhost.com/v1/cats'
And request { name: 'Billie' }
When method post
Then status 201
And match response == { id: '#notnull', name: 'Billie' }
Given path response.id
When method get
Then status 200
Hello World
Intuitive DSL
for HTTP
Payload
assertion in
one line
Second HTTP
call using
response data
JSON is ‘native’
to the syntax
Demo: GraphQL
10
@ptrthomas | @KarateDSL
Performance Testing
The Old Way With Karate
Write
Functional
Tests
Write
Performance
Tests
Tool
A
Tool
B
Execute
Execute
Write
Functional
Tests
Execute
Load Model
re-use
Execute delegate
11
@ptrthomas | @KarateDSL
load any Java
class in one line
re-usable JS
function to “glue”
to Java method
value appears in
HTML report / title
supports all JSON
data-types e.g.
string, number.
boolean, null, even
in-line JSON !
Data Driven / Java Interop
12
@ptrthomas | @KarateDSL
Full-Stack with Karate
13
@ptrthomas | @KarateDSL
HTTP / WebSocket
Client
Mock Servlet
Perf Testing
Web
Browser
Automation
Desktop App
Automation
API Test
Doubles
CLI
Java API
VS Code
Extension
Docker Image
Data Transforms / Assertions
JSON, XML, YAML, CSV
API Testing
JUnit Support
HTTP Server
Parallel / Distributed Executor
Mock HTTP Client
Native Interop
Microsoft UI
Automation Adapter
Gatling Adapter
Chrome / DevTools
Protocol
WebDriver
Adapter
Playwright
Adapter
JS Engine / Java Interop
Process Builder / Async
HTML Templating
Image Processing
OCR
Reports / Output / Logging
IDE Debug Adapter Protocol
Gherkin / Cucumber Compat.
Cucumber
IDE Support
Standalone
JAR
Maven
Artifacts
Custom / 3rd
Party Reports
Appium
Support
Cross
Browser
Support Run
time
core
optional
Karate
Capability
Map
Tags, Life-Cycle, Hooks
Config. / Env. Switching
Custom
Hooks / Code
https://github.com/intuit/karate
@ptrthomas | @KarateDSL
Thank You !
15
@ptrthomas | @KarateDSL
Comparison with REST-assured
http://tinyurl.com/karatera
@Test public void
lotto_resource_returns_200_with_expected_id_and_winners() {
when().
get("/lotto/{id}", 5).
then().
statusCode(200).
body("lotto.lottoId", equalTo(5),
"lotto.winners.winnerId", containsOnly(23, 54));
Scenario: lotto resource returns 200 with expected id and winners
Given path ‘lotto’, 5
When method get
Then status 200
And match $.lotto.lottoId == 5
And match $.lotto.winners[*].winnerId contains only [23, 54]
given().
param("key1", "value1").
param("key2", "value2").
when().
get("/somewhere").
then().
body(containsString("OK"));
Given param key1 = ‘value1’
And param key2 = ‘value2’
And path ‘somewhere’
When method get
Then response contains ‘OK’
Detailed Comparison
Capability REST-assured Karate
No Java coding / compilation required ❌ ✅
Parallel Execution ? (partial) ✅
Data Driven Testing ❌ (needs TestNG etc.) ✅
Config / Environment Switching ❌ ✅
Match entire nested payload in one
step. “Fuzzy Matching” / “Deep Equals”
❌ (partially solved with
Hamcrest etc.)
✅
Mutate given JSON / response object ❌ ✅
WebSocket / Async Support ❌ ✅
HTTP Mocks / Test-Doubles ❌ (needs Wiremock etc.) ✅
Re-use as Performance / Load Tests ❌ ✅
16
@ptrthomas | @KarateDSL
Metrics – port of an open-source example
Reduction of lines of code from 431 à 67
Link: https://github.com/mwinteringham/api-framework/pull/3
17
@ptrthomas | @KarateDSL
The “missing” Cucumber Features that Karate adds
Capability Cucumber Karate
Step Definitions built-in, no Java code needed ❌ ✅
Re-Use Feature files from other Features ❌ ✅
Dynamic Data-Driven Testing / e.g. CSV files ❌ ✅
Parallel Test Execution and Reporting ? ✅
Option to run routines only once per Feature ❌ ✅
Embedded JS Engine, Config / Env. Switching ❌ ✅
https://stackoverflow.com/questions/47795762/in-
karate-how-we-can-collaboratively-work-along-with-
ba-to-automate-business-sce
https://hackernoon.com/the-world-needs-an-alternative-to-selenium-so-we-built-one-zrk3j3nyr

apidays LIVE India - Karate for REST and GraphQL test automation by Peter Thomas, Intuit

  • 1.
    Peter Thomas |Distinguished Engineer | Intuit @ptrthomas | @KarateDSL Karate Test Automation Made Simple
  • 2.
    Intuit Open Source ReleasedFeb-2017 A DSL for Test-Automation Inspired by Cucumber | BDD syntax Karate github.com/intuit/karate API Testing Test Doubles Perf Testing UI Automation
  • 3.
    3 @ptrthomas | @KarateDSL ThoughtWorksTech Radar April 2019: “Assess” May 2020: “Trial”
  • 4.
    4 @ptrthomas | @KarateDSL Evolutionof Web User Experiences 4
  • 5.
    5 @ptrthomas | @KarateDSL AJAX! https://thehistoryoftheweb.com/what-does-ajax-even-stand-for/
  • 6.
  • 7.
  • 8.
    8 @ptrthomas | @KarateDSL Scenario:create and retrieve a cat Given url 'http://myhost.com/v1/cats' And request { name: 'Billie' } When method post Then status 201 And match response == { id: '#notnull', name: 'Billie' } Given path response.id When method get Then status 200 Hello World Intuitive DSL for HTTP Payload assertion in one line Second HTTP call using response data JSON is ‘native’ to the syntax
  • 9.
  • 10.
    10 @ptrthomas | @KarateDSL PerformanceTesting The Old Way With Karate Write Functional Tests Write Performance Tests Tool A Tool B Execute Execute Write Functional Tests Execute Load Model re-use Execute delegate
  • 11.
    11 @ptrthomas | @KarateDSL loadany Java class in one line re-usable JS function to “glue” to Java method value appears in HTML report / title supports all JSON data-types e.g. string, number. boolean, null, even in-line JSON ! Data Driven / Java Interop
  • 12.
  • 13.
    13 @ptrthomas | @KarateDSL HTTP/ WebSocket Client Mock Servlet Perf Testing Web Browser Automation Desktop App Automation API Test Doubles CLI Java API VS Code Extension Docker Image Data Transforms / Assertions JSON, XML, YAML, CSV API Testing JUnit Support HTTP Server Parallel / Distributed Executor Mock HTTP Client Native Interop Microsoft UI Automation Adapter Gatling Adapter Chrome / DevTools Protocol WebDriver Adapter Playwright Adapter JS Engine / Java Interop Process Builder / Async HTML Templating Image Processing OCR Reports / Output / Logging IDE Debug Adapter Protocol Gherkin / Cucumber Compat. Cucumber IDE Support Standalone JAR Maven Artifacts Custom / 3rd Party Reports Appium Support Cross Browser Support Run time core optional Karate Capability Map Tags, Life-Cycle, Hooks Config. / Env. Switching Custom Hooks / Code
  • 14.
  • 15.
    15 @ptrthomas | @KarateDSL Comparisonwith REST-assured http://tinyurl.com/karatera @Test public void lotto_resource_returns_200_with_expected_id_and_winners() { when(). get("/lotto/{id}", 5). then(). statusCode(200). body("lotto.lottoId", equalTo(5), "lotto.winners.winnerId", containsOnly(23, 54)); Scenario: lotto resource returns 200 with expected id and winners Given path ‘lotto’, 5 When method get Then status 200 And match $.lotto.lottoId == 5 And match $.lotto.winners[*].winnerId contains only [23, 54] given(). param("key1", "value1"). param("key2", "value2"). when(). get("/somewhere"). then(). body(containsString("OK")); Given param key1 = ‘value1’ And param key2 = ‘value2’ And path ‘somewhere’ When method get Then response contains ‘OK’ Detailed Comparison Capability REST-assured Karate No Java coding / compilation required ❌ ✅ Parallel Execution ? (partial) ✅ Data Driven Testing ❌ (needs TestNG etc.) ✅ Config / Environment Switching ❌ ✅ Match entire nested payload in one step. “Fuzzy Matching” / “Deep Equals” ❌ (partially solved with Hamcrest etc.) ✅ Mutate given JSON / response object ❌ ✅ WebSocket / Async Support ❌ ✅ HTTP Mocks / Test-Doubles ❌ (needs Wiremock etc.) ✅ Re-use as Performance / Load Tests ❌ ✅
  • 16.
    16 @ptrthomas | @KarateDSL Metrics– port of an open-source example Reduction of lines of code from 431 à 67 Link: https://github.com/mwinteringham/api-framework/pull/3
  • 17.
    17 @ptrthomas | @KarateDSL The“missing” Cucumber Features that Karate adds Capability Cucumber Karate Step Definitions built-in, no Java code needed ❌ ✅ Re-Use Feature files from other Features ❌ ✅ Dynamic Data-Driven Testing / e.g. CSV files ❌ ✅ Parallel Test Execution and Reporting ? ✅ Option to run routines only once per Feature ❌ ✅ Embedded JS Engine, Config / Env. Switching ❌ ✅ https://stackoverflow.com/questions/47795762/in- karate-how-we-can-collaboratively-work-along-with- ba-to-automate-business-sce
  • 18.