A step by step guide to creating a DSL to test rest web services.
I presented this talk to the Scala Days in Berlin the 18th of June 2014
http://www.scaladays.org/#schedule/RESTTest--exploring-DSL-design-in-Scala
Watch this presentation online
https://www.parleys.com/play/53a7d2d1e4b0543940d9e56c
2. Who am I?
Iain Hull
Email: iain.hull@workday.com
Twitter: @IainHull
Web: http://IainHull.github.io
Workday
3. HTTP Clients … What’s the problem?
http://www.hdwpapers.com/pretty_face_of_boxer_dog_wallpaper_hd-wallpapers.html
4. Simple HTTP Client
case class Request(
method: Method,
url: URI,
headers: Map[String, List[String]],
body: Option[String])
case class Response(
statusCode: Int,
headers: Map[String, List[String]],
body: Option[String])
type HttpClient = Request => Response
5. Simple HTTP Client
case class Request(
method: Method,
url: URI,
headers: Map[String, List[String]],
body: Option[String])
case class Response(
statusCode: Int,
headers: Map[String, List[String]],
body: Option[String])
type HttpClient = Request => Response
6. Simple HTTP Client
case class Request(
method: Method,
url: URI,
headers: Map[String, List[String]],
body: Option[String])
case class Response(
statusCode: Int,
headers: Map[String, List[String]],
body: Option[String])
type HttpClient = Request => Response
7. Simple HTTP Client
case class Request(
method: Method,
url: URI,
headers: Map[String, List[String]],
body: Option[String])
case class Response(
statusCode: Int,
headers: Map[String, List[String]],
body: Option[String])
type HttpClient = Request => Response
8. Using the Client
val request = Request(
GET,
new URI("http://api.rest.org/person",
Map(), None))
val response = httpClient(request)
assert(response.code1 === Status.OK)
assert(jsonAsList[Person](response.body.get) ===
EmptyList)
9. Using the Client
val request = Request(
GET,
new URI("http://api.rest.org/person",
Map(), None))
val response = httpClient(request)
assert(response.code === Status.OK)
assert(jsonAsList[Person](response.body.get) ===
EmptyList)
10. Using the Client
val request = Request(
GET,
new URI("http://api.rest.org/person",
Map(), None))
val response = httpClient(request)
assert(response.code === Status.OK)
assert(jsonAsList[Person](response.body.get) ===
EmptyList)
11. Using the Client
val request = Request(
GET,
new URI("http://api.rest.org/person",
Map(), None))
val response = httpClient(request)
assert(response.code === Status.OK)
assert(jsonAsList[Person](response.body.get) ===
EmptyList)
12. Sample API
GET /person List all the persons
POST /person Create a new person
GET /person/{id} Retrieve a person
DELETE /person/{id} Delete a person
14. Create a new person
GET /person Verify list is empty
POST /person Verify it succeeds and returns an id
15. Retrieve the person
GET /person Verify list is empty
POST /person Verify it succeeds and returns an id
GET /person/{id} Verify details are the same
16. Retrieve the list
GET /person Verify list is empty
POST /person Verify it succeeds and returns an id
GET /person/{id} Verify details are the same
GET /person Verify list contains the person
17. Delete the person
GET /person Verify list is empty
POST /person Verify it succeeds and returns an id
GET /person/{id} Verify details are the same
GET /person Verify list contains the person
DELETE /person/{id} Verify the status code
18. Retrieve the list
GET /person Verify list is empty
POST /person Verify it succeeds and returns an id
GET /person/{id} Verify details are the same
GET /person Verify list contains the person
DELETE /person/{id} Verify the status code
GET /person Verify list is empty
19. val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
20. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
21. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
22. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
23. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
24. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
25. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
assert(r1.statusCode === Status.OK)
r1.body match {
Some(body) =>
assert(jsonAsList[Person](body) === EmptyList)
None => fail("Expected a body"))
}
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
assert(r2.statusCode === Status.Created)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
assert(r3.statusCode === Status.OK)
r3.body match {
Some(body) =>
assert(jsonAs[Person](body) === Jason)
None => fail("Expected a body"))
}
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
assert(r4.statusCode === Status.OK)
r4.body match {
Some(body) =>
assert(jsonAsList[Person](body) === Seq(Jason))
None => fail("Expected a body"))
}
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
assert(r5.statusCode === Status.OK)
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
assert(r6.statusCode === Status.OK)
r6.body match {
Some(body) =>
assert(jsonAsList[Person](body) === EmptyList)
None => fail("Expected a body"))
}
35. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
36. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
37. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
38. val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r2 = httpClient(Request(
POST, new URI("http://api.rest.org/person/"),
Map(), Some(personJson)))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(Request(
GET, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r4 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
val r5 = httpClient(Request(
DELETE, new URI("http://api.rest.org/person/" + id),
Map(), None))
val r6 = httpClient(Request(
GET, new URI("http://api.rest.org/person/"),
Map(), None))
39.
40. Applying the builder
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET).toRequest)
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson).toRequest)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id)
.toRequest)
val r4 = httpClient(rb.withMethod(GET).toRequest)
val r5 = httpClient(rb.withMethod(DELETE).addPath(id)
.toRequest)
val r6 = httpClient(rb.withMethod(GET).toRequest)
41. Applying the builder
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET).toRequest)
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson).toRequest)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id)
.toRequest)
val r4 = httpClient(rb.withMethod(GET).toRequest)
val r5 = httpClient(rb.withMethod(DELETE).addPath(id)
.toRequest)
val r6 = httpClient(rb.withMethod(GET).toRequest)
42. Applying the builder
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET).toRequest)
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson).toRequest)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id)
.toRequest)
val r4 = httpClient(rb.withMethod(GET).toRequest)
val r5 = httpClient(rb.withMethod(DELETE).addPath(id)
.toRequest)
val r6 = httpClient(rb.withMethod(GET).toRequest)
43. Applying the builder
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET).toRequest)
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson).toRequest)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id)
.toRequest)
val r4 = httpClient(rb.withMethod(GET).toRequest)
val r5 = httpClient(rb.withMethod(DELETE).addPath(id)
.toRequest)
val r6 = httpClient(rb.withMethod(GET).toRequest)
44. Implicit Conversion
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET).toRequest)
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson).toRequest)
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id)
.toRequest)
val r4 = httpClient(rb.withMethod(GET).toRequest)
val r5 = httpClient(rb.withMethod(DELETE).addPath(id)
.toRequest)
val r6 = httpClient(rb.withMethod(GET).toRequest)
implicit def toRequest(b: RequestBuilder): Request =
b.toRequest
45. Implicit Conversion
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/"
val r1 = httpClient(rb.withMethod(GET))
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id))
val r4 = httpClient(rb.withMethod(GET))
val r5 = httpClient(rb.withMethod(DELETE).addPath(id))
val r6 = httpClient(rb.withMethod(GET))
implicit def toRequest(b: RequestBuilder): Request =
b.toRequest
46. Implicit Conversion
val personJson = """{ "name": "Jason" }"""
val rb = RequestBuilder()
.withUrl("http://api.rest.org/person/")
val r1 = httpClient(rb.withMethod(GET))
val r2 = httpClient(rb.withMethod(POST)
.withBody(personJson))
val id = r2.headers("X-Person-Id").head
val r3 = httpClient(rb.withMethod(GET).addPath(id))
val r4 = httpClient(rb.withMethod(GET))
val r5 = httpClient(rb.withMethod(DELETE).addPath(id))
val r6 = httpClient(rb.withMethod(GET))
48. Narrate your code
• Get from url http://api.rest.org/person/
• Post personJson to url http://api.rest.org/person/
• Get from url http://api.rest.org/person/personId
• Delete from url http://api.rest.org/person/personId
49. Narrate your code
• Get from url http://api.rest.org/person/
• Post personJson to url http://api.rest.org/person/
• Get from url http://api.rest.org/person/personId
• Delete from url http://api.rest.org/person/personId
57. Initial DSL
val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(
GET withUrl "http://api.rest.org/person/")
val r2 = httpClient(
POST withUrl "http://api.rest.org/person/"
withBody personJson)
val id = r2.headers.get("X-Person-Id").get.head
val r3 = httpClient(
GET withUrl "http://api.rest.org/person/" addPath id)
val r4 = httpClient(
GET withUrl "http://api.rest.org/person/")
val r5 = httpClient(
DELETE withUrl "http://api.rest.org/person/" addPath id)
val r6 = httpClient(
GET withUrl "http://api.rest.org/person/")
58. Initial DSL
val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(
GET withUrl "http://api.rest.org/person/")
val r2 = httpClient(
POST withUrl "http://api.rest.org/person/"
withBody personJson)
val id = r2.headers.get("X-Person-Id").get.head
val r3 = httpClient(
GET withUrl "http://api.rest.org/person/" addPath id)
val r4 = httpClient(
GET withUrl "http://api.rest.org/person/")
val r5 = httpClient(
DELETE withUrl "http://api.rest.org/person/" addPath id)
val r6 = httpClient(
GET withUrl "http://api.rest.org/person/")
59. Initial DSL
val personJson = """{ "name": "Jason" }"""
val r1 = httpClient(GET withUrl "http://api.rest.org/person/")
val r2 = httpClient(POST withUrl "http://api.rest.org/person/"
withBody personJson)
val id = r2.headers.get("X-Person-Id").get.head
val r3 = httpClient(GET withUrl "http://api.rest.org/person/" addPath id)
val r4 = httpClient(GET withUrl "http://api.rest.org/person/")
val r5 = httpClient(DELETE withUrl "http://api.rest.org/person/"
addPath id)
implicit class RichRequestBuilder(builder: RequestBuilder) {
def execute()(implicit httpClient: HttpClient): Response = {
httpClient(builder)
}
}
60. Initial DSL
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
val r1 = GET withUrl "http://api.rest.org/person/" execute ()
val r2 = POST withUrl "http://api.rest.org/person/" withBody personJson
execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET withUrl "http://api.rest.org/person/" addPath id execute ()
val r4 = GET withUrl "http://api.rest.org/person/" execute ()
val r5 = DELETE withUrl "http://api.rest.org/person/" addPath id execute ()
val r6 = GET withUrl "http://api.rest.org/person/" execute ()
implicit class RichRequestBuilder(builder: RequestBuilder) {
def execute()(implicit httpClient: HttpClient): Response = {
httpClient(builder)
}
}
61. Initial DSL
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
val r1 = GET withUrl "http://api.rest.org/person/" execute ()
val r2 = POST withUrl "http://api.rest.org/person/" withBody personJson
execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET withUrl "http://api.rest.org/person/" addPath id execute ()
val r4 = GET withUrl "http://api.rest.org/person/" execute ()
val r5 = DELETE withUrl "http://api.rest.org/person/" addPath id execute ()
val r6 = GET withUrl "http://api.rest.org/person/" execute ()
implicit class RichRequestBuilder(builder: RequestBuilder) {
def execute()(implicit httpClient: HttpClient): Response = {
httpClient(builder)
}
}
62. Initial DSL
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
val r1 = GET withUrl "http://api.rest.org/person/" execute ()
val r2 = POST withUrl "http://api.rest.org/person/" withBody personJson
execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET withUrl "http://api.rest.org/person/" addPath id execute ()
val r4 = GET withUrl "http://api.rest.org/person/" execute ()
val r5 = DELETE withUrl "http://api.rest.org/person/" addPath id execute ()
val r6 = GET withUrl "http://api.rest.org/person/" execute ()
implicit class RichRequestBuilder(builder: RequestBuilder) {
def execute()(implicit httpClient: HttpClient): Response = {
httpClient(builder)
}
}
63. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }""”
val r1 = GET withUrl "http://api.rest.org/person/" execute ()
val r2 = POST withUrl "http://api.rest.org/person/" withBody personJson
execute ()
val id = r2.headers.get("X-Person-Id”)
val r3 = GET withUrl "http://api.rest.org/person/" addPath id execute ()
val r4 = GET withUrl "http://api.rest.org/person/" execute ()
val r5 = DELETE withUrl "http://api.rest.org/person/" addPath id execute ()
val r6 = GET withUrl "http://api.rest.org/person/" execute ()
64. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
val r1 = GET withUrl "http://api.rest.org/person/" execute ()
val r2 = POST withUrl "http://api.rest.org/person/" withBody personJson
execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET withUrl "http://api.rest.org/person/" addPath id execute ()
val r4 = GET withUrl "http://api.rest.org/person/" execute ()
val r5 = DELETE withUrl "http://api.rest.org/person/" addPath id execute ()
val r6 = GET withUrl "http://api.rest.org/person/" execute ()
implicit def methodToRequestBuilder(m: Method): RequestBuilder
= RequestBuilder().withMethod(m)
65. Common Configuration
implicit val httpClient = ...
implicit val rb = RequestBuilder() withUrl "http://api.rest.org/person/"
val personJson = """{ "name": "Jason" }"""
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
implicit def methodToRequestBuilder(m: Method)
(implicit builder: RequestBuilder): RequestBuilder
= builder.withMethod(m)
66. Common Configuration
implicit val httpClient = ...
implicit val rb = RequestBuilder() withUrl "http://api.rest.org/person/"
val personJson = """{ "name": "Jason" }"""
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
67. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
68. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
def using(config: RequestBuilder => RequestBuilder)
(process: RequestBuilder => Unit)
(implicit builder: RequestBuilder): Unit = {
process(config(builder))
}
69. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
def using(config: RequestBuilder => RequestBuilder)
(process: RequestBuilder => Unit)
(implicit builder: RequestBuilder): Unit = {
process(config(builder))
}
70. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
def using(config: RequestBuilder => RequestBuilder)
(process: RequestBuilder => Unit)
(implicit builder: RequestBuilder): Unit = {
process(config(builder))
}
71. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
def using(config: RequestBuilder => RequestBuilder)
(process: RequestBuilder => Unit)
(implicit builder: RequestBuilder): Unit = {
process(config(builder))
}
73. Common Configuration
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ withUrl "http://api.rest.org/person") { implicit rb =>
val r1 = GET execute ()
val r2 = POST withBody personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET addPath id execute ()
val r4 = GET execute ()
val r5 = DELETE addPath id execute ()
val r6 = GET execute ()
}
79. A Spoonful of sugar
implicit val httpClient = ...
val personJson = """{ "name": "Jason" }"""
using(_ url "http://api.rest.org") { implicit rb =>
val r1 = GET / "person" execute ()
val r2 = POST / "person" body personJson execute ()
val id = r2.headers.get("X-Person-Id").get.head
val r3 = GET / "person" / id execute ()
val r4 = GET / "person" execute ()
val r5 = DELETE / "person" / id execute ()
val r6 = GET / "person" :? ('page -> 2, 'per_page -> 100)
execute ()
}
80. Extracting values
val r1 = GET url "http://api.rest.org/person/" execute ()
val code1 = r1.statusCode
val list1 = jsonToList[Person](Json.parse(r1.body), JsPath)
81. Extracting values
val r1 = GET url "http://api.rest.org/person/" execute ()
val code1 = r1.statusCode
val list1 = jsonToList[Person](Json.parse(r1.body), JsPath)
Get from url http://api.rest.org/person/ returning the
status code and the body as list of persons
82. Extracting values
val r1 = GET url "http://api.rest.org/person" execute ()
val code1 = r1.statusCode
val list1 = jsonToValue[Person](Json.parse(r1.body), JsPath)
val code1 = GET url "http://api.rest.org/person"
returning (StatusCode)
83. Extracting values
val r1 = GET url "http://api.rest.org/person" execute ()
val code1 = r1.statusCode
val list1 = jsonToValue[Person](Json.parse(r1.body), JsPath)
val (code1, list1) = GET url "http://api.rest.org/person"
returning (StatusCode, BodyAsPersonList)
105. Asserting values
val (code1, list1) = GET / "person" returning (
StatusCode, BodyAsPersonList)
assert (code1 === Status.OK)
assert (list1 === EmptyList)
Get from url http://api.rest.org/person/ asserting the
status code is OK and and the body is an empty list of
persons
123. Code is Communication
using(_ url "http://api.rest.org") { implicit rb =>
GET / "person" asserting (StatusCode === Status.OK,
BodyAsPersonList === EmptyList)
val id = POST / "person" body personJson
asserting (StatusCode === Status.Created)
returning (header("X-Person-Id"))
GET / "person" / id asserting (StatusCode === Status.OK,
BodyAsPersonList === Jason)
GET asserting (StatusCode === Status.OK,
BodyAsPersonList === Seq(Jason))
DELETE / "person" / id asserting (StatusCode === Status.OK)
GET / "person" asserting (StatusCode === Status.OK,
BodyAsPersonList === EmptyList)
}
124. DSL Resources - Books
DSLs in Action
By Debasish Ghosh
ISBN: 9781935182450
http://www.manning.com/ghosh/
Scala in Depth
By Joshua D. Suereth
ISBN: 9781935182702
http://www.manning.com/suereth/
The problem
Narrate your you code
DSL should just orchestrate an existing API
Understand your context
Documentation and Tests
We are going to look at testing REST web services – How a DSL can help
Not demo of a new open source library – using it explore DSL design
But there is one question
Standard Http Clients
Fine for business logic – Boxing and unboxing data
Talking to other systems like databases or other web services
Not REST specific – REST system tests are very verbose
Lets look at…
This could be: Jersey, Dispatch, Rapture
What does a system test using this client look like?
How do test written in this style scale?
Lets look at simple use case…
Swagger
What would a simple system test look like?
Now lets look at the code
This is just the first request there are no code to test the response
Now we verify the results
I don’t know about you but I’m like…
And I wrote it
So the question is why?
What is wrong with this??
Random pie chart from the internet
We all know that we read code more than we write it, so
But what is wrong with the code?
Boiler plate code
Hides the intent of your code
But its harder to read code than write it
Copy and Paste – you could write that test in 15-20 mins
I program in Scala and expect to eat cake
Worked on services whose system tests have grown to 1500 lines
New to the team – small modification – read and understand all the test code?
What can we do?
Mini programming language
Specific problem domain
Standalone
Embedded in general purpose language
Thinly Embedded or deeply embedded
Expresses the solution in the vocabulary of the domain
Highlights the intent
Reduces the boiler plate code
We are gong to look at two approaches for designing a DSL
How does they improve the intent of the code
And reduce the boiler plate
Scala has a lot of features and techniques to enable DSL construction
Going to show some of these techniques and how they are assembled?
(Not how those techniques work)
Our test code created 6 http requests and validated their responses, here are the requests
Lets use this code to design our DSL
Start by – Calling out any code obscuring the intent
Executing the request
Positional parameters and unused parameters
Repeating prefix
These are the boiler plate we want our DSL to address
The most popular tool for addressing these issues is …
The builder pattern.
Create a default builder with repeating properties
Override this default and execute the request
Little easier to write and maybe easier to read
But I think we can do better …
.. I want to point out
The calls to toRequest are don’t add anything to our understanding
If answer is NO, we should remove them, but how?
Create an implicit conversion
We are going to use implicits three ways today
* converting one type to another – implicit views
* add a new method to an existing types – pimp my library
* pass smart default parameters into methods
Now when we remove the toRequest method call
The compiler sees a Request where is expects a RequestBuilder
And our implicit conversion is in scope
And calls our conversion automatically
* Implicit conversions can reduces typing and boilerplate, and help expose the intent of code
* Easier to read than write – must be discoverable – like worm holes to people reading your code
Not here to learn the builder pattern
The first and MOST IMPORTANT step – Writing the code in boilerplate
Narrate the code as if explaining it to a colleague – now look for patterns
** Remember, Good design is just discovering the design that is already present in problem
Notice that each sentence begins with the HTTP Method
Make these the starting point to our DSL
Starting with a single request
How can we remove the highlighted code to bring the method to the front?
Adding an implicit function converting http methods to RequestBuilders
Explain function
Now the HTTP method is closer to the start of our expression
We can start all requests by calling a RequestBuilder operation on the appropriate Method object
Now when we call a withUrl on the GET object the compiler expands the code tothe bubble
This punctuation does not improve the intent of our code
Infix notation
Now lets look at the complete use case
When narrating our code
Do you execute the request before or after you describe it?
Pimp my library pattern
Create a new type to add the execute method to RequestBuilder
The httpClient parameter is marked implicit
The empty parameter list is required to terminate the expression
It also highlights the side effects
Can we pass this implicitly?
Create an implicit RequestBuilder with the common url
Add an implicit RequestBuilder parameter to the function that converts methods to RequestBuilders
Now the common configuration is used to bootstrap each expression
Nice but, no visual connection
No where to document this feature
We can solve both of …
… these problems with a new method which applies the configuration to a block of code
I would like like to call the function“with” but that’s a key word
The using function takes 3 parameters
The config parameter applies the common configuration
This is simply a function that takes a takes a RequestBuilder and returns a new RequestBuilder with the configuration applied
This function is easily constructed with scala’s underscore notation
The process parameter is the code bock which is just a function taking a RequetBuilder and returning Unit
Finally the implicit builder parameter allows using statements to be nested
However where does the first implicit RequestBuilder come from?
We also modify the RequestBuilder companion object, making the emptyBuilder available implicitly, when there is not other builder defined implicitly.
The using function improves intent
The code block provides visual connection
Avoid pollution of implicit values
Finally it provides a name for this feature - documentation
Our DSL is starting to take shape
Remember the RichRequestBuilder? Where we defined our execute method?
Shorten the builder method names
Use slash to concatenate url paths
Question mark to add query parameters
Colon is required to fiddle Scala operator precedents
Now the intent of our requests is clear
** Approach rooted in the implantation – working at syntax level – have to be careful - personal specific language**
What about the response?
Starting with the first request – what do we do with the response
Lets narrate it….
Now lets just make up some code to express this
Narrating the code has exposed a new domain concept – so what are StatusCode and BodyAsPersonList??
These are Extractors – we haven’t decided on an impl
Initially simple functions – then more powerful objects
What do they look like?
ExtractorLike specifies the core operations of an Extractor
Extractor is the basic implementation of the trait.
It takes a function from Response to any type A
And implements the value method, with error handling.
AndThen enables to be composed of other functions
As renames composed extractors
Now we can define some Extractors
An Extractor has a name and a function to perform the extraction
Here “r” is the response and it returns the statusCode
Calling get on an Option is not safe
However it is ok here because its context
Header is a method to generate extractors for specified header
Of course the standard headers are defined as constants.
rfc2616
jsonBodyAs
generates an extractor that converts a json document
Or part there of
To any object
Overloading to provide default parameters – view bounds
How does our DSL use these?
The returning method executes the Request in a RequestBuilder
And extracts a result from the response
An overloaded returning function takes a pair of extractors and returns their values in a pair
There are versions of this method for 3 and 4 member tuples as well.
Now instead of just returning a response
Our DSL extracts values we can use
But what are these values for?
Lets narrate this code
Again we can make up some code to express this
So StatusCode is an extractors – but what is StatusCode is Status.OK
We have discovered a new domain concept: an assertion
But what is it?
So what does an Assertion looking
Is simply a function that takes a Response and returns an optional failure message
Now we need a way to construct them
This is the pimp my library pattern again
RichExtractor adds the is method to extractors to convert them to assertions
Also support the not equals and comparison operators
Interesting – these operations do not perform the assertion – they create a function that performs the assertion later.
We need something to execute these assertions
Asserting first executes the request to retrieve the response.
Then evaluates a sequence of Assertions looking for failures
If any fail the assertionFailues sequence will be nonEmpty
Now lets take some time to read the results
Lets compare this with the original test
We changed our design methodology to discover Extractors and Assertions
** Second approach rooted in the domain – relies more on narration – and invented syntax then though of implementation **
We have designed our DSL – Why is it important
Samples
User guide
Reference documentation
DSLs are complicated not mix them with business logic
Implement the business logic in a generic OO or functional API
The DSL should only orchestrate this API – Simple as possible / Testing / Code reuse
I also kept the extractors separate to the DSL,
However the DSL includes the vanilla HTTP Extractors
This means custom extractors are orthogonal to the DSL
Json and XML extractors are not part of the code DSL
Other extractors can be defined, similarly
This enables others to extend the DSL
maybe adding support for Oauth and Json
Covers our structure
Why is smaller code better??
Why is it important that the intent is easily to follow?
52 v 16
Why is smaller code better??
Why is it important that the intent is easily to follow?
100 lines – so break even after 3 times
Code is communication
The next person to work on your service
Your testing team
Your Product manager or customer
QA don’t check 404 after deleting the person
Architect, custom header is not very resty should use the location header
We are going to look at testing REST web services
And how a DSL can help
But there is one question