There is no doubt that Web APIs have become a fundamental aspect of modern architectures. However how to distinct a “good” API from an “evil” one is a topic for debate and one that often ends up in a “religious” debate. However, the one thing that is clear to most people, is that it all starts with a design, and getting that design right up-front will likely result in less headaches down the line.
So how to avoid “evil” APIs?
In this presentation, I will talk about the 7 most common pitfalls I’ve come across when doing API design in large implementations. With real-life examples, I will describe why such pitfalls deserved to be called “evil” and how to remediate them with “good" design practices.
The presentation will also touch upon how API-design first techniques can be applied to solve prevent some of the common issues.
It will be a highly interactive session with good balance of theory and practice.
Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...
7 deadly sins of API at DevoxxUK 2019
1. 1THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
DEADLY SINS
OF {API} DESIGN
THE
2. 2THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
LUIS WEIR
Director Software Development
Oracle Hospitality
luis.weir@oracle.com
uk.linkedin.com/in/lweir
@luisw19
http://www.soa4u.co.uk
tinyurl.com/eapim19
July 2019
apiplatform.cloud/
May 2018
ABOUT ME @luisw19
3. 3THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
This is an opinionated presentation
expressing my own views.
4. 4THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
APIs are doors to information and functionality.
5. 5THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
But if you get the design and implementation
wrong…
https://www.boredpanda.com/funny-architecture-construction-fails/?utm_source=google&utm_medium=organic&utm_campaign=organic
6. 6THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
So avoid common API-design pitfalls by not
committing any of the 7 Deadly Sins
1- LUST
2- GLUTTONY
3- GREED
4- SLOTH
5- WRATH
6- ENVY
7- PRIDE
7. 7THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
So avoid common API-design pitfalls by not
committing git commit(ing) any of these sins
1- LUST
2- GLUTTONY
3- GREED
4- SLOTH
5- WRATH
6- ENVY
7- PRIDE
8. 8THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
1- LUST
Unrestrained
desire for
something
Try
this
tool
9. 9THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
1- LUST: The Perfectly Useless API
Focusing mainly on runtime,
implementation tech, CICD
(e.g. K8s, APIG, APIMGW,
Mesh, Pipelines, etc) and
forgetting about the API
usability.
The motivations:
Typically:
• Fashion,
• CV building,
• Comfort zones,
• Even office politics?
10. 10THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
1- LUST: Deliver us from evil
API design first, implementation
second. It doesn’t take fancy
tools or complex steps to do
this:
1. First capture what the needs
really are in a design,
2. Quickly mock API endpoints,
share, get feedback,
3. Once design is complete,
ensure implementation
consistency by testing design
against service.
Design Build &
Contract Test
Try
01
• API docs and mock
endpoints available
in the Dev Portal
• API consumers try the
API and give feedback
02 03
Feedback-loops
inspector
OpenAPI-to-Postman
11. 11THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
2- GLUTTONY
The over-
indulge
specially by
over eating
12. 12THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
2- GLUTTONY: The API Sandwich Architecture
Purposely adding layers on top of
layers of API middleware unnecessarily
adding complexity and costs without
clear business benefits. Backend for
Frontend
API Consumer
Service
The motivations:
There are many but commonly:
• In the name of abstraction and
decoupling??,
• Unawareness of, or avoidance to use,
existing API infrastructure,
• Solving a network problem (e.g. geo-
routing),
• Other sins? e.g. envy, pride?.
LB
Internal API
Gateway
API Micro-
Gateway
LB
LB
LB
13. 13THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
2- GLUTTONY: Deliver us from evil
Remove unnecessary layers by:
1. Seriously questioning what is
the unique value each layer
adds,
2. Adopt domain-driven design
principles. Get the bounded
contexts right,
3. Consider a service mesh instead
more layers of API Gateways (or
event sourcing),
4. Checkout API Gateway Pattern.
API Consuming Applications
Customers
Service
Orders
Service
Logistics
Service
Service
Registry
Frontend
Specific
Service
API Gateway
Backend
Frontends
discovery
discovery
14. 14THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
3- GREED
Intense and
selfish desire
for something
Derek Michael Brennan
15. 15THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
3- GREED: The Chatty API
Abusive use of network resources
resulting in bad API consumption
experience:
1. A client application has to make
several API calls just to build up
a single page,
2. HTTP GET as a strategy to do batch
data downloads.
The motivations:
• HATEOAS to the extreme…
• Extremely granular resource designs
• Batch downloads via paginated payloads
APIGateway
++ bandwidth = $$ & << UX
16. 16THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
3- GREED: Deliver us from evil
Optimise number of API calls by
adopting a different approach:
1. Webhooks as means to
implement web events (push
not pull –only send data
changes),
2. Implement an API Composition
pattern (e.g. consider
GraphQL?).
APIGateway
EventHub
APIGateway
API Composition
Webhooks
subscribe
Web events
Composition
Service
17. 17THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
4- SLOTH Laziness, lack
of effort
18. 18THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
4- SLOTH: REST In Peace (RIPfull)
Ignoring REST architectural principles. E.g.:
• Operations in URIs:
[GET] https://eg.com/getAllorders
[GET] https://eg.com/getOrder?id=xxx
[POST] https://eg.com/newOrder
[PUT|DELETE] https://eg.com/updateOrder?id=xxx
• Unbounded responses?
[GET] https://eg.com/orders so, send back 1m orders?
• Use of both singulars and plurals in a single API
[GET] https://eg.com/orders
[GET] https://eg.com/order/1234
• Ignoring HTTP status codes
[POST] https://eg.com/orders
{…}
Response:
HTTP 200 Ok Really?
19. 19THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
4- SLOTH: REST In Peace (RIPfull)..continues
• Go figure out how to find a related
resource
[GET] https://eg.com/orders
[orders{
order:{
"id":"or001",
customer: {
"id":"cust123" where is it?
}
},
…
}]
The motivations:
Still a mystery to me… but perhaps:
• Using SOAP to REST convertors?
• Lack of understanding of REST?
• Or simply SLOTH…
20. 20THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
4- SLOTH: Deliver us from evil
Lots of content online for best practices..
So no excuse Homer. In any case:
• Use HTTP verbs
Search, Create: [GET|POST] /orders?{search filters}
Read, Update, Partly update, Remove:
[GET|PUT|PATCH|DELETE] /orders/{id}
• Plural nouns are best (see above)
• Use limits/offsets to constraint response
/orders?limit=50&offset=150&{additional filters}
• Respect HTTP status codes e.g.
2xx – Success, 4xx – Client errors, 5xx – Server errors. check this link
• HATEOAS (but not to the extream). Check Richardson’s maturity model
• Idempotency on methods: [GET|PUT|PATCH|DELETE]
21. 21THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
5- WRATH
Uncontrolled
feelings of
hatred and
anger
22. 22THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
5- WRATH: No (documented?) API
What can be worse than not having
an API? Well, an undocumented API
or even worse, a poorly documented
API…
The motivations:
Most likely:
• An accidental API (those that
weren’t meant to used but sort
of did),
• Time pressures (deadlines),
• Other sins .. Sloth
API Doc
Angry developer Is this
supposed to
be an API
Doc?
23. 23THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
5- WRATH: Deliver us from evil
An API is only as good
as its doc.
The ABC of a Good API:
• Absent API doc =
No API,
• Bad documentation =
Angry Developers,
• Can’t find it =
re-inventing wells
(duplicate APIs)
A good API Doc should include
API Page
(e.g. doc.myapi.com)
Nav: Get Started
Intro
Authentication
Errors
Constraints
Change Log
Resource Name
Nav: Resources
Resource Name
Search
Create
Read
Update
Delete
Curl JS | Java |..
API Mock URL
Sample Requests
URI / Params
description
Sample Responses
Description of
resource. What
is it? What
does it do?
GET /samples?{}
Description of
the endpoint.
Search
POST /samples
Description of
the endpoint.
Create
Resource Name
24. 24THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
5- WRATH: Deliver us from evil
An API is only as good
as its doc.
The ABC of a Good API:
• Absent API doc =
No API,
• Bad documentation =
Angry Developers,
• Can’t find it =
re-inventing wells
(duplicate APIs)
https://ordersms.docs.apiary.io/#
25. 25THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
5- WRATH: Recommend
Good examples of API docs:
• Stripe API: https://stripe.com/docs/api
• Strava API: https://developers.strava.com/docs/reference/
Some good tools:
• Apiary.io: supports OAS, API Blueprint, rich documentation,
Mocking server, spec testing with Dredd,
• Snowboard & Aglio: really nice API blueprint
parsers/renderers.
26. 26THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
6- ENVY
Jealousy
towards
another's
happiness
27. 27THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
6- ENVY: The Disloyal API Designer
So you’re a REST fan. Considering GraphQL as an
alternative it’s just unconceivable. Even if UI
developers favour it.
The motivations:
Multiple valid reasons:
• Existing investment in REST
APIs,
• Current skills in the
organisation,
• Reluctance to change?,
• Other since: Pride?
https://imgur.com/a/J3ttg
{REST}
28. 28THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
Let’s put it into perspective
https://trends.google.com/trends/explore?date=2004-01-10%202019-05-09&q=GraphQL,REST%20API,OData,WSDL
trend
WSDLGraphQL REST API OData
Feb 2004 May 2019
29. 29THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
6- ENVY: Deliver us from evil
GraphQL IS NOT a silver bullet. But it has some
great features:
• Best for API usability (specially with
GraphiQl, get only the data you want),
• Best for API composition pattern,
• It can perfectly co-exist with REST. REST REST REST
API
GraphQL
Define Schema
type Country {
id: ID!
name: String!
code: String!
}
type query {
countries:
[Country]
}
GraphQL Service GraphQL Client
Quickly write and
run queries
{
getCountries(name:"great")
{
name
}
}
GraphQL Client
Get exactly what
you asked for
{
"data": {
"countries": [
{
"name": "United
Kingdom"
}
]
}
}
30. 30THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
6- ENVY: Recommend
Browser
GraphiQL
{
query
}
GraphQL Server:
Apollo/Express
GraphQL Service
Graphiql
GraphQL Schema
GraphQL Endpoint
Query Operation {JSON}
[HTTP/POST]
{JSON}
{
data
}
[HTTP/GET]
https://restcountries.eu/rest/v2/{resource}
{JSON}
REST COUNTRIES
[HTTP/POST]
https://www.google.co.uk/search?q={search}
{JSON}
• Devoxx London: GraphQL as an alternative approach to REST,
• 4 part code samples:
https://github.com/luisw19/graphql-samples
31. 31THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
THE 7 DEADLY SINS
7- PRIDE
Inflated sense
of one's
accomplishments
32. 32THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
7- PRIDE: Bottom-up API Design
Unanimously deciding what the
interface should be. No design
feedback loops.
The motivations:
Most likely:
• Easier/quicker to auto-generate
endpoints from data sources,
• Time pressures (deadlines),
• Or just Pride!
http://www.constructionhunter.com.au/blog/industry-news/20-
photos-that-will-make-you-question-your-faith-in-humanity
33. 33THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
7- PRIDE: Deliver us from evil
No better accomplishment as an
API designer/developer to create
an API that is fit for purpose
and actually used.
Avoid the picture on the right
Design your API first with
usability in mind.
https://www.pinterest.co.uk/pin/504543964485602355
https://www.boredpanda.com/funny-architecture-construction-
fails/?utm_source=google&utm_medium=organic&utm_campaign=organic
34. 34THE 7 DEADLY SINS OF {API} DESIGN @LUISW19#DevoxxUk
Questions?
A similar presentation is available at www.soa4u.co.uk :
www.soa4u.co.uk/2018/10/the-se7en-deadly-sins-of-api-design.html
Editor's Notes
The theme of this presentation is based on the movie Seven. How many of you has seem the movie?
Ok cool, so hopefully you find the next slide funny
Raise your hands if you’ve ever made a mistake (e.g. in coding, in a script, in a design, in testing?)
We all have made mistakes, as that’s human nature. We learn by trial and error. That’s how we learn to walk and speak.
Who has seen the movie seven? Don’t worry if you haven’t this is the API version of it…
The 7 deadly sins, although may refer to Christian religion, they also exist in one form or another in other religions. The 7 deadly sins are basically a “perverse” and ‘eveil” desire for something or someone. So it’s a good way to show common pitfalls in the context of API design and also how to prevent such pitalls.
Hinduism
Literally speaking, pāpam means evil, wicked, mischievous, destructive, vile, low, vice, guilt, and so on. Pa means to drink, inhale or absorb. Apa means water. However, in some constructs it also means bad, inferior, corrupt. Thus, pāpam means taking in or drinking impure water, polluted water or poison.
The word Pāpam (paap) is mostly used to denote sin in the Vedas and scriptures such as the Bhagavadgita. Its opposite (antonym) is Punyam (punya). There is no equivalent word to the latter in English, since the notion is absent in Christianity and western culture. It may loosely be translated into English as virtue or merit. Pāpam and Punyam are the twin fruits of karma (actions), which arise in conjunction with dharma and adharma. They determine the fate or destiny of beings upon earth.
Islam
Muslims see sin as anything that goes against the commands of Allah (God), a breach of the laws and norms laid down by religion.[1] Islam teaches that sin is an act and not a state of being.
Lust (Lujuria)
Use case: excessive use of network calls to just pool for changes
Solution/demo: talk about web-hooks / web events as a best way to avoid just calling for changes
Ref: https://blog.vsoftconsulting.com/blog/common-api-design-mistakes-and-how-to-fix-them
Use case: unbounded responses….
Solution: please implement pagination
Gluttony (glotonería)
Use case: 2 or 3 layer API Gateway architecture
Solution/demo: just show why is a bad idea. Show a sample of a simple architecture of UI > APIG > Service.
Use case: make the scope of an API too large and keep adding to it
Solution/demo: talk about bounded context and DDD. How to properly define the size of an API. How Apiary can help with this
Greed (Avaricia)
Use case: no API documentation or worst no API-design. Bottom up generated Swagger….
Solution/demo: how to use API-design first
Ref: good practices when documenting: https://www.programmableweb.com/news/web-api-documentation-best-practices/2010/08/12
Use case: designing APIs for machines not humans (e.g. autogenerated swagger from code)
Solution/demo: talk about API-design first
Ref: https://nordicapis.com/designing-apis-humans/
https://medium.com/api-product-management/asset-discovery-and-api-product-ideation-7f23ccde6939
Sloth (Pereza)
Use case: Ignoring the HTTP. e.g. HTTP Operations as URIs and not resource centric design e.g. /api/getcustomers, bad (or no) use of HTTP Verbs and error codes
Solution/demo: talk about best practices to design REST endpoints, verbs and HTTP response codes
Ref: https://apigee.com/about/blog/technology/restful-api-design-nouns-are-good-verbs-are-bad
https://techcrunch.com/2009/11/08/the-top-10-most-common-api-pitfalls/
https://medium.com/@schneidenbach/restful-api-best-practices-and-common-pitfalls-7a83ba3763b5
Wrath (ira)
Use case: being a strict RESTafarian and blow up in anger if an API is not L3 Richardson/Hypermedia for example..
Solution/demo: not all APIs have to be level 3. Level 2 is also ok in many cases.
Envy (Envidia)
Use case: GraphQL as an alternative to REST
Solution/demo: ???
Use case: Re-invent a well simply because you can. What if that API already exists? have you verified?
Solution/demo: show how APIs can be discovered in Apiary and API Developer Portal
Pride (orgullo)
use case: not taking feedback on an API design
solution: API-design first
use case: database driven design but it’s easier. Push complexity up stream
solution: API-design first
Use case: Stick to an architectural style (e.g. REST) because is the best in the world… even if it’s inefficient from a client point of view.
Solution/demo: talk about GraphQL and when is a good fit over REST (e.g. API Composition)
Good links:
https://www.programmableweb.com/news/top-10-api-worst-practices/2012/08/03
https://medium.com/@anthonypjshaw/8-reasons-your-api-sucks-7f6ff60ddc04
https://hackernoon.com/anti-patterns-of-web-apis-33d2a28534f2
https://dzone.com/articles/5-rookie-mistakes-that-can-destroy-your-api-strate
https://www.programmableweb.com/news/top-10-api-worst-practices/2012/08/03
https://dzone.com/articles/curated-list-best-and-worst
I really spend time in the picture so if nothing I hope you at least give me credit for the photoshop!!!
Creative commons: https://www.flickr.com/photos/hoffarth/3385758449
Source of image: http://www.constructionhunter.com.au/blog/industry-news/20-photos-that-will-make-you-question-your-faith-in-humanity/
Also known as the Capital Vices or Cardinal Sins, according to Christian tradition these are perverse or corrupt versions of love for something or someone –in this case APIs. They originate from the desert fathers, especially Evagrius Ponticus who identified seven or eight evil thoughts or spirits that one needed to overcome in order to become a better person.
The Desert Fathers (along with Desert Mothers) were early Christian hermits, ascetics, and monks who lived mainly in the Scetes desert of Egypt beginning around the third century AD.
Also known as the Capital Vices or Cardinal Sins, according to Christian tradition these are perverse or corrupt versions of love for something or someone –in this case APIs. They originate from the desert fathers, especially Evagrius Ponticus who identified seven or eight evil thoughts or spirits that one needed to overcome in order to become a better person.
The Desert Fathers (along with Desert Mothers) were early Christian hermits, ascetics, and monks who lived mainly in the Scetes desert of Egypt beginning around the third century AD.
Also known as the Capital Vices or Cardinal Sins, according to Christian tradition these are perverse or corrupt versions of love for something or someone –in this case APIs. They originate from the desert fathers, especially Evagrius Ponticus who identified seven or eight evil thoughts or spirits that one needed to overcome in order to become a better person.
The Desert Fathers (along with Desert Mothers) were early Christian hermits, ascetics, and monks who lived mainly in the Scetes desert of Egypt beginning around the third century AD.
What a consumer really wants to access is information/functionality delivered by a service. Only add layers that truly add value but not on top of each other unless it can’t be avoided. Instead focus on ensuring that the API contract is not broken.
https://zapier.com/engineering/webhook-design/
No, because it’s HTTP it doesn’t mean it’s RESTful.
https://www.infoq.com/news/2011/06/RestAPIs
No, because it’s HTTP it doesn’t mean it’s RESTful.
https://www.infoq.com/news/2011/06/RestAPIs
http://seleniumwebdriver.org/rest-api-best-practices-2-http-and-crud
Request Methods
GET and POST should not be used in place of other request methods
GET is used to retrieve a representation of a resource
HEAD is used to retrieve response headers
PUT is used to insert or update a stored resource
POST is used to create a new resource in a collection
DELETE is used to remove a resource
Response Status Codes
200 “OK” indicates general success
200 “OK” shouldn’t be used to return error messages
201 “Created” indicates a resource was successfully created
202 “Accepted” indicates that an asynch operation was started
204 “No Content” indicates success but with an intentionally empty response body
301 “Moved Permanently” is used for relocated resources
303 “See Other” tells the client to query a different URI
304 “Not Modified” is used to save bandwidth
307 “Temporary Redirect” means resubmit the query to a different URI
400 “Bad Request” indicates a general failure
401 “Unauthorized” indicates bad credentials
403 “Forbidden” denies access regardless of authentication
404 “Not Found” means the URI doesn’t map to a resource
405 “Method Not Allowed” means the HTTP method isn’t supported
406 “Not Acceptable” indicates the requested format isn’t available
409 “Conflict” indicates a problem with the state of the resource
412 “Precondition Failed” is used for conditional operations
415 “Unsupported Media Type” means the type of payload can’t be processed
500 “Internal Server Error” indicates an API malfunction
A note about the PATCH method. There are good reasons to consider using the HTTP PATCH method for partial updates of resources, but because it’s not supported everywhere, and because there are workarounds, I haven’t added it to my cheat sheet yet.
http://www.aleax.it/pycon11_adap.pdf
https://blog.stoplight.io/components-of-killer-api-docs-ebbf41c13450
Trying to use an API without clear and accurate documentation is like trying to read a book without any punctuation
the worst API is -- no API at all check e.g. stackoverflow -- most common Q: about spidering and scraping websites about simulating keystroke & mouse acts some of those Qs are about system testing most of them are about "missing APIs" the APIs may not actually be there or they may be undocumented which is almost as bad a bad, weak API is still better than no API
https://alistapart.com/article/the-ten-essentials-for-good-api-documentation
https://zapier.com/engineering/great-documentation-examples/
http://apidocjs.com/
Good API Doc samples:
https://stripe.com/docs/api
https://developers.strava.com/docs/reference/
https://clearbit.com/docs
https://alistapart.com/article/the-ten-essentials-for-good-api-documentation
https://zapier.com/engineering/great-documentation-examples/
http://apidocjs.com/
Good API Doc samples:
https://stripe.com/docs/api
https://developers.strava.com/docs/reference/
https://clearbit.com/docs
SOAP was designed as an object-access protocol in 1998 by Dave Winer, Don Box, Bob Atkinson, and Mohsen Al-Ghosein for Microsoft, where Atkinson and Al-Ghosein were working.[3] The specification was not made available until it was submitted to IETF 13 September 1999.
Version 1.2 of the specification, however, became a W3C recommendation on June 24, 2003.
What a consumer really wants to access is information/functionality delivered by a service. Only add layers that truly add value but not on top of each other unless it can’t be avoided. Instead focus on ensuring that the API contract is not broken.
What a consumer really wants to access is information/functionality delivered by a service. Only add layers that truly add value but not on top of each other unless it can’t be avoided. Instead focus on ensuring that the API contract is not broken.