Choisir entre une API
RPC, SOAP, REST, GraphQL?


Et si le problème était ailleurs ?
François-Guillaume Ribreau
—
Architect & Head of development @Ouest-France
🌟 SaaS founder of _______ ____ MotionDynamic,
Mailpopin
🚀 Trainer @EPSI_Nantes @UnivNantes
📢 Twitter/Github: @FGRibreau
<quick>
<history>
- pattern: procedure(arguments...): [result]
- ~1980
- principle: simplest form of API interaction
- goal: run a task on another address space
- often requires an IDL (Interface Definition Language)
- well-known implementations:
- XML-RPC (XML to encode calls & HTTP as transport)
- JSON-RPC (JSON to encode calls & HTTP as transport)
- Thrift, gRPC, Avro... and *lots* of other implementation
RPC - Remote Procedure Call
client api
procedure(arguments...)
result
- ~successor of XML-RPC
- XML-based messaging framework
- can work with any transport protocol (just need XML)
- XML everywhere
- IDL: WSDL, a schema of every available operations
- 1998 (+18 yrs after RPC principle)
- Spec WG closed on 10 July 2009 (+11 yrs)
SOAP - Simple Object Access Protocol
client api
envelope(message(call))
envelope(message(response))
- Roy Fielding, 2000 (20 yrs after RPC)
- based on the uniform and predefined set of stateless
operations from HTTP protocol
- request/response
- server-side data are made available through
representations of data in simple formats (e.g. JSON, XML)
- IDL: no schema required (state-of-the-art: OpenAPI)
REST - Representational State Transfer
client api
HTTP request
HTTP response
More power on the client-side at request time
GET /ressources/?fields=property1,property2
GET /ressources/:ressource_id?fields=property1,property2
(e.g. Google APIs, Facebook APIs)
REST - Representational State Transfer
client api
HTTP request
HTTP response
GET /fql?
q=SELECT+uid2+FROM+friend+WHERE+uid1=me()&a
ccess_token=…
FQL @ Facebook (2007)
https://query.yahooapis.com/v1/public/yql?
q=select title, link from rss
where url='https://engt.co/2JyTaQP'
&format=json
&env=store://datatables.org/alltableswithkeys
YQL @ Yahoo (2008)
- Another RPC protocol (not a Graph db)
- 3 verbs (in the REST sense):
- fetch (implicit, read requests)
- subscription (explicit, read streams)
- mutation (writes requests)
- introduce inversion of control between client & server
- client only request needed data
GraphQL
client api
HTTP request
HTTP response
</history>
</quick>
Wow
much work
so specifications
client api
Current
State of the art
3-tier
Database
API
Client
3-tier
Database
(Tables / Views)
API
(Models)
Client
Validation
Database
(Schema (constraint))
API
(Models (validation))
Client
(validation)
Authorization
Database
(Users/roles, policies)
API
(Authorization
middleware)
Client
Etc… 🕰
Database
…
API
…
Client
Only for
data
persistence
Without
data
persistence
Mostly for
data
persistence
(+ some
MQueing)
What kind of API do you write the most?
~90% of the time
it's. just. CRUD.
effort =>
data
api
client
user
data
api
client
user
effort =>
Let's build a
persistence API
api.motiondynamic.tech
Fast highly dynamic video generation at scale 🚀
api
SQLHTTP
data
(PostgreSQL)
• HTTP request handling
• Authentication
• Authorization
• Request Parsing
• Request Validation
• Database Communication
• Database Response Handling (deserialization)
• Output format serialization
• HTTP Response Building
Persistence API
our job
api
SQLHTTP
Persistence API
TL;DR: HTTP <-> SQL mapping
… with a lot of space for potential mistakes. our job
data
(PostgreSQL)
Database Modelling 101
v1_0 schema authentication schema
theme schema
video schema
….
view schemastored
fn
generations
signIn signUp
themes
api
(PostgREST)
data
(PostgreSQL)
Persistence API
<= our job
#SSoT #DRY
v1_0
schema
client
HTTP / REST / (JSON,CSV)
Are we
serious?
Schema modelling
public private
v1_0 schema authentication schema
theme schema
video schema
….
view schemastored
fn
projects signIn signUp
Postgrest
videos
generations
startVideoGeneration
Read / Write requests
(read, view) GET /themes
(read, view) GET /videos
(read, view) GET /generations
(write, stored function) POST /rpc/signUp
(write, stored function) POST /rpc/signIn
(write, stored function) POST /rpc/startVideoGeneration
DBv1_0
schema
api
(PostgREST)
GET /themes?downloads=gte.1000&isMine=is.true
GET /themes?select=downloads::text,name,description
GET /stuff?metadata->a->>b=eq.2
GET /themes?select=id,name,generations{createdAt}
&order=name.asc&generations.order=createdAt.desc
How do you manage
projection, filtering, ordering?
Read / Write requests
(read, view) GET /themes
(read, view) GET /videos
(read, view) GET /generations
(write, stored function) POST /rpc/signUp
(write, stored function) POST /rpc/signIn
(write, stored function) POST /rpc/startVideoGeneration
(write) POST /generations
INSTEAD OF
create trigger generations_mutation
INSTEAD OF INSERT OR UPDATE OR DELETE
ON v1_0.generations
for each row execute procedure
video.mutation_generations_trigger()
Trigger
create or replace function video.mutation_generations_trigger() returns trigger as $$
declare
begin
if (tg_op = 'DELETE') then
-- delete generation
return new;
elsif (tg_op = 'UPDATE') then
-- update generation
return new;
elsif (tg_op = 'INSERT') then
-- create generation
return new;
end if;
end;
$$ security definer language plpgsql;
pl/pgsql ? It's 2018: pl/v8 (javascript) - pl/python - pl/java - pl/tcl
State-of-the-art API performance
api
data
(PostgreSQL)
client
HTTP / REST / JSON
deserialize db response
serialize http JSON response
serialize to internal representation
API performance 🚀
api
(PostgREST)
data
(PostgreSQL)
client
HTTP / REST / JSON
serialize result to JSON
“PostgreSQL JSON encoder is fast, it’s C fast.
For a large JSON object graph PostgreSQL JSON
generation can offer well over 12x the throughput.
It can be 2x/10x faster than ActiveRecord or even 160x
for large responses”
https://hashrocket.com/blog/posts/faster-json-generation-with-postgresql
Versioning
public private
v1_0 schema
v2_0 schema
authentication schema
theme schema
video schema
….
view schemastored
fn
videos signIn signUp
generations logIn signUp
How do you manage authentication?
How do you manage authorization?
CREATE ROLE authenticator NOINHERIT LOGIN;
CREATE ROLE anonymous;
CREATE ROLE authenticated_user;
GRANT anonymous, authenticated_user TO authenticator;
How do you manage authorization?
Row Level Security (PG 9.5+)
ALTER TABLE video.generation ENABLE ROW LEVEL SECURITY;
create policy generation_access_policy on video.generation to api
-- a user can only see its own generations, admin can see everything
using (
(request.user_role() = 'videoRequester' and user_id =
request.user_id()) or request.user_role() = 'admin')
-- only admin can change anyone generation data
with check (request.user_role() = 'admin');
3 lines of SQL
Reliable security model (closed by default)
Declarative
Expressive
Imperative
Programming
"how to do"
%📺'
Declarative
Programming
"what to do"
(🚀)
How do you manage
emails/3rd parties?
pg_notify (PG 9.2+)
http://bit.ly/2oNbaKy
How do you manage
emails/3rd parties?
pg_notify (PG 9.2+)
http://bit.ly/2JDCsQu
Rate-limiting? Headers?
Monitoring? Tracing? Custom-logic ?
Respect the separation of concerns
(that's what Service Mesh is all about btw)
data
api
client
user
proxy
(nginx/openresty | istio proxy | ...)
How to manage documentation?
OpenAPI (Swagger) format
automatically extracted from schema
How to manage
code-reviews, tests?
It’s just SQL.
SQL unit test: pgTAP
Functional tests: Supertest / Jest
How to manage migrations?
Full migration management
sqitch/apgdiff
How to build & deploy?
local env. + docker 🐳
(watch + SQL auto-reload)
Cloud (PostgreSQL + PostgREST + OpenResty)
apply migrations
rolling deploy PostgREST/OpenResty
run unit & functional tests
test migrations
One
more
thing
REST & GraphQL api
(PostgREST + SubZero)
data
(PostgreSQL)
<= our job
#SSoT #DRY
v1_0
schema
client
GraphQL ?
subZero ❤
GraphQL & REST (PostgREST) API for your database
This philosophy is
gaining traction
PostgraphQL - A GraphQL API created by reflection
over a PostgreSQL schema: NodeJS + PostgreSQL
Prisma - Prisma turns your database into a realtime
GraphQL API: Scala + MySQL/Postgres (upcoming: MongoDB,
ElasticSearch)
Graphql-engine - Instant GraphQL APIs on Postgres
with fine grained access control: NodeJS + PostgreSQL
Et si le problème était ailleurs ?
API
data
(BDD)
v1_0
schema
client
REST GraphQL Tomorrow?
Free plans for Redis
administration & monitoring
at redsmin.com
We are looking for Front-end Developers
twitter.com/iadvizetech
Questions?
@FGRibreau
No more server-side rendering pain,
1 url = 1 chart
image-charts.com
https://apple.github.io/foundationdb/transaction-manifesto.html
Performance and scalability?
We know of no practical limits to the scalability or performance of systems supporting
transactions. When the movement toward NoSQL databases began, early systems, such
as Google Bigtable, adopted minimal designs tightly focused on the scalability and
performance. Features familiar from relational databases had been aggressively shed and
the supposition was that the abandoned features were unnecessary or even harmful
for scalability and performance goals.

Those suppositions were wrong. It is becoming clear that supporting transactions is a
matter of engineering effort, not a fundamental tradeoff in the design space.
Algorithms for maintaining transactional integrity can be distributed and scale out like
many other problems. 

Transactional integrity does come at a CPU cost, but in our experience that cost is less
than 10% of total system CPU. This is a small price to pay for transactional integrity and
can easily be made up elsewhere.
I am quite convinced that in fact computing will
become a very important science. (🐶💩)
But at the moment we are in a very
primitive state of development; we don't
know the basic principles yet and we must
learn them first. 

If universities spend their time teaching the
state of the art, they will not discover
these principles and that, surely, is what
academics should be doing.



— Christopher Strachey, 1969 (49 yrs ago)

(quote by Edsger W. Dijkstra)
https://bit.ly/2pMI7aJ
“
”

Choisir entre une API RPC, SOAP, REST, GraphQL? 
Et si le problème était ailleurs ?

  • 1.
    Choisir entre uneAPI RPC, SOAP, REST, GraphQL? 
 Et si le problème était ailleurs ?
  • 2.
    François-Guillaume Ribreau — Architect &Head of development @Ouest-France 🌟 SaaS founder of _______ ____ MotionDynamic, Mailpopin 🚀 Trainer @EPSI_Nantes @UnivNantes 📢 Twitter/Github: @FGRibreau
  • 3.
  • 4.
    - pattern: procedure(arguments...):[result] - ~1980 - principle: simplest form of API interaction - goal: run a task on another address space - often requires an IDL (Interface Definition Language) - well-known implementations: - XML-RPC (XML to encode calls & HTTP as transport) - JSON-RPC (JSON to encode calls & HTTP as transport) - Thrift, gRPC, Avro... and *lots* of other implementation RPC - Remote Procedure Call client api procedure(arguments...) result
  • 5.
    - ~successor ofXML-RPC - XML-based messaging framework - can work with any transport protocol (just need XML) - XML everywhere - IDL: WSDL, a schema of every available operations - 1998 (+18 yrs after RPC principle) - Spec WG closed on 10 July 2009 (+11 yrs) SOAP - Simple Object Access Protocol client api envelope(message(call)) envelope(message(response))
  • 6.
    - Roy Fielding,2000 (20 yrs after RPC) - based on the uniform and predefined set of stateless operations from HTTP protocol - request/response - server-side data are made available through representations of data in simple formats (e.g. JSON, XML) - IDL: no schema required (state-of-the-art: OpenAPI) REST - Representational State Transfer client api HTTP request HTTP response
  • 7.
    More power onthe client-side at request time GET /ressources/?fields=property1,property2 GET /ressources/:ressource_id?fields=property1,property2 (e.g. Google APIs, Facebook APIs) REST - Representational State Transfer client api HTTP request HTTP response
  • 8.
    GET /fql? q=SELECT+uid2+FROM+friend+WHERE+uid1=me()&a ccess_token=… FQL @Facebook (2007) https://query.yahooapis.com/v1/public/yql? q=select title, link from rss where url='https://engt.co/2JyTaQP' &format=json &env=store://datatables.org/alltableswithkeys YQL @ Yahoo (2008)
  • 9.
    - Another RPCprotocol (not a Graph db) - 3 verbs (in the REST sense): - fetch (implicit, read requests) - subscription (explicit, read streams) - mutation (writes requests) - introduce inversion of control between client & server - client only request needed data GraphQL client api HTTP request HTTP response
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    Only for data persistence Without data persistence Mostly for data persistence (+some MQueing) What kind of API do you write the most? ~90% of the time it's. just. CRUD.
  • 19.
  • 20.
  • 21.
  • 22.
    api.motiondynamic.tech Fast highly dynamicvideo generation at scale 🚀
  • 23.
    api SQLHTTP data (PostgreSQL) • HTTP requesthandling • Authentication • Authorization • Request Parsing • Request Validation • Database Communication • Database Response Handling (deserialization) • Output format serialization • HTTP Response Building Persistence API our job
  • 24.
    api SQLHTTP Persistence API TL;DR: HTTP<-> SQL mapping … with a lot of space for potential mistakes. our job data (PostgreSQL)
  • 25.
    Database Modelling 101 v1_0schema authentication schema theme schema video schema …. view schemastored fn generations signIn signUp themes
  • 26.
    api (PostgREST) data (PostgreSQL) Persistence API <= ourjob #SSoT #DRY v1_0 schema client HTTP / REST / (JSON,CSV)
  • 27.
  • 28.
    Schema modelling public private v1_0schema authentication schema theme schema video schema …. view schemastored fn projects signIn signUp Postgrest videos generations startVideoGeneration
  • 29.
    Read / Writerequests (read, view) GET /themes (read, view) GET /videos (read, view) GET /generations (write, stored function) POST /rpc/signUp (write, stored function) POST /rpc/signIn (write, stored function) POST /rpc/startVideoGeneration DBv1_0 schema api (PostgREST)
  • 30.
    GET /themes?downloads=gte.1000&isMine=is.true GET /themes?select=downloads::text,name,description GET/stuff?metadata->a->>b=eq.2 GET /themes?select=id,name,generations{createdAt} &order=name.asc&generations.order=createdAt.desc How do you manage projection, filtering, ordering?
  • 31.
    Read / Writerequests (read, view) GET /themes (read, view) GET /videos (read, view) GET /generations (write, stored function) POST /rpc/signUp (write, stored function) POST /rpc/signIn (write, stored function) POST /rpc/startVideoGeneration (write) POST /generations
  • 32.
    INSTEAD OF create triggergenerations_mutation INSTEAD OF INSERT OR UPDATE OR DELETE ON v1_0.generations for each row execute procedure video.mutation_generations_trigger()
  • 33.
    Trigger create or replacefunction video.mutation_generations_trigger() returns trigger as $$ declare begin if (tg_op = 'DELETE') then -- delete generation return new; elsif (tg_op = 'UPDATE') then -- update generation return new; elsif (tg_op = 'INSERT') then -- create generation return new; end if; end; $$ security definer language plpgsql; pl/pgsql ? It's 2018: pl/v8 (javascript) - pl/python - pl/java - pl/tcl
  • 34.
    State-of-the-art API performance api data (PostgreSQL) client HTTP/ REST / JSON deserialize db response serialize http JSON response serialize to internal representation
  • 35.
    API performance 🚀 api (PostgREST) data (PostgreSQL) client HTTP/ REST / JSON serialize result to JSON “PostgreSQL JSON encoder is fast, it’s C fast. For a large JSON object graph PostgreSQL JSON generation can offer well over 12x the throughput. It can be 2x/10x faster than ActiveRecord or even 160x for large responses” https://hashrocket.com/blog/posts/faster-json-generation-with-postgresql
  • 36.
    Versioning public private v1_0 schema v2_0schema authentication schema theme schema video schema …. view schemastored fn videos signIn signUp generations logIn signUp
  • 37.
    How do youmanage authentication?
  • 38.
    How do youmanage authorization? CREATE ROLE authenticator NOINHERIT LOGIN; CREATE ROLE anonymous; CREATE ROLE authenticated_user; GRANT anonymous, authenticated_user TO authenticator;
  • 39.
    How do youmanage authorization? Row Level Security (PG 9.5+) ALTER TABLE video.generation ENABLE ROW LEVEL SECURITY; create policy generation_access_policy on video.generation to api -- a user can only see its own generations, admin can see everything using ( (request.user_role() = 'videoRequester' and user_id = request.user_id()) or request.user_role() = 'admin') -- only admin can change anyone generation data with check (request.user_role() = 'admin'); 3 lines of SQL Reliable security model (closed by default) Declarative Expressive
  • 40.
  • 41.
    How do youmanage emails/3rd parties? pg_notify (PG 9.2+) http://bit.ly/2oNbaKy
  • 42.
    How do youmanage emails/3rd parties? pg_notify (PG 9.2+) http://bit.ly/2JDCsQu
  • 43.
    Rate-limiting? Headers? Monitoring? Tracing?Custom-logic ? Respect the separation of concerns (that's what Service Mesh is all about btw) data api client user proxy (nginx/openresty | istio proxy | ...)
  • 45.
    How to managedocumentation? OpenAPI (Swagger) format automatically extracted from schema
  • 46.
    How to manage code-reviews,tests? It’s just SQL. SQL unit test: pgTAP Functional tests: Supertest / Jest
  • 47.
    How to managemigrations? Full migration management sqitch/apgdiff
  • 48.
    How to build& deploy? local env. + docker 🐳 (watch + SQL auto-reload) Cloud (PostgreSQL + PostgREST + OpenResty) apply migrations rolling deploy PostgREST/OpenResty run unit & functional tests test migrations
  • 49.
  • 50.
    REST & GraphQLapi (PostgREST + SubZero) data (PostgreSQL) <= our job #SSoT #DRY v1_0 schema client GraphQL ?
  • 51.
    subZero ❤ GraphQL &REST (PostgREST) API for your database
  • 52.
  • 53.
    PostgraphQL - AGraphQL API created by reflection over a PostgreSQL schema: NodeJS + PostgreSQL Prisma - Prisma turns your database into a realtime GraphQL API: Scala + MySQL/Postgres (upcoming: MongoDB, ElasticSearch) Graphql-engine - Instant GraphQL APIs on Postgres with fine grained access control: NodeJS + PostgreSQL
  • 54.
    Et si leproblème était ailleurs ? API data (BDD) v1_0 schema client REST GraphQL Tomorrow?
  • 55.
    Free plans forRedis administration & monitoring at redsmin.com We are looking for Front-end Developers twitter.com/iadvizetech Questions? @FGRibreau No more server-side rendering pain, 1 url = 1 chart image-charts.com
  • 56.
    https://apple.github.io/foundationdb/transaction-manifesto.html Performance and scalability? Weknow of no practical limits to the scalability or performance of systems supporting transactions. When the movement toward NoSQL databases began, early systems, such as Google Bigtable, adopted minimal designs tightly focused on the scalability and performance. Features familiar from relational databases had been aggressively shed and the supposition was that the abandoned features were unnecessary or even harmful for scalability and performance goals. Those suppositions were wrong. It is becoming clear that supporting transactions is a matter of engineering effort, not a fundamental tradeoff in the design space. Algorithms for maintaining transactional integrity can be distributed and scale out like many other problems. Transactional integrity does come at a CPU cost, but in our experience that cost is less than 10% of total system CPU. This is a small price to pay for transactional integrity and can easily be made up elsewhere.
  • 57.
    I am quiteconvinced that in fact computing will become a very important science. (🐶💩) But at the moment we are in a very primitive state of development; we don't know the basic principles yet and we must learn them first. 
 If universities spend their time teaching the state of the art, they will not discover these principles and that, surely, is what academics should be doing. — Christopher Strachey, 1969 (49 yrs ago) (quote by Edsger W. Dijkstra) https://bit.ly/2pMI7aJ “ ”