A talk given by Jakub Draganek during the GraphQL Wroclaw meetup on 16 September 2019. It's a brief introduction to GraphQL Data Loaders, a middle layer that helps you get and buffer data for API.

1. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARECOPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
GraphQL Data Loaders
How to feed your GraphQL API with data the smart way

2. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Jakub Draganek
Python Developer @ Mirumee Software
Ariadne
Saleor

3. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
GRAPHQL API SETUP EXAMPLE

4. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Schema

5. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
GraphQL query execution

6. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
GraphQL query execution
CLIENT
QUERY
getData()
getOtherData()

7. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
GraphQL query

8. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Building response
■ Parse
■ Validate
■ Execute

9. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Resolver execution tree

10. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
name
kingdoms
ruler
name
house
name
KINGDOM PERSON HOUSEQUERY

11. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Resolvers execution flow
■ Receive parent and arguments
■ Fetch / transform data in resolver
■ Wait until finished and iterate on children

12. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Simple and elegant, what is wrong with that?
Things start to get very ugly quickly…

13. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Give me more data!

14. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Execution steps
■ run Query.kingdoms
■ for Kingdom run Kingdom.name and Kingdom.ruler
■ for ruler run Person.name, Person.parents, Person.house
■ for kingdom ruler’s parents run Person.name and Person.house
■ for each parent house run House.name
■ …
■ fetch House.name a few more times
■ fetch same Person a few times with everything once again!

15. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE

16. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
N+1 and other problems
■ Repetition of the same data fetching
■ Suboptimal database usage
■ Overloaded external services
■ Data inconsistency

17. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Solution?
■ Batching
■ Caching
■ Data layer abstraction

18. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Data layer abstraction
QUERY
resolver
resolver
DB
Services
Data layer
abstraction

19. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
DataLoader
Generic data fetching layer

20. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
DataLoader
■ Collect keys from every .load()
■ Batch data fetch on event loop tick
■ Each .load() revolves to data
■ Cache data for a single request

21. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
DataLoader loop
LOOP
load(1)
load(2)
load(5)
batch([1,2,5])resolver DBCACHE
resolver

22. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Async resolver flow
load(1)
load(2)
batch([1,2])
resolver 2
resolver 1
promise
promise
tick!
⌛
DB
{1: “data_1”, 2: “data_2}
resolver 1
resolve promise!
SOON...

23. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
What is important?
■ Asynchronous environment
■ Single instance of Data Loader per request
■ One DataLoader per GraphQL Type
■ No business logic in the data layer!
■ All data fetching in the batch function

24. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
Where now?
■ Data Loader implementation for your environment
■ Persistent caching
■ Different execution strategies
■ Field level query analysis
■ The sooner the better. Keep the data layer separate

25. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE
THANK YOU!
QUESTIONS?