ORM and..
Bad habits
Patryk Szlagowski
Patryk Szlagowski
Backend Developer at Logisfera Nova - lsn.io
@abdulklarapl
Captain at FunkyMonkeyLabs
1. ORM - what is it?

2. Problem 1 - greedy fetching

3. Problem 2 - „lets map everything”

4. Performance table

5. Conclusions

6. Conclusions - moar
1. ORM
In one picture
ORMDB applicationDBAL*
/**

* @Route("/", name="homepage")

*/

public function indexAction(Request $request)

{

return $this->render('default/index.html.twig', array(

'news' => $this->getDoctrine()
->getRepository('AppBundle:Post')
->findAll()

));

}
2. Problem 1 - greedy fetching
/**

* @return Post[]

*/

public function getAll()

{

$qb = $this->createQueryBuilder("post");

$qb

->select("post, author, role,
category, comments,
commentsAuthor,
commentsAuthorRole")

->join("post.category", "category")

->join("post.author", "author")

->join("author.role", "role")

->join("post.comments", "comments")

->join("comments.author", "commentsAuthor")

->join("commentsAuthor.role", "commentsAuthorRole");



return $qb->getQuery()->getResult();

}
That is nice. But wait.. How is possible, that results can be
mapped to relations from this?
Let’s add custom method in repository and control the joins
2. Problem 1 - greedy fetching
2. Problem 1 - greedy fetching
SELECT {fields}

FROM post p0_

INNER JOIN dictionary_item d1_ ON p0_.category_id = d1_.id

INNER JOIN person p2_ ON p0_.author_id = p2_.id

INNER JOIN dictionary_item d3_ ON p2_.role_id = d3_.id

INNER JOIN COMMENT c4_ ON p0_.id = c4_.post_id

INNER JOIN person p5_ ON c4_.author_id = p5_.id

INNER JOIN dictionary_item d6_ ON p5_.role_id = d6_.id
That is I expected. 3 rows for 1 post (3 comments). ORM mapped columns to many objects,
grouped by primary key.

But let’s imagine, what if we have 100 records (A) with relation to B (10 records each) AND B-C
(with another 10 records for each B)? 10000 records to map.
2. Problem 1 - greedy fetching
What’s going on?
What if..
I tell you, that you can get all of the rows by one query, without multiply results rows number?
IMPOSSIBRU
Cons
• PostgreSQL
• JSON functions
• complicated query
• slower than simple

query
Pros
• JSON response
• whole object in one row
• what you see in results, 

it’s what you get in app
SELECT array_to_json(

array_agg(

row_to_json({alias})

)

)

FROM (

{subquery}

) AS {alias}
SELECT array_to_json(array_agg(row_to_json(results)))

FROM (

SELECT

post.id, post.create_date, post.content, post.title,

(SELECT row_to_json(author)

FROM (

SELECT

person.*,

(

SELECT row_to_json(role)

FROM (

SELECT * FROM dictionary_item di WHERE di.id = person.role_id

) AS role

) AS role

FROM person WHERE person.id = post.author_id

) as author) AS author,

(

SELECT row_to_json(category)

FROM dictionary_item category WHERE category.id = post.category_id

) as category,

(

SELECT array_to_json(array_agg(row_to_json(comments)))

FROM (

SELECT comment.title, comment.content, (

SELECT row_to_json(comment_author)

FROM (

SELECT person.*, (

SELECT row_to_json(role)

FROM (

SELECT *

FROM dictionary_item di

WHERE di.id = person.role_id

) AS role

) AS role

FROM person

WHERE person.id = comment.author_id

) AS comment_author

) AS author

FROM comment where comment.post_id = post.id

) AS comments

) as comments

FROM post

) AS results
3. Problem 2 - results mapping
ORMDataSource
response
application
model
REST
response
In most cases
for what?
3. Problem 2 - results mapping
DataSource
response
array/hashmap response REST
response
In most cases
Built-in findAll
function
Simple query with
joins, mapped to
objects
array_to_json query,
mapped to objects
array_to_json query,
no mapping, return
in REST Api
query time 2326ms 52ms 101ms 111ms
memory usage 82MB 51MB 33.5MB 17.2MB
total (with render)
time
7464ms 2100ms 1755ms 364ms
number of queries 4405 1 1 1
4. All about performance
* tested with 400 posts, 10 comments per post, one author per comment, author per post, label per category
5. Conclusions
Query builders
Query builders
$this->repository->findAll()
4405
trolololo
vs
$this->repository->customFindAll()
1
1. Maybe we should start using aggregating functions to get data from DB

2. If you don’t have to to transform DB response to object - don’t do it

3. Don’t use built-in functions such as findBy, findAll, findOne, sth, sth

4. Write your own functions with joins, groups to take control over the query

5. Remember about lazy loading
6. Conclusions
Thank youI want to talk with you

ORM - tuningujemy podejście do mapowania

  • 1.
  • 2.
    Patryk Szlagowski Backend Developerat Logisfera Nova - lsn.io @abdulklarapl Captain at FunkyMonkeyLabs
  • 3.
    1. ORM -what is it? 2. Problem 1 - greedy fetching 3. Problem 2 - „lets map everything” 4. Performance table 5. Conclusions 6. Conclusions - moar
  • 4.
    1. ORM In onepicture ORMDB applicationDBAL*
  • 5.
    /**
 * @Route("/", name="homepage")
 */
 publicfunction indexAction(Request $request)
 {
 return $this->render('default/index.html.twig', array(
 'news' => $this->getDoctrine() ->getRepository('AppBundle:Post') ->findAll()
 ));
 } 2. Problem 1 - greedy fetching
  • 6.
    /**
 * @return Post[]
 */
 publicfunction getAll()
 {
 $qb = $this->createQueryBuilder("post");
 $qb
 ->select("post, author, role, category, comments, commentsAuthor, commentsAuthorRole")
 ->join("post.category", "category")
 ->join("post.author", "author")
 ->join("author.role", "role")
 ->join("post.comments", "comments")
 ->join("comments.author", "commentsAuthor")
 ->join("commentsAuthor.role", "commentsAuthorRole");
 
 return $qb->getQuery()->getResult();
 } That is nice. But wait.. How is possible, that results can be mapped to relations from this? Let’s add custom method in repository and control the joins 2. Problem 1 - greedy fetching
  • 7.
    2. Problem 1- greedy fetching
  • 8.
    SELECT {fields}
 FROM postp0_
 INNER JOIN dictionary_item d1_ ON p0_.category_id = d1_.id
 INNER JOIN person p2_ ON p0_.author_id = p2_.id
 INNER JOIN dictionary_item d3_ ON p2_.role_id = d3_.id
 INNER JOIN COMMENT c4_ ON p0_.id = c4_.post_id
 INNER JOIN person p5_ ON c4_.author_id = p5_.id
 INNER JOIN dictionary_item d6_ ON p5_.role_id = d6_.id That is I expected. 3 rows for 1 post (3 comments). ORM mapped columns to many objects, grouped by primary key. But let’s imagine, what if we have 100 records (A) with relation to B (10 records each) AND B-C (with another 10 records for each B)? 10000 records to map. 2. Problem 1 - greedy fetching What’s going on?
  • 9.
    What if.. I tellyou, that you can get all of the rows by one query, without multiply results rows number? IMPOSSIBRU
  • 10.
    Cons • PostgreSQL • JSONfunctions • complicated query • slower than simple
 query Pros • JSON response • whole object in one row • what you see in results, 
 it’s what you get in app SELECT array_to_json(
 array_agg(
 row_to_json({alias})
 )
 )
 FROM (
 {subquery}
 ) AS {alias} SELECT array_to_json(array_agg(row_to_json(results)))
 FROM (
 SELECT
 post.id, post.create_date, post.content, post.title,
 (SELECT row_to_json(author)
 FROM (
 SELECT
 person.*,
 (
 SELECT row_to_json(role)
 FROM (
 SELECT * FROM dictionary_item di WHERE di.id = person.role_id
 ) AS role
 ) AS role
 FROM person WHERE person.id = post.author_id
 ) as author) AS author,
 (
 SELECT row_to_json(category)
 FROM dictionary_item category WHERE category.id = post.category_id
 ) as category,
 (
 SELECT array_to_json(array_agg(row_to_json(comments)))
 FROM (
 SELECT comment.title, comment.content, (
 SELECT row_to_json(comment_author)
 FROM (
 SELECT person.*, (
 SELECT row_to_json(role)
 FROM (
 SELECT *
 FROM dictionary_item di
 WHERE di.id = person.role_id
 ) AS role
 ) AS role
 FROM person
 WHERE person.id = comment.author_id
 ) AS comment_author
 ) AS author
 FROM comment where comment.post_id = post.id
 ) AS comments
 ) as comments
 FROM post
 ) AS results
  • 11.
    3. Problem 2- results mapping ORMDataSource response application model REST response In most cases for what?
  • 12.
    3. Problem 2- results mapping DataSource response array/hashmap response REST response In most cases
  • 13.
    Built-in findAll function Simple querywith joins, mapped to objects array_to_json query, mapped to objects array_to_json query, no mapping, return in REST Api query time 2326ms 52ms 101ms 111ms memory usage 82MB 51MB 33.5MB 17.2MB total (with render) time 7464ms 2100ms 1755ms 364ms number of queries 4405 1 1 1 4. All about performance * tested with 400 posts, 10 comments per post, one author per comment, author per post, label per category
  • 14.
    5. Conclusions Query builders Querybuilders $this->repository->findAll() 4405 trolololo vs $this->repository->customFindAll() 1
  • 15.
    1. Maybe weshould start using aggregating functions to get data from DB 2. If you don’t have to to transform DB response to object - don’t do it 3. Don’t use built-in functions such as findBy, findAll, findOne, sth, sth 4. Write your own functions with joins, groups to take control over the query 5. Remember about lazy loading 6. Conclusions
  • 16.
    Thank youI wantto talk with you