With increasing levels of abstraction in frameworks, it is rare that web developers interact directly with databases. However, ORMs can slow seemingly simple queries down to a crawl. Instead of trying to maintain complex cache keys with the latest fashionable key-value store, let's revisit this 30 year old technology.

1. Revenge of the ORMs
Why SQL skills still matter in 2015
Lightning strikes © flickr.com/photos/snowpeak CC-BY

2. First, An Intro

3. Megan Bowra-Dean !
• Rails/JS/Android/iOS/
kitchen sink developer
at Rabid Tech
• 2 years .NET
enterprise web dev
• 2 years C/C++
embedded dev
• Ruby NZ Committee
Member

4. –Roy Batty, Bladerunner
“I've seen things you people wouldn't believe.”

5. Reintroducing ORMs

6. ORMs
• “Object Relational Mappers”
• Translates and serialises objects to a relational
database
• Generally database agnostic

7. Cat.new(fur: 'calico', name: 'Ms Tibbles')
id fur name
1 calico Ms Tibbles

8. ms_tibbles.owner = Owner.new(name: 'Megan')
id fur name owner_id
1 calico Ms Tibbles 1
cats table
id name
1 Megan
owners table

9. Examples
Entity FrameworkActiveRecord django.db

10. The Good 👍

11. Saves development time & easier to maintain
SELECT
*
FROM
books
INNER JOIN
libraries ON
books.library_id =
libraries.id
WHERE
libraries.name = 'Wellington
City Library'
Book
.joins('libraries')
.where(libraries: { name:
'Wellington City Library' })
VS

12. Security
• Most ORMs stop SQL injection attacks 💉
• Some restrict columns that can be updated by
user input (e.g. Rails 4’s strong_params)

13. BUT $

14. When and How ORMs
Can Breakdown

15. Ultimately ORMs are an
Abstraction
• Simplified for specific use cases
• What happens when the relationships between
your models get more complex?
• What happens when you need data not tied to a
model’s fields? 📊

16. N+1 Problem
A book has an editor and magazines are a type of
book, how do we find all the editors belonging to
magazines to print them out?
📚'(

17. Naive Way
magazines = Book
.where(type: 'magazine')
magazines.each do |magazine|
puts magazine.editor
end

18. Resultant SQL
SELECT * FROM books WHERE type = 'magazine'
SELECT * FROM editors WHERE book_id = 1
SELECT * FROM editors WHERE book_id = 2
..
SELECT * FROM editors WHERE book_id = n

19. We end up with 1 + n queries, where n is the number
of magazines, hence the N+1 problem.

20. Optimised Way
magazines = Book
.includes('editor')
.where(type: 'magazine')
magazines.each do |magazine|
puts magazine.editor
end

21. Resultant SQL
SELECT * FROM books WHERE type = 'magazine'
SELECT * FROM editors WHERE (book_id IN (1,2,3..n))

22. • Not always as obvious as this
• Can have a lot of things happening between
fetching the parent model and the child models
• Still, (most) ORMs have the capability to help

23. Modern Scripting
Languages are SLOW* 🐢
* At handling large data sets

24. • Page load times can slow down noticeably with
just a few thousand instances of models.
• May expect to run operations on hundreds of
thousands.
• Is it WEBSCALE?

25. We Usually Deal With This
by Over-engineering
• Adding extra caching layers
• Load balancing with horizontal scaling
• Progressive page loading

26. Subverting Your ORM
for Fun and Profit

27. • We can wrest the raw database connection from
the ORM
• With this we can improve performance without
greatly increasing complexity

28. A real world example
• Web app for client that surveyed organisational
performance
• Produced an online report with several different
breakdowns of statistics from the survey
• Was surprisingly slow - hit web server timeout

29. Looking closer
Most of the time spent outside of the database

30. • 100,532 calls to Class#new ‼
• A simple page was only creating ~900 objects
• One suspect was a function calculating the
average of responses to a group of questions (a
“domain”)

31. sum = 0.0
count = 0.0
domain.questions.each do |q|
response = q.response_for(respondent)
sum += response.value
count += 1.0
end
if count > 0
return sum / count
else
return 1.0
end

32. conn = ActiveRecord::Base.connection
result = conn.execute <<-SQL
SELECT
SUM(responses.value) as sum,
COUNT(*) as count
FROM
domains
INNER JOIN
questions ON questions.domain_id = domains.id
INNER JOIN
responses ON responses.question_id = questions.id
AND responses.respondent_id = #{respondent.id}
WHERE
domains.id = #{domain.id}
SQL
score = 1.0
sum = result[0]['sum'].to_f
count = result[0]['count'].to_f
score = sum / count if count > 0

33. • Reduced page load time by more than a half
• Reduced number of objects created by 30%

34. Words of Caution ⚠

35. • Not as maintainable
• Need to keep an eye on security.
Never insert user provided values into raw SQL.
• Not as portable.
XKCD #327 https://xkcd.com/327/

36. sql = Cat.where(name: 'Ms Tibbles').to_sql
ActiveRecord::Base.connection.execute sql
Can sometimes get ORM
to help you defeat itself

37. Cool Things Beyond
Performance 😎

38. • Database functions
• Common table expressions
• Views
• GIS extensions for geographical data 🌏
• Non-standard data types

39. Finally:
Remember your
indexes!

40. To Summarise
• ORMs bring great benefits much of the time
• However being aware of what they’re doing is
essential
• If need be, it is possible to work around them.
• Databases are your friend.