Revenge of the ORMs

735 views

Published on

With increasing levels of abstraction in frameworks, it is rare that web developers interact directly with databases. However, ORMs can slow seemingly simple queries down to a crawl. Instead of trying to maintain complex cache keys with the latest fashionable key-value store, let's revisit this 30 year old technology.

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
735
On SlideShare
0
From Embeds
0
Number of Embeds
44
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Revenge of the ORMs

  1. 1. Revenge of the ORMs Why SQL skills still matter in 2015 Lightning strikes © flickr.com/photos/snowpeak CC-BY
  2. 2. First, An Intro
  3. 3. Megan Bowra-Dean ! • Rails/JS/Android/iOS/ kitchen sink developer at Rabid Tech • 2 years .NET enterprise web dev • 2 years C/C++ embedded dev • Ruby NZ Committee Member
  4. 4. –Roy Batty, Bladerunner “I've seen things you people wouldn't believe.”
  5. 5. Reintroducing ORMs
  6. 6. ORMs • “Object Relational Mappers” • Translates and serialises objects to a relational database • Generally database agnostic
  7. 7. Cat.new(fur: 'calico', name: 'Ms Tibbles') id fur name 1 calico Ms Tibbles
  8. 8. ms_tibbles.owner = Owner.new(name: 'Megan') id fur name owner_id 1 calico Ms Tibbles 1 cats table id name 1 Megan owners table
  9. 9. Examples Entity FrameworkActiveRecord django.db
  10. 10. The Good 👍
  11. 11. Saves development time & easier to maintain SELECT * FROM books INNER JOIN libraries ON books.library_id = libraries.id WHERE libraries.name = 'Wellington City Library' Book .joins('libraries') .where(libraries: { name: 'Wellington City Library' }) VS
  12. 12. Security • Most ORMs stop SQL injection attacks 💉 • Some restrict columns that can be updated by user input (e.g. Rails 4’s strong_params)
  13. 13. BUT $
  14. 14. When and How ORMs Can Breakdown
  15. 15. Ultimately ORMs are an Abstraction • Simplified for specific use cases • What happens when the relationships between your models get more complex? • What happens when you need data not tied to a model’s fields? 📊
  16. 16. N+1 Problem A book has an editor and magazines are a type of book, how do we find all the editors belonging to magazines to print them out? 📚'(
  17. 17. Naive Way magazines = Book .where(type: 'magazine') magazines.each do |magazine| puts magazine.editor end
  18. 18. Resultant SQL SELECT * FROM books WHERE type = 'magazine' SELECT * FROM editors WHERE book_id = 1 SELECT * FROM editors WHERE book_id = 2 .. SELECT * FROM editors WHERE book_id = n
  19. 19. We end up with 1 + n queries, where n is the number of magazines, hence the N+1 problem.
  20. 20. Optimised Way magazines = Book .includes('editor') .where(type: 'magazine') magazines.each do |magazine| puts magazine.editor end
  21. 21. Resultant SQL SELECT * FROM books WHERE type = 'magazine' SELECT * FROM editors WHERE (book_id IN (1,2,3..n))
  22. 22. • Not always as obvious as this • Can have a lot of things happening between fetching the parent model and the child models • Still, (most) ORMs have the capability to help
  23. 23. Modern Scripting Languages are SLOW* 🐢 * At handling large data sets
  24. 24. • Page load times can slow down noticeably with just a few thousand instances of models. • May expect to run operations on hundreds of thousands. • Is it WEBSCALE?
  25. 25. We Usually Deal With This by Over-engineering • Adding extra caching layers • Load balancing with horizontal scaling • Progressive page loading
  26. 26. Subverting Your ORM for Fun and Profit
  27. 27. • We can wrest the raw database connection from the ORM • With this we can improve performance without greatly increasing complexity
  28. 28. A real world example • Web app for client that surveyed organisational performance • Produced an online report with several different breakdowns of statistics from the survey • Was surprisingly slow - hit web server timeout
  29. 29. Looking closer Most of the time spent outside of the database
  30. 30. • 100,532 calls to Class#new ‼ • A simple page was only creating ~900 objects • One suspect was a function calculating the average of responses to a group of questions (a “domain”)
  31. 31. sum = 0.0 count = 0.0 domain.questions.each do |q| response = q.response_for(respondent) sum += response.value count += 1.0 end if count > 0 return sum / count else return 1.0 end
  32. 32. conn = ActiveRecord::Base.connection result = conn.execute <<-SQL SELECT SUM(responses.value) as sum, COUNT(*) as count FROM domains INNER JOIN questions ON questions.domain_id = domains.id INNER JOIN responses ON responses.question_id = questions.id AND responses.respondent_id = #{respondent.id} WHERE domains.id = #{domain.id} SQL score = 1.0 sum = result[0]['sum'].to_f count = result[0]['count'].to_f score = sum / count if count > 0
  33. 33. • Reduced page load time by more than a half • Reduced number of objects created by 30%
  34. 34. Words of Caution ⚠
  35. 35. • Not as maintainable • Need to keep an eye on security. 
 Never insert user provided values into raw SQL. • Not as portable. XKCD #327 https://xkcd.com/327/
  36. 36. sql = Cat.where(name: 'Ms Tibbles').to_sql ActiveRecord::Base.connection.execute sql Can sometimes get ORM to help you defeat itself
  37. 37. Cool Things Beyond Performance 😎
  38. 38. • Database functions • Common table expressions • Views • GIS extensions for geographical data 🌏 • Non-standard data types
  39. 39. Finally: Remember your indexes!
  40. 40. To Summarise • ORMs bring great benefits much of the time • However being aware of what they’re doing is essential • If need be, it is possible to work around them. • Databases are your friend.

×