Defense Against the Dark Arts Protecting Your Data Against ORMs
Object-Relational Mappers <ul><li>&quot;An object that wraps a row in a database table or view, encapsulates the database ...
Defense Against the Dark Arts <ul><li>Overview </li></ul><ul><ul><li>Relationships </li></ul></ul><ul><ul><li>Inheritance ...
Relationships <ul><li>Object Relationships </li></ul><ul><li>Maintained in application code  (transparent to developers) <...
Inheritance <ul><li>Concrete vs Single Table Inheritance </li></ul><ul><li>Caution: These are TOTALLY DIFFERENT. And confu...
Inheritance <ul><li>Postgres Table Inheritance  with  Abstract  parent class  </li></ul><ul><ul><li>Parent structure is re...
Inheritance <ul><li>Single Table Inheritance  for  Small Data  sets </li></ul><ul><ul><li>All child classes physically in ...
Data Types <ul><li>Standard Data Types </li></ul><ul><li>Port to new DBMS easily </li></ul><ul><li>Developers don’t have t...
Memory Usage <ul><li>RDBMSs store rows ORMs retrieve objects </li></ul><ul><li>Every time you use any piece of that object...
Data Integrity <ul><li>Safeguards & Dark Arts Trickery </li></ul><ul><li>Foreign Key / Relationship enforcement </li></ul>...
Version Control <ul><li>Schema Management </li></ul><ul><li>Ideally correlated with application changes </li></ul><ul><li>...
Connections <ul><li>Connection Persistence </li></ul><ul><li>Configuration is only evaluated at deploy time </li></ul><ul>...
Strategy: Know Your Data <ul><li>&quot;Trust is for people with poor surveillance” </li></ul><ul><li>  –  Col. James R. Tr...
Never Fight Alone
Upcoming SlideShare
Loading in …5
×

Defense Against the Dark Arts: Protecting Your Data from ORMs

5,217 views

Published on

Object-relational mappers, or ORMs, enable rapid software development by allowing application developers to treat database entities similar to objects within an application. This can increase productivity drastically, but has unfortunate implications for DBAs and anyone who actually looks at data created by the application. This talk will help DBAs, Developers-turned-DBAs, and anyone in between understand how to leverage and limit, as necessary, ORMs. The talk covers what types of data ORMs are really great for, and how to look for and understand the nuances that may impact performance or compromise data integrity in an application.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Defense Against the Dark Arts: Protecting Your Data from ORMs

  1. 1. Defense Against the Dark Arts Protecting Your Data Against ORMs
  2. 2. Object-Relational Mappers <ul><li>&quot;An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.” </li></ul><ul><li>Fundamental Perspective Shift => </li></ul><ul><li>Inevitably, something will be lost </li></ul><ul><ul><li>Enables Rapid Development </li></ul></ul><ul><ul><li>Simplifies Application Code </li></ul></ul><ul><ul><li>Standardizes Relationships </li></ul></ul><ul><ul><li>Standardizes Data Structures </li></ul></ul><ul><ul><li>Sometimes Sucks </li></ul></ul>
  3. 3. Defense Against the Dark Arts <ul><li>Overview </li></ul><ul><ul><li>Relationships </li></ul></ul><ul><ul><li>Inheritance </li></ul></ul><ul><ul><li>Data Types </li></ul></ul><ul><ul><li>Memory Usage </li></ul></ul><ul><ul><li>Data Integrity </li></ul></ul><ul><ul><li>Version Control </li></ul></ul><ul><ul><li>Connections </li></ul></ul>
  4. 4. Relationships <ul><li>Object Relationships </li></ul><ul><li>Maintained in application code (transparent to developers) </li></ul><ul><li>Simplified Validation (easier for developers to remember) </li></ul><ul><li>Cheap, but viable alternative to foreign key constraints </li></ul><ul><li>Examples </li></ul><ul><li>1 : ∞ | one-to-many = A has_many B </li></ul><ul><li>1 : ∞ | one-to-many = A has_many B through A_B </li></ul><ul><li>1 : 1 | one-to-one = A has_one C </li></ul><ul><li>∞ : 1 | many-to-one = B belongs_to A </li></ul><ul><li>1 : 1 | one-to-one = C belongs_to A </li></ul><ul><li>∞ : ∞ | many-to-many = D has_and_belongs_to_many A </li></ul>
  5. 5. Inheritance <ul><li>Concrete vs Single Table Inheritance </li></ul><ul><li>Caution: These are TOTALLY DIFFERENT. And confusing. </li></ul><ul><li>Use PG Table Inheritance with Abstract parent class </li></ul><ul><li>(or Partitioning) </li></ul><ul><ul><li>Parent structure allows code reuse and some helpful queries </li></ul></ul><ul><ul><li>Child tables are physically separate, so have their own performance metadata - indexes, keys, etc. </li></ul></ul><ul><li>Use Single Table Inheritance for Small Data sets </li></ul><ul><ul><li>All Child classes are physically in Parent table with “type” attribute </li></ul></ul><ul><ul><li>This sucks for a lot of data, and is hard to maintain & extend </li></ul></ul>
  6. 6. Inheritance <ul><li>Postgres Table Inheritance with Abstract parent class </li></ul><ul><ul><li>Parent structure is really for app code reuse, not giant tables </li></ul></ul><ul><ul><li>Child tables have their own performance metadata – indexes, etc. </li></ul></ul><ul><ul><li>class Weapon < ActiveRecord::Base </li></ul></ul><ul><ul><li>self.abstract_class = true </li></ul></ul><ul><ul><li>class Wand < Weapon </li></ul></ul><ul><li>CREATE TABLE Weapon( </li></ul><ul><li>id int, name text); </li></ul><ul><li>CREATE TABLE Wand( </li></ul><ul><li>wood text, length int, core text) </li></ul><ul><li>INHERITS (Weapon); </li></ul><ul><li>SELECT * FROM Weapons; --wands and other weapons </li></ul><ul><li>SELECT * FROM Wands; --only wands </li></ul>
  7. 7. Inheritance <ul><li>Single Table Inheritance for Small Data sets </li></ul><ul><ul><li>All child classes physically in Parent table with “type” attribute </li></ul></ul><ul><ul><li>This sucks for a lot of data, and is hard to maintain & extend </li></ul></ul><ul><ul><li>class Weapon < ActiveRecord::Base </li></ul></ul><ul><ul><li>class Wand < Weapon </li></ul></ul><ul><li>CREATE TABLE Weapon( </li></ul><ul><li>id int, name text, </li></ul><ul><li>wood text, length int, core text, </li></ul><ul><li>type text </li></ul><ul><li>); </li></ul><ul><li>SELECT * FROM Weapons WHERE type = ‘Wand’; </li></ul>
  8. 8. Data Types <ul><li>Standard Data Types </li></ul><ul><li>Port to new DBMS easily </li></ul><ul><li>Developers don’t have to learn new data types </li></ul><ul><li>Use tools written for any DBMS without modification </li></ul><ul><li>Miss out on Postgres awesomeness </li></ul><ul><li>Waste space & memory </li></ul><ul><li>Compromise data integrity </li></ul><ul><li>Examples </li></ul><ul><li>Custom Data Types: INTERVAL, smallint, floating point </li></ul><ul><li>Size Limitation: Zip code, Phone number, Email Address </li></ul>
  9. 9. Memory Usage <ul><li>RDBMSs store rows ORMs retrieve objects </li></ul><ul><li>Every time you use any piece of that object’s (row’s) data, you get back everything you ever added on to that model (table). </li></ul><ul><li>> Accounts.find(2) </li></ul><ul><li>SELECT * FROM &quot;accounts&quot; WHERE (&quot;accounts&quot;.&quot;id&quot; = 2) </li></ul><ul><li>> Accounts.find(2).updated_at </li></ul><ul><li>SELECT * FROM &quot;accounts&quot; WHERE (&quot;accounts&quot;.&quot;id&quot; = 2) </li></ul><ul><li>Number and data type of attributes per table DO matter </li></ul><ul><li>Watch out for large fields, TOAST data especially </li></ul>
  10. 10. Data Integrity <ul><li>Safeguards & Dark Arts Trickery </li></ul><ul><li>Foreign Key / Relationship enforcement </li></ul><ul><li>Standardized Validation in model (& thus across application) </li></ul><ul><li>NULL vs Empty String </li></ul><ul><ul><li>Defense: Look at data created in all scenarios. </li></ul></ul><ul><ul><li>The slightest application code difference can mean different data. </li></ul></ul><ul><ul><li>Varies by ORM. </li></ul></ul><ul><li>Object–to–row updates can Nullify an entire row </li></ul><ul><ul><li>Defense: Add NULL constraints to database </li></ul></ul><ul><ul><li>Specify if and how fields can be updated </li></ul></ul><ul><ul><li> (e.g. keys can’t be set to NULL) </li></ul></ul>
  11. 11. Version Control <ul><li>Schema Management </li></ul><ul><li>Ideally correlated with application changes </li></ul><ul><li>Rails db migrations stay in branch with dependent code </li></ul><ul><li>Migration scripts include up & down to reverse effects </li></ul><ul><li>Data Migrations </li></ul><ul><li>YMMV - Find the right tool for the job </li></ul><ul><ul><li>Iterative or set-based? </li></ul></ul><ul><ul><li>How much time do I have at run-time? </li></ul></ul><ul><ul><li>How will this impact the production site? </li></ul></ul><ul><li>Small dynamic migrations stay with the schema change logic </li></ul><ul><li>Adding/updating custom data should be separate </li></ul>
  12. 12. Connections <ul><li>Connection Persistence </li></ul><ul><li>Configuration is only evaluated at deploy time </li></ul><ul><li>Expense of creating & dropping connections is limited </li></ul><ul><li>Every call gets wrapped in a transaction </li></ul><ul><ul><li>Very important to remember for migrations and callback-style background processes sometimes naively launched in parallel </li></ul></ul>
  13. 13. Strategy: Know Your Data <ul><li>&quot;Trust is for people with poor surveillance” </li></ul><ul><li> – Col. James R. Trahan, USMC </li></ul><ul><li>Don't be at the mercy of your application code </li></ul><ul><li>Run Bad Data Checks </li></ul><ul><li>Cron job/Rake task to run stored checks & email results </li></ul><ul><li>CREATE TABLE data_checks( </li></ul><ul><li>id int, name text, </li></ul><ul><li>description text, check_sql text, fix_sql text); </li></ul><ul><li>Don't guess what’s happening, find out Monitor logs with PgFouine to find problem queries </li></ul><ul><li>System Tables (index usage, pg_stat, pg_stat_io, null fill) </li></ul>
  14. 14. Never Fight Alone

×