Rob Sullivan at Heroku's Waza 2013: Your Database -- A Story of Indifference


Published on

Rob Sullivan took the stage at this year's Waza 2013 to present "Your Database: A Story of Indiffence." For more from Rob, ping him at @datachomp.

For Waza videos stay tuned at or visit

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Rob Sullivan at Heroku's Waza 2013: Your Database -- A Story of Indifference

  1. 1. Your Database a story of indifference
  2. 2. Vanity PlateI’m RobEnterprise Data Magistrate (Ego Ridden Stable)SET twitter = @datachompSET blog = datachomp.comSET = @datachomp, balance = balance-$50
  3. 3. PreludeLife Imitates ArtDBA AppDev
  4. 4. Angry Love MakingWe are paid to care about different things
  5. 5. Tender Hug Making But we have a common goal
  6. 6. First World ProblemsWhat happens when you can’t afford a DBA?Who is the primary care provider for the DB? This is where our story begins
  7. 7. CharactersYour DB Your AppThe Hero Knuckle Dragger
  8. 8. ~Story 1~You’re not my Data Type
  9. 9. ORM As A Service:mycolumn, :integer:mydate, :timestampints are 4 bytes for every row - 2Bsmallints are 2 bytes for every row - 32Kdate = 4 bytestimestamp = 8 bytes
  10. 10. The SettingCREATE TABLE thinner (id smallserial, things smallint, CONSTRAINT thinner_pkey PRIMARY KEY (id));CREATE TABLE bloaty (id bigserial, things bigint, CONSTRAINT bloaty_pkey PRIMARY KEY (id)); Table Size Index Insert 10K thinner 384kb 240kb records bloaty 464kb 240kb
  11. 11. Healthy Diet of SufferingWhen they key is added to a larger tableCREAT TABLE smallertime (id serial, fk_numbers smallint, stringers varchar(80), recents timestamp default now()) And indexes are added I need sweat pants Table Size Index smaller 97MB 113MB larger 111MB 121MB
  12. 12. Death by 1000 BytesDisks are cheap, but good disks are notBigger DB = Longer Backups/Recovery Don’t waste your buffer cache Partitions are free in Postgres
  13. 13. ~Story 2~You’re Making Me Sick
  14. 14. Sipping on Gin and GiST B-Tree, Gin, GiST, Partial, Expression, Unique Full Text Index / Searching
  15. 15. SargabilityIndexes are only awesome when you can use them Btree < <= = >= >WHERE sucks LIKE ‘%tablescan’WHERE suckyfunc(column) = ‘pain’
  16. 16. So Worthless Indexes are not a get out of perf jail free card They have to be maintained They have to have writesSELECT * FROM pg_stat_all_indexes WHERE schemaname<>pg_catalog
  17. 17. So WorthlessIndexes are not a get out of perf jail free card They have to be maintained They have to have writes There Can Be Dupes!SELECT pg_size_pretty(sum(pg_relation_size(idx))::bigint) AS size, (array_agg(idx))[1] AS idx1, (array_agg(idx))[2] AS idx2, (array_agg(idx))[3] AS idx3, (array_agg(idx))[4] AS idx4FROM ( SELECT indexrelid::regclass AS idx, (indrelid::text ||En|| indclass::text ||En||indkey::text ||En|| coalesce(indexprs::text,)||En ||coalesce(indpred::text,)) AS KEY FROM pg_index) subGROUP BY KEY HAVING count(*)>1ORDER BY sum(pg_relation_size(idx)) DESC;
  18. 18. ~Story 3~No One Is Listening My tuples are 43 doing the 5  Harlem Shake
  19. 19. Do you even add bro? R × S = {(r1, r2, ..., rn, s1, s2, ..., sm) | (r1, r2, ..., rn) ∈ R, (s1, s2, ..., sm) ∈ S} U := T- R Cartesian ProductsRelational Algebra R ⟗ S = (R ⟕ S) (R ⟖ S) Outer Join R φ S = σφ(R × S) equijoins
  20. 20. Explain YourselfThe engine can talk:EXPLAIN SELECT * FROM sites; Seq Scan on sites (cost=0.00..1.09 rows=9 width=46)EXPLAIN (BUFFERS true, ANALYZE true, FORMATYAML) SELECT * FROM sites; - Plan: Node Type: "Seq Scan" Relation Name: "sites" Alias: "sites" Startup Cost: 0.00 Total Cost: 1.09 Plan Rows: 9 Plan Width: 46 Actual Total Time: 0.011 Actual Rows: 9
  21. 21. Kind of a big dealStats - they are a pretty big dealSELECT * FROM pg_stat_all_tablesVACUUM ANALYZE <tablename>
  22. 22. Real big deal There are lots of helpful tables pg_stat_activity pg_stat_all_indexes pg_stat_databse pg_statio_user_tablespg_stat_user_tables pg_stat_user_functions Lots of other tools to show you stat info Lots of good existing queries Don’t have to reinvent the wheel
  23. 23. Set Based Operations Marry me Active Record N+1 is a performance death sentence RBAR - Row By Agonizing Row Always profile your App/ORM The App has context
  24. 24. Choose Your Own AdventureCare about your dataSo we can all live App’ily ever afterArtsy thank you - @tenderlove and @derekgrapeThank you - Heroku and other sponsorsThank you - the person making a differenceHappy Hour?!?!