Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

92 grand prix_2013


Published on

  • Be the first to comment

  • Be the first to like this

92 grand prix_2013

  1. 1. Josh Berkus LCA 2013
  2. 2. Write Cascading Read Performance Replication Scalability Index-only Scans Start OtherFeatures JSON DDL Migrations Range Types SP-GiST Admin Tools New Feature Grand Prix
  4. 4. Read Scalability● fast path sorting● reduced overhead frequent table locks● prproc shared memory optimization● laxy vxid locks● heap hot search buffer
  5. 5. Read Scalability 350,000 q.p.s.
  6. 6. Write Performance● better group commit● reduce WalInsertLock contention● batch tuple copy● reduced checkpoint activity
  7. 7. Write Scalability +50% more writes/second
  8. 8. Parallel Bulk Load 10-15 parallel streams up to 3X load speed
  9. 9. Index-Only Scans Col3ms A Cos table books Title C D Dea Dem 300ms Will There Be Faith? State of Wonder Sphere 18c Flr A H Hea Dead Eyes H I Het The Greatest Music N Stories Never Told Rea S N Skeleton Canyon Res U R The Long Snapper Ska Super Immunity S Sut The Barbary Pirates T Heat Tab Confessions of a The U Prairie Bitch W Uvh Talking Pictures Woz Reggie Jackson
  10. 10. “count(*) is slow in postgres”
  11. 11. Index-Only Scans Col3ms A Cos 30ms C Dea D Dem A H Hea H I Het N Rea S N Res U R Ska S Sut T Tab The U W Uvh Visibility Woz Map
  12. 12. Cascading Replication walrecieverwalsender walreciever walreciever
  13. 13. Cascading ReplicationSydney North Carolina walsender walsender walreciever walreciever walreciever
  14. 14. Replication Improvements● pg_receivexlog an xlog stream reader● standby-only backup no more snapshotting the master● recv vs. write modes for synchronous replication
  15. 15. JSON● JSON data type● array-to-json, row-to-json functions● get query results as JSON!
  16. 16. PL/v8
  17. 17. PL/v8● Based on Googles v8 JS engine● write fast stored procedures in JavaScript● create ad-hoc attribute indexes on JSON dataA data type and PL made for eachother!
  18. 18. create or replace functionfib(n int) returns int as $$ function fib(n) { return n<2 ? n : fib(n-1) + fib(n-2) } return fib(n)$$ LANGUAGE plv8 IMMUTABLESTRICT;
  19. 19. pl/coffee
  20. 20. PL/coffee=# CREATE OR REPLACE FUNCTIONpublic.fibonacci(n integer) RETURNS integer LANGUAGE plcoffeeIMMUTABLE STRICTAS $function$fibonacci = (x)-> return 0 if x == 0 return 1 if x == 1 return fibonacci(x-1) + fibonacci(x-2)return fibonacci n$function$;CREATE FUNCTION
  21. 21. Indexing7.1: Generalized Search Tree (GiST)8.1: Generalized Inverted Index (GIN)9.1: K-Nearest Neighbor (KNN)
  22. 22. 9.2: Space-GiST
  23. 23. SP-GiST● Based on “Space-Partitioning Trees”● Faster to read, faster to update than GiST ● Will cause PostGIS 2.1!
  24. 24. SP-GiSTknn=# create index pt_gist_idx on geousing gist(point);CREATE INDEXTime: 36672.283 msSize: 153,124,864knn=# create index pt_spgist_idx on geousing spgist(point);CREATE INDEXTime: 12805.530 ms ~ 3 times faster !Size: 153,788,416 ~ the same size
  25. 25. Range Types● Temporal range: [ 2012-04-10, 2012-04-12 )● Alpha Index: ( Abbe, Babel ]● Linear distance: ( 375.453, 374.441 )
  26. 26. DDL pit stop
  27. 27. runtime DDL made easy● DROP INDEX CONCURRENTLY● NOT VALID CHECK constraints● less rewriting for ALTER TABLE● ALTER IF EXISTS● RENAME domains, FDWs● vacuum skips locked pages
  28. 28. Instrumentation
  29. 29. better monitoring● new pg_stat_statements● more autovacuum logging● track_io_timing● deadlock, temp file counters● checkpoint timing in pg_stat_bgwriter
  30. 30. better EXPLAIN● track buffers dirtied/written● report filtered-out rows● no-TIMING option
  31. 31. Other Features● lower CPU wakeups (power saving)● more sepgsql security● optimizer improvements● pg_hba.conf fixes● XML improvements● exportable snapshots
  32. 32. Other Features● more pg_dump/pg_restore options● more psql features● more pg_upgrade fixes, improvements● dropped support for antiques ● dgux, nextstep, sunos4, svr4, ultrix4, univel, bsdi
  33. 33. Finish Line!
  34. 34. The Next Race: 9.3
  35. 35. 9.3 Features (so far)● More JSON goodness! ● accessor methods ● constructor methods● Writable Foreign Data Wrappers● Event Triggers ● write a trigger on anything● Updatable VIEWs
  36. 36. 9.3 Features (so far)● create your own background workers● LATERAL JOINs● Streaming-only Remastering● Improved FK locking● pg_stat_plans
  38. 38. More Information● Josh Berkus: ●● Upcoming Events ● Melbourne pgDay, Monday Feb 5 – ● PyPgDay at PyCon 2013 ● pgCon, Ottawa, May 21-25 Text and graphics of this talk are licensed Creative Commons Share-Alike, except where otherwise noted. Most racing photos in the presentation are stock photos which may only be reproduced and reused as a part of this presenstation. Graphs courtesy Robert Haas and Heikki Linnakangas. SP-Gist examples from Oleg Bartunov, PL/v8 and PL/Coffee examples courtesy of Will Leinweber.