92 grand prix_2013


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

92 grand prix_2013

  1. 1. Josh Berkus LCA 2013
  2. 2. Write Cascading Read Performance Replication Scalability Index-only Scans Start OtherFeatures JSON DDL Migrations Range Types SP-GiST Admin Tools New Feature Grand Prix
  4. 4. Read Scalability● fast path sorting● reduced overhead frequent table locks● prproc shared memory optimization● laxy vxid locks● heap hot search buffer
  5. 5. Read Scalability 350,000 q.p.s.
  6. 6. Write Performance● better group commit● reduce WalInsertLock contention● batch tuple copy● reduced checkpoint activity
  7. 7. Write Scalability +50% more writes/second
  8. 8. Parallel Bulk Load 10-15 parallel streams up to 3X load speed
  9. 9. Index-Only Scans Col3ms A Cos table books Title C D Dea Dem 300ms Will There Be Faith? State of Wonder Sphere 18c Flr A H Hea Dead Eyes H I Het The Greatest Music N Stories Never Told Rea S N Skeleton Canyon Res U R The Long Snapper Ska Super Immunity S Sut The Barbary Pirates T Heat Tab Confessions of a The U Prairie Bitch W Uvh Talking Pictures Woz Reggie Jackson
  10. 10. “count(*) is slow in postgres”
  11. 11. Index-Only Scans Col3ms A Cos 30ms C Dea D Dem A H Hea H I Het N Rea S N Res U R Ska S Sut T Tab The U W Uvh Visibility Woz Map
  12. 12. Cascading Replication walrecieverwalsender walreciever walreciever
  13. 13. Cascading ReplicationSydney North Carolina walsender walsender walreciever walreciever walreciever
  14. 14. Replication Improvements● pg_receivexlog an xlog stream reader● standby-only backup no more snapshotting the master● recv vs. write modes for synchronous replication
  15. 15. JSON● JSON data type● array-to-json, row-to-json functions● get query results as JSON!
  16. 16. PL/v8
  17. 17. PL/v8● Based on Googles v8 JS engine● write fast stored procedures in JavaScript● create ad-hoc attribute indexes on JSON dataA data type and PL made for eachother!
  18. 18. create or replace functionfib(n int) returns int as $$ function fib(n) { return n<2 ? n : fib(n-1) + fib(n-2) } return fib(n)$$ LANGUAGE plv8 IMMUTABLESTRICT;
  19. 19. pl/coffee
  20. 20. PL/coffee=# CREATE OR REPLACE FUNCTIONpublic.fibonacci(n integer) RETURNS integer LANGUAGE plcoffeeIMMUTABLE STRICTAS $function$fibonacci = (x)-> return 0 if x == 0 return 1 if x == 1 return fibonacci(x-1) + fibonacci(x-2)return fibonacci n$function$;CREATE FUNCTION
  21. 21. Indexing7.1: Generalized Search Tree (GiST)8.1: Generalized Inverted Index (GIN)9.1: K-Nearest Neighbor (KNN)
  22. 22. 9.2: Space-GiST
  23. 23. SP-GiST● Based on “Space-Partitioning Trees”● Faster to read, faster to update than GiST ● Will cause PostGIS 2.1!
  24. 24. SP-GiSTknn=# create index pt_gist_idx on geousing gist(point);CREATE INDEXTime: 36672.283 msSize: 153,124,864knn=# create index pt_spgist_idx on geousing spgist(point);CREATE INDEXTime: 12805.530 ms ~ 3 times faster !Size: 153,788,416 ~ the same size
  25. 25. Range Types● Temporal range: [ 2012-04-10, 2012-04-12 )● Alpha Index: ( Abbe, Babel ]● Linear distance: ( 375.453, 374.441 )
  26. 26. DDL pit stop
  27. 27. runtime DDL made easy● DROP INDEX CONCURRENTLY● NOT VALID CHECK constraints● less rewriting for ALTER TABLE● ALTER IF EXISTS● RENAME domains, FDWs● vacuum skips locked pages
  28. 28. Instrumentation
  29. 29. better monitoring● new pg_stat_statements● more autovacuum logging● track_io_timing● deadlock, temp file counters● checkpoint timing in pg_stat_bgwriter
  30. 30. better EXPLAIN● track buffers dirtied/written● report filtered-out rows● no-TIMING option
  31. 31. Other Features● lower CPU wakeups (power saving)● more sepgsql security● optimizer improvements● pg_hba.conf fixes● XML improvements● exportable snapshots
  32. 32. Other Features● more pg_dump/pg_restore options● more psql features● more pg_upgrade fixes, improvements● dropped support for antiques ● dgux, nextstep, sunos4, svr4, ultrix4, univel, bsdi
  33. 33. Finish Line!
  34. 34. The Next Race: 9.3
  35. 35. 9.3 Features (so far)● More JSON goodness! ● accessor methods ● constructor methods● Writable Foreign Data Wrappers● Event Triggers ● write a trigger on anything● Updatable VIEWs
  36. 36. 9.3 Features (so far)● create your own background workers● LATERAL JOINs● Streaming-only Remastering● Improved FK locking● pg_stat_plans
  38. 38. More Information● Josh Berkus: josh@pgexperts.com ● www.databasesoup.com● Upcoming Events ● Melbourne pgDay, Monday Feb 5 – 2013.pgday.org.au ● PyPgDay at PyCon 2013 ● pgCon, Ottawa, May 21-25 Text and graphics of this talk are licensed Creative Commons Share-Alike, except where otherwise noted. Most racing photos in the presentation are stock photos which may only be reproduced and reused as a part of this presenstation. Graphs courtesy Robert Haas and Heikki Linnakangas. SP-Gist examples from Oleg Bartunov, PL/v8 and PL/Coffee examples courtesy of Will Leinweber.