PostgreSQL 9.6 새 기능 소개

587 views

Published on

PostgreSQL 9.6 새 기능에 대한 소개입니다. ᅟParallel Query 및 복제 등 다수의 새 기능에 대한 내용이 담겨 있습니다.

Copyright: 서치(이지호) search5@gmail.com
License CC BY-NC-ND

Published in: Technology

PostgreSQL 9.6 새 기능 소개

  1. 1. PostgreSQL 9.6 새 기능 이지호(search5@gmail.com)
  2. 2. 강연자 소개 u 이름: 이지호, 호: 서치(슬기 서, 기다릴 치), 영문 중간 이름: persy u 소속: 한국방송통신대학교 대학원 정보과학과, (주)위비즈넷, 사단법인 E.S.C u 98년 알짜레드햇 6.0에서 PostgreSQL 6.4를 보고 아!!! 이거다!!
  3. 3. PostgreSQL은 어떤 데이터베이스에요? u PostgreSQL은 1986년 캘리포니아 버클리에서 시작해 1988년 만들어진 BSD 기반의 오픈소 스 데이터베이스 u 이후 여러번의 지속적인 개선을 통해 1996년 PostgreSQL 발표. u PostgreSQL이 제공하는 특징 u 연산자, 복합 자료형, 집계 함수, 자료형 변환자, 확장 함수 등 제공 u 테이블 생성 및 조회시 상속 기능 사용 가능 u 함수(프로시저) 사용 가능(pl/pgsql이 공식이나 python, java, php, perl, tcl도 가능) u 이름은 어떻게 부르나요? -> 여러분이 부르고 싶은대로 부르세요.
  4. 4. PostgreSQL 커뮤니티 u 한국 PostgreSQL 사용자 모임은 u http://www.facebook.com/groups/postgres.kr u 2016년 10월 10일 기준 1,030명에 이르는 회원수를 자랑합니다. u 모임은? u 매달 넷째주 일요일에 애 안봐도 되고, 애인 안 만나도 되고, 다른 할 일도 없는 분들이 모여 정기 기술 세미나를 오후 2시 ~ 오후 5시까지 하고, u 주말의 필수품목인 ‘알콜’ Drinking을 하지 않는 정말 건전한 모임을 합니다.
  5. 5. Parallel execution of sequential scans, joins and aggregates
  6. 6. Parallel execution of sequential scans, joins and aggregates u max_parallel_workers_per_gather (integer) - 병렬 쿼리를 실행할 작업자 수. 기본값 0 u max_worker_processes (integer) - 백그라운드 프로세스의 최대 수. 기본값: 시스템 코어 수 u parallel_setup_cost (floating point) - Sets the planner's estimate of the cost of launching parallel worker processes. The default is 1000. u parallel_tuple_cost (floating point) - Sets the planner's estimate of the cost of transferring one tuple from a parallel worker process to another process. The default is 0.1. u min_parallel_relation_size (integer) - Sets the minimum size of relations to be considered for parallel scan. The default is 8 megabytes (8MB).
  7. 7. Parallel execution of sequential scans, joins and aggregates =# EXPLAIN SELECT count(*) FROM partest; QUERY PLAN ----------------------------------------------------------------------- Aggregate (cost=37059.38..37059.39 rows=1 width=8) -> Seq Scan on partest (cost=0.00..31417.50 rows=2256750 width=0)
  8. 8. Parallel execution of sequential scans, joins and aggregates template1=# set max_parallel_workers_per_gather = 1; template1=# EXPLAIN SELECT count(*) FROM partest; QUERY PLAN ----------------------------------------------------------------------- Finalize Aggregate (cost=24556.00..24556.01 rows=1 width=8) -> Gather (cost=24555.88..24555.99 rows=1 width=8) Workers Planned: 1 -> Partial Aggregate (cost=23555.88..23555.89 rows=1 width=8) -> Parallel Seq Scan on partest (cost=0.00..20614.71 rows=1176471 width=0)
  9. 9. Parallel execution of sequential scans, joins and aggregates template1=# set max_parallel_workers_per_gather = 3; template1=# EXPLAIN SELECT count(*) FROM partest; QUERY PLAN ----------------------------------------------------------------------- Finalize Aggregate (cost=20266.88..20266.89 rows=1 width=8) -> Gather (cost=20266.67..20266.88 rows=1 width=8) Workers Planned: 2 -> Partial Aggregate (cost=19266.67..19266.68 rows=1 width=8) -> Parallel Seq Scan on partest (cost=0.00..17183.33 rows=833333 width=0)
  10. 10. Parallel execution of sequential scans, joins and aggregates template1=# set max_parallel_workers_per_gather = 5; template1=# explain analyze select min(col1) from partest; QUERY PLAN ----------------------------------------------------------------------- Finalize Aggregate (cost=127493.90..127493.91 rows=1 width=8) … -> Gather (cost=127493.37..127493.88 rows=5 width=8) … Workers Planned: 5 Workers Launched: 5 -> Partial Aggregate (cost=126493.37..126493.38 rows=1 width=8) … -> Parallel Seq Scan on partest (cost=0.00..121493.30 rows=…) … Planning time: 0.110 ms Execution time: 1422.139 ms
  11. 11. Parallel execution of sequential scans, joins and aggregates template1=# set max_parallel_workers_per_gather = 6; template1=# explain analyze select min(col1) from partest; QUERY PLAN ----------------------------------------------------------------------- Finalize Aggregate (cost=127493.90..127493.91 rows=1 width=8) … -> Gather (cost=127493.37..127493.88 rows=5 width=8) … Workers Planned: 6 Workers Launched: 6 -> Partial Aggregate (cost=126493.37..126493.38 rows=1 width=8) … -> Parallel Seq Scan on partest (cost=0.00..121493.30 rows=…) … Planning time: 0.110 ms Execution time: 1422.139 ms
  12. 12. Autovacuum no longer performs repetitive scanning of old data u PostgreSQL 9.6 부터 성능 향상을 위해 영구보관용 데이터에 대해서 auto vacuum 중지 u PostgreSQL 9.6 이전까지 모든 힙 페이지 스캔 u 9.6은 마지막 동결 이후 수정된 페이지만 스캔 u 이 결정 덕분에 쓰기가 드문 테이블 성능 향상 혜택
  13. 13. Synchronous replication now allows multiple standby servers for increased reliability How replication works
  14. 14. Synchronous replication now allows multiple standby servers for increased reliability synchronous_commit = off
  15. 15. Synchronous replication now allows multiple standby servers for increased reliability synchronous_commit = local
  16. 16. Synchronous replication now allows multiple standby servers for increased reliability synchronous_commit = on (default)
  17. 17. Synchronous replication now allows multiple standby servers for increased reliability synchronous_commit = remote_write
  18. 18. Synchronous replication now allows multiple standby servers for increased reliability synchronous_commit = remote_apply
  19. 19. Synchronous replication now allows multiple standby servers for increased reliability u synchronous_standby_names =  ‘standby_name [,  …]’ u synchronous_standby_names =  ‘num (standby_name [,  …])’ u ex)  synchronous_standby_names =  ‘2(*)’ u synchronous_standby_names is  empty u If  this parameter  is  empty as  shown it  changes behaviour of   setting synchronous_commit to on, remote_write or remote_apply to  behave same   as local (ie,  the COMMIT will only wait for  flushing to  local disk).
  20. 20. postgres_fdw now supports remote joins, sorts, UPDATEs, and DELETEs FDW Foreign   Server Foreign   Server Foreign   Server SQL 쿼리 Join, Sort, Updates, Deletes
  21. 21. Allow Limiting of Snapshot Age old_snapshot_threshold 값은 서버를 시작할때만 설정 가능(DEFAULT: -1, 0, 1min - 60d) Session  1 Session  2 show  old_snapshot_threshold;; create  table  snaptest(x  int);; insert  into  snaptest values  (1);; begin  work;; set  transaction  isolation  level  serializable;; select  *  from  snaptest;; update  snaptest set  x  =  2;; select  pg_sleep(100);; vacuum  snaptest;; select  *  from  snaptest;; ERROR:snapshot  too  old;;
  22. 22. Allow Sessions with Long-Idle Transactions To Be Terminated u 오래 기다리고 있는 세션에 대해서 연결을 기다리지 않고 세션 자동 종료. default: 0 =# SET idle_in_transaction_session_timeout = ’2s’; =# BEGIN WORK; -- sit idle for 3 seconds =# SELECT 1; FATAL: terminating connection due to idle-in-transaction timeout server closed the connection unexpectedly
  23. 23. CREATE  EXTENSION  CASCADE  Support u PostgreSQL 9.6 이전에 EXTENSION 설치 시 종속성 부터 설치 =# create extension cube; =# create extension earthdistance; u PostgreSQL 9.6 부터는 EXTENSION의 control 파일에 종속성 정의 =# create extension earthdistance cascade; NOTICE: installing required extension "cube" CREATE EXTENSION
  24. 24. CREATE  EXTENSION  CASCADE  Support # blackhole extension comment = 'Minimal Extension Template’ default_version = '1.0’ module_pathname = '$libdir/blackhole’ relocatable = true requires = ’cube, jsonlog’ =# create extension blackhole cascade
  25. 25. SUPPORT  CROSSTABVIEW  IN  PSQL u PostgreSQL 9.6 이전엔 crosstab 쿼리는 tablefunc 모듈 사용. =# create table test2 as select ('{a,b,c,d,e,f}'::text[])[1 + floor(random() * 6)] as type_1, ('{m,n,o,p,q,r}'::text[])[1 + floor(random() * 6)] as type_2, random() * 1000 as some_float from generate_series(1,20000);
  26. 26. SUPPORT  CROSSTABVIEW  IN  PSQL u =# select type_1, type_2, count(*) from test2 group by type_1, type_2 crosstabview type_1 | q | n | o | r | m | p --------+-----+-----+-----+-----+-----+----- f | 519 | 569 | 515 | 593 | 519 | 560 e | 605 | 539 | 547 | 555 | 587 | 577 c | 574 | 590 | 534 | 549 | 541 | 556 d | 569 | 487 | 569 | 521 | 523 | 553 b | 543 | 594 | 547 | 573 | 548 | 590 a | 586 | 576 | 568 | 555 | 559 | 510 (6 rows) type_1  |  type_2  |  some_float -­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ 1  |          b                |            m          |409.59995565936 2  |   e                |            n            |453.834847547114 3  |   f                  |            o            |113.634209148586 4  |   b                |            n            |332.8404747881 5  |   d                |            n            |832.656755112112 6  |   b                |            o            |525.860982947052 7  |   b                |              r            |873.10529500246
  27. 27. Generic  WAL  facility u PostgreSQL 9.6은 외부 개발자가 CUSTOM EXTENSION 개발을 쉽게 할 수 있도록 일반화된 WAL 인터페이스 제공 u GenericXLogStart u GenericXLogRegisterBuffer u GenericXLogFinish u GenericXLogAbort
  28. 28. Allow COPY to copy the output of an INSERT/UPDATE/DELETE ... RETURNING query =# COPY (INSERT INTO tab_data VALUES (1, 'aa'), (2, 'bb') RETURNING a, b) TO stdout; 1 aa 2 bb =# COPY (UPDATE tab_data SET b = 'cc' WHERE a = 1 RETURNING *) TO stdout; 1 cc =# COPY (DELETE FROM tab_data WHERE a = 2 RETURNING *) TO stdout; 2 bb
  29. 29. Prompt variable to show PId of backend u psql 프롬프트에 Postmaster PID 표시 u 주의)  pg_bouncer, pg_pool 등 사용시 PID 예측 불가 =# set PROMPT1 '%n@%/ : %p $ ' depesz@depesz : 26853 $ u prompt에 사용 가능한 모든 옵션: u https://www.postgresql.org/docs/current/static/app-­psql.html#APP-­PSQL-­PROMPTING
  30. 30. Monitoring Features u Add System Catalog u Vacuum progress reporting u pg_config System View u System view to monitor WAL receiver status u Modified System Catalog u pg_stat_activity wait-type reporting u Add System Function u pg_control Values Exposed u Notification queue monitoring
  31. 31. Monitoring Features - pg_stat_activity wait-type reporting u PostgreSQL 9.6에서 pg_stat_activity catalog 컬럼 추가 u wait_event_type - The type of event for which the backend is waiting u wait_event - Wait event name if backend is currently waiting u The value of wait_event_type column • LWLockNamed - Waiting by a particular lightweight lock • LWLockTranche - Waiting by the lightweight lock for the group • Lock - Waiting by weight lock (LOCK TABLE statement etc) • BufferPin - PIN waiting for the buffer u wait_event • https://www.postgresql.org/docs/9.6/static/monitoring-stats.html
  32. 32. Monitoring Features - Vacuum progress reporting u This catalog provides information about the state of the progress of vacuum processing. This catalog is readable by the general superuser. Column  Name Data  Type Description   pid integer   Process  ID  of  backend datid oid OID  of  database datname name Connected  database  name relid oid OID  of  the  table  being  vacuumed phase text Current  processing  phase  of  vacuum heap_blks_total bigint Total  number  of  heap  blocks  in  the  table heap_blks_scanned bigint Number  of  heap  blocks  scanned heap_blks_vacuumed bigint Number  of  heap  blocks  vacuumed index_vacuum_count bigint Number  of  completed  index  vacuum  cycles max_dead_tuples bigint Number  of  dead  tuples  that  we  can  store  before  needing  to  perform  an  index   vacuum  cycle num_dead_tuples bigint Number  of  dead  tuples  collected  since  the  last  index  vacuum  cycle
  33. 33. Monitoring Features - Vacuum progress reporting postgres=# SELECT * FROM pg_stat_progress_vacuum ; -[ RECORD 1 ]-----------+-------------- pid | 3184 datid | 16385 datname | demodb relid | 16398 phase | scanning heap heap_blks_total | 10052 heap_blks_scanned | 2670 heap_blks_vacuumed | 2669 index_vacuum_count | 0 max_dead_tuples | 291 num_dead_tuples | 185
  34. 34. Monitoring Features - pg_config System View u pg_config를 시스템 유저 권한으로 SQL로 조회 u Column: name, setting postgres=#  SELECT  *  FROM  pg_config  ;   name   |  setting -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐+-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ BINDIR   |  /usr/local/pgsql/bin DOCDIR   |  /usr/local/pgsql/share/doc HTMLDIR   |  /usr/local/pgsql/share/doc INCLUDEDIR   |  /usr/local/pgsql/include PKGINCLUDEDIR  |  /usr/local/pgsql/include
  35. 35. Monitoring Features - pg_control Values Exposed Name Return  Type Description pg_control_checkpoint() record Returns  information  about  current  checkpoint   state. pg_control_system() record Returns  information  about  current  control  file  s tate. pg_control_init() record Returns  information  about  cluster  initialization   state. pg_control_recovery() record Returns  information  about  recovery  state. They also show information about write-ahead logging and checkpoint processing. This information is cluster-wide, and not specific to any one database
  36. 36. Monitoring Features - System view to monitor WAL receiver status This catalog provides information about the state of a slave instance's WAL receiver process in replication environment. This catalog is readable by the general user. pid integer Process  ID  of  the  WAL  receiver  process status text Activity  status  of  the  WAL  receiver  process receive_start_lsn pg_lsn First  transaction  log  position  used  when  WAL   receiver  is  started receive_start_tli integer First  timeline  number  used  when  WAL  receiver  is   started received_lsn pg_lsn Last  transaction  log  position  already  received  and   flushed  to  disk,  the  initial  value received_tli integer Timeline  number  of  last  transaction  log  position   received  and  flushed  to  disk,  the  initial  value last_msg_send_time timestamp  with  time  zone Send  time  of  last  message  received  from  origin  WAL   sender
  37. 37. Monitoring Features - System view to monitor WAL receiver status (…) This catalog provides information about the state of a slave instance's WAL receiver process in replication environment. This catalog is readable by the general user. pid integer Process  ID  of  the  WAL  receiver  process last_msg_receipt_time timestamp  with  time  zone Receipt  time  of  last  message  received  from  origin   WAL  sender latest_end_lsn pg_lsn Last  transaction  log  position  reported  to  origin  WAL   sender latest_end_time timestamp  with  time  zone Time  of  last  transaction  log  position  reported  to   origin  WAL  sender slot_name text Replication  slot  name  used  by  this  WAL  receiver conninfo text Connection  string  used  by  this  WAL  receiver,  with   security-­sensitive  fields  obfuscated.
  38. 38. Monitoring Features - Notification queue monitoring u PostgreSQL 큐는 8GB까지의 제한 존재 u DBA 관리자가 큐에 남은 작업이 있는지 확인할 필요 =# select pg_notification_queue_usage(); u 이 함수가 반환하는 실수 양에 따라 남아있는 작업량 판단 가능
  39. 39. 감사합니다. Q&A

×