Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PostgreSQLでスケールアウト

8,952 views

Published on

スケールするSQLシステム勉強会での講演資料です。
<https://connpass.com/event/64558/>

Published in: Technology
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ,DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

PostgreSQLでスケールアウト

  1. 1. Copyright©2017 NTT corp. All Rights Reserved. PostgreSQLでスケールアウト NTT OSSセンタ 澤田 雅彦 スケールアウトするSQLシステム (8/22)
  2. 2. 2Copyright©2017 NTT corp. All Rights Reserved. 自己紹介 澤田 雅彦 Twitter : @sawada_masahiko GitHub: MasahikoSawada NTT OSSセンタ勤務 PostgreSQL Contributor Freeze Map(PG9.6) Multiple Synchronous Replication(PG9.6) Quorum-based Synchronous Replication(PG10) 本日紹介する機能も開発中 PostgreSQL技術サポート PostgreSQL周辺ツールの開発
  3. 3. 3Copyright©2017 NTT corp. All Rights Reserved. 様々な方法がある • PostgreSQLのフォーク版 • Amazon Redshift (Amazon Web Service) • Greenplum Database (Pivotal社) • Postgres Pro (Postgres Professional社) • Postgres-XL (2nd Quadrant社) • PostgreSQL+拡張モジュール • Citus (Citus data社) • PostgreSQLのみ • FDW-based Sharding スケールアウトするために
  4. 4. 4Copyright©2017 NTT corp. All Rights Reserved. Postgres-XL https://www.postgres-xl.org/overview/
  5. 5. 5Copyright©2017 NTT corp. All Rights Reserved. Citus https://docs.citusdata.com/en/v6.1/performance/query_processing.html
  6. 6. 6Copyright©2017 NTT corp. All Rights Reserved. • FDW(Foreign Data Wrapper)とは? • PostgreSQLを”ハブ”のようにして、外部データ(他のDBMS、 NoSQL、ファイル、Webサービス等)と連携できる機能 FDW-based Sharding Oracle PostgreSQL CSV ファイル SELECT * FROM oracle_tbl WHERE amount < 100; oracle_tbl pg_tbl file_tbl PostgreSQL postgres_fdw oracle_fdw fdw_fdw
  7. 7. 7Copyright©2017 NTT corp. All Rights Reserved. 商用 DBMS 異種DB間連携 FDWの使い方の例 ○○部署 △△部署 △△部署のデータ も見れるようした いけど、向こうは 商用DBを使って る。データを二重 でも持つのは避け たいし・・ oracle_fdw
  8. 8. 8Copyright©2017 NTT corp. All Rights Reserved. • FDW-based Shardingとは? • FDW機能とテーブルパーティショニングを組み合わせてデータ ベースシャーディングを行うというアイディア FDW-based Sharding SELECT * FROM oracle_tbl WHERE amount < 100; PostgreSQL PostgreSQL postgres_fdw postgres_fdw PostgreSQL PostgreSQL postgres_fdw
  9. 9. 9Copyright©2017 NTT corp. All Rights Reserved. • FDW機能の特性上、連携先はPostgreSQLでなくても良い • どちらの機能もPostgreSQLが標準機能として提供 • シャードノードを増やすことでスケールアウト可能 FDW-based Shardingの特徴① PostgreSQL PostgreSQL PostgreSQL PostgreSQL postgres_fdw PostgreSQL・・・
  10. 10. 10Copyright©2017 NTT corp. All Rights Reserved. • クライアントは一つのサーバにのみアクセスするだけで良い • アプリケーションの変更がいらない • WHERE句、JOIN、SORT等はシャードサーバへプッシュダウンされる FDW-based Shardingの特徴② PostgreSQL PostgreSQL PostgreSQL PostgreSQL postgres_fdw PostgreSQL・・・
  11. 11. 11Copyright©2017 NTT corp. All Rights Reserved. FDW-based Shardingの仕組み PostgreSQL 親テーブル 子 外部テーブル 子 外部テーブル PostgreSQLテーブル PostgreSQLテーブル postgres_fdw テーブル パーティショニン グ機能 子テーブル FDW機能
  12. 12. 12Copyright©2017 NTT corp. All Rights Reserved. FDW-based Shardingの仕組み PostgreSQL 親テーブル 子 外部テーブル 子 外部テーブル PostgreSQLテーブル PostgreSQLテーブル postgres_fdw 子テーブル テーブルパーティショニング機能によっ て、SELECTでは振り分けられる
  13. 13. 13Copyright©2017 NTT corp. All Rights Reserved. FDW-based Shardingの仕組み PostgreSQL 親テーブル 子 外部テーブル 子 外部テーブル PostgreSQLテーブル PostgreSQLテーブル postgres_fdw 子テーブル 振り分けられた先が外部テーブルなら、 外部のサーバへ処理を依頼。 ローカルにあるテーブルなら、自分で処 理する。
  14. 14. 14Copyright©2017 NTT corp. All Rights Reserved. • 現在PostgreSQLコミュニティにて鋭意開発中なため、い くつか制約がある(PostgreSQL 9.6現在) • パーティションテーブルの運用が面倒※ • 集約処理はプッシュダウンされない※ • クラスタリング機能(高可用性)はコミュニティから提供 されていない • 分散トランザクション機能がない など ※PostgreSQL 10で一部改善 FDW-based Shardingは鋭意開発中
  15. 15. 15Copyright©2017 NTT corp. All Rights Reserved. FDW-based Shardingは鋭意開発中 PostgreSQL PostgreSQL PostgreSQL PostgreSQL postgres_fdw • マルチマスター構成も組もうと思えば組める • が、一貫性のある結果が返ってくる保証はない(更新が混ざった場合) • また、耐障害性について別途作り込む必要がある PostgreSQL postgres_fdw
  16. 16. 16Copyright©2017 NTT corp. All Rights Reserved. • PostgreSQLもスケールアウトできる • 商用製品、OSS製品など様々 • PostgreSQLのみを使ったスケールアウト構成も可能 • 開発中でもあるので制約をよく把握して使うこと まとめ
  17. 17. 17Copyright©2017 NTT corp. All Rights Reserved. ご清聴ありがとうございました
  18. 18. 18Copyright©2017 NTT corp. All Rights Reserved. Sort Push Down -- 9.5 Sort Output: p.col Sort Key: p.col -> Append -> Seq Scan on public.p Output: p.col -> Foreign Scan on public.s1 Output: s1.col Remote SQL: SELECT col FROM public.s1 -> Foreign Scan on public.s2 Output: s2.col Remote SQL: SELECT col FROM public.s2 -- 9.6 Merge Append Sort Key: p.col -> Sort Output: p.col Sort Key: p.col -> Seq Scan on public.p Output: p.col -> Foreign Scan on public.s1 Output: s1.col Remote SQL: SELECT col FROM public.s1 ORDER BY col ASC NULLS LAST -> Foreign Scan on public.s2 Output: s2.col Remote SQL: SELECT col FROM public.s2 ORDER BY col ASC NULLS LAST =# EXPLAIN (verbose on, costs off) SELECT * FROM p ORDER BY col;

×