Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Our answer to Uber

1,742 views

Published on

Uber’s blog post about migration from PostgreSQL to MySQL made a lot of buzz in PostgreSQL community. Many of Developers PostgreSQL community realized shortcoming of our table engine (which is the only one yet). As result, many of patches were developer in order to overcome the shortcomings mentioned by Uber. Some of those patches are overlapping, even some of them are in contradictory. Those patches include: indirect indexes (indexes which references primary key value), WARM (write-amplification reduction method), RDS (recently dead storage). Also there are discussions about pluggable table engines and undo log.

In this talk I’ll consider points of Uber’s blog post from PostgreSQL developer point of view. I’ll tell which points I agree, which points I disagree and which points I partially agree. Also I’ll consider developments of PostgreSQL community, and how them can overcome mentioned shortcomings from my point of view.

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Our answer to Uber

  1. 1. Our answer to Uber Alexander Korotkov Postgres Professional April 7, 2017 Alexander Korotkov Our answer to Uber 1 / 31
  2. 2. Russian developers of PostgreSQL: Alexander Korotkov, Teodor Sigaev, Oleg Bartunov ▶ Speakers at PGCon, PGConf: 20+ talks ▶ GSoC mentors ▶ PostgreSQL commi ers (1+1 in progress) ▶ Conference organizers ▶ 50+ years of expertship: development, audit, consul ng ▶ Postgres Professional co-founders PostgreSQL CORE ▶ Locale support ▶ PostgreSQL extendability: GiST(KNN), GIN, SP-GiST ▶ Full Text Search (FTS) ▶ NoSQL (hstore, jsonb) ▶ Indexed regexp search ▶ Create AM & Generic WAL ▶ Table engines (WIP) Extensions ▶ intarray ▶ pg_trgm ▶ ltree ▶ hstore ▶ plantuner ▶ jsquery ▶ RUM ▶ imgsmlr Alexander Korotkov Our answer to Uber 2 / 31
  3. 3. Disclaimer ▶ I’m NOT a MySQL expert. I didn’t even touch MySQL since 2011... ▶ This talk express my own opinion, not PostgreSQL community posi on, not even Postgres Professional official posi on. ▶ Uber’s guys knows be er which database they should use. Alexander Korotkov Our answer to Uber 3 / 31
  4. 4. What happened? ▶ Uber migrated from MySQL to PostgreSQL in 2012. ▶ Uber migrated from PostgreSQL to MySQL in 2016. ▶ PostgreSQL to MySQL migra on made a log of buzz in PostgreSQL community. Alexander Korotkov Our answer to Uber 4 / 31
  5. 5. Why did it happen? ▶ Uber migrated from MySQL to PostgreSQL for “a bunch of reasons, but one of the most important was availability of PostGIS” ¹ ▶ Uber migrated from PostgreSQL to MySQL “some of the drawbacks they found with Postgres” ² ¹https://www.yumpu.com/en/document/view/53683323/migrating-uber-from-mysql-to-postgresql ²https://eng.uber.com/mysql-migration/ Alexander Korotkov Our answer to Uber 5 / 31
  6. 6. Uber’s complaints to PostgreSQL Uber claims following “PostgreSQL limita ons”: ▶ Inefficient architecture for writes ▶ Inefficient data replica on ▶ Issues with table corrup on ▶ Poor replica MVCC support ▶ Difficulty upgrading to newer releases Alexander Korotkov Our answer to Uber 6 / 31
  7. 7. PostgreSQL’s vs. InnoDB’s storage formats Alexander Korotkov Our answer to Uber 7 / 31
  8. 8. PostgreSQL storage format ▶ Both primary and secondary indexes point to loca on (blkno, offset) of tuple (row version) in the heap. ▶ When tuple is moved to another loca on, all corresponding index tuples should be inserted to the indexes. ▶ Heap contains both live and dead tuples. ▶ VACUUM cleans up dead tuples and corresponding index tuples in a bulk manner. Alexander Korotkov Our answer to Uber 8 / 31
  9. 9. Update in PostgreSQL ▶ New tuple is inserted to the heap, previous tuple is marked as deleted. ▶ Index tuples poin ng to new tuple are inserted to all indexes. Alexander Korotkov Our answer to Uber 9 / 31
  10. 10. Heap-Only-Tuple (HOT) PostgreSQL ▶ When no indexed columns are updated and new version of row can fit the same page, then HOT is used and only heap is updated. ▶ Microvacuum can be used to free required space in the page for HOT. Alexander Korotkov Our answer to Uber 10 / 31
  11. 11. Update in MySQL ▶ Table rows are placed in the primary index itself. Updates are performed in-place. Old version of rows are placed to special segment (undo log). ▶ When secondary indexed column is updated, then new index tuple is inserted while previous index tuple is marked as deleted. Alexander Korotkov Our answer to Uber 11 / 31
  12. 12. Updates: InnoDB in comparison with PostgreSQL Pro: ▶ Update of few indexed columns is cheaper. ▶ Update, which don’t touch indexed columns, doesn’t depend on page free space in the page Cons: ▶ Update of majority of indexed columns is more expensive. ▶ Secondary index scan is slower. ▶ Primary key update is disaster. Alexander Korotkov Our answer to Uber 12 / 31
  13. 13. Uber example for write-amplifica on in PostgreSQL CREATE TABLE users (id SERIAL PRIMARY KEY, first TEXT, last TEXT, birth_year INTEGER); CREATE INDEX ix_users_first_last ON users (first, last); CREATE INDEX ix_users_birth_year ON users (birth_year); UPDATE users SET birth_year = 1986 WHERE id = 1; 1. Write the new row tuple to the tablespace 2. Update the primary key index to add a record for the new tuple 3. Update the (first, last) index to add a record for the new tuple 4. Update the birth_year index to add a record for the new tuple 5. Previous ac ons are protected by WAL log. Alexander Korotkov Our answer to Uber 13 / 31
  14. 14. Uber example for write-amplifica on: MySQL vs. PostgreSQL Alexander Korotkov Our answer to Uber 14 / 31
  15. 15. Uber example for write-amplifica on: MySQL vs. PostgreSQL PostgreSQL 1. Write the new row tuple to the tablespace 2. Insert new tuple to primary key index 3. Insert new tuple to (first, last) index 4. Insert new tuple to birth_year index 5. Previous ac ons are protected by WAL log. MySQL 1. Update row in-place 2. Write old version of row to the rollback segment 3. Insert new tuple to birth_year index 4. Mark old tuple of birth_year index as obsolete 5. Previous ac ons are protected by innodb log 6. Write update record to binary log Assuming we have replica on turned on Alexander Korotkov Our answer to Uber 15 / 31
  16. 16. Pending patches: WARM (write-amplifica on reduc on method) ▶ Behaves like HOT, but works also when some of index columns are updated. ▶ New index tuples are inserted only for updated index columns. https://www.postgresql.org/message-id/flat/20170110192442.ocws4pu5wjxcf45b%40alvherre.pgsql Alexander Korotkov Our answer to Uber 16 / 31
  17. 17. Pending patches: indirect indexes ▶ Indirect indexes are indexes which points to primary key value instead of pointer to heap. ▶ Indirect index is not updates un l corresponding column is updated. https://www.postgresql.org/message-id/20161018182843.xczrxsa2yd47pnru@alvherre.pgsql Alexander Korotkov Our answer to Uber 17 / 31
  18. 18. Ideas: RDS (recently dead store) ▶ Recently dead tuples (deleted but visible for some transac ons) are displaced into special storage: RDS. ▶ Heap tuple headers are le in the heap. Alexander Korotkov Our answer to Uber 18 / 31
  19. 19. Idea: undo log ▶ Displace old version of rows to undo log. ▶ New index tuples are inserted only for updated index columns. Old index tuples are marked as expired. ▶ Move row to another page if new version doesn’t fit the page. https://www.postgresql.org/message-id/flat/CA%2BTgmoZS4_CvkaseW8dUcXwJuZmPhdcGBoE_ GNZXWWn6xgKh9A%40mail.gmail.com Alexander Korotkov Our answer to Uber 19 / 31
  20. 20. Idea: pluggable table engines Owns ▶ Ways to scan and modify tables. ▶ Access methods implementa ons. Shares ▶ Transac ons, snapshots. ▶ WAL. https://www.pgcon.org/2016/schedule/events/920.en.html Alexander Korotkov Our answer to Uber 20 / 31
  21. 21. Types of replica on ▶ Statement-level – stream wri ng queries to the slave. ▶ Row-level – stream updated rows to the slave. ▶ Block-level – stream blocks and/or block deltas to the slave. Alexander Korotkov Our answer to Uber 21 / 31
  22. 22. Replica on types in PostgreSQL vs. MySQL Replica on Type MySQL PostgreSQL Statement-level buil n pgPool-II Row-level buil n pgLogical Londiste Slony ... Block-level N/A buil n Alexander Korotkov Our answer to Uber 22 / 31
  23. 23. Uber’s replica on comparison ▶ Uber compares MySQL replica on versus PostgreSQL replica on. ▶ Actually, Uber compares MySQL row-level replica on versus PostgreSQL block-level replica on. ▶ That happened because that me PostgreSQL had buil n block-level replica on, but didn’t have buil n row-level replica on. Simultaneously, MySQL had buil n row-level replica on, but didn’t have buil n block-level replica on. Alexander Korotkov Our answer to Uber 23 / 31
  24. 24. Uber’s complaints to PostgreSQL block-level replica on ▶ Replica on stream transfers all the changes at block-level including “write-amplifica on”. Thus, it requires very high-bandwidth channel. In turn, that makes geo-distributed replica on harder. ▶ There are MVCC limita ons for read-only requires on replica. Apply of VACUUM changes conflicts with read-only queries which could see the data VACUUM is going to delete. Alexander Korotkov Our answer to Uber 24 / 31
  25. 25. Is row-level replica on superior over block-level replica on? Alibaba works on adding block-level replica on to InnoDB. Zhai Weixiang, database developer from Alibaba considers following advantages of block-level replica on: ³ ▶ Be er performance: higher throughput and lower response me ▶ Write less data (turn off binary log and g d), and only one fsync to make transac on durable ▶ Less recovery me ▶ Replica on ▶ Less replica on latency ▶ Ensure data consistency (most important for some sensi ve clients) ³https://www.percona.com/live/data-performance-conference-2016/sessions/ physical-replication-based-innodb Alexander Korotkov Our answer to Uber 25 / 31
  26. 26. Replica read-only query MVCC conflict with VACUUM Possible op ons: ▶ Delay the replica on, ▶ Cancel read-only query on replica, ▶ Provide a feedback to master about row versions which could be demanded. Undo log would do be er, we wouldn’t have to choose... Alexander Korotkov Our answer to Uber 26 / 31
  27. 27. More about replica on and write-amplifica on MySQL row-level replica on PostgreSQL row-level replica on (pgLogical) Alexander Korotkov Our answer to Uber 27 / 31
  28. 28. Major version upgrade with pg_upgrade Alexander Korotkov Our answer to Uber 28 / 31
  29. 29. Major version upgrade with pgLogical https://www.depesz.com/2016/11/08/major-version-upgrading-with-minimal-downtime/ Alexander Korotkov Our answer to Uber 29 / 31
  30. 30. Other Uber notes ▶ PostgreSQL 9.2 had data corrup on bug. It was fixed long me ago. Since that me PostgreSQL automated tests system was significantly improved to evade such bugs in future. ▶ pread is faster than seek + read. Thats really gives 1.5% accelera on on read-only benchmark. ⁴ ▶ PostgreSQL advises to setup rela vely small shared_buffers and rely on OS cache, while “InnoDB storage engine implements its own LRU in something it calls the InnoDB buffer pool”. PostgreSQL also implements its own LRU in something it calls the shared buffers. And you can setup any shared buffers size. ▶ PostgreSQL uses mul process model. So, connec on is more expensive since unless you use pgBouncer or other external connec on pool. ⁴https://www.postgresql.org/message-id/flat/a86bd200-ebbe-d829-e3ca-0c4474b2fcb7%40ohmu.fi Alexander Korotkov Our answer to Uber 30 / 31
  31. 31. Thank you for a en on! Alexander Korotkov Our answer to Uber 31 / 31

×