Migrating from PostgreSQL to MySQL Without Downtime

9,217 views
8,917 views

Published on

How Etsy migrates live data into MySQL shards without downtime.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
9,217
On SlideShare
0
From Embeds
0
Number of Embeds
29
Actions
Shares
0
Downloads
84
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Migrating from PostgreSQL to MySQL Without Downtime

  1. 1. Migrating from PostgreSQL to MySQL ... without downtimeMatthew GrahamPercona Live NYC 2011
  2. 2. Global marketplace for buying and selling handmade goods Total April 2011• 8 million members • $38.7 million goods sold• 800k active shops • 2 million items sold• 1 billion page views / month • 1.8 million new items listed • 390k new members joined
  3. 3. Global marketplace for buying and selling handmade goods Total April 2011• 8 million members • $38.7 million = $900 / min• 800k active shops • 2 million items sold• 1 billion page views / month • 1.8 million new items listed • 390k new members joined
  4. 4. Downtime = -$
  5. 5. $320m$240m$160m$80m $0m 2006 2007 2008 2009 2010 Gross Merchandise Sales
  6. 6. Reasons To Migrate • Horizontal Scaling • Reduce Types of Databases • Licensing Costs • Functional Partitioning • Schema Refactor
  7. 7. Deciding to Switch expected quality - transition ----------------- net quality quality > actual
  8. 8. Deciding to Switch expected quality - transition----------------- net quality > actual quality
  9. 9. Deciding to Switch expected quality - transition----------------- net quality > actual quality
  10. 10. Source Migration Target
  11. 11. Why ?
  12. 12. Foundational Principles • Migrate One Table at a Time • Progressive Ramp Up • Data Duplication During Transition
  13. 13. 5 Steps Per Table 1. Create Target Tables 2. Tee Writes 3. Backfill 4. Read from Target Tables 5. Wrap Up
  14. 14. Create the Target
  15. 15. Merge into the Target
  16. 16. Separate the Target
  17. 17. Source OnlySource Target Application Writes
  18. 18. Teed WritesSource Target Application Writes
  19. 19. Generating IDs Source TargetID Name ID Name1 Adam 1 Adam2 Bill 2 Bill3 Charlie 3 David4 David 4 Charlie Ticket Server: http://bit.ly/dbtickets
  20. 20. Backfill
  21. 21. Application Code Backfillfunction backfill_user(user_id) { old_row = find_old_user(user_id); new_row = create_new_user(); new_row.id = old_row.id; ... store_new_user(new_row);}
  22. 22. ETL BackfillSource Target Load Extract Transform
  23. 23. Don’t Overload • Script bulk inserts 100-2000 row batchesINSERT INTO account (id, firstname, lastname)VALUES(1, ‘Alan’, ‘Alda’),(2, ‘Barry’, ‘Bonds’),(3, ‘Charlie’, ‘Chaplin’);
  24. 24. Backfill SpeedApplication Code ETL• Easier to Write • Faster Run Time
  25. 25. Backfill ExtractApplication Code ETL• Easier to Write • Faster Run Time• Less Likely to Get Out • Needs an Extract Source Of Sync
  26. 26. Backfill RerunsApplication Code ETL• Easier to Write • Faster Run Time• Less Likely to Get Out • Needs an Extract Source Of Sync• Handles Duplicates from • REPLACE and INSERT Multiple Executions ON DUPLICATE KEY UPDATE
  27. 27. Teed Writes then Backfill orBackfill then Teed Writes
  28. 28. Verification by Diff$ diff_user 111222333target user row is missing$ diff_user 123456789- source.user.address_state = ‘CA’+ target.user.address_state = ‘NY’
  29. 29. Verification by DiffCOMPARING 200 ROWS From: 111222197User Id: 111222333target user row is missingUser Id: 111222345- source.user.address_state = ‘CA’+ target.user.address_state = ‘NY’SUMMARY: total rows with errors: 2/200
  30. 30. Read from Target
  31. 31. Progressive Ramp Up 100% 0%Source Target Application Reads
  32. 32. Progressive Ramp Up 99% 1%Source Target Application Reads
  33. 33. Progressive Ramp Up 95% 5%Source Target Application Reads
  34. 34. Progressive Ramp Up 75% 25%Source Target Application Reads
  35. 35. Progressive Ramp Up 0% 100%Source Target Application Reads
  36. 36. Ramp Up Example# global configurationuse_new_tables: employees: true percent: 1# application codeif (enabled(‘use_new_tables’)) { $result = read_from_target();} else { $result = read_from_source();}
  37. 37. Enabled or Disabled?• Check cookies if user already assigned• $enabled = configured threshold > random %• Store $enabled in a cookie for future requests
  38. 38. Continuous Deployment • Ramp Up / Ramp Down • Backfill Fixes • Need Code Running on Prod to Proceed • Makes it Easier
  39. 39. No Foreign Keys
  40. 40. Wrapping Up
  41. 41. Analytical Data
  42. 42. Teed WritesSource Target Application Writes
  43. 43. Target OnlySource Target Application Writes
  44. 44. Things to Remove• Code to use old tables• Code to switch on configuration• Configuration• Drop the old tables... eventually
  45. 45. 5 Steps Per Table 1. Create Target Tables 2. Tee Writes 3. Backfill 4. Read from Target Tables 5. Wrap Up
  46. 46. Foundational Principles • Migrate One Table at a Time • Progressive Ramp Up • Data Duplication During Transition
  47. 47. Questions? • Yes, we’re hiring! Specifically MySQL Ops http://bit.ly/etsywantsawesome • http://codeascraft.etsy.com • http://twitter.com/lapsuMatthew GrahamPercona Live NYC 2011

×