Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Database Wizardry for Legacy Applications

1,755 views

Published on

This is the presentation from the PHP UK conference

Published in: Software, Technology
  • Hey! Stop when you hit £700 profit on the week! Want to get started and make £700 this week? He's running a special introductory offer for the start of 2020 but this ends soon. CLICK HERE TO JOIN.. ♣♣♣ http://t.cn/A6vAxKsh
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Real people just like you are kissing the idea of punching the clock for someone else goodbye, and embracing a new way of living. The internet economy is exploding, and there are literally THOUSANDS of great earnings opportunities available right now, all just one click away. ●●● http://ishbv.com/ezpayjobs/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • This Single Mother Makes Over $700 per Week Helping Businesses with their Facebook and Twitter Accounts! and Now You Can Too! ★★★ http://t.cn/AieX6y8B
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Database Wizardry for Legacy Applications

  1. 1. Database Wizardry for Legacy Applications Gabriela D’Ávila gabriela.io #PHPUK15 FEBRUARY, 2015The Brewery At London 19th-20th
  2. 2. WHOAMI
  3. 3. Huge fan… really! Almost 25k pieces.
 I told you I was a fan. Majored in Digital Game
 Development Sr. Software
 Engineer
  4. 4. x
  5. 5. What to expect? • Waging options • Spaghetti code x Legacy code • Real life examples
  6. 6. Respect
  7. 7. Legacy & Spaghetti Legacy
 code zone Spaghetti 
 code zone Turn programmer
 crazy zone
  8. 8. How can one be different from another?
  9. 9. Legacy x Spaghetti • No longer supported • Usually old technology • Development freeze • Tangled structure • May have current technology • Too many bugs to track
  10. 10. How to optimize your database?
  11. 11. Wait 10 years. paraphrasing Joe Armstrong, erlang creator
  12. 12. Moore’s Law “The number of transistors per square inch on integrated circuits doubles every year”.
  13. 13. Use structures correctly
  14. 14. JSON,Serialized Array/Object option_id option_name option_value option_value 1 24 time_format "g:i a" yes 2 43 comment_whitelist 1 yes 3 45 comment_registration 0 yes 4 76 default_comments_page newest yes 5 95 sidebars_widgets "a:3:{s:19:""wp_inactive_widgets"";a:0:{}s:9:""sidebar-1"";a:6:{i:0;s:8:""searcyes 6 96 cron "a:3:{i:1423456489;a:3:{s:16:""wp_version_check"";a:1:{s:32:""40cd750bba9870f18yes 7 100 _site_transient_update_plugins "O:8:""stdClass"":4:{s:12:""last_checked"";i:1423413299;s:8:""response"";a:0:{}yes
  15. 15. Another exampleLet’s see another example
  16. 16. Binary Tree • Materialized Path • Field size limit • Low performance on search • TEXT field Index: • Length limited • FULLTEXT only in MyISAM parent_id user_id level asc_path node 1 0 1 1 <null> <null> 2 1 2 2 ,1, L 3 1 13 2 ,1, R 4 2 3 3 ,2,1, L 5 2 61 3 ,2,1, R 6 13 23 3 ,13,1, L 7 13 22 3 ,13,1, R 8 3 4 4 ,3,2,1, L 9 3 156 4 ,3,2,1, R 10 22 1568 4 ,22,13,1, L 11 22 26 4 ,22,13,1, R 12 23 1476 4 ,23,13,1, L 13 23 690716 4 ,23,13,1, R 14 61 1051 4 ,61,2,1, L 15 61 62 4 ,61,2,1, R
  17. 17. sql> SELECT b.parent_id, b.user_id, b.level, b.asc_path, b.node FROM binary_tree b JOIN users u ON b.user_id = u.id WHERE b.asc_path LIKE ‘%,1,%’ OR b.user_id = 1 ORDER BY b.level, b.parent_id, b.node LIMIT 0, 16 [2015-02-10 21:27:43] 15 row(s) retrieved starting from 1 in 16905/16937 ms
  18. 18. sql> SELECT b.parent_id, b.user_id, b.level, b.asc_path, b.node FROM binary_tree b JOIN users u ON b.user_id = u.id WHERE (b.asc_path LIKE ‘%,1,%’ OR b.user_id = 1) AND (b.level BETWEEN 1 AND 4) ORDER BY b.level, b.parent_id, b.node [2015-02-13 23:39:27] 15 row(s) retrieved starting from 1 in 326/348 ms
  19. 19. 51x faster
  20. 20. Hierarchical Data Storage
  21. 21. Adjacent List (RDBMS) user_id lft rgt parent_id 1 1 2 13 <null> 2 2 3 61 1 3 13 23 22 1 4 3 4 156 2 5 61 1052 62 2 6 23 1476 690716 13 7 22 1568 26 13
  22. 22. Graph Database 1 2 361 13 23 22 4 1561051 62 1476 690716 1568 26
  23. 23. What is to learn? • Use the right structures • Avoid LIKE statements on a TEXT field • Review your query, the problem may be easier to solve than it seems • Different tools come with a trade-off
  24. 24. Let’s see a particular query
  25. 25. SELECT
 trans.id,
 trans.transaction_type,
 trans.value,
 trans.user_receiver_id, user_receiver.username,
 trans.user_sender_id,
 user_sender.username, trans.status,
 CASE trans.status
 WHEN 1 THEN 'COMPLETED'
 WHEN 2 THEN 'CANCELED'
 END
 AS transaction_status,
 trans.date_transaction
 FROM
 transactions AS trans
 INNER JOIN users AS user_receiver ON trans.user_receiver_id = user_receiver.id
 INNER JOIN users AS user_sender ON trans.user_sender_id = user_sender.id
 WHERE
 (trans.user_sender_id = 1
 OR trans.user_receiver_id = 1)
 AND (trans.transaction_type IN ('withdraw', ‘cancelWithdraw', ‘transactionFee', 'refundOderPaidWithBalance',
 ‘transferBetweenUsers', ‘creditcardFee', ‘creditcardWithdraw', 'creditDeposit',
 ‘creditWithdraw', ‘orderPayment’, ‘monthlyPayment', 'shippingFee'))
 AND trans.date_transaction >= '2013-05-01 00:00:00'
 AND trans.date_transaction <= '2013-05-31 23:59:59'
 ORDER BY trans.date_transaction DESC LIMIT 100;
  26. 26. How long did that query take?
  27. 27. • 2 INNER JOIN • Using foreign-key primary key relationship • Using varchar field as filter • This field is indexed • ORDER BY using a datetime field DESC • This field is a B-tree index, created DESC
  28. 28. DESC Index http://dev.mysql.com/doc/refman/5.5/en/create-index.html An index_col_name specification can end with ASC or DESC. These keywords are permitted for future extensions for specifying ascending or descending index value storage. Currently, they are parsed but ignored; index values are always stored in ascending order.
  29. 29. Same query in MongoDB
  30. 30. db.financial_report.find({ $or: [{ user_receiver_id: 1 }, { user_sender_id: 1 }], transaction_type: { $in: [ 'withdraw', 'cancelWithdraw', 'transactionFee', 'refundOderPaidWithBalance', 'transferBetweenUsers', 'creditcardFee', 'creditcardWithdraw', 'creditDeposit', 'creditWithdraw', 'orderPayment', 'monthlyPayment', 'shippingFee' ] }, transaction_date: { $gte: new ISODate('2013-03-01 00:00:00'), $lte: new ISODate('2013-03-31 23:59:59') } }).sort({ transaction_date: -1 }).limit(100)
  31. 31. SELECT
 [FIELDS] FROM
 transactions AS trans
 INNER JOIN users AS user_receiver ON trans.user_receiver_id = user_receiver.id
 INNER JOIN users AS user_sender ON trans.user_sender_id = user_sender.id
 WHERE
 (trans.user_sender_id = 1
 OR trans.user_receiver_id = 1)
 AND (trans.transaction_type IN ('withdraw', ‘cancelWithdraw', ‘transactionFee', 'refundOderPaidWithBalance',
 ‘transferBetweenUsers', ‘creditcardFee', ‘creditcardWithdraw', 'creditDeposit',
 ‘creditWithdraw', ‘orderPayment’, ‘monthlyPayment', 'shippingFee'))
 AND trans.date_transaction >= '2013-05-01 00:00:00'
 AND trans.date_transaction <= '2013-05-31 23:59:59'
 ORDER BY trans.date_transaction DESC LIMIT 100;
  32. 32. db.financial_report.find({ $or: [{ user_receiver_id: 1 }, { user_sender_id: 1 }], transaction_type: { $in: [ 'withdraw', 'cancelWithdraw', 'transactionFee', 'refundOderPaidWithBalance', 'transferBetweenUsers', 'creditcardFee', 'creditcardWithdraw', 'creditDeposit', 'creditWithdraw', 'orderPayment', 'monthlyPayment', 'shippingFee' ] }, transaction_date: { $gte: new ISODate('2013-03-01 00:00:00'), $lte: new ISODate('2013-03-31 23:59:59') } }).sort({ transaction_date: -1 }).limit(100)
  33. 33. What is to learn? • MySQL has faulty DESC indexes • Sometimes you need to use another tool to solve the problem
  34. 34. How does it work?
  35. 35. Passive version
  36. 36. App data transactions pending_sync_transactions
  37. 37. App pending_sync_transactions transactions data
  38. 38. App pending_sync_transactions transactions Triggerdata
  39. 39. App pending_sync_transactions transactions data
  40. 40. App pending_sync_transactions transactions data
  41. 41. Active version
  42. 42. Queueing App
  43. 43. data Queueing App
  44. 44. Queueing App data
  45. 45. data Queueing data App
  46. 46. data data Queueing data App
  47. 47. data data Queueing App
  48. 48. data data Queueing App
  49. 49. data data Queueing App
  50. 50. data data Queueing App
  51. 51. datadata Queueing App
  52. 52. What is to learn? • The best solution may not be the text-book solution • The fancy solution may be overkill
  53. 53. Making data searchable
  54. 54. App AWS DynamoDB
  55. 55. Getting from the database
  56. 56. App
  57. 57. Getting from a text file
  58. 58. App S3
  59. 59. What is to learn? • Add layer for search purposes • Using text-files as a manner of storing data to make it searchable
  60. 60. Summary
  61. 61. Summary • Always analyse the trade-off • Use the structures correctly • Simple solutions can be the perfect ones
  62. 62. Thank you! • Please rate this talk at joind.in: https://joind.in/13377 • Twitter: @gabidavila • Blog: gabriela.io

×