Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Effective DB Interaction


Published on

Effective DB Interaction - presentation by Thamarai K. in Core Java meetup #7.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Effective DB Interaction

  1. 1. Effective programming to interact with DB
  2. 2. Introduction •Thamaraikanni Panneerselvam •10+ years of IT experience • •
  3. 3. DB & ORM
  4. 4. Interaction which need attention •Fetching huge data •Interacting via ORM framework •Database connection maintenance •Misc.
  5. 5. Fetching Huge Data •Indexing •Avoid searching with non-indexed columns •Partitioning the huge table •Fetch the data in specific sequence from related table
  6. 6. Indexing •Index page of the book. Most common Index types are B-Tree and Hash. •B-Tree Index type will have separate table with indexed columns sorted in ascending order. •Hash Index works with memory backend supports, it supports Key-value stores.
  7. 7. Indexing Benefits •An index can filter and/or sort values •A leftmost prefix can be used •Indexes on several columns are useful •Order of columns in composite keys is important
  8. 8. Indexing Overhead •MySQL only uses 1 index per table per query •Can't index fullTEXT fields •Can speed up queries (good) •Increases the size of your dataset (bad) •Slows down writes (bad)
  9. 9. Indexing – Explain Plan
  10. 10. Partition
  11. 11. Partition •Partitioning divides a table into smaller parts called “partitions” •Queries works with specific memory blocks •Only works with Range and List partitioning •Ease of purging old data
  12. 12. Partition
  13. 13. Partition
  14. 14. Partition Overhead •More indexes! •Wrong partition key •Partitions over 124 will slow down queries
  15. 15. Partition - issue •Table had 13core records, which had 24 partitions, out of which 4 had values others are blank. •Response time was 800ms on load it crosses 1s. •Removed all the unused . •Response time reduced 400ms. •Resource utilization on DB for this specific query reduced by 35%
  16. 16. DB connection pool
  17. 17. DB connection pool •Readily available DB connections •Reduces the new DB connection creation •Minimizes the stale connections •Connection reused
  18. 18. DB connection Pool Parameters •Initial Pool •Max Size •Timeout •Validator query
  19. 19. DB connection Pool - Issues •All connections are in use. •Idle timeout •Maximum time exceed •User cancel the operation
  20. 20. ORM framework - Issues •Loading all the child objects •Many select queries for related tables •OneToMany relationship •Hibernate object are used only for the persistence, for business service use transactional objects.
  21. 21. Misc. •Caching •EXISTS •OUTER Join benefits •IN & IS NULL limitations •LIKE operator will not use Index
  22. 22. Example SELECT * FROM Employee WHERE department_id IN (SELECT department_id FROM department WHERE department_name = ‘Analyst’);
  23. 23. EXISTS SELECT * FROM Employee e WHERE EXISTS (SELECT 1 FROM department d WHERE department_name = ‘Analyst’ AND d.department_id = e.department_id);
  24. 24. Example SELECT * FROM Department d WHERE d.department_id not in (SELECT department_id FROM employee e);
  25. 25. Outer Joins SELECT * FROM Department d left outer join Employee e ON e.department_id = d.department_id WHERE d.department_id is null;
  26. 26. LIKE Operator SELECT * FROM Employee where name like ‘John%’; => Index will be used it is range search. SELECT * FROM Employee where name like ‘%Denslin’; => Index will not be used
  27. 27. Better Computation Product PromotionInventoryProduct Spec
  28. 28. Better Computation SELECT p.product_id , COUNT(DISTINCT ( CASE WHEN ps.ProductSpec_id IS NOT NULLTHEN ps.ProductSpec_id END ))T2_COUNT , COUNT(DISTINCT ( CASE WHEN i.inventory_id IS NOT NULLTHEN i.inventory_id END ))T3_COUNT , COUNT(DISTINCT ( CASE WHEN pr.promotion_id IS NOT NULLTHEN pr.promotion_id END ))T4_COUNT FROM (SELECTColumn1 FROM Product WHERE product_id IN ( :values ) ) p LEFT OUTER JOIN ProductSpec ps ON ps.product_id = p.product_id LEFT OUTER JOIN Inventory i ON i.product_id = p.product_id LEFT OUTER JOIN Promotion pr ON pr.product_id = p.product_id GROUP BY p.product_id
  29. 29. Better Computation - example
  30. 30. Better Computation - example
  31. 31. Better Computation - example SELECT p.product_name , Min(case when pa.Type = ‘Category’ then pa.Name end) Category , Min(case when pa.Type = ‘Seller’ then pa.Name end) Seller FROM Product p, Product_association pa WHERE p.product_id = pa.product_id GROUP BY p.product_name
  32. 32. Q & A
  33. 33. Thank you