Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime. -- The First "Table Centric" Approach to Solving SQL Performance Problems


Published on

"Too many calls to the database" is the #1 performance problem is server-side software.

wuqiSpank's unique "Table Centric" SQL visualizations show the repetitious SQL anti-patterns that kill system performance (like SELECT N+1).

In 2013, $2.4 Billion was spent on APM (Application Performance Monitoring) tools. None of them get to the heart of the problem like wuqiSpank.

See for more.

Published in: Technology
  • Be the first to comment

  • Be the first to like this -- The First "Table Centric" Approach to Solving SQL Performance Problems

  1. 1. / Slide 1 / STLCMG
  2. 2. / Slide 2 / STLCMG Performance Center of Excellence at FIS -- #1 in the FinTech100
  3. 3. / Slide 3 / STLCMG Why wuqiSpank?
  4. 4. / Slide 4 / STLCMG Who needs wuqiSpank? We should be able to code without this kind of an effort:
  5. 5. / Slide 5 / STLCMG We should be able to code without these kinds of obvious performance defects – 2,256 SQL invocations with a single button click.
  6. 6. / Slide 6 / STLCMG The #1 Server Side Perf Problem Sampling of Performance Defects ● 2,272 SELECTs to table [BLAH] (single button click) ● The code is querying [TABLE BLAH] 15 times a second for data the rarely, if ever, changes ● [TABLE BLAH] hit 37k times for 250 logons. ● Both of these SQL are executed 548,976 times in a 15 minutes duration ● 82 sql stmts with single button click. Many of the tables are accessed repeatedly during the request.Here are counts of times each table is accessed with a SELECT: 11 – BLAH 11 – OTHER_BLAH 16 -- BLAH_BLAH
  7. 7. / Slide 7 / STLCMG The #1 Server Side Perf Problem Other examples (industry 1) ● 2003 -- Martin Fowler ● “Try to pull back multiple rows at once. In particular, never do repeated queries on the same table to get multiple rows.”
  8. 8. / Slide 8 / STLCMG Martin Fowler says: Never do repeated queries on the same table. Does this graph help you any? No.
  9. 9. / Slide 9 / STLCMG Martin Fowler says: Never do repeated queries on the same table. Does this SQL text help you any? No. select custom0_.TC_9RUI as J1_84_, custom0_.TC_9RVT as J2_84_, custom0_.TC_9RWT as J3_84_, custom0_.XC_RAPI as J4_84_, custom0_.TC_9SZI as J5_84_, custom0_.TC_9SCI as J6_84_, custom0_.XC_CVAC as J7_84_, custom0_.TC_9SGL as J8_84_, '10' as Pormula0_, custom0_.TC_9SCI as Pormula1_ FROM XRTADLS.XC_CUMS custom0_ where custom0_.TC_9RUI in ('00000117') order by custom0_.TC_9RVT||custom0_.TC_9RWT;select Rankor0_.XC_NOPC as J1_121_0_, Rankor0_.XC_NOGI as J2_121_0_, Rankor0_.XC_NOHT as J3_121_0_, Rankor0_.XC_CYWT as J4_121_0_ FROM XRTADLS.XC_CUSTM Rankor0_ where Rankor0_.XC_NOPC='10' and Rankor0_.XC_NOGI='AT_OPPICER';select adeRus0_.WRKDWNC as WRKDWNC138_, adeRus0_.WRKDWOC as WRKDWOC138_, adeRus0_.WRKDWPI as WRKDWPI138_, adeRus0_.WRKDWQT as WRKDWQT138_, adeRus0_.WRKDWRC as WRKDWRC138_, adeRus0_.WRKDWST as WRKDWST138_ FROM XRTADLS.WRKGNER adeRus0_ where adeRus0_.WRKDWNC='SECURITY_USERS' and adeRus0_.WRKDWOC=' ' and adeRus0_.WRKDWPI='PARIS';select mst0_.XC_RAAI as J1_88_0_, mst0_.XC_RACI as J2_88_0_, mst1_.XC_RPAI as J1_90_1_, mst1_.XC_RPCI as J2_90_1_, mst1_.XC_RPDI as J3_90_1_, mst1_.XC_CQJC as J4_90_1_, mst1_.XC_RPEI as J5_90_1_, mst1_.XC_RPPC as J6_90_1_, mst1_.XC_RPGI as J7_90_1_, mst0_.XC_CQYI as J3_88_0_, mst0_.XC_CQZI as J4_88_0_, mst0_.XC_CRAI as J5_88_0_, mst0_.XC_RARI as J6_88_0_, mst0_.XC_RAGT as J7_88_0_, mst0_.XC_RAH8 as J8_88_0_, mst0_.XC_RAN8 as J9_88_0_, mst0_.XC_RAO8 as J10_88_0_, mst0_.XC_RAZC as J11_88_0_, mst0_.XC_GIZI as J12_88_0_, mst0_.XC_CROI as J13_88_0_, mst0_.XC_CRCC as J14_88_0_, mst0_.XC_CRDC as J15_88_0_, mst0_.XC_RDRC as J16_88_0_, mst0_.XC_RADI as J17_88_0_, mst0_.XC_CRRC as J18_88_0_, mst0_.XC_CRPI as J19_88_0_, mst0_.XC_RRGI as J20_88_0_, mst0_.XC_RRHI as J21_88_0_, mst0_.XC_RRDI as J22_88_0_, mst0_.XC_RRNI as J23_88_0_, mst0_.XC_RRPI as J24_88_0_, mst1_.XC_RPS8 as J8_90_1_, mst1_.XC_RPXL as J9_90_1_, mst1_.XC_RGQL as J10_90_1_, mst1_.XC_RPRI as J11_90_1_, mst1_.XC_RPMC as J12_90_1_, mst1_.XC_RGA_ as J13_90_1_, mst1_.XC_RGR_ as J14_90_1_, mst1_.XC_RGC_ as J15_90_1_, mst1_.XC_RGD_ as J16_90_1_, mst1_.XC_RGOP as J17_90_1_, mst1_.WRKDUR8 as WRKDUR18_90_1_, mst1_.WRKDUQ8 as WRKDUQ19_90_1_, mst1_.XC_RPVC as J20_90_1_, mst1_.XC_CQKI as J21_90_1_, mst1_.XC_CQLT as J22_90_1_, mst1_.XC_CQMN as J23_90_1_, mst1_.XC_RAKT as J24_90_1_ FROM XRTADLS.XC_CAMS mst0_, XRTADLS.XC_ORDER mst1_ where mst0_.XC_CRCC=mst1_.XC_CQJC and mst0_.XC_RAAI=mst1_.XC_RPAI and mst0_.XC_RACI=mst1_.XC_RPCI and mst1_.XC_RPEI='00' and mst1_.XC_RPPC='00' and (mst0_.XC_RARI='00000117' or mst0_.XC_CQZI='00000117') and mst1_.XC_RPVC='A'; select acctma0_.TC_9RSI as J1_100_0_, acctma0_.TC_9RTI as J2_100_0_, acctmi1_.TC_9RSI as J1_101_1_, acctmi1_.TC_9RTI as J2_101_1_, acctac2_.TC_9EYI as J1_102_2_, acctac2_.TC_9EZI as J2_102_2_, acctac2_.TC_9PAI as J3_102_2_, acctac2_.TC_9PRI as J4_102_2_, acctac2_.XC_CJMC as J5_102_2_, acctma0_.TC_9RWI as J3_100_0_, acctma0_.TC_9RXI as J4_100_0_, acctma0_.XC_CMKI as J5_100_0_, acctma0_.XC_CMMI as J6_100_0_, acctma0_.XC_CMLI as J7_100_0_, acctma0_.TC_9RUI as J8_100_0_, acctma0_.TC_9CH8 as J9_100_0_, acctma0_.TC_9CI8 as J10_100_0_, acctma0_.TC_9CJ8 as J11_100_0_, acctma0_.TC_9CK8 as J12_100_0_, acctma0_.XC_DHD8 as J13_100_0_, acctma0_.TC_9CPC as J14_100_0_, acctma0_.XC_AWAC as J15_100_0_, acctma0_.TC_9EP8 as J16_100_0_, acctma0_.XC_GJAI as J17_100_0_, acctma0_.XC_CMPI as J18_100_0_, acctma0_.TC_9RVL as J19_100_0_, acctma0_.TC_9EE_ as J20_100_0_, acctma0_.TC_9EP_ as J21_100_0_, acctma0_.TC_9EG_ as J22_100_0_, acctma0_.XC_CMS_ as J23_100_0_, acctma0_.XC_CNPC as J24_100_0_, acctma0_.XC_CNGC as J25_100_0_, acctma0_.XC_DUPC as J26_100_0_, acctma0_.XC_CMOC as J27_100_0_, acctma0_.TC_9CQI as J28_100_0_, acctma0_.TC_9CPI as J29_100_0_, acctma0_.TC_9CRI as J30_100_0_, acctma0_.TC_9COI as J31_100_0_, acctma0_.TC_9DLC as J32_100_0_, acctmi1_.XC_DUEC as J3_101_1_, acctmi1_.TC_9CCC as J4_101_1_, acctmi1_.TC_9CRC as J5_101_1_, acctmi1_.XC_EGPI as J6_101_1_, acctmi1_.XC_EJY8 as J7_101_1_, acctac2_.TC_9PM8 as J6_102_2_, acctac2_.TC_9PEI as J7_102_2_, acctac2_.XC_RLJC as J8_102_2_, acctac2_.TC_9HK_ as J9_102_2_, acctac2_.TC_9HL_ as J10_102_2_, acctac2_.TC_9HM_ as J11_102_2_, acctac2_.TC_9HN_ as J12_102_2_, acctac2_.WRKCVVA as WRKCVVA102_2_, acctac2_.WRKCZCA as WRKCZCA102_2_, acctac2_.WRKDEIA as WRKDEIA102_2_, acctac2_.WRKECKA as WRKECKA102_2_, acctac2_.XC_CKU_ as
  10. 10. / Slide 10 / STLCMG The #1 Server Side Perf Problem Other examples (industry 2) 2010 -- “Too many Database Calls” #1 in DynaTrace's top 10 perf problems
  11. 11. / Slide 11 / STLCMG The #1 Server Side Perf Problem Other examples (industry 3) 2012 -- “Too many external system calls are performed in a synchronous and sequential manner.” #4 in DZone's top 10 perf problems
  12. 12. / Slide 12 / STLCMG The #1 Server Side Perf Problem Other examples (industry 4) 2013 -- #1 server-side problem (DynaTrace again)
  13. 13. / Slide 13 / STLCMG The #1 Server Side Perf Problem Other examples (industry 5) 2013 “SELECT N+1” Part of AppDynamic's "Common Application Problem" series
  14. 14. / Slide 14 / STLCMG The #1 Server Side Perf Problem Other examples (industry 6) Jack Shirazi, author of “Java Performance Tuning” says “There are ways in which you can paint yourself in a corner so that the only alternative is to throw away the application and start again”
  15. 15. / Slide 15 / STLCMG The #1 Server Side Perf Problem The Cost of Retreat is High
  16. 16. / Slide 16 / STLCMG Demo
  17. 17. / Slide 17 / STLCMG What is wuqiSpank?
  18. 18. / Slide 18 / STLCMG What is so special? ● The first Table-Centric Visualization ● Visually identify repetitious patterns ● Navigate the Perilous Voyage from * Small development database * To gargantuan producticon database
  19. 19. / Slide 19 / STLCMG How do find root cause? ● Familiarize yourself with known SQL performance anti-patterns ● Then provide a tool (wuqiSpank) that makes it easy to see anti-patterns ● Apply a known solution
  20. 20. / Slide 20 / STLCMG Anti-Pattern #1: SELECT N+1 ● Code iterates through the result set of the parent table and executes SELECTs to the child table. The extra queries incur unnecessary overhead. ResultSet parentRs = myStatement.executeQuery(strParentSELECT); while( { String myId = rs.getString(1); ResultSet childRs = statement.executeQuery( "select * from child where id = " + myId); //do something with childRs }
  21. 21. / Slide 21 / STLCMG Anti-Pattern #1: SELECT N+1 ● Known Solution: Reduce query count by joining parent and child table.
  22. 22. / Slide 22 / STLCMG Anti-Pattern #1: SELECT N+1 problem
  23. 23. / Slide 23 / STLCMG Anti-Pattern #2: Uncached Static Data ● Definition of static table: less than 1 INSERT/UPDATE/DELETE an hour ● Source code repeatedly SELECTs static data, even though that data rarely changes. The extra queries incur unnecessary overhead.
  24. 24. / Slide 24 / STLCMG Anti-Pattern #2: Uncached Static Data Known Solution: ● Reduce query count with use EHCache or similar caching facility to retrieve data from memory instead of from disk
  25. 25. / Slide 25 / STLCMG Anti-Pattern #3: Uncached Dynamic Data Source code retrieves dynamic data from db and re- queries that same data before the business process has completed. The extra queries incur unnecessary overhead.
  26. 26. / Slide 26 / STLCMG Anti-Pattern #3: Uncached Dynamic Data Known Solution: Reduce query count by temporarily storing the initial query results so they are available to subsequent code. ● For example, say that two web service calls are made after a user clicks a button. Both calls require the same five SQL queries for initialization. Solution: Refactor code into a single web service call instead of two.
  27. 27. / Slide 27 / STLCMG Table Roles ● Parent and Child
  28. 28. / Slide 28 / STLCMG Table Roles ● Teams 'flag' certain tables as 'growth' tables – see red circle? ● large row counts like TRAN_HIST ● Reminder for developers to avoid multiple queries to these tables. Voyage from small to large.
  29. 29. / Slide 29 / STLCMG Quiz What happens when we move functionally-complete code from a small development database to a large production database?
  30. 30. / Slide 30 / STLCMG Quiz: Answer The software developers go on vacation.
  31. 31. / Slide 31 / STLCMG Table Roles ● Teams 'flag' certain tables as 'static' tables – see blue circle? ● For data that changes seldom. Perhaps once an hour or less. ● Reminders for developers to provide caching
  32. 32. / Slide 32 / STLCMG Table Roles Patterns
  33. 33. / Slide 33 / STLCMG Root Cause - Summary ● We find 'Root Cause' for a defect by identifying the roles the tables play in known anti-patterns ● Parent ● Child ● Growth - Red ● Static – Blue, like frozen, static glacier
  34. 34. / Slide 34 / STLCMG Why Bother?
  35. 35. / Slide 35 / STLCMG Chunky Outperforms Chatty
  36. 36. / Slide 36 / STLCMG Chunky Outperforms Chatty The performance problems with chatty code are easy to demonstrate: chunky-outperforms-chatty.html
  37. 37. / Slide 37 / STLCMG SQL Sequence Diagrams wuqiSpank does Data access strategy comparisons ● Scenario 5 is the chattiest ● Scenario 1 is the chunkiest ● 2, 3 and 4 line up inbetween
  38. 38. / Slide 38 / STLCMG
  39. 39. / Slide 39 / STLCMG
  40. 40. / Slide 40 / STLCMG Premature Optimization Some early tuning is warranted when: ● When you’re addressing the worst server-side performance problem in enterprise software ● When the cost of retreat is high.
  41. 41. / Slide 41 / STLCMG Premature Optimization Some early tuning is warranted when: ● When the problem is easy to demonstrate ● When the response of development teams to identify the problem has been meager ● Effort required to catch worst problems: trivial
  42. 42. / Slide 42 / STLCMG SPE Whining and Complaining: 1 ● We've got the wrong staffing model for performance. ● Why are tiny SPE teams cleaning up huge messes left by much larger development teams?
  43. 43. / Slide 43 / STLCMG SPE Whining and Complaining: 2 ● Don't quite have the right tools. No tool visibility into the 1 month of work done by a developer on a SQL strategy.
  44. 44. / Slide 44 / STLCMG SPE Whining and Complaining: 3 ● Performance is placed in the wrong part of the SDLC ● That last lead-up segment of the plan before production ● --> that was meant for LOW RISK activities ● --> It was not meant for "rewrite it all because we got a late start on performance."
  45. 45. / Slide 45 / STLCMG Conclusion wuqiSpank is an open-source SQL tracing/visualization tool for software teams.
  46. 46. / Slide 46 / STLCMG Conclusion It provides a Missing Road Map for Refactoring SQL ...saving many hours of research when a performance defect is opened.
  47. 47. / Slide 47 / STLCMG Conclusion Chunky outperforms Chatty We already know the chatty pattern doesn't perform. Why wait to optimize when the cost to fix is so low?
  48. 48. / Slide 48 / STLCMG Conclusion Some platforms are more equal than others. wuqiSpank reads SQL from any platform, but has very nice live tracing for Java SQL (JDBC).
  49. 49. / Slide 49 / STLCMG Conclusion A safer Voyage from Small-to-Large database
  50. 50. / Slide 50 / STLCMG Conclusion The exact line of code that executed the SQL ....but don't get wrapped up in that.
  51. 51. / Slide 51 / STLCMG Conclusion Retreat has varying degrees of difficulty
  52. 52. / Slide 52 / STLCMG Conclusion Yes we must avoid Premature Optimization. ● But it does not excuse developers from all performance concerns. ● We Must Finally Draw the line and start making a dent on the biggest server-side performance problem.
  53. 53. / Slide 53 / STLCMG Thank You
  54. 54. / Slide 54 / STLCMG Help
  55. 55. / Slide 55 / STLCMG Help Submit wuqiSpank blog entry to one of these sites: LinkedIn: Software Performance Engineering LinkedIn: Software Test & Performance Group LinkedIn: Performance Specialists
  56. 56. / Slide 56 / STLCMG Help Provide server to host an internet discussion group ● Review wuqiSpank on your blog. ● Submit wuqiSpank blog entry to one of these sites:
  57. 57. / Slide 57 / STLCMG Help Vote on new Features: ● Security ● Move from Create Commons Non-Commercial ● Angular JS UI with Infinite Scroller ● "Tuning Recommendations" ● Generate JMeter load project with SQL
  58. 58. / Slide 58 / STLCMG Help Coding ● Java - Command line program to connect intrace- agent.jar ● Multi-threaded issues ● 1) ● AngularJS Guru to build new prototype for UI
  59. 59. / Slide 59 / STLCMG Therefore, consider a situation where you need to get 50 people that you can identify by a primary key in your domain model, but you can only construct a query such that you get 200 people, from which you'll do some further logic to isolate the 50 you need. It's usually better to use one query that brings back unnecessary rows than to issue 50 individual queries. --Martin Fowler