Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sangam 18 - Database Development: Return of the SQL Jedi

135 views

Published on

A look at the techniques that middle tier developers can employ to get greater value out of their applications, simply by having an understanding of how the database works and how to make it sing.

Published in: Technology
  • Be the first to comment

Sangam 18 - Database Development: Return of the SQL Jedi

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. The Complete Developer Meaner, faster code via SQL! 1
  2. 2. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Connor McDonald
  3. 3. 3
  4. 4. 4
  5. 5. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | Stuff youtube bit.ly/youtube-connor blog bit.ly/blog-connor twitter bit.ly/twitter-connor 400+ posts mainly on database & SQL 250 technical videos, new uploads every week my thoughts on tech and the world :-)
  6. 6. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. etc... facebook bit.ly/facebook-connor linkedin bit.ly/linkedin-connor instagram bit.ly/instagram-connor slideshare bit.ly/slideshare-connor
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 7 https://asktom.oracle.com
  8. 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved.https://asktom.oracle.com/officehours
  9. 9. why talk about this ? 9 # 1
  10. 10. remember when ... 10
  11. 11. ... life was simple 11
  12. 12. 12
  13. 13. 13 Java html/css node/angular git/svn logstash/MQ Junit/Selenium Cloud/REST/etc etc etc etc
  14. 14. 14 Java html/css node/angular git/svn logstash/MQ Junit/Selenium Cloud/REST/etc ???ORM Containers KubernetesDocker
  15. 15. "SQL is hard" 15
  16. 16. "SQL is old" 16
  17. 17. 17
  18. 18. but perhaps ... 18
  19. 19. ... SQL can be cool again? 19
  20. 20. we developers love cool stuff 20
  21. 21. MICROSERVICES 21
  22. 22. SQL is microservices ! 22
  23. 23. 23 "fine-grained to perform a single function" "Each service is ... minimal, and complete" https://en.wikipedia.org/wiki/Microservices select COUNT(*) from PEOPLE where GENDER = 'MALE'
  24. 24. even cooler stuff 24
  25. 25. API 25
  26. 26. SQL is entirely APIs ! 26
  27. 27. 27 "By abstracting the underlying implementation" "describes the expected behaviour ... but can have multiple implementations" https://en.wikipedia.org/wiki/Application_programming_interface select NAME, STREET_NO, ZIP_CODE from PEOPLE p, ADDRESS a where p.AGE > 50 and p.ADDRESS_ID = a.ADDRESS_ID;
  28. 28. don’t believe just me! 28
  29. 29. 29
  30. 30. this session is about ... 30
  31. 31. database knowledge = 31
  32. 32. ... faster, simpler, better applications 32
  33. 33. 33 Java html/css node/angular git/svn logstash/MQ Junit/Selenium Cloud/REST/etc SQL Database
  34. 34. most applications ... 34
  35. 35. ... do two things with data 35
  36. 36. 36
  37. 37. that's it! 37
  38. 38. 38 PART #1 reading data
  39. 39. two things 39
  40. 40. a) reading data needs ... 40
  41. 41. ... smart application 41
  42. 42. a quick primer on (database) memory
  43. 43. "chains"
  44. 44. memory access = controlled access
  45. 45. 45 metaphor
  46. 46. 50 same with memory
  47. 47. SGA
  48. 48. SGA protected by
  49. 49. SGA protected by 1) get latch
  50. 50. SGA protected by 2) access memory
  51. 51. SGA protected by 3) release latch
  52. 52. latch contention
  53. 53. SGA protected by someone must wait ...
  54. 54. active wait
  55. 55. spinning can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? can I have the latch ? 59
  56. 56. latch contention....
  57. 57. hurts CPU... 62
  58. 58. hurts concurrency
  59. 59. you ... get ... nothing ... done
  60. 60. “But how is that related to reading data?”
  61. 61. querying data
  62. 62. data is stored in blocks
  63. 63. to read data, you read blocks...
  64. 64. SQL> select * 2 from EMP 3 where ... /d01/ts/my_data1.dat Cache
  65. 65. but (database) memory access is complex 70 multi-process
  66. 66. cache coherency
  67. 67. SQL> select * 2 from EMP 3 where … SQL> update EMP 2 set … 3 where … SQL> select * 2 from EMP 3 where … SQL> insert into 2 EMP 3 values (…) SQL> select * 2 from CUSTOMER 3 / SQL> select max(EMP) 2 from DEPT SQL> delete 2 from DEPT 3 /
  68. 68. we have to latch !
  69. 69. • Get latch • Search along list of blocks in memory (or "chain") • Read block • Extract row of interest • Release latch • Give row back to client "fetch a row" 74
  70. 70. lots of rows
  71. 71. lots of latching
  72. 72. typical program
  73. 73. rs = stmt.executeQuery("..."); while(rs.next()) { v1 = rs.getInt(1); v2 = rs.getString(2); ... } • Get latch • Walk along chain • Get block • Extract single row • Release latch • Give row back to client • Get latch (again) • Walk along chain (again) • Get block (again) • Extract single row • Release latch (again) • Give row back to client (again) Row #1 Row #2 • Get latch (again) • Walk along chain (again) • Get block (again) • Extract single row • Release latch (again) • Give row back to client (again) Row #3 78
  74. 74. a better strategy.... “pinning”
  75. 75. "fetch a row" • Get latch • Walk along chain • Get block • Pin the block • Release latch • Give row 1 back to client • Give row 2 back to client … • Give row n to client • Get latch • Walk along chain • Remove my pin on the block • Release latch “and .. I’ll will need several rows from this block”
  76. 76. how do you say “I may want many rows” ?
  77. 77. 82 FetchDemo nn
  78. 78. b) reading data needs ... 83
  79. 79. ... smart database 84
  80. 80. controversial :-) 85
  81. 81. 86 DBA
  82. 82. DBA relationship is important 88 oc00.sql
  83. 83. PART #2 89 running SQL
  84. 84. back to the last demo 90
  85. 85. Statistics ------------------------------- 13 recursive calls 1 db block gets 527 consistent gets 91 Statistics ------------------------------ 0 recursive calls 0 db block gets 377 consistent gets versus
  86. 86. the most expensive SQL ... 92
  87. 87. ... is one we've never seen 93
  88. 88. parsing 94
  89. 89. Syntax Validity Optimization Rowsourcing Execution (Fetch) SQL> select * 2 frmo emp; frmo emp * ERROR at line 2: ORA-00923: FROM keyword not found where expected 95
  90. 90. Syntax Validity Optimization Rowsourcing Execution (Fetch) SQL> select empnoo 2 from emp; select empnoo * ERROR at line 1: ORA-00904: invalid column name
  91. 91. Syntax Validity Optimization Rowsourcing Execution (Fetch) PLAN ------------------------------------- SELECT STATEMENT TABLE ACCESS BY INDEX ROWID EMP INDEX RANGE SCAN EMP_PK
  92. 92. EMP_PK EMP Syntax Validity Optimization Rowsourcing Execution (Fetch)
  93. 93. Syntax Validity Optimization Rowsourcing Execution (Fetch) 99
  94. 94. Syntax Validity Optimization Rowsourcing Execution (Fetch)
  95. 95. lots of preliminaries
  96. 96. could be most of execution time
  97. 97. lots of memory access 103
  98. 98. lots of latching ! 104
  99. 99. SQL> select name, gets 2 from v$latch 3 where name like 'library%' or name like 'shared%'; NAME GETS ------------------------------ ---------- shared pool 333907 shared pool simulator 7191 shared pool sim alloc 133 shared server configuration 6 shared server info 1 SQL> select 0 from dual; 0 ---------- 0 GETS ---------- 333991 = 84 7196 = 5 135 = 2 6 1
  100. 100. library cache
  101. 101. previously executed statements 107
  102. 102. select surname, firstname from emp where empno = 123 probability of reuse low ?
  103. 103. select surname, firstname from emp where empno = ? binding parse this... now run it with ? = 7362 now run it with ? = 123
  104. 104. demo ParseDemo ParseDemoBind
  105. 105. "performance still looks ok"
  106. 106. let's make it real ParseDemo2 nn ParseDemo2Bind nn
  107. 107. much more serious ...
  108. 108. building SQL by concatenation
  109. 109. it takes 5 minutes to hack you
  110. 110. PART #3 119 writing data
  111. 111. the same rules apply 120
  112. 112. think blocks not rows 121 when appropriate
  113. 113. commit when it makes sense to 122
  114. 114. demo 123 InsertDemo3 InsertDemo4 nn
  115. 115. 124 PART #4 simpler coding
  116. 116. so far ... 125
  117. 117. query ... fetch 126
  118. 118. maybe that's enough! 127
  119. 119. more in SQL ... less code 128
  120. 120. Tomorrow 9am ! 129
  121. 121. wrap up 130
  122. 122. more knowledge with SQL 131
  123. 123. faster applications 132
  124. 124. more secure applications 133
  125. 125. the default data access language 134
  126. 126. 135 Hive Spark Tez Presto Drill Hawq Cloudera Vertica BigSQL Jethro
  127. 127. 136
  128. 128. ksql> CREATE STREAM ATM_TXNS ( account_id VARCHAR, atm VARCHAR, location STRUCT, amount INT, transaction_id VARCHAR) WITH ( KAFKA_TOPIC='atm_txns', VALUE_FORMAT='JSON'); ksql> SELECT ACCOUNT_ID, TRANSACTION_ID, AMOUNT, ATM FROM ATM_TXNS; 137
  129. 129. never too early to start 138
  130. 130. Thank you youtube bit.ly/youtube-connor blog connor-mcdonald.com twitter @connor_mc_d

×