Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Why SmartDB?

560 views

Published on

With presenter talking at https://www.youtube.com/watch?v=1MfEC6WwrjI (start at 23 min).

A presentation by Toon Koppelaars on why the SmartDB approach is the best fit for most common transactional database applications. It offers a path to the most secure, highest performing and easiest-to-maintain APIs to your data.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Why SmartDB?

  1. 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Why SmartDB? Toon Koppelaars Real-World Performance Oracle Server Technologies Winter 2018
  2. 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | SmartDB Definition • SmartDB is about how you use, or fail to use, the database • Do you use the database just as a persistence layer? or • Do you use the database as a processing engine? 3
  4. 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Context of This Presentation • DB: processing engine, or persistence layer? Our scope is: • Transactional (oltp) enterprise applications – Data store as foundation  for which we use Oracle’s DBMS – Much/complex CRUD functionality on top – User interfaces, workflows, batches, reports, API's to other applications – Potentially many users • Not: data-warehouse, business-intelligence, big-data, ... 4 Retrieval Manipulation Data store Preliminaries
  5. 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Enterprise Applications • A big component of these applications is “Business Logic” • What is “Business Logic”? • And where do you position it to run? – Inside application servers? – Inside a “processing engine” database? 5 Preliminaries
  6. 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Transactional Business Applications • Conceptually three tiers – Functionality exposed via interfaces • GUI's for human interaction • REST, Soap or otherwise, for software interaction – Business logic – Data store, relational database User Interfaces Software Interfaces Business Logic Table Data Store 6 Preliminaries
  7. 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Wiki 7 Business Logic SQL SQLSQL SQL SQL SQL “If-then-else-loop” code with embedded data access statements (= SQL) in it The way the business requires this to be done Preliminaries
  8. 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | We See Two Mutually Distinct Approaches 8 DBMS = Persistence Layer "NoPlsql" Approach DBMS = Processing Engine "SmartDB" Approach Preliminaries
  9. 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Why SmartDB? 1. Requirements 2. Reasoning 3. Recommendations 9
  10. 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Requirements • We need to develop OLTP Enterprise Application • It needs to be: – Maintainable – Good performance – State of the art (user) interfaces 10
  11. 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning • It’s the data, stupid! • Business logic, you say? • I have to write SQL? • Where is the volatility? • Database is always 1st bottleneck, didn’t you know? 11
  12. 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: It’s the Data, Stupid! 12 • Conceptually three tiers – Functionality exposed via interfaces • GUI's for human interaction • REST, Soap or otherwise, for software interaction – Business logic – Data store: heart of the application, crown jewels User Interfaces Software Interfaces Business Logic Table Data Store
  13. 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: It’s the Data, Stupid! • The heart of these applications, the crown jewels  data • The way it’s organized = blueprint/foundation on which all else rests • Different ways to structure this foundation • History of different models of organizing data: – Hierarchical data model – Network data model – Relational data model (what E.F. Codd gave us) 13
  14. 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: It’s the Data, Stupid! • Object oriented divagation to model data + behavior as one – Hierarchical domain models have failed in maintainability aspect (déja-vu) • In relational database design: every table is 1st class citizen • For maintainability, it’s best to base your application on a sound relational database design – One that is genuinely designed relationally – Not one that is derived from a OO domain model 14
  15. 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning • It’s the data, stupid! • Business logic, you say? • I have to write SQL? • Where is the volatility? • Database is always 1st bottleneck, didn’t you know? 15
  16. 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Business Logic You Say? 16 • Conceptually three tiers – Functionality exposed via interfaces • GUI's for human interaction • REST, Soap or otherwise, for software interaction – Business logic: guts of the application – Data store, relational database User Interfaces Software Interfaces Business Logic Table Data Store
  17. 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Business Logic You Say? • Viewed abstractly, core essence of business logic is this: Two types of use-cases, code that – Composes and executes transactions against data store – Composes and executes queries on data store Both in the way business requires this to be done • Lets look at an example 17 Business Logic SQL SQLSQL SQL SQL SQL “If-then-else-loop” code with embedded data access statements in it
  18. 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Business Logic You Say? 18 Single-row data access Business logic Row fetching (data access) Business logic Row-by-row updating (data access) “if-then-else-loop” code Primitive data access (single table, row-by-row) Business logic
  19. 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Business Logic You Say? Point to be made  business logic can appear: – As code-lines in some programming language that issues simple (poor) SQL – As part of (rich) SQL – Or a mix of both 19 Set-based data processing, aka “rich”SQL Multiple tables referenced Multiple rows processed You declare what needs to be done. Optimizer will translate this into code equivalent with code on previous page
  20. 20. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Business Logic You Say? 20 • Is there an impact on: – Maintainability? – Performance?
  21. 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning • It’s the data, stupid! • Business logic, you say? • I have to write SQL? • Where is the volatility? • Database is always 1st bottleneck, didn’t you know? 21
  22. 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: I Have to Write SQL? • There are nearly always opportunities for your business logic to be pushed into set-based SQL • Why is this the case? • There’s a fundamental reason for this... 22
  23. 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: I Have to Write SQL? 23 The Real World Us, living in the real world, using natural language to reason with each other about the real world We reason in this model using rich, set-based, SQL Application: model of a small part of the real world about which we want to reason using computers SQL is based on logic and set theory Logic and set theory are based on natural language, particularly the parts of it that deal with reasoning
  24. 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: I Have to Write SQL? 24 The Real World Us, living in the real world, using natural language to reason with each other about the real world We reason in this model using rich, set-based, SQL Application: model of a small part of the real world about which we want to reason using computers SQL is based on logic and set theory Logic and set theory are based on natural language, particularly the parts of it that deal with reasoning So we reason in the model using language that was based on how we reason in the real-world Ergo, SQL fundamentally fits what we want to achieve
  25. 25. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: I Have to Write SQL? 25 • Yes you do... – TX’s and Q’s simply need to be hand-written – The essence of them is always very business specific • And preferably write SQL with business logic in it  set-based SQL • It is natural fit for requirement at hand, being: – Manipulating and retrieving data from database design
  26. 26. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning • It’s the data, stupid! • Business logic, you say? • I have to write SQL? • Where is the volatility? • Database is always 1st bottleneck, didn’t you know? 26
  27. 27. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Where Is the Volatility? 27  Here it is, the volatility… – New hip tech-stacks every 1-2 years – Paradigm-shifts every 6-7 years – Never upwards compatible • Lots of if-then-else-loop languages  Stable technology SQL – Upwards compatible since three decades User Interfaces Software Interfaces Business Logic Table Data Store
  28. 28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Reasoning: Where Is the Volatility? • We want state-of-the-art user interfaces – Since that layer is most volatile – This is the layer where you want to be agile to the extreme – We should architect such that we can easily swap out, swap in new tooling • So *only* do UI in these tools – If other stuff done in it (e.g. business logic), will make swap more expensive • Question arises: how to interact between UI  business logic layer? 28
  29. 29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | • Well, currently REST and JSON are in-swung • Web server acts as proxy • And using tools such as ORDS or node.js  just translate REST request to DB-API call 29 DB-API’s Web server REST ORDS Reasoning: Where Is the Volatility?
  30. 30. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Where Is the Volatility? 30 • Agility here with respect to tech tooling • Agility here with respect to functional changes (maintainability) – What do we do here? • Java, JavaScript, C#, PHP, Python, PL/SQL • Agility here with respect to functional changes (maintainability) User Interfaces Software Interfaces Business Logic Table Data Store Since we’re already handwriting our SQL, PL/SQL seems obvious choice Will provide easiest impact-analysis
  31. 31. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning • It’s the data, stupid! • Business logic, you say? • I have to write SQL? • Where is the volatility? • Database is always 1st bottleneck, didn’t you know? 31
  32. 32. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck • Everybody knows this, right? • This “common sense knowledge” has driven business logic out of the DBMS – Away from PL/SQL, into other “outside” programming languages • Flash news! – The common sense knowledge, was a misconception – SQL from outside is more expensive (2X) than SQL from inside (PL/SQL) – The added expenses are higher than the gains of moving business logic out – If-then-else-loop business logic is 20X more expensive in Java 32
  33. 33. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 33 Youtube: “toon koppelaars” https://www.youtube.com/watch?v=8jiJDflpw4Y
  34. 34. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck • Four ways of doing SQL: – Row-by-row from NoPlsql (ubiquitous) – Row-by-row from Plsql (sometimes necessary) – Set-based from NoPlsql (hardly ever seen, as ORM’s don’t work this way) – Set-based from Plsql (very natural, successful 90’s approach) 34 Row-by-row SQL Set-based SQL SQL from NoPLSQL Slow Not tested SQL from PL/SQL Faster (2X) Superfast (20X) You wanted performance?
  35. 35. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 35 App server Database is always first bottleneck100 TX/Sec N O P L S Q L
  36. 36. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 36 App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec N O P L S Q L
  37. 37. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 37 App server App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec300 TX/Sec N O P L S Q L
  38. 38. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 38 App server App server App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec300 TX/Sec400 TX/Sec N O P L S Q L
  39. 39. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 39 App server App server App server App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec300 TX/Sec400 TX/Sec500 TX/Sec N O P L S Q L
  40. 40. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 40 App server App server App server App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec300 TX/Sec400 TX/Sec500 TX/Sec SmartDB will outscale other approaches N O P L S Q L S M A R T D B 500 TX/Sec500 TX/Sec
  41. 41. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Reasoning: Database Is Always 1st Bottleneck 41 App server App server App server App server App server Database is always first bottleneck100 TX/Sec200 TX/Sec300 TX/Sec400 TX/Sec500 TX/Sec SmartDB will outscale other approaches N O P L S Q L S M A R T D B 500 TX/Sec Set-based SQL… 500 TX/Sec
  42. 42. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Conclusions • Requirements for our OLTP Enterprise Application: – Maintainable – Good performance – State of the art (user) interfaces • Lots of reasoning • Recommendations 42
  43. 43. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Recommendations • Have relational database design as blueprint/foundation •  Maintainability – “Refactoring”: database design easiest in relational model •  Performance – Follow sound relational database design principles – Don’t be scared to introduce yet-another-table • “More tables = bad” myth • Employ people who are educated and experienced in designing databases 43
  44. 44. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Recommendations • Use database as processing engine instead of persistence-layer only •  Performance – SmartDB architecture will outscale any other architecture •  Maintainability – Easiest when business logic in PL/SQL with set-based SQL •  State of the art (user) interfaces – Proper demarcation enables agility in top-layer • Employ people who are experienced in (set-based) SQL and PL/SQL 44
  45. 45. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Recommendations 45 "SmartDB" Approach DBMS = Processing Engine
  46. 46. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | #SmartDB • You’re doing #SmartDB project: SQL and PL/SQL are your main tools – And use [insert-some-tech-here] to generate UI – And use [insert-some-tech-here] to handle workflows – And use [insert-some-tech-here] to handle event-processing – And use [insert-some-tech-here] to handle batches • All those [some-tech]’s just call database API’s 46
  47. 47. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 47
  48. 48. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 48

×