Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Developers vs DBAs - How to win the war

519 views

Published on

This presentation demonstrates how developers and DBAs can work together to provide better quality software and more value to the business rather than hindering each other on getting the job done.

Published in: Technology
  • Be the first to comment

Developers vs DBAs - How to win the war

  1. 1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | Developers versus DBAs: How to Win the War! Maria Colgan & Gerald Venzl Oracle Database Systems April, 2017 @GeraldVenzl @SQLMaria
  2. 2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | The following is intended to outline our general product direcQon. It is intended for informaQon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcQonality, and should not be relied upon in making purchasing decisions. The development, release, and Qming of any features or funcQonality described for Oracle’s products remains at the sole discreQon of Oracle. Safe Harbor Statement
  3. 3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 3 Public Developer DBA Oh no! What does he want now? Why don’t you just use Oracle? We need to build a new web applica?on for the coffee shops we recently acquired in a hurry, so we want to use a flexible schema using JSON documents. Can you setup a MongoDB database for us? What is she smoking …. and where can I get some?
  4. 4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 4 Oracle Database 12c as a Document Store 12.2 JSON SQL> CREATE TABLE orders  (order_details BLOB 
 CONSTRAINT is_json CHECK (order_details IS JSON)       ); Table containing JSON documents Public •  Oracle stores JSON in table columns •  No special data type •  Can be VARCHAR, CLOB or BLOB •  JSON supported by all Oracle features •  AnalyQcs, EncrypQon, In-Memory, RAC, ReplicaQon, Parallel SQL, … •  Plus can index any JSON element
  5. 5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 5 Public But the JSON document is BLOB. That’s not what we need. We need to be able to query the elements in the JSON documents? Don’t panic! Oracle offers naQve SQL support for JSON. Let me show you
  6. 6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 6 NaQve SQL Support for JSON 12.2 JSON Public •  JSON can be queried using simple SQL dot notaQon SELECT o.order_details.location.city
 FROM orders o;
 Location -------------------- Boston Los Angeles New York San Francisco Washington SQL> desc ORDERS NAME TYPE ------------------ ----------- ORDER_DETAILS BLOB {"date": "2016-05-03 10:40:36", "loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. } } Table containing JSON documents
  7. 7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | A few hours later …......... Public 7
  8. 8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 8 Public Dude, REST is out of the box with Oracle! Let me show you Hey, we have decided to do everything via REST and we know that MongoDB can speak REST. So it looks like we need a document store aSer all
  9. 9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Configuring REST with Oracle Exadata Express Cloud Service 9
  10. 10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Configuring REST with Oracle Exadata Express Cloud Service 10 Give the service a name and specify the prefix
  11. 11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Configuring REST with Oracle Exadata Express Cloud Service 11 Specify the rest of the URL demo/geraldorders
  12. 12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Configuring REST with Oracle Exadata Express Cloud Service 12 Specify the type of service Specify the actual command to be used by the service
  13. 13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | REST Endpoint For Developer hkps://mydatabase.could.oracle.com/apex/coffeeshop/orders Oracle ConfidenQal – Internal/Restricted/Highly Restricted 13
  14. 14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 14 Public Go ahead, send your REST calls
  15. 15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Checking result of REST SELECT Count(*) FROM orders; COUNT(*) -------- 1 15
  16. 16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 16 Public This is awesome! You just reduced our Qme to market by two weeks. Thanks a million! You’re welcome I’m a genius!
  17. 17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | A few days later …......... Public 17
  18. 18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 18 Public How’s it going? Isn’t the CRM data already in the database with the JSON data? Yeah, exactly. Now we just need to build some classes to get that corresponding relaQonal data for the JSON orders Hey, we are working on integraQng the CRM data with that JSON data we stored
  19. 19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 19 Public There is no need to integrate at the app level. You can use SQL to run a query against the JSON & the CRM data. Let me show you but first I’ll need to check what the structure of your JSON looks like.
  20. 20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Data Guide : Understanding Your JSON Documents •  Metadata discovery: discovers the structure of collecQon of JSON documents – OpQonal: deep analysis of JSON for List of Values, ranges, sizing etc. •  AutomaQcally Generates – Virtual columns – RelaQonal views •  De-normalized relaQonal views for arrays – Reports/Synopsis of JSON structure 20
  21. 21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 21 SQL> SELECT JSON_DATAGUIDE(o.order_details) FROM orders o; JSON_DATAGUIDE(O.ORDER_DETAILS) ---------------------------------------------------------------------- [ {"o:path": "$.date", "type": "string", "o:length": 32 }, {"o:path": "$.order", "type": "array", "o:length": 256 }, {"o:path": "$.location", "type": "object", "o:length": 32}, {"o:path": "$.location.city", "type": "string", "o:length": 8}, ... ... { "o:path": "$.salesAmount", "type": "number", "o:length": 4 }, { "o:path": "$.loyaltyCardNo","type": "number", "o:length": 8}] Data Guide : Understanding Your JSON Documents
  22. 22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Query JSON with RelaQonal Data SQL> SELECT c.c_name,          o.order_details.location.city,          SUM(To_number(o.order_details.salesAmount)) total  FROM   customers c,          orders o  WHERE  c.c_custid = o.order_details.loyaltyCardNo   GROUP  BY c.c_name,             o.order_details.location.city ORDER BY total Desc FETCH FIRST 10 ROWS ONLY;  22
  23. 23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 23 Public Developer DBA To make it easier for you, I can create a view so the developers don’t need to know all of the join condi?ons
  24. 24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | CreaQng a View on Top of JSON & RelaQonal Data SQL> CREATE OR replace VIEW crm_data AS  SELECT c.c_name,          o.order_details.location.city,          SUM(To_number(o.order_details.salesAmount)) total  FROM   customers c,          orders o  WHERE  c.c_custid = o.order_details.loyaltyCardNo   GROUP  BY c.c_name,             o.order_details.location.city ORDER BY total Desc FETCH FIRST 10 ROWS ONLY 24
  25. 25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 25 Public That's great but how will that work when we have GBs of data in there. Surely just selecQng the JSON first and then retrieving the corresponding data is faster and safer performance wise No way, that just leads to a lot of round trips between app and database, which will make things even slower. And besides, this problem has already been solved. Ever heard of Materialized Views?
  26. 26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 26 Public Oh boy, this one is really clueless! A materialized what?
  27. 27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | What is a Materialized View? •  A materialized view is a database object that contains the results of a query •  The query can be base on tables, view or other materialized views •  Typically used to avoid repeaQng costly aggregaQons •  OpQmizer transforms queries against original tables into a semanQcally equivalent query against the materialized view •  A trade off between disk space and CPU Oracle ConfidenQal – Internal/Restricted/Highly Restricted 27
  28. 28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | CreaQng an MV on Top of JSON & RelaQonal Data SQL> CREATE MATERIALIZED VIEW crm_data_mv  ENABLE query rewrite  AS SELECT c.c_name,          o.order_details.location.city,          SUM(To_number(o.order_details.salesAmount)) total  FROM   customers c,          orders o  WHERE  c.c_custid = o.order_details.loyaltyCardNo   GROUP  BY c.c_name,             o.order_details.location.city; 28
  29. 29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Queries AutomaQcally Redirected to Use Materialized View 29 SQL> select c.c_name, 2 o.order_details.location.city, 3 sum(to_number(o.order_details.salesamount)) total 4 from customers c, 5 orders o 6 where c.c_custid = o.order_details.loyaltycardno 7 group by c.c_name, 8 o.order_details.location.city 9 order by total desc 10 fetch first 10 rows only;--------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | --------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10 | 20550 | |* 1 | VIEW | | 10 | 20550 | |* 2 | WINDOW SORT PUSHED RANK | | 1 | 2029 | | 3 | MAT_VIEW REWRITE ACCESS FULL| TOP_CUSTOMERS_MV | 1 | 2029 | ---------------------------------------------------------------------------
  30. 30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 30 Public I’m sQll not sure this approach will scale as we need the queries to be returned in sub-seconds. Can we try it with a bigger data set? Sure. Where is the bigger data set?
  31. 31. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Test MV Query Against Larger Dataset •  3.2 million JSON documents Oracle ConfidenQal – Internal/Restricted/Highly Restricted 31
  32. 32. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 32 Public Oh that’s great for the known reports but the markeQng folks want to run ad-hoc reports too. What can we do to make sure they are fast? Don’t panic, we can put the tables into the In-Memory column store The what?
  33. 33. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Oracle Database In-Memory: Dual Format Architecture •  BOTH row and column formats for same table •  Simultaneously acQve and transacQonally consistent •  OLTP uses proven row format •  AnalyQcs & reporQng use new in-memory Column format •  Seamlessly built into Database •  All enterprise features work - RAC, Dataguard, Flashback, etc 33 Buffer Cache New In-Memory Column Store SALES SALES Row Format Column Format SALES
  34. 34. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | •  Full JSON documents populated using an opQmized binary format •  AddiQonal expressions can be created on JSON columns (e.g. JSON_VALUE) & stored in column store •  Queries on JSON content or expressions automaQcally directed to In-Memory format Superfast / MulQ-Model AnalyQcs: In-Memory JSON RelaQonal In-Memory Colum Store In-Memory Virtual Columns In-Memory JSON Format RelaQonal Virtual JSON 34 {"date": "2016-05-03 10:40:36", "loyaltyCardNo": 1230, "location": { "city": "Dallas", "state": "TX”}, "salesAmount": 7.55, "order": [ { "item": "Espresso”, …. } }
  35. 35. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | SQL> ALTER TABLE orders_history ADD city AS (JSON_VALUE(order_details, '$.location.city')); SQL> ALTER TABLE orders_history ADD state AS (JSON_VALUE(order_details, '$.location.state')); SQL> ALTER TABLE orders_history ADD salesamount AS (JSON_VALUE(order_details, '$.salesAmount')); Create Virtual Columns on the Orders_History table 35
  36. 36. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | SQL> ALTER TABLE customers INMEMORY; SQL> ALTER TABLE orders_history INMEMORY; SQL> SELECT count(*) FROM customers; SQL> SELECT count(*) FROM orders_history; SQL> SELECT segment_name, populate_status, bytes_not_populated  FROM   v$im_segments;  Place Tables into IM Column Store 36
  37. 37. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 37 Public Wow, that’s fantasQc. Will it automaQcally stay In-Memory? Yup, all done for you by the database! This is awesome! That's exactly what you want from a database. Schema flexibility, analyQcs and performance.
  38. 38. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 38 Public Hey one more thing, Dave has given us some ridiculous security requirements. Nobody is allowed to see the loyalty card number and stuff like that. Can we do something there too? Sure we can Let's implement data redacQon.
  39. 39. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Define a RedacQon Policy to Redact Employee Last Name SQL> BEGIN     dbms_redact.add_policy( object_schema => ’coffeeshop',                            object_name => ’customers',                            column_name => ’c_custid',                            policy_name => 'redact_Loyal_crd',                            function_type => DBMS_REDACT.FULL,                            expression => '1=1');  END; / Oracle ConfidenQal – Internal/Restricted/Highly Restricted 39
  40. 40. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 40 Public This is just awesome, can you also give me a REST endpoint to that? Of course!
  41. 41. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Developer DBA 41 Public Man, I wish we would have talked to you sooner. You preky much got our go-live down from 6 to 1 week! Hey, can we just include you going forward in our discussions and project planning? I'd be delighted to! I always thought we should work much closer together. It's also great for me to have beker understand what apps you are building in order to support them.
  42. 42. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | Summary: Oracle Database 12c - Universal Peace Maker Oracle ConfidenQal – Internal/Restricted/Highly Restricted 42 Flexibility NaQve JSON support REST Services Scalability Scale-out or Scale-up Real-Time AnalyQcs JSON in the In-Memory Column Store Security Data encrypted at rest Data RedacQon

×