Lucas Jellema (AMIS, The Netherlands)THE VERY, VERY LATEST INORACLE DATABASE DEVELOPMENTOracle Open World 2012, San Franci...
THE VERY VERY VERY LATEST…<Secret Code>
THE DATABASEIN MODERN ARCHITECTURESOracle Open World 2012, San Francisco
NO SQL
THE TOP-3 EARNING EMPLOYEES• What can you say about the result of this query with  respect to the question: “Who are our t...
IN-LINE VIEWS
SPECIAL „BUSINESS RULE‟: DEFAULT VALUE• The default values is the value that should be inserted  for a column when the cli...
COLUMN DEFAULT• Columns can have default values   – Static or literals   – SQL expressions evaluating to a static   – Pseu...
APPLICATION CONTEXT  • Memory area that enables application developers to    define, set, and access key/value pairs      ...
APPLICATION CONTEXT APPEARANCES• Per session (default)   – Stored in UGA, just like package state• Globally Accessible (sh...
TYPICAL WEB ARCHITECTURE USINGCONNECTION POOL               JDBC Connection Pool Session 1     Session 2        Session 3 ...
PACKAGE STATE IS TIED TO DATABASESESSION               JDBC Connection Pool Session 1     Session 2         Session 3     ...
PACKAGE STATE IS TIED TO DATABASESESSION – NOT WEB SESSION               JDBC Connection Pool Session 1     Session 2     ...
APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS               JDBC Connection Pool Session 1     Session 2  ...
APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS               JDBC Connection Pool Session 1     Session 2  ...
APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS               JDBC Connection Pool Session 1     Session 2  ...
PACKAGE GLOBALS: THE STATE OF THEPACKAGE IN A SESSION• This state is lost when the package is recompiled   – That is undes...
PACKAGE GLOBALS CAN BE REPLACED BY APPLICATION CONTEXT • The Application Context is untouched by   recompilation of the pa...
EBR TO KILL PLANNED DOWNTIME    (BECAUSE OF APPLICATION UPGRADE)                         ApplicationApplication X         ...
TIME TRAVELLING
FLASHBACK• Introduced in 9i• Based on UNDO• Initially only for recovery• As of 11g – Total Recall option with  Flashback D...
DATABASE IN MODERN ARCHITECTURE               Mobile                   WS                        Business Tier         Cac...
MULTI TIER ARCHITECTURE                        Mobile                           WS                                  Busine...
APPLICATION ARCHITECTURE:DRIVE APPLICATION FROM META DATA• Agility• Design Time at Run Time• Define part of the applicatio...
SEPARATEBASE DATA AND CUSTOMIZED DATA• If a value is changed during site-level implementation    – Or run time customizati...
REPLACE THE ORIGINAL SINGLE TABLEWITH A TWO-TABLE BASE/CUSTOM SPLIT• rename <original> to <base>• create table <customizat...
REPLACE THE ORIGINAL SINGLE TABLEWITH A TWO-TABLE BASE/CUSTOM SPLIT (2)•   create or replace trigger handle_insert_trg    ...
APPLICATION ARCHITECTURE: NO SQL• NO SQL   – Complex SQL is     hidden away inside     the database   – Cache to not have ...
QUERY RESULT CHANGE NOTIFICATION• Continuous Query Notification:   – Send an event when the result set for a query changes...
CONTINUOUS PROCESSING OF DATASTREAMS USING CQL• Aggregation, Spot deviation, Match on complex  patterns
WHO IS AFRAID OF RED, YELLOW AND BLUE• Table Events   – Column Seq number(5)   – Column Payload varchar2(200)
SOLUTION USING LEAD • With LEAD it is easy to compare a row with its   successor(s)    – As long as the pattern is fixed, ...
THE SHOPPING ALGORITHM
THE SHOPPING ALGORITHM•   shopForItem Item ( String itemName) {        driveToShop;        Item item = buyItemAtShop ( ite...
GET THIS WEEK‟S GROCERIESgetGroceries Item[] ( String[] shoppingList) {    Item[] items = new Item[ shoppingList.length]; ...
PENSION FUND – SEPTEMBER 2012      Employer             <    >      Participants      Job & Benefits
FETCHING THE DATA OF THE PENSIONFUND FOR THE WEB APPLICATION              select *                                    1 re...
REPORTING ON MANY EMPLOYERS           select *                                    100s records           from   employers ...
APPLICATION ARCHITECTURE –BULK RETRIEVE• Have the database bulk up the data retrieval• Return Ref Cursor, Types and Collec...
APPLICATION ARCHITECTURE –   SERVICE ENABLING     WebLogic Server                                         Database        ...
XML/JSON                                                                   Relational/Oracle Type                         ...
THE TALKING DATABASE                  Details on the Employee. Employee                  name is Smith, his job is Analyst...
THE TALKING DATABASE                   PL/SQL         exposed through dbms_epg
BUSINESS RULES• Data Oriented Rules or Data Constraints• Declarative support in database   – For referential integrity    ...
BUSINESS RULES –WHERE AND HOW TO IMPLEMENT• Criteria:   – Safe   – Well performant   – Reusable and maintainable   – Produ...
RDBMS NOT ALWAYS EXCLUSIVELY     ACCESSED THROUGH ONE LAYER   SOA, ESB,  WebServices   Batch Bulk   Processes             ...
11G VIRTUAL COLUMNS      • Add columns to a table based on an        expression         – Using ‘real’ columns, SQL Functi...
UNIQUENESS RULES    USING VIRTUAL COLUMNS    • Business Rule:       – Not more than one manager per departmentalter table ...
CHALLENGE: ORDERS BELONG TO A    CUSTOMER IN ONE OF TWO TABLES    • The Orders table contains Order records for      custo...
USING VIRTUAL COLUMNS     IN FOREIGN KEY RELATIONS     • A foreign key can be created on a Virtual Column        – That me...
USING VIRTUAL COLUMNS             IN FOREIGN KEY RELATIONSalter table orders             alter table ordersadd (australian...
FOREIGN KEY SHOULD ONLY REFER TOCERTAIN RECORDS USING VC• Foreign Key can reference a UK based on a Virtual  Column• That ...
RESTRICTED FOREIGN KEYS USING        VIRTUAL COLUMNS                                                   alter table domain_...
LACK OF WATERTIGHTNESSIN TRIGGER BASED RULE VALIDATOIN
VALIDATION      •   Statement time validation means:                            DML in different sessionDML               ...
SAFE SOLUTION: USE CUSTOM LOCKS      • Prior to validating a certain business rule for a specific        record – acquire ...
SUMMARY•   Inline Views•   Defaulting•   Application Context•   Flashback and the time dimension•   NoSQL means smart SQL ...
Upcoming SlideShare
Loading in …5
×

The Very Very Latest in Database Development - Oracle Open World 2012

5,935 views

Published on

This is the official slide deck from my presentation at OOW 2012 on Database Development.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,935
On SlideShare
0
From Embeds
0
Number of Embeds
4,389
Actions
Shares
0
Downloads
30
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Process
  • http://technology.amis.nl/blog/?p=2530
  • http://technology.amis.nl/blog/2424/oracle-11g-virtual-columns-to-publish-derived-column-values-without-storing-them-and-more
  • http://technology.amis.nl/blog/2424/oracle-11g-virtual-columns-to-publish-derived-column-values-without-storing-them-and-more
  • http://technology.amis.nl/blog/2424/oracle-11g-virtual-columns-to-publish-derived-column-values-without-storing-them-and-more
  • http://technology.amis.nl/blog/2424/oracle-11g-virtual-columns-to-publish-derived-column-values-without-storing-them-and-more
  • The Very Very Latest in Database Development - Oracle Open World 2012

    1. 1. Lucas Jellema (AMIS, The Netherlands)THE VERY, VERY LATEST INORACLE DATABASE DEVELOPMENTOracle Open World 2012, San Francisco
    2. 2. THE VERY VERY VERY LATEST…<Secret Code>
    3. 3. THE DATABASEIN MODERN ARCHITECTURESOracle Open World 2012, San Francisco
    4. 4. NO SQL
    5. 5. THE TOP-3 EARNING EMPLOYEES• What can you say about the result of this query with respect to the question: “Who are our top three earning employees?” A. Correct Answer B. Sometimes correct C. Correct if there are never duplicate salaries D. Not Correct
    6. 6. IN-LINE VIEWS
    7. 7. SPECIAL „BUSINESS RULE‟: DEFAULT VALUE• The default values is the value that should be inserted for a column when the client has ignored the column – not provided a value nor indicated NULL• The default value is applied prior to the execution of the Before Row trigger – So :new.<column_value> has the value that will be inserted – The Before Row trigger has no built in way to telling whether the value was provided by the client or supplied as default by the database• Default value is typically used for auditing purposes – Note: default values for columns exposed in UI should be set in the client
    8. 8. COLUMN DEFAULT• Columns can have default values – Static or literals – SQL expressions evaluating to a static – Pseudo-columns like USER and CURRENT_DATE • DO NOT USE SYSDATE! DO NOT USE USER! – References to Application Context parameters • sys_context(‘USERENV’, ‘IP_ADDRESS’).. – Some funny value to let the before row trigger know that the real (complex) default must be calculated create table citizens ( name varchar2(100) default John Doe , birthdate date default current_date - 1 , city varchar2(50) default sys_context(KANE_CTX, DEFAULT_CITY ) , zipcode varchar2(8) default XYXYXYXYXQQ )
    9. 9. APPLICATION CONTEXT • Memory area that enables application developers to define, set, and access key/value pairs Application • Rapid access in SQL and PL/SQL Context Attribute Value Attribute Value Pairs Attribute Valueselect sys_context(USERENV, SESSION_USER)from duall_user:= sys_context(USERENV, SESSION_USER) • Two Application Contexts are always around: – CLIENTCONTEXT and USERENV
    10. 10. APPLICATION CONTEXT APPEARANCES• Per session (default) – Stored in UGA, just like package state• Globally Accessible (shared across all sessions) – Stored in SGA• Associated with a Client Identifier – Attributes in a Globally Accessible Application Context can explicitly be tied to the Client Identifier – And are only accessible to sessions with that Client Identifier
    11. 11. TYPICAL WEB ARCHITECTURE USINGCONNECTION POOL JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 Package A Package B Package C globals
    12. 12. PACKAGE STATE IS TIED TO DATABASESESSION JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 globals Package A Package B Package C globals
    13. 13. PACKAGE STATE IS TIED TO DATABASESESSION – NOT WEB SESSION JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 globals Package A Package B Package C globals
    14. 14. APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 globals ? Package A Package B Package C globals
    15. 15. APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 USERENV USERENV Package A Global Context Package C globals globals globals
    16. 16. APPLICATION CONTEXT TO RETAINSTATE FOR LIGHT WEIGHT END USERS JDBC Connection Pool Session 1 Session 2 Session 3 Session 4 USERENV USERENV USERENV Package A Global Context Package C globals globals globals
    17. 17. PACKAGE GLOBALS: THE STATE OF THEPACKAGE IN A SESSION• This state is lost when the package is recompiled – That is undesirable in a highly available environmentPackage
    18. 18. PACKAGE GLOBALS CAN BE REPLACED BY APPLICATION CONTEXT • The Application Context is untouched by recompilation of the package – All ‘globals’ in the application context retain their valuesPackage Application Context
    19. 19. EBR TO KILL PLANNED DOWNTIME (BECAUSE OF APPLICATION UPGRADE) ApplicationApplication X XVERSION 1 VERSION 2
    20. 20. TIME TRAVELLING
    21. 21. FLASHBACK• Introduced in 9i• Based on UNDO• Initially only for recovery• As of 11g – Total Recall option with Flashback Data Archive – Controlled history keeping• Look back into history – Query trends (version history) – Difference reporting – Audit trails (Replace journaling tables) • Require trick for transaction history: WHO?• Also: when is the start of history?
    22. 22. DATABASE IN MODERN ARCHITECTURE Mobile WS Business Tier Cache/Grid Enterprise (L1, L2, L3) Service Bus Services Standard ApplicationDatabase s Database Legacy Application s
    23. 23. MULTI TIER ARCHITECTURE Mobile WS Business Tier Cache/GridEnterprise HTTP REST JDBC (L1, L2, L3)Service Bus HTTP SOAP JPA (H/EL) FTP/WEBDAV Services DB QRCN HTTP JMX, JMX Monitor, Tra ce, Audit Stored EncapsulationDatabase Procedures Decoupling Authentication & Caching Fine Grained Business Logic Authorization
    24. 24. APPLICATION ARCHITECTURE:DRIVE APPLICATION FROM META DATA• Agility• Design Time at Run Time• Define part of the application behavior and appearance through meta-data (outside the base source code) – The default settings are defined by developers and deployed along with the application – Read and interpreted at run time – Manipulated and re-read and re-interpreted at run time Application• Note: very similar to the way the database operates: – Data Dictionary is the meta-data driving the behavior of the database meta
    25. 25. SEPARATEBASE DATA AND CUSTOMIZED DATA• If a value is changed during site-level implementation – Or run time customization• It should be kept apart from the base „meta-data‟ – To prevent overwriting customized data when the new release arrives – To allow for (temporarily) reverting to base data• A simple solution: the Complex View with two underlying tables approach – Note: Select… For Update Of is not allowed ORIGINAL_NAME IO trg Customized Values New release BaseValues
    26. 26. REPLACE THE ORIGINAL SINGLE TABLEWITH A TWO-TABLE BASE/CUSTOM SPLIT• rename <original> to <base>• create table <customizations> as select * from base where rownum = 0• create or replace view <original> as select * from <customizations> union all select * from <base> b left outer join <customizations> c on (b.id = c.id) where c.rowid is null
    27. 27. REPLACE THE ORIGINAL SINGLE TABLEWITH A TWO-TABLE BASE/CUSTOM SPLIT (2)• create or replace trigger handle_insert_trg instead of insert on original for each row begin insert into <customizations> (col, col2,…) values(:new.col, :new.col2,…); end;• create or replace trigger handle_update_trg instead of update on original for each row begin update <customizations> set col = :new.col, … where id = :new.id ; if sql%rowcount = 0 then insert into <customizations> (id, col, col2,…) (select id, :new.col, :new.col2 from base where id = :new.id); end if; end;
    28. 28. APPLICATION ARCHITECTURE: NO SQL• NO SQL – Complex SQL is hidden away inside the database – Cache to not have Web Browser to query all the time from the database – … and to not take the overhead of a commit for not so important data JEE Application Server NO SQL – Process first – in memory, on middle tier (BigData and CEP) - and only persist what is useful RDBMS SQL
    29. 29. QUERY RESULT CHANGE NOTIFICATION• Continuous Query Notification: – Send an event when the result set for a query changes – Background process calls PL/SQL Handler or Java Listener or OCI client when the Java commit has occurred Listener – Event contains rowid of changed rows• Used for: – Refreshing specific data caches (middle tier, global context) – (custom) Replication PL/SQL
    30. 30. CONTINUOUS PROCESSING OF DATASTREAMS USING CQL• Aggregation, Spot deviation, Match on complex patterns
    31. 31. WHO IS AFRAID OF RED, YELLOW AND BLUE• Table Events – Column Seq number(5) – Column Payload varchar2(200)
    32. 32. SOLUTION USING LEAD • With LEAD it is easy to compare a row with its successor(s) – As long as the pattern is fixed, LEAD will sufficewith look_ahead_events as( SELECT e.* , lead(payload) over (order by seq) next_color , lead(payload,2) over (order by seq) second_next_color FROM events e)select seqfrom look_ahead_eventswhere payload =redand next_color =yellowand second_next_color=blue
    33. 33. THE SHOPPING ALGORITHM
    34. 34. THE SHOPPING ALGORITHM• shopForItem Item ( String itemName) { driveToShop; Item item = buyItemAtShop ( itemName); driveHomeFromShop; return item; }
    35. 35. GET THIS WEEK‟S GROCERIESgetGroceries Item[] ( String[] shoppingList) { Item[] items = new Item[ shoppingList.length]; for (int i=0; i < shoppingList.length; i++) { items[i] = shopForItem (shoppingList[i]); } return items;}
    36. 36. PENSION FUND – SEPTEMBER 2012 Employer < > Participants Job & Benefits
    37. 37. FETCHING THE DATA OF THE PENSIONFUND FOR THE WEB APPLICATION select * 1 record < > from employers where id = < 324> select * 100s records from participants where employer_id = < 324> select * 10s records from benefits where participant_id = <#>
    38. 38. REPORTING ON MANY EMPLOYERS select * 100s records from employers 1 query select * 10k records from participants 100s queries where employer_id = <#> select * 100k records from benefits 10k queries where participant_id = <#>
    39. 39. APPLICATION ARCHITECTURE –BULK RETRIEVE• Have the database bulk up the data retrieval• Return Ref Cursor, Types and Collections or JSON/XML Benefits Packageselect *from employerswhere id in <some set> select * from participants where employer_id in <some set> select b.* from benefits b join participants p on (p.id = b.participant_id) where p.employer_id in <some set>
    40. 40. APPLICATION ARCHITECTURE – SERVICE ENABLING WebLogic Server Database Native DB WebService HTTP EPG Java/JEE PL/SQL packageSOA Suite JDBC AQ View Oracle Other TableService Bus (Email, FTP/File, XML DB XMPP/Chat) Chat/IM XMPP Server File/FTP Server Email Server
    41. 41. XML/JSON Relational/Oracle Type JEE Server Database 11g Native DB WebService HTTP 10g http ADF BC EPG JSON/ CSVJava App SDO /SDO WS XML PL/SQL XML & XSD WS JAX-WS package Ref Cursor JPublisher DB Types & Coll SOA WS JDBC XML 8i Types Suite AQ Adapters AQ JMS Queue JMS utl_file, View JMS BFILE,Oracle EJB EJB/JPA URITYPEService Pojo Other Table (Email, FTP/File, 9i XML Bus File XMPP/Chat) DB FTP UMS Chat/IM XMPP Server File/FTP Server Email Server
    42. 42. THE TALKING DATABASE Details on the Employee. Employee name is Smith, his job is Analyst. He works in department 20…EMP
    43. 43. THE TALKING DATABASE PL/SQL exposed through dbms_epg
    44. 44. BUSINESS RULES• Data Oriented Rules or Data Constraints• Declarative support in database – For referential integrity • Order must be for a Customer – For attribute and tuple rules • Salary must be numeric, • Hiredate may not be in the future, • End date must come after begin date• No declarative support for complex data rules – across multiple records and tables – A department in France may not have less then 20% female employees – Order items of type weapon may not be part of an order that ships around Christmas
    45. 45. BUSINESS RULES –WHERE AND HOW TO IMPLEMENT• Criteria: – Safe – Well performant – Reusable and maintainable – Productive to implement• Options – Client side • JavaScript – Middle-tier • Java, Enterprise Service Bus – Database • Constraints and triggers are statement level – i/o transaction level
    46. 46. RDBMS NOT ALWAYS EXCLUSIVELY ACCESSED THROUGH ONE LAYER SOA, ESB, WebServices Batch Bulk Processes Standard Applications DatabaseData Replication & Legacy Synchronization Applications
    47. 47. 11G VIRTUAL COLUMNS • Add columns to a table based on an expression – Using ‘real’ columns, SQL Function and User Defined Functions – No data is stored for Virtual Columns, only meta-data VIRTUAL – Virtual Columns can be indexedalter table empADD(income AS (sal + nvl(comm,0)))
    48. 48. UNIQUENESS RULES USING VIRTUAL COLUMNS • Business Rule: – Not more than one manager per departmentalter table empadd constraint only_one_mgr_in_dept_ukunique (one_mgr_flag) alter table emp ADD ( one_mgr_flag as ( case when job =MANAGER then deptno end ) )
    49. 49. CHALLENGE: ORDERS BELONG TO A CUSTOMER IN ONE OF TWO TABLES • The Orders table contains Order records for customers – either Dutch or Australian customers • These customers are stored in two different tables • Can we implement referential integrity to ensure that the order‟s customer exists? OZ_CUSTOMERORDER ? Id NameCountry DUTCH_CUSTOMERCustomer_Id…. Id Name
    50. 50. USING VIRTUAL COLUMNS IN FOREIGN KEY RELATIONS • A foreign key can be created on a Virtual Column – That means for example we can have a single column with some id – And two virtual columns with CASE expressions that produce NULL or the ID value – With Foreign Keys on the Virtual Columns OZ_CUSTOMERORDER IdCountry Name DUTCH_CUSTOMERCustomer_IdDutch_id (VC) IdAustralian_id (VC) Name
    51. 51. USING VIRTUAL COLUMNS IN FOREIGN KEY RELATIONSalter table orders alter table ordersadd (australian_ctr_id as add constraint odr_ocr_fk (case country foreign key (australian_ctr_id) when OZ references oz_customer (id) then customer_id end)) OZ_CUSTOMER ORDER Id Country Name DUTCH_CUSTOMER Customer_Id Dutch_id (VC) Id Australian_id (VC) Namealter table ordersadd (dutch_ctr_id as alter table orders (case country add constraint odr_dcr_fk when NL foreign key (dutch_ctr_id) then customer_id references dutch_customer end)) (id)
    52. 52. FOREIGN KEY SHOULD ONLY REFER TOCERTAIN RECORDS USING VC• Foreign Key can reference a UK based on a Virtual Column• That allows a „conditional foreign key‟ or a foreign key that can only reference specific records in the referenced table – Only refer to Women in the PEOPLE table for the Mother Foreign Key – Only refer to Values in the Domain Values table for the Domain Name == ‘COLORS’
    53. 53. RESTRICTED FOREIGN KEYS USING VIRTUAL COLUMNS alter table domain_values alter (country_value as add table domain_values (case domain_name add (country_value as alter table domain_values when COUNTRIES (case domain_name then domain_value when COUNTRIES add (color_value as end)) then domain_value (case domain_name end)) when COLORS then domain_value DOMAIN_VALUES end))CARS Id Domain_Name Domain_ValueID Color_ValueMake Gender_ValueType OrderStatus_ValueColor Country_ValueYear ShipmentMethod_Value alter table cars add constraint car_clr_fk foreign key (color) references domain_values (color_value)
    54. 54. LACK OF WATERTIGHTNESSIN TRIGGER BASED RULE VALIDATOIN
    55. 55. VALIDATION • Statement time validation means: DML in different sessionDML More DML Commit validation validation • To prevent leakage we should validate at commit time – Logically correct as transaction is the logical unit – Effects from other sessions between statement and commit are taken into account • However: Oracle unfortunately does not provide us with a pre-commit or on-commit trigger • Workarounds: – Dummy Table with Materialized View On Commit Refresh and Trigger on Materialized View – Do a soft-commit by calling a package to do the actual commit – that will first do transaction level checks • Supported by a deferred check constraint that is violated by each operation that potentially violates a business rule
    56. 56. SAFE SOLUTION: USE CUSTOM LOCKS • Prior to validating a certain business rule for a specific record – acquire a custom lock – That identifies both Rule and Record – Using dbms_lock DML in different sessionDML More DML Commit validation validation • When a record is being validated for a certain rule, other sessions have to wait • The commit (or rollback) releases all locks • Validation in a different session will include all committed data
    57. 57. SUMMARY• Inline Views• Defaulting• Application Context• Flashback and the time dimension• NoSQL means smart SQL – Cache refresh driven by change notification – Streaming analysis before persisting• Decoupling galore – Bulk retrieval – Service enabling• Business Rules• EBR• 12c promises even more

    ×