Lucas Jellema
15th July 2013
Oracle Database 12c
for Developers
Overview
• The Fourth Dimension: Time
• Invisibility
• Security
• Data masking (data redaction)
• The evolution of SQL
• P...
The Fourth Dimension
The Fourth Dimension
• Oracle Database 12c is time aware in two ways
– Transaction time – through flashback
– Business tim...
Flashback
• Introduced in 9i
• Based on UNDO
• Initially only for recovery
• As of 11g – Total Recall option with
Flashbac...
Total Recall
• Flashback Data Archive Improvements The following
improvements have been made to Flashback Data Archive
(FD...
Total Recall (2)
• Import and export of history
– Support for import and export using Data Pump for FDA-enabled
tables. Da...
Flashback Data Archive in all
database editions
Valid time temporal modeling
• Validity (or effectivity) of facts recorded in a database is frequently
specified through d...
Creating a table with valid time
dimension
• Table with explicit valid time columns:
• Table with valid time dimension and...
Valid time aware flashback
queries
• Select all employees who were employed at a certain moment in time
• Perform all quer...
Scope of Valid time temporal
modeling in 12c database
• Valid time aware DML
• Flashback Queries that do not drive from Tr...
Invisibility…
Nebulous features
White List
• A white list of allowed invokers can be defined for a PL/SQL unit
– supports the robust implementation of a m...
accessible by clause
package Helper authid Definer accessible by (Good_Guy, Bad_Guy)
is
procedure p;
end Helper;
package b...
Invisible columns
• You can make individual table columns invisible and visible again.
– CREATE TABLE EMP (…, SAL NUMBER(5...
More visible…
• PL/SQL DBMS_UTILITY.EXPAND_SQL_TEXT can be used to
uncover the real SQL executed for a given query
• It re...
SQL Text Expansion
ops$tkyte%ORA12CR1> variable x clob
ops$tkyte%ORA12CR1> begin
2 dbms_utility.expand_sql_text
3 ( input_...
SQL Text Expansion
ops$tkyte%ORA12CR1> print x
X
-------------------------------------------------------------------------...
SQL Text Expansion
ops$tkyte%ORA12CR1> create or replace
2 function my_security_function( p_schema in varchar2,
3 p_object...
SQL Text Expansion
ops$tkyte%ORA12CR1> create table my_table
2 ( data varchar2(30),
3 OWNER varchar2(30) default USER
4 )
...
SQL Text Expansion
ops$tkyte%ORA12CR1> begin
2 dbms_rls.add_policy
3 ( object_schema => user,
4 object_name => 'MY_TABLE',...
SQL Text Expansion
ops$tkyte%ORA12CR1> begin
2 dbms_utility.expand_sql_text
3 ( input_sql_text => 'select * from my_table'...
API for inspecting the PL/SQL
callstack
• New PL/SQL Package UTL_CALL_STACK provides API for inspecting the
PL/SQL Callsta...
API for inspecting the PL/SQL
callstack
create or replace package body callstack_demo
as
function b( p1 in number, p2 in n...
UTL_CALL_STACK
• Functions for retrieving
– BACKTRACE:
• DEPTH, LINE and UNIT
– ERROR:
• DEPTH, MSG and NUMBER
– OWNER, SU...
Security
View with invoker’s rights
• As of Oracle Database Release 12c, a view can be either
– BEQUEATH DEFINER (the default), whi...
Invoker’s rights function results
can be cached
• An Invoker's Rights Function Can Be Result-Cached
– Through Oracle Datab...
Ability to Attach Roles to Program
Units
• You now can attach database roles to the program units functions,
procedures, p...
Inherit or not in Invoker rights
program units
• When a user runs an invoker's rights procedure (or program unit), it runs...
Inherit or not in Invoker rights
program units
• In previous releases, the invoking user had no control over who could
hav...
SYS_SESSION_ROLES
• A new built-in namespace, SYS_SESSION_ROLES, allows you to
determine if a specified role is enabled fo...
Audit the real privilege
requirements of an application
• Objective: restrict privileges for a role to those that are real...
Data Masking
Data Redaction
• At runtime, you can optionally have the query results modified to
reset/scramble/randomize sensitive data...
My first Data redaction policy
• Access to DBMS_REDACT package
• Create Data Redaction Policy for SAL column in EMP table ...
Querying EMP with DATA
REDACTIOn in place
• Note: drop Redaction Policy
DBMS_REDACT.DROP_POLICY
( object_schema => 'scott'...
Partial Column masking –
mask Day and month from
hiredate
BEGIN
DBMS_REDACT.ADD_POLICY(
object_schema => 'scott',
object_n...
The Evolution of Oracle SQL
The Top-3 Earning Employees
• What can you say about the result of this query with respect to the
question: “Who are our t...
In-Line Views
TOP-N Queries in 12c
• Last part of a query to be evaluated – to fetch only selected rows from
the result set:
– To select...
Pagination is just a convenient
syntax…
• OFFSET and FETCH NEXT are replaced by optimizer with Analytic
Functions such as ...
BOTTOM-N QUERY in 12c
• Return only the last three rows in the ordered result set (in the proper
order)
– or:
select *
fro...
TOP-n% querying
• To query for a percentage of the result set (rather than an absolute
number of rows)
• And the next batc...
Ties
• When ordering rows, multiple rows may have the same ‘ranking’
– For example order by sal will be unable to order re...
In-line PL/SQL Functions and
procedures
• Procedures are also allowed in-line
• In-Line Functions and Procedures can invok...
In-line PL/SQL Functions and
procedures
• In-Line Functions and Procedures can invoke each other
– And themselves (recursi...
Faculty calculation
• Inline PL/SQL
function
• Recursively invoked
• Using APEX as IDE
Dynamic (PL/)SQL is allowed
inside inline functions
• EXECUTE IMMEDIATE can be used inside an inline PL/SQL function to
dy...
PL/SQL Functions That Run
Faster in SQL
• As of Oracle Database Release 12c, two kinds of PL/SQL functions
might run faste...
Lateral inline view
• ANSI SQL feature: A lateral view is an inline view that contains correlation
referring to other tabl...
Lateral inline view enrich joins
• Lateral In-line view can be
used for ‘enrichment’ inside
multi-table join
SELECT *
FROM...
Apply for joining
• APPLY is used to join with a Collection
• The function employees_in_department
returns a collection (T...
Function that produces the
collection
create or replace type string_tbl as table of varchar2(200)
create or replace
functi...
OUTER Apply for joining
• APPLY is used to join with a Collection
• With OUTER APPLY, each DEPT record is
produced at leas...
Rules for APPLY
• The APPLY keyword allows you to specify a table_reference or
collection_expression on the right side of ...
Pattern Matching
Who is afraid of Red, Yellow
and blue
• Table Events
– Column Seq number(5)
– Column Payload varchar2(200)
Solution using Lead
• With LEAD it is easy to compare a row with its successor(s)
– As long as the pattern is fixed, LEAD ...
Find the pattern red, yellow and
blue
• Using the new 12c Match Recognize operator for finding patterns in
relational data...
Match_recognize for finding
patterns in relational data
• The expression MATCH_RECOGNIZE provides native SQL support to
fi...
Did we ever consecutively hire
three employees in the same job?
• Find a string of three subsequent hires where each hire ...
Row Pattern Matching
c##tkyte%CDB1> select symbol, tstamp, price,
2 rpad('*',price,'*') hist
3 from stocks
4 order by symb...
Row Pattern Matching
c##tkyte%CDB1> SELECT *
2 FROM stocks MATCH_RECOGNIZE
3 ( PARTITION BY symbol
4 ORDER BY tstamp
5 MEA...
Pattern clause is a regular
expression
• Supported operators for the pattern clause include:
– * for 0 or more iterations
...
Find the longest sequence of
related observations
• Records are ordered
• Each record is qualified: assigned
to a certain ...
Find the longest sequence of
related observations
SELECT section_category
, section_start
FROM observations
MATCH_RECOGNIZ...
Solution using lead
with observed_sections as
( select seq
, case lag(category) over (order by seq)
when category then 'N'...
Suppose we allow a single
interruption of a sequence
• One record with a different category
will not end the sequence – it...
Find sequence (with one accepted
interruption) from all records
SELECT substr(section_category,1,1) cat
, section_start
, ...
Miscellaneous
Miscellaneous
• Default
• Varchar2
• Java in the Database
• SQL Translation
• Export View as Table
• DICOM support for Web...
Export View as Table
• Through the Data Pump export facility, a view can be exported as a table
• Upon import, a table is ...
Varchar2 expansion
• VARCHAR2 maximum size to 32Kb
– In Column definition
– In Type definition
• On par with VARCHAR2 in P...
Database Java Engine (JVM)
• The Java engine (JVM) in the database is now JDK 6 (default) or JDK 7
(when explicitly set)
•...
SQL Statement Preprocessor
• A new mechanism is provided to allow the text of a SQL statement,
submitted from a client pro...
SQL Statement Preprocessor
• The translation code is named and is installed in the database using a
PL/SQL API. It can be ...
SQL Translation
• Objective: turn SQL issued by client applications into better Oracle SQL
• Package DBMS_SQL_TRANSLATOR
•...
DEFAULT
DEFAULT
• New in 12c
– Default based on Sequence
– Default applied (also) when NULL was explicitly specified
– Identity Co...
Specify default to be applied even
when insert stipulated NULL
• For example:
– When no salary is provided for a new emplo...
DERIVE DEFAULT value from
a sequence
• For example:
– The default value for a column that is specified as DEFAULT ON NULL ...
Define column as ‘identity
column’
• Specify an identity column: .the identity column will be assigned an
increasing or de...
Improved Defaults – metadata
only defaults
c##tkyte%CDB1> create table t
2 as
3 select *
4 from stage;
Table created.
c##t...
Improved Defaults – metadata
only defaults
c##tkyte%CDB1> set timing on
c##tkyte%CDB1> alter table t add (data char(2000) ...
Improved Defaults – metadata
only defaults
c##tkyte%CDB1> exec show_space('T')
…
Full Blocks ..................... 1,437
T...
Summary
Best Take Aways
• Flashback and Temporal Validity
• In-line PL/SQL Functions
• SQL Pattern Match
• Default value derived f...
Overview of Oracle database12c for developers
Upcoming SlideShare
Loading in...5
×

Overview of Oracle database12c for developers

362

Published on

AMIS organiseerde op maandagavond 15 juli het seminar ‘Oracle database 12c revealed’. Deze avond bood AMIS Oracle professionals de eerste mogelijkheid om de vernieuwingen in Oracle database 12c in actie te zien! De AMIS specialisten die meer dan een jaar bèta testen hebben uitgevoerd lieten zien wat er nieuw is en hoe we dat de komende jaren gaan inzetten!
Deze presentatie is deze avond gegeven in de vorm van een parallelsessie.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
362
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Overview of Oracle database12c for developers

  1. 1. Lucas Jellema 15th July 2013 Oracle Database 12c for Developers
  2. 2. Overview • The Fourth Dimension: Time • Invisibility • Security • Data masking (data redaction) • The evolution of SQL • Pattern Matching • Miscellaneous
  3. 3. The Fourth Dimension
  4. 4. The Fourth Dimension • Oracle Database 12c is time aware in two ways – Transaction time – through flashback – Business time – through temporal validity (brand new in 12c)
  5. 5. Flashback • Introduced in 9i • Based on UNDO • Initially only for recovery • As of 11g – Total Recall option with Flashback Data Archive – Controlled history keeping • Look back into history – Query trends (version history) – Difference reporting – Audit trails (Replace journaling tables) • Require trick for transaction history: WHO? • Also: when is the start of history? • By the way: Flashback Data Archive requires EE & Advanced Compression database option
  6. 6. Total Recall • Flashback Data Archive Improvements The following improvements have been made to Flashback Data Archive (FDA): – Complete schema evolution support All table definition, partitioning, and space management DDLs are supported on FDA-enabled tables. – The metadata information for tracking transactions including the user context is now tracked. The addition of user-context tracking makes it easier to determine which user made which changes to a table. • This could mean that journaling tables are now officially deprecated • Also given that the current contents of journaling tables can even be migrated to Flashback Data Archive
  7. 7. Total Recall (2) • Import and export of history – Support for import and export using Data Pump for FDA-enabled tables. Data Pump can now be used to export and import an FDA- enabled base table along with its schema-evolution metadata and historical row versions. • User generated history – Support for importing user-generated history has been added. Customers who have been maintaining history using other mechanisms, such as triggers, can now import that history into Total Recall. • Database Hardening – Register “Application” (a group of tables) and enable/disable flashback data archive for the application (also available: lock an application – make all tables read only)
  8. 8. Flashback Data Archive in all database editions
  9. 9. Valid time temporal modeling • Validity (or effectivity) of facts recorded in a database is frequently specified through dates or timestamps – For example begin date and [derived] end date of a price, membership, allocation, certificate, agreement • This valid time can differ from the transaction time at which a record is entered into the database • Multiple entries with different, non-overlapping valid-time periods can exist for a single entity • In 12c the notion of Valid Time is introduced into the Oracle Database – The valid-time dimension consists of two date-time columns specified in the table definition (create or alter) – These Valid Time columns specify the period during which a record is valid – A table can have multiple valid_time markers • Valid Time provides the optimizer with additional information and makes/will make many query operations easier to perform
  10. 10. Creating a table with valid time dimension • Table with explicit valid time columns: • Table with valid time dimension and implicit columns: columns valid_time_start and valid_time_end (TIMESTAMP) are added implicitly CREATE TABLE EMP ( employee_number NUMBER , salary NUMBER , department_id NUMBER , name VARCHAR2(30) , hiredate TIMESTAMP , firedate TIMESTAMP , PERIOD FOR user_time (hiredate, firedate) ); CREATE TABLE EMP ( employee_number NUMBER , salary NUMBER , department_id NUMBER , name VARCHAR2(30) , PERIOD FOR contract_time );
  11. 11. Valid time aware flashback queries • Select all employees who were employed at a certain moment in time • Perform all queries for records that are valid at a certain point in time • Return all records currently (session time) valid • Return all records (default) SELECT * FROM EMP AS OF PERIOD FOR user_time TO_TIMESTAMP('01-JUN-2012 12.00.01 PM') EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time ( 'ASOF' , TO_TIMESTAMP('29-JUL-12 12.00.01 PM') ); EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('CURRENT'); EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('ALL');
  12. 12. Scope of Valid time temporal modeling in 12c database • Valid time aware DML • Flashback Queries that do not drive from Transaction Time but from Valid Time (or a mix) • Temporal Primary and Unique Key constraints that allow multiple non- overlapping entries • Temporal referential constraints that take into account the valid-time during which the rows exist. – Child needs to have a valid Master at any time during its own validity • Temporal Aggregation - group or order by valid-time • Normalization - coalescing rows which are in adjacent or overlapping time periods • Temporal joins – joins between tables with valid-time semantics based on ‘simultaneous validity’ • For Information Lifecycle Management (ILM), the Valid Time information is used to assess records to move
  13. 13. Invisibility…
  14. 14. Nebulous features
  15. 15. White List • A white list of allowed invokers can be defined for a PL/SQL unit – supports the robust implementation of a module, consisting of a main unit and helper units, by allowing the helper units to be inaccessible from anywhere except the unit they are intended to help.
  16. 16. accessible by clause package Helper authid Definer accessible by (Good_Guy, Bad_Guy) is procedure p; end Helper; package body Good_Guy is procedure p is begin Helper.p(); ... end p; end Good_Guy; package body Bad_Guy is procedure p is begin Helper.p(); ... end p; end Bad_Guy; PLS-00904: insufficient privilege to access object HELPER _______
  17. 17. Invisible columns • You can make individual table columns invisible and visible again. – CREATE TABLE EMP (…, SAL NUMBER(5) INVISIBLE,…) – ALTER TABLE EMP MODIFY (b VISIBLE); • Any generic access of a table does not show the invisible columns in the table. – SELECT * FROM statements in SQL – DESCRIBE commands in SQL*Plus – %ROWTYPE attribute declarations in PL/SQL – Describes in Oracle Call Interface (OCI) • Invisible columns are not the same as system-generated hidden columns. You can make invisible columns visible, but you cannot make hidden columns visible. • Note: select * from table order by 1,3 can suffer when columns are made (in)visible • When a column is made visible again, it will show up at the end of the column list (in select * and desc)
  18. 18. More visible… • PL/SQL DBMS_UTILITY.EXPAND_SQL_TEXT can be used to uncover the real SQL executed for a given query • It returns the query in full – All views replaced by their core table based queries – All VPD policies explicitly added to the query • This features provides true insight in what a query actually does – Functionality – From a performance analysis point of view
  19. 19. SQL Text Expansion ops$tkyte%ORA12CR1> variable x clob ops$tkyte%ORA12CR1> begin 2 dbms_utility.expand_sql_text 3 ( input_sql_text => 'select * from all_users', 4 output_sql_text => :x ); 5 end; 6 / PL/SQL procedure successfully completed.
  20. 20. SQL Text Expansion ops$tkyte%ORA12CR1> print x X -------------------------------------------------------------------------------- SELECT "A1"."USERNAME" "USERNAME","A1"."USER_ID" "USER_ID","A1"."CREATED" "CREAT ED","A1"."COMMON" "COMMON" FROM (SELECT "A4"."NAME" "USERNAME","A4"."USER#" "US ER_ID","A4"."CTIME" "CREATED",DECODE(BITAND("A4"."SPARE1",128),128,'YES','NO') " COMMON" FROM "SYS"."USER$" "A4","SYS"."TS$" "A3","SYS"."TS$" "A2" WHERE "A4"."DA TATS#"="A3"."TS#" AND "A4"."TEMPTS#"="A2"."TS#" AND "A4"."TYPE#"=1) "A1"
  21. 21. SQL Text Expansion ops$tkyte%ORA12CR1> create or replace 2 function my_security_function( p_schema in varchar2, 3 p_object in varchar2 ) 4 return varchar2 5 as 6 begin 7 return 'owner = USER'; 8 end; 9 / Function created.
  22. 22. SQL Text Expansion ops$tkyte%ORA12CR1> create table my_table 2 ( data varchar2(30), 3 OWNER varchar2(30) default USER 4 ) 5 / Table created.
  23. 23. SQL Text Expansion ops$tkyte%ORA12CR1> begin 2 dbms_rls.add_policy 3 ( object_schema => user, 4 object_name => 'MY_TABLE', 5 policy_name => 'MY_POLICY', 6 function_schema => user, 7 policy_function => 'My_Security_Function', 8 statement_types => 'select, insert, update, delete' , 9 update_check => TRUE ); 10 end; 11 / PL/SQL procedure successfully completed.
  24. 24. SQL Text Expansion ops$tkyte%ORA12CR1> begin 2 dbms_utility.expand_sql_text 3 ( input_sql_text => 'select * from my_table', 4 output_sql_text => :x ); 5 end; 6 / PL/SQL procedure successfully completed. ops$tkyte%ORA12CR1> print x X -------------------------------------------------------------------------------- SELECT "A1"."DATA" "DATA","A1"."OWNER" "OWNER" FROM (SELECT "A2"."DATA" "DATA", "A2"."OWNER" "OWNER" FROM "OPS$TKYTE"."MY_TABLE" "A2" WHERE "A2"."OWNER"=USER@!) "A1"
  25. 25. API for inspecting the PL/SQL callstack • New PL/SQL Package UTL_CALL_STACK provides API for inspecting the PL/SQL Callstack – Complements the DBMS_ UTILITY.FORMAT_CALL_STACK that returns a pretty print human readable overview of the callstack procedure tell_on_call_stack is l_prg_uqn UTL_CALL_STACK.UNIT_QUALIFIED_NAME; begin dbms_output.put_line('==== TELL ON CALLSTACK ==== ' ||UTL_CALL_STACK.DYNAMIC_DEPTH ); for i in 1..UTL_CALL_STACK.DYNAMIC_DEPTH loop l_prg_uqn := UTL_CALL_STACK.SUBPROGRAM(i); dbms_output.put_line( l_prg_uqn(1) ||' line '||UTL_CALL_STACK.UNIT_LINE(i) ||' ' ||UTL_Call_Stack.Concatenate_Subprogram ( UTL_Call_Stack.Subprogram(i)) ); end loop; end tell_on_call_stack;
  26. 26. API for inspecting the PL/SQL callstack create or replace package body callstack_demo as function b( p1 in number, p2 in number) return number is l number:=1; begin tell_on_call_stack; return l; end b; procedure a ( p1 in number, p2 out number) is begin tell_on_call_stack; for i in 1..p1 loop p2:= b(i, p1); end loop; end a; function c( p_a in number) return number is l number; begin tell_on_call_stack; a(p_a, l); return l; end c; end callstack_demo;
  27. 27. UTL_CALL_STACK • Functions for retrieving – BACKTRACE: • DEPTH, LINE and UNIT – ERROR: • DEPTH, MSG and NUMBER – OWNER, SUBPROGRAM, UNIT_LINE – LEXICAL DEPTH (NESTING LEVEL)
  28. 28. Security
  29. 29. View with invoker’s rights • As of Oracle Database Release 12c, a view can be either – BEQUEATH DEFINER (the default), which behaves like a Definer’s Rights unit (functions in the view are executed using the view owner’s rights) – or BEQUEATH CURRENT_USER, which behaves somewhat like an invoker’s rights unit (functions in the view are executed using the current user’s rights) create or replace view managers ( name, sal, deptno, experience) BEQUEATH CURRENT_USER as select ename, sal, deptno, some_function(hiredate) from emp where job = ‘MANAGER’
  30. 30. Invoker’s rights function results can be cached • An Invoker's Rights Function Can Be Result-Cached – Through Oracle Database 11g Release 2 (11.2), only definer's rights PL/SQL functions could be result cached. • Now, invoker's rights PL/SQL functions can also be result cached. – The identity of the invoking user is implicitly added to the key of the result
  31. 31. Ability to Attach Roles to Program Units • You now can attach database roles to the program units functions, procedures, packages, and types. • The role then becomes enabled during execution of the program unit (but not during compilation of the program unit). • This feature enables you to temporarily escalate privileges in the PL/SQL code without granting the role directly to the user. The benefit of this feature is that it increases security for applications and helps to enforce the principle of least privilege. • Also: the new INHERIT PRIVILEGES privilege GRANT clerk_admin TO procedure scott.process_salaries;
  32. 32. Inherit or not in Invoker rights program units • When a user runs an invoker's rights procedure (or program unit), it runs with the privileges of the invoking user. • As the procedure runs, the procedure’s owner temporarily has access to the invoking user's privileges. • [If the procedure owner has fewer privileges than an invoking user,] the procedure owner could use the invoking user’s privileges to perform operations procedure owner Invoker’s rights invoker procedure Special_ Table Tap_ Table
  33. 33. Inherit or not in Invoker rights program units • In previous releases, the invoking user had no control over who could have this access when he or she runs an invoker’s rights procedure. • Starting with 12c, invoker’s rights procedure calls only can run with the privileges of the invoker if the procedure’s owner has the INHERIT PRIVILEGES privilege on the invoker or if the procedure’s owner has the INHERIT ANY PRIVILEGES privilege. – This gives invoking users control over who has access to their privileges when they run invoker’s rights procedures or query BEQUEATH CURRENT_USER views. • Any user can grant or revoke the INHERIT PRIVILEGES privilege on themselves to the user whose invoker’s rights procedures they want to run. GRANT INHERIT PRIVILEGES ON USER invoking_user TO procedure_owner REVOKE INHERIT PRIVILEGES ON invoking_user FROM procedure_owner;
  34. 34. SYS_SESSION_ROLES • A new built-in namespace, SYS_SESSION_ROLES, allows you to determine if a specified role is enabled for the querying user. • The following example determines if the DBA role is enabled for user oe: • CONNECT OE/password • SELECT SYS_CONTEXT('SYS_SESSION_ROLES', 'DBA') FROM DUAL; • SYS_CONTEXT('SYS_SESSION_ROLES','DBA') -------- FALSE
  35. 35. Audit the real privilege requirements of an application • Objective: restrict privileges for a role to those that are really required for using an application • A privilege capture displays privilege usage for a database according to a specified condition – such as privileges to run an application module – or privileges used in a given user session. • When a user performs an action and you want to monitor the privileges that are used for this action, you can create a privilege capture to capture privilege usage. • Afterwards, you can view a report that describes the behavior the privilege capture defined. – The privilege capture includes both system privileges and object privileges. • Part of Database Vault option
  36. 36. Data Masking
  37. 37. Data Redaction • At runtime, you can optionally have the query results modified to reset/scramble/randomize sensitive data – Through ‘data redaction’ policies associated with tables and view and applied at query time • Because the data is masked in real-time, Data Redaction is well suited to environments in which data is constantly changing. • You can create the Data Redaction policies in one central location and easily manage them from there. SQL engine SQL POLICY POLICY RESULTS
  38. 38. My first Data redaction policy • Access to DBMS_REDACT package • Create Data Redaction Policy for SAL column in EMP table – hide salaries from view • Find that querying EMP has changed forever… – Note: the expression can be used to dynamically decide whether or not to apply the policy grant execute on dbms_redact to scott; BEGIN DBMS_REDACT.ADD_POLICY( object_schema => 'scott', object_name => 'emp', column_name => 'sal', policy_name => 'hide_salary', function_type => DBMS_REDACT.FULL, expression => '1=1' ); END;
  39. 39. Querying EMP with DATA REDACTIOn in place • Note: drop Redaction Policy DBMS_REDACT.DROP_POLICY ( object_schema => 'scott' , object_name => 'emp' , policy_name => 'hide_salary' );
  40. 40. Partial Column masking – mask Day and month from hiredate BEGIN DBMS_REDACT.ADD_POLICY( object_schema => 'scott', object_name => 'emp', column_name => 'hiredate', policy_name => 'partially mask hiredate', function_type => DBMS_REDACT.PARTIAL, function_parameters => 'm1d31YHMS', expression => '1=1' ); END;
  41. 41. The Evolution of Oracle SQL
  42. 42. The Top-3 Earning Employees • What can you say about the result of this query with respect to the question: “Who are our top three earning employees?” A. Correct Answer B. Sometimes correct C. Correct if there are never duplicate salaries D. Not Correct
  43. 43. In-Line Views
  44. 44. TOP-N Queries in 12c • Last part of a query to be evaluated – to fetch only selected rows from the result set: – To select the next set of rows: select * from emp order by sal desc FETCH FIRST 3 ROWS ONLY; select * from emp order by sal desc OFFSET 3 FETCH NEXT 4 ROWS ONLY;
  45. 45. Pagination is just a convenient syntax… • OFFSET and FETCH NEXT are replaced by optimizer with Analytic Functions such as ROW_NUMBER() c##tkyte%CDB1> select /*+ first_rows(5) */ owner, object_name, object_id 2 from t 3 order by owner, object_name 4 OFFSET 5 ROWS FETCH NEXT 5 ROWS ONLY; … --------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | --------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 1450 | 7 (0)| 00:00:01 | |* 1 | VIEW | | 5 | 1450 | 7 (0)| 00:00:01 | |* 2 | WINDOW NOSORT STOPKEY | | 5 | 180 | 7 (0)| 00:00:01 | | 3 | TABLE ACCESS BY INDEX ROWID| T | 87310 | 3069K| 7 (0)| 00:00:01 | | 4 | INDEX FULL SCAN | T_IDX | 5 | | 3 (0)| 00:00:01 | --------------------------------------------------------------------------------------- Predicate Information (identified by operation id): --------------------------------------------------- 1 - filter("from$_subquery$_003"."rowlimit_$$_rownumber"<=CASE WHEN (5>=0) THEN 5 ELSE 0 END +5 AND "from$_subquery$_003"."rowlimit_$$_rownumber">5) 2 - filter(ROW_NUMBER() OVER ( ORDER BY "OWNER","OBJECT_NAME")<=CASE WHEN (5>=0) THEN 5 ELSE 0 END +5)
  46. 46. BOTTOM-N QUERY in 12c • Return only the last three rows in the ordered result set (in the proper order) – or: select * from emp order by sal desc OFFSET ((select count(*) from emp)-3) ROWS FETCH NEXT 3 ROWS ONLY select * from ( select * from emp order by sal asc FETCH FIRST 3 ROWS ONLY ) order by sal desc;
  47. 47. TOP-n% querying • To query for a percentage of the result set (rather than an absolute number of rows) • And the next batch select * from emp order by sal desc FETCH FIRST 30 PERCENT ROWS ONLY; select * from emp order by sal desc OFFSET (0.3*(select count(*) from emp)) ROWS FETCH NEXT (0.3*(select count(*) from emp)) ROWS ONLY;
  48. 48. Ties • When ordering rows, multiple rows may have the same ‘ranking’ – For example order by sal will be unable to order records with the same salary in a meaningful ways • Using WITH TIES instead of ONLY will make sure that if one record with a certain position in the order is included, they all are select * from emp order by sal desc FETCH FIRST 30 PERCENT ROWS WITH TIES;
  49. 49. In-line PL/SQL Functions and procedures • Procedures are also allowed in-line • In-Line Functions and Procedures can invoke each other WITH procedure increment( operand in out number , incsize in number) is begin operand:= operand + incsize; end; FUNCTION inc(value number) RETURN number IS l_value number(10):= value; BEGIN increment(l_value, 100); RETURN l_value; end; SELECT inc(sal) from emp
  50. 50. In-line PL/SQL Functions and procedures • In-Line Functions and Procedures can invoke each other – And themselves (recursively) WITH FUNCTION inc(value number) RETURN number IS BEGIN if value < 6000 then return inc(value+100); else return value + 100; END if; end; SELECT inc(sal) from emp
  51. 51. Faculty calculation • Inline PL/SQL function • Recursively invoked • Using APEX as IDE
  52. 52. Dynamic (PL/)SQL is allowed inside inline functions • EXECUTE IMMEDIATE can be used inside an inline PL/SQL function to dynamically construct and execute SQL and PL/SQL WITH FUNCTION EMP_ENRICHER(operand varchar2) RETURN varchar2 IS sql_stmt varchar2(500); job varchar2(500); BEGIN sql_stmt := 'SELECT job FROM emp WHERE ename = :param'; EXECUTE IMMEDIATE sql_stmt INTO job USING operand; RETURN ' has job '||job; END; SELECT ename || EMP_ENRICHER(ename) from emp
  53. 53. PL/SQL Functions That Run Faster in SQL • As of Oracle Database Release 12c, two kinds of PL/SQL functions might run faster in SQL: – PL/SQL functions that are defined in the WITH clauses of SQL SELECT statements, described in Oracle Database SQL Language Reference – PL/SQL functions that are defined with the "UDF Pragma“ • Pragma UDF means: compile in the ‘SQL way’ as to eliminate SQL  PL/SQL context switch FUNCTION inc(string VARCHAR2) RETURN VARCHAR2 IS PRAGMA UDF; value number(10):= to_number(string); BEGIN if value < 6000 then return inc(value+100); else return to_char(value + 100); end if; end;
  54. 54. Lateral inline view • ANSI SQL feature: A lateral view is an inline view that contains correlation referring to other tables that precede it in the FROM clause – You can specify this left correlation anywhere within subquery (such as the SELECT, FROM, and WHERE clauses), and at any nesting level. SELECT * FROM emp e , LATERAL( SELECT * FROM dept d WHERE e.deptno = d.deptno )
  55. 55. Lateral inline view enrich joins • Lateral In-line view can be used for ‘enrichment’ inside multi-table join SELECT * FROM emp e cross join LATERAL( SELECT * FROM dept d WHERE e.deptno = d.deptno ) cross join LATERAL( SELECT e2.ename manager FROM emp e2 WHERE e2.deptno = e.deptno and e2.job ='MANAGER' )
  56. 56. Apply for joining • APPLY is used to join with a Collection • The function employees_in_department returns a collection (TABLE OF VARCHAR2 in this case) • The function takes a value from the DEPT records as input • Only when the returned collection is not empty will the DEPT record be produced by this join SELECT * FROM DEPT d CROSS APPLY employees_in_department(deptno) staff
  57. 57. Function that produces the collection create or replace type string_tbl as table of varchar2(200) create or replace function employees_in_department ( p_deptno in number ) return string_tbl as l_enames string_tbl; begin select cast(collect(ename) as string_tbl) into l_enames from emp where deptno = p_deptno ; return l_enames; end;select dname , deptno , employees_in_department(deptno) staff from dept
  58. 58. OUTER Apply for joining • APPLY is used to join with a Collection • With OUTER APPLY, each DEPT record is produced at least once – depending on the collection returned by the function for the DEPT record, multiple joins may be produced SELECT * FROM DEPT d OUTER APPLY employees_in_department(deptno) staff
  59. 59. Rules for APPLY • The APPLY keyword allows you to specify a table_reference or collection_expression on the right side of a join clause. – The table_reference can be a table, inline view, or TABLE collection expression; it cannot be a lateral inline view. – The collection_expression can be a subquery, a column, a function, or a collection constructor. • Regardless of its form, it must return a collection value—that is a nested table or varray. • Left correlations are allowed on the right side of APPLY. • With keyword CROSS – only rows from table that are joined with result in right side are returned • With keyword OUTER – both rows that join and those that do not are returned from the table (on the left)
  60. 60. Pattern Matching
  61. 61. Who is afraid of Red, Yellow and blue • Table Events – Column Seq number(5) – Column Payload varchar2(200)
  62. 62. Solution using Lead • With LEAD it is easy to compare a row with its successor(s) – As long as the pattern is fixed, LEAD will suffice with look_ahead_events as ( SELECT e.* , lead(payload) over (order by seq) next_color , lead(payload,2) over (order by seq) second_next_color FROM events e ) select seq from look_ahead_events where payload ='red' and next_color ='yellow' and second_next_color='blue'
  63. 63. Find the pattern red, yellow and blue • Using the new 12c Match Recognize operator for finding patterns in relational data SELECT * FROM events MATCH_RECOGNIZE ( ORDER BY seq MEASURES RED.seq AS redseq , MATCH_NUMBER() AS match_num ALL ROWS PER MATCH PATTERN (RED YELLOW BLUE) DEFINE RED AS RED.payload ='red', YELLOW AS YELLOW.payload ='yellow', BLUE AS BLUE.payload ='blue' ) MR ORDER BY MR.redseq , MR.seq;
  64. 64. Match_recognize for finding patterns in relational data • The expression MATCH_RECOGNIZE provides native SQL support to find patterns in sequences of rows • Match_recognize returns Measures for selected (pattern matched) rows – Similar to MODEL clause • Match Conditions are expressed in columns from the Table Source, aggregate functions and pattern functions FIRST, PREV, NEXT, LAST • Patterns are regular expressions using match conditions to express a special sequence of rows satisfying the conditions Table Source & Where Match_ Recognize Process and Filter Select & Order By
  65. 65. Did we ever consecutively hire three employees in the same job? • Find a string of three subsequent hires where each hire has the same job • Order by hiredate, pattern is two records that each have the same job as their predecessor SELECT * FROM EMP MATCH_RECOGNIZE ( ORDER BY hiredate MEASURES SAME_JOB.hiredate AS hireday , MATCH_NUMBER() AS match_num ALL ROWS PER MATCH PATTERN (SAME_JOB{3}) DEFINE SAME_JOB AS SAME_JOB.job = FIRST(SAME_JOB.job) ) MR
  66. 66. Row Pattern Matching c##tkyte%CDB1> select symbol, tstamp, price, 2 rpad('*',price,'*') hist 3 from stocks 4 order by symbol, tstamp; SYMBOL TSTAMP PRICE HIST ---------- --------- ---------- ---------------------------------------- ORCL 01-SEP-12 35 *********************************** ORCL 02-SEP-12 34 ********************************** ORCL 03-SEP-12 33 ********************************* ORCL 04-SEP-12 34 ********************************** ORCL 05-SEP-12 35 *********************************** ORCL 06-SEP-12 36 ************************************ ORCL 07-SEP-12 37 ************************************* ORCL 08-SEP-12 36 ************************************ ORCL 09-SEP-12 35 *********************************** ORCL 10-SEP-12 34 ********************************** ORCL 11-SEP-12 35 *********************************** ORCL 12-SEP-12 36 ************************************ ORCL 13-SEP-12 37 ************************************* 13 rows selected.
  67. 67. Row Pattern Matching c##tkyte%CDB1> SELECT * 2 FROM stocks MATCH_RECOGNIZE 3 ( PARTITION BY symbol 4 ORDER BY tstamp 5 MEASURES 6 STRT.tstamp AS start_tstamp, 7 LAST(DOWN.tstamp) AS bottom_tstamp, 8 LAST(UP.tstamp) AS end_tstamp 9 ONE ROW PER MATCH 10 AFTER MATCH SKIP TO LAST UP 11 PATTERN (STRT DOWN+ UP+) 12 DEFINE 13 DOWN AS DOWN.price < PREV(DOWN.price), 14 UP AS UP.price > PREV(UP.price) 15 ) MR 16 ORDER BY MR.symbol, MR.start_tstamp; SYMBOL START_TST BOTTOM_TS END_TSTAM ---------- --------- --------- --------- ORCL 01-SEP-12 03-SEP-12 07-SEP-12 ORCL 07-SEP-12 10-SEP-12 13-SEP-12 SYMBOL TSTAMP PRICE HIST ---------- --------- ---------- ---------------------------------------- ORCL 01-SEP-12 35 *********************************** ORCL 02-SEP-12 34 ********************************** ORCL 03-SEP-12 33 ********************************* ORCL 04-SEP-12 34 ********************************** ORCL 05-SEP-12 35 *********************************** ORCL 06-SEP-12 36 ************************************ ORCL 07-SEP-12 37 ************************************* ORCL 08-SEP-12 36 ************************************ ORCL 09-SEP-12 35 *********************************** ORCL 10-SEP-12 34 ********************************** ORCL 11-SEP-12 35 *********************************** ORCL 12-SEP-12 36 ************************************ ORCL 13-SEP-12 37 ************************************* 13 rows selected.
  68. 68. Pattern clause is a regular expression • Supported operators for the pattern clause include: – * for 0 or more iterations – + for 1 or more iterations – ? for 0 or 1 iterations – { n } for exactly n iterations (n > 0) – { n, } for n or more iterations (n >= 0) – { n, m } for between n and m (inclusive) iterations (0 <= n <= m, 0 < m) – { , m } for between 0 and m (inclusive) iterations (m > 0) – reluctant qualifiers - *?, +?, ??, {n}?, {n,}?, { n, m }?, {,m}? – | for alternation (OR) – grouping using () parentheses – exclusion using {- and -} – empty pattern using () – ^ and $ for start and end of a partition
  69. 69. Find the longest sequence of related observations • Records are ordered • Each record is qualified: assigned to a certain category • Examples: – Voting records – Ball possession in football – Days with or without rain – Passing vehicles (make and model or category) – DNA records • The challenge: find the longest string of consecutive observations in the same category
  70. 70. Find the longest sequence of related observations SELECT section_category , section_start FROM observations MATCH_RECOGNIZE ( ORDER BY seq MEASURES SAME_CATEGORY.category as section_category , FIRST(SAME_CATEGORY.seq) as section_start ONE ROW PER MATCH PATTERN (SAME_CATEGORY* DIFFERENT_CATEGORY) -- as many times as possible DEFINE SAME_CATEGORY AS SAME_CATEGORY.category = FIRST(SAME_CATEGORY.category) , DIFFERENT_CATEGORY AS DIFFERENT_CATEGORY.category != NEXT(DIFFERENT_CATEGORY.category) ) MR order by rows_in_section desc )
  71. 71. Solution using lead with observed_sections as ( select seq , case lag(category) over (order by seq) when category then 'N' -- next row has same category as current row else 'Y' end start_of_next_category – is row beginning of a section? , category from observations o ) , sectioned_observations as ( select category , seq , lead( case start_of_next_category when 'Y' then seq end) ignore nulls over (order by seq) start_next from observed_sections ) select so.category , so.seq start_position , so.start_next -1 last_position , start_next - seq section_size from sectioned_observations so order by start_next - seq desc nulls last FETCH FIRST 1 ROWS ONLY
  72. 72. Suppose we allow a single interruption of a sequence • One record with a different category will not end the sequence – it might after all be a fluke or an incident • Rewrite the pattern match to also accept one entry with a different category ONE ROW PER MATCH AFTER MATCH SKIP TO NEXT ROW -- a next row in the current match may be start of a next string PATTERN (SAME_CATEGORY* DIFFERENT_CATEGORY{0,1} SAME_CATEGORY* ) DEFINE SAME_CATEGORY AS SAME_CATEGORY.category = FIRST(SAME_CATEGORY.category) , DIFFERENT_CATEGORY AS DIFFERENT_CATEGORY.category != SAME_CATEGORY.category
  73. 73. Find sequence (with one accepted interruption) from all records SELECT substr(section_category,1,1) cat , section_start , seq FROM observations MATCH_RECOGNIZE ( ORDER BY seq MEASURES SAME_CATEGORY.category as section_category , FIRST(SAME_CATEGORY.seq) as section_start , seq as seq ONE ROW PER MATCH AFTER MATCH SKIP TO NEXT ROW -- a next row in the current match may be -- start of a next string PATTERN (SAME_CATEGORY* DIFFERENT_CATEGORY{0,1} SAME_CATEGORY* ) DEFINE SAME_CATEGORY AS SAME_CATEGORY.category = FIRST(SAME_CATEGORY.category) , DIFFERENT_CATEGORY AS DIFFERENT_CATEGORY.category != SAME_CATEGORY.category ) MR order by rows_in_section desc
  74. 74. Miscellaneous
  75. 75. Miscellaneous • Default • Varchar2 • Java in the Database • SQL Translation • Export View as Table • DICOM support for WebCenter Content • New package dbms_monitor for fine grained trace collection
  76. 76. Export View as Table • Through the Data Pump export facility, a view can be exported as a table • Upon import, a table is created – With the columns exposed by the view – And with all the data returned by the view • This provides a lot of control over what exactly is exported – and will subsequently be imported • Available in PL/SQL DBMS_DATAPUMP package and in expdp command line – Also available in network mode import
  77. 77. Varchar2 expansion • VARCHAR2 maximum size to 32Kb – In Column definition – In Type definition • On par with VARCHAR2 in PL/SQL programs • Note: anything over 4000 bytes is stored out of line, just like CLOBs are
  78. 78. Database Java Engine (JVM) • The Java engine (JVM) in the database is now JDK 6 (default) or JDK 7 (when explicitly set) • Starting from Oracle Database 12c Release 1, Oracle JVM provides support for multiple JDK versions, including the latest JDK version. – Typically, the supported versions are a default version and the next higher version. – Oracle Database 12c Release 1 (12.1) supports JDK 6 and JDK 7, where JDK 6 is the default JDK version. So, if you do not set the JDK version explicitly, the JDK version in effect is JDK 6. • You can set the JDK version prior to database creation – Or update an already created database
  79. 79. SQL Statement Preprocessor • A new mechanism is provided to allow the text of a SQL statement, submitted from a client program using an open application programming interface (API) such as ODBC or JDBC, to be translated by user-supplied code before it is submitted to the Oracle Database SQL compiler. Application SQLPre proce ssor SQL engine SQL
  80. 80. SQL Statement Preprocessor • The translation code is named and is installed in the database using a PL/SQL API. It can be implemented programmatically, or by look-up, or by a suitable mixture of these. • The mechanism also allows Oracle error codes and American National Standards Institute (ANSI) SQLSTATES to be translated by user-supplied code. • The motivating use case is to allow extant client-side application code, written for a different vendor's database (and therefore for a SQL dialect other than Oracle's), to run unchanged against an Oracle Database by emulating the syntax and semantics of the other SQL dialect thereby greatly reducing the cost of migration. • Additionally, this feature can satisfy any other use case where it is expedient to intervene between the SQL statement that the client submits and what is actually executed. • See: Oracle Database SQL Translation Installation, Configuration, and User's Guide for details
  81. 81. SQL Translation • Objective: turn SQL issued by client applications into better Oracle SQL • Package DBMS_SQL_TRANSLATOR • Register SQL for Profile and specify the SQL to execute in that case • Register Error for Profile and specify the Error Code to return in that case • Questions – Deal with bind parameters?
  82. 82. DEFAULT
  83. 83. DEFAULT • New in 12c – Default based on Sequence – Default applied (also) when NULL was explicitly specified – Identity Column that is automatically assigned generated sequence number value – Meta Data Only Defaults
  84. 84. Specify default to be applied even when insert stipulated NULL • For example: – When no salary is provided for a new employee, either because the salary is missing from the INSERT or when it is provided but set to NULL, then set a default value [of 1000] alter table emp modify (sal number(10,2) DEFAULT ON NULL 1000 )
  85. 85. DERIVE DEFAULT value from a sequence • For example: – The default value for a column that is specified as DEFAULT ON NULL can be derived from a sequence – so no before row insert trigger is needed for this! alter table emp modify (empno number(5) NOT NULL DEFAULT ON NULL EMPNO_SEQ.NEXTVAL )
  86. 86. Define column as ‘identity column’ • Specify an identity column: .the identity column will be assigned an increasing or decreasing integer value from a sequence generator for each subsequent INSERT statement – Use ALWAYS instead of BY DEFAULT to force the generated value over any value that may have been provided in the insert statement create table emp ( empno NUMBER GENERATED BY DEFAULT AS IDENTITY (START WITH 100 INCREMENT BY 10) , ... )
  87. 87. Improved Defaults – metadata only defaults c##tkyte%CDB1> create table t 2 as 3 select * 4 from stage; Table created. c##tkyte%CDB1> exec show_space('T') … Full Blocks ..................... 1,437 Total Blocks............................ 1,536 Total Bytes............................. 12,582,912 Total MBytes............................ 12 … PL/SQL procedure successfully completed.
  88. 88. Improved Defaults – metadata only defaults c##tkyte%CDB1> set timing on c##tkyte%CDB1> alter table t add (data char(2000) default 'x'); Table altered. Elapsed: 00:00:00.07 ops$tkyte%ORA11GR2> set timing on ops$tkyte%ORA11GR2> alter table t add (data char(2000) default 'x'); Table altered. Elapsed: 00:00:28.59 11g 12c
  89. 89. Improved Defaults – metadata only defaults c##tkyte%CDB1> exec show_space('T') … Full Blocks ..................... 1,437 Total Blocks............................ 1,536 Total Bytes............................. 12,582,912 Total MBytes............................ 12 … PL/SQL procedure successfully completed. ops$tkyte%ORA11GR2> exec show_space('T') … Total MBytes............................ 9 <<<= before Total MBytes............................ 192 <<<= after … PL/SQL procedure successfully completed. 11g 12c
  90. 90. Summary
  91. 91. Best Take Aways • Flashback and Temporal Validity • In-line PL/SQL Functions • SQL Pattern Match • Default value derived from sequence (identity column) • Top-N Query (“SQL pagination”) • Varchar2(32768) • Data Masking to create representative test data sets • Multitenant architecture allowing for fast cloning of databases • DBMS_UTILITY.EXPAND_SQL_TEXT for full query discovery

×