Time has always been an important dimension for data in any database with topics like when was data created, when are records valid, how did records evolve over time, can we compare with yesteryear or even travel through time and data. The Oracle Database 12c release added a number of features in this area of time and history. The powerful Flashback mechanism is enhanced in many ways, such as to allow history to be constructed, for example from existing journaling tables and to capture the transaction context as well as the data change. Now for the first time, Flashback (Query & Data Archive) will become a key element in database design and application implementation. The support for Valid Time Modeling (aka Temporal Database) makes the database aware of the fact that records have a business lifetime with start and expiry date. This awareness results in many new features that will be discussed and demonstrated. Slides from the Singapore Oracle Sessions presentation on July 13th 2015, sponsored by the Oracle ACE Program and organized by Doug Burns.

1. Lucas Jellema
Singapore Oracle Sessions III - 14th July 2015
On Flashback and Temporal Validity –
analyzing the time dimension
in the Oracle 12c database

2. 2
Time

3. 3
The Pension Fund

4. 4
The Pension Fund
• Administration for 2 milion participants
• Record their complete employment history
– employer, job role, salary and benefits
• Used to calculate invoices for monthly premium as well as the benefits
that have been accrued
• MN has to be able to
– Report on the situation per end of each quarter
– Account for any invoice that has been sent (and the way it was calculated)
– Provide an audit trail of all mutations in the employment history including the nature
and timestamp of the mutation and the identity of the perpetrator
– Retain history for specified periods of time

5. 5
The Pension Fund
Current approach
• Journal tables
• Audit columns to record identity of person responsible for change
• DML Triggers on all relevant tables to record history
• Long reporting batches at the end of the quarter
• Complex queries to reconstruct a certain moment in time that caused a
certain invoice
• Logic to exclude data that is no longer relevant/allowed
• [lack of] scheduled batches to remove data once beyond retention

6. Glimpses of the past
Session 1 Session 2

7. Glimpses of the past
Session 1 Session 2

8. Glimpses of the past – same
session

9. Glimpses from the past
How a session can see its own past
Session 1

10. Formalized Glimpses of the past
Flashback Query

11. Compare current with previous

12. Powered by UNDO
UNDO

13. FLASHBACK facilities
• Flashback Database
• Flashback Standby Database
• Flashback Table
• Flashback Table to Before Undrop
(Recycle Bin)
• Flashback Query
• Flashback Versions Query
• Flashback Transaction
DBA
Developer

14. 11g Flashback Data Archive
1 Month
10 Years
UNDO5 Days

15. Configuring Total Recall for
Table EMP
• Set up Flashback Data Archive
• Enable Total Recall for EMP
• Some requirements:
– Tablespace under Automatic Segment Space Management to create the Flashback
Data Archive in
– Automatic UNDO management must be enabled
– The FLASHBACK ARCHIVE ADMINISTER privilege
CREATE FLASHBACK ARCHIVE LONGTERM_ARCHIVE
TABLESPACE HISTORY
RETENTION 15 YEAR
ALTER TABLE EMP
FLASHBACK ARCHIVE LONGTERM_ARCHIVE;

16. Sessions Travel back in time
• Using Flashback Query and Flashback Versions Query, the past can be
inspected
– Queries are adapted
– Every table referenced in a query needs to be explicitly ‘flashbacked’
• Non intrusive time travelling for an entire session is available too –
through dbms_flashback
– Only travelling to the past
– The past is read-only
– Travel to Timestamp or SCN
• For example to set from after-logon trigger or as option in a client
application to set the “time scope”
– For example to create reports for the end of the year – no need to stop new data
coming in!

17. Move entire session to point in time

18. Flashback Row History aka
flashback versions query
• See all flashback data for each row, including:
– SCN range the row was “valid for”
– Time range (approx) the row was valid for
– The transaction id (XID) that modified the row
– The operation (I/U/D) that was performed on the row
– special constants: minvalue and maxvalue
select ename, sal,
versions_operation,
versions_starttime,
versions_endtime,
versions_startscn,
versions_endscn,
versions_xid
from emp versions between timestamp &A and &B
order by versions_startscn nulls first
from emp versions between timestamp minvalue and maxvalue

19. Trac-king’s Salary History

20. Provide insight in trends and recent
changes using flashback versions
10 Years

21. MOST_RECENT_SALARY_CHANGES

22. “Flashback Transaction”
• Select the UNDO_SQL for a certain transaction
select xid
, start_scn
, commit_scn
, operation
, undo_sql
, table_name
from
flashback_transaction_query
where xid = '0004000400000003D';

23. Transaction Back Out
• To undo the effects of one or more transactions, we can invoke
DBMS_FLASHBACK.TRANSACTION_BACKOUT
– with a list of Transaction IDs
– and options for dealing with dependent transactions (later transactions partially on
the same records)
• NonConflict_Only: only backout those parts of selected transaction that do not conflict with later transactions
• Nocascade: raise error if there is dependent transaction
• NoCascade_Force: only backout selected transaction
• Cascade: back out of dependent transaction too
– Backout does not commit: it performs compensating DML, constraints are enforced,
holds locks and reports
– Back out reports are available in views
*_FLASHBACK_TXN_STATE and *_FLASHBACK_TXN_REPORT

24. …_JOURNAL…
Date_modified
User_modified
Date_modified
User_modified
+ columns for
… columns
…_JOURNAL…
Date_modified
User_modified
Date_modified
User_modified
+ columns for
… columns
History is frequently kept in Audit
columns and journaling tables
EMP_JOURNALEMP
Date_modified
User_modified
Date_modified
User_modified
+ columns for
… columns
…_JOURNAL…
Date_modified
User_modified
Date_modified
User_modified
+ columns for
… columns

25. Flashback data archive could mean
The end of journaling
• Get rid of all journaling tables & triggers
• Productivity Gain
• Ease of use of archived data
– AS OF queries
– DBMS_FLASHBACK – transparent
• Performance Benefits
– Maintain flashback data archive is much more efficient than maintaining journaling
tables with triggers
10 YearsEMP
… columns
Date_modified
… columns

26. Overhead of populating Flashback
Data archive compared to triggers
Total Recall vs Triggers
6.0%
8.9%
2.5%
4.5%
54.0%53.2%
56.9%
52.8%
0
10
20
30
40
50
60
70
100k/1k 100k/10k 1M/1k 1M/10k
# of Rows / Commit Interval
%IncreaseinResponseTime
Total Recall Triggers

27. Whodunnit?
• Flashback Data Archive does not record WHO made the change – only
the changes that were made!
• Database Auditing does record the username and the client identifier for
each completed transaction
– AUDIT INSERT,UPDATE,DELETE ON EMP
– Join Flashback Versions Query with user_audit_trail on audit.transactionid =
version.xid
10 YearsEMP
… columns xid Date_modified
… columns
USER_AUDIT
_TRAIL
Client Identifier

28. Flashback Weak Spots
• Whodunnit?
• Also:
– when is the start of history?
– What went on before? What to do with existing archives?
• By the way: Flashback Data Archive requires EE & Advanced
Compression database option

29. 29
License change in 12c
• Flashback Data Archive in all database editions

30. 30
• Flashback Data Archive also available in 11.2.0.4 – all editions

31. Total Recall 12c
• Flashback Data Archive Improvements:
– Complete schema evolution support All table definition, partitioning,
and space management DDLs are supported on FDA-enabled
tables.
– The metadata information for tracking transactions including the
user context is now tracked. The addition of user-context tracking
makes it easier to determine which user made which changes to a
table.
• This could mean that journaling tables are now officially deprecated
• Also given that the current contents of journaling tables can even be migrated to
Flashback Data Archive

32. 32
Ensure transaction context is
recorded (and set)
exec dbms_flashback_archive.set_context_level(level=> 'ALL');
exec dbms_session.set_identifier('The Creepy User from Finance ');
update oow.emp
set sal = sal * 1.4
where ename = 'ALLEN'
/
commit;
exec dbms_session.set_identifier('Scary Janitor from the Annex');
update oow.emp
set sal = sal * 0.7
where ename = 'MILLER'
/
commit;

33. 33
Audit the generated history
SELECT versions_xid
, versions_starttime
, empno, ename, sal new_sal
, s.client_identifier
FROM oow.emp VERSIONS BETWEEN TIMESTAMP minvalue
AND maxvalue
, sys.sys_fba_context_aud s
where versions_xid = s.xid

34. 34
Alternative: retrieve context with
dbms_flashback_archive.get_sys_context
SELECT versions_xid
, versions_starttime
, empno, ename, sal new_sal
, dbms_flashback_archive.get_sys_context
(versions_xid,'USERENV','CLIENT_IDENTIFIER') who
FROM emp VERSIONS BETWEEN TIMESTAMP minvalue
AND maxvalue

35. Total Recall (2)
• Import and export of history
– Support for import and export using Data Pump for FDA-enabled
tables. Data Pump can now be used to export and import an FDA-
enabled base table along with its schema-evolution metadata and
historical row versions.
• User generated history
– Support for importing user-generated history has been added.
Customers who have been maintaining history using other
mechanisms, such as triggers, can now import that history into Total
Recall.

36. 36
Generate History –
All actions by SYS
create table oow.emp as select * from scott.emp
grant execute on dbms_flashback_archive to oow;
grant execute on dbms_flashback to oow;
CREATE FLASHBACK ARCHIVE DEFAULT one_year TABLESPACE users
QUOTA 100M RETENTION 1 YEAR;
grant flashback archive on one_year to oow;
exec dbms_flashback_archive.create_temp_history_table('OOW', 'EMP');
-- This statement once in a database instance
--This will extend mappings to the past so that import of old
history can be done. Goes back to 01-JAN-88.
EXEC DBMS_FLASHBACK_ARCHIVE.extend_mappings();
---
-- after some history has been created:
EXEC DBMS_FLASHBACK_ARCHIVE.IMPORT_HISTORY('oow','EMP');

37. 37
Generate History –
Actions by Application
• Insert records describing each stage of history that has existed
– Including start and end time of historic state
insert into temp_history
(RID , STARTSCN , ENDSCN , XID, OPERATION
,EMPNO, ename, job, hiredate, sal, deptno )
values (NULL, timestamp_to_scn(to_date('01-04-2001', 'DD-MM-YYYY')),
timestamp_to_scn(to_date('01-07-2003', 'DD-MM-YYYY')), NULL, 'I'
,1567,'SELLERS','CLERK',to_date('01-04-2001','DD-MM-YYYY'),2200, 10);
insert into temp_history
(RID , STARTSCN , ENDSCN , XID, OPERATION
,EMPNO, ename, job, hiredate, sal, deptno)
values (NULL, timestamp_to_scn(to_date('01-07-2003', 'DD-MM-YYYY')),
timestamp_to_scn(to_date('01-10-2006', 'DD-MM-YYYY')), NULL, 'U'
,1567,'SELLERS','CLERK',to_date('01-04-2001','DD-MM-YYYY'),2200, 20);
…

38. 38
Query the generated history
select ename
, job
from emp as of timestamp (sysdate - INTERVAL '10' YEAR)
minus
select ename
, job
from emp
select ename
, job
from emp as of timestamp (systimestamp - INTERVAL '3' YEAR)
minus
select ename
, job
from emp

39. 39
The Pension Fund
Future approach
• Flashback Data Archives
– Per retention period
• Associate relevant tables to FDA
• Configure tracking of meta-data
– To ensure transaction context is retained
• Report batches can be run at any
moment against any moment
– Using dbms_flashback.enable_at_time
• Queries can easily be performed to
reproduce a moment in the past
• FDA management takes care of
deprecated data
• Journal tables
• Audit columns
• DML Journaling Triggers
• Batches at the end of the
quarter
• Complex queries to reconstruct
• Logic to exclude deprecated
data
• batches to purge data

40. 40
Looking into the future…
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products

41. 41
Looking further into the
future…
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'ASOF'
, query_time => TO_TIMESTAMP('01-10-2018', 'DD-MM-YYYY')
);
end;

42. 42
Current situation …
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'CURRENT'
);
end;

43. 43
All data in the table
(the default setting)
OUR_PRODUCTS
NAME PRICE
select name, price
from our_products
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'ALL'
);
end;

44. 44
All data in the table
(the default setting)
OUR_PRODUCTS
NAME PRICE
select name, price, start_date, end_date
from our_products
order
by start_date
START_DATE END_DATE
begin
DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME (
level => 'ALL'
);
end;

45. 45
Part of SQL 2011 standard:
Temporal Database

46. Make the database aware of the time
based business validity of records
• Add timestamp columns indicating start and end of valid time for a record
• Specify a PERIOD for the table
• Note:
– A table can have multiple sets of columns, describing multiple types of validness
create table our_products
( name varchar2(100)
, price number(7,2)
, start_date timestamp
, end_date timestamp
, PERIOD FOR offer_time (start_date, end_date)
);

47. Valid time aware
flashback queries
• Select all product prices on offer at a certain moment in time
• Perform all queries for records that are valid at a certain point in time –
past or future
• Return all records currently (session time) valid
• Return all records (default)
SELECT *
FROM OUR_PRODUCTS AS OF PERIOD FOR offer_time
TO_TIMESTAMP('01-10-2014','DD-MM-YYYY')
EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time
( 'ASOF'
, TO_TIMESTAMP('01-05-2016','DD-MM-YYYY')
);
EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('CURRENT');
EXECUTE DBMS_FLASHBACK_ARCHIVE.enable_at_valid_time('ALL');

48. 48
The future of temporal validity
Beyond 12.1.0.2 many Temporal Validity enhancements are expected:
• Gap and overlap checks
• Valid time aware DML
• Temporal Primary and Unique Key constraints that allow multiple non-
overlapping entries
• Temporal referential constraints that take into account the valid-time
during which the rows exist.
– Child needs to have a valid Master at any time during its own validity
• Temporal Aggregation - group or order by valid-time
• Normalization - coalescing rows which are in adjacent or overlapping time
periods
• Temporal joins – joins between tables with valid-time semantics based on
‘simultaneous validity’
• For Information Lifecycle Management (ILM), the Valid Time information is
used to assess records to move

49. 49
Summary
• Time and History are supported in Oracle Database 12c along two
dimensions:
– Transaction time – what been the committed state of the database at any point
– Business (valid) time – what was, is or will be the relevant data at any point
• Flashback is concerned with transaction time. As of 12c:
– Available in all editions (no license restrictions)
– Includes transaction (session) context at time of commit (whodunnit)
– Supports user defined history (as well as import and export)
– Flashback Query, Flashback Versions Query, Enable database session at timestamp
 Flashback is ready for inclusion in database design and application development
• Valid Time Modeling (Temporal Database) is concerned with business
validity of data – begin date and end date
– Based on SQL 2011
– Low impact use: alter table add (period for …)
– Easy querying across time – past, present and future
– Next steps are expected in enriching temporal database support (12.2?)