SQL Server 2016 CTP2 introduced support for temporal tables as a database feature that provides built-in support for provide information about data stored in the table at any point in time rather than only the data that is correct at the current moment in time.
Topics will cover:
What is a Temporal Table?, Why Temporal? How does this work?, When to use (use cases) and demos...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Back to the future - Temporal Table in SQL Server 2016
1. Back to the future – Temporal Tables
in
SQL Server 2016
Stéphane Fréchette
Thursday September 17, 2015
2. My name is Stéphane Fréchette
SQL Server MVP | Data & Business Intelligence Solutions Architect | Consultant |
Speaker | Big Data | NoSQL | Data Science. Drums, good food and fine wine.
I have a passion for architecting, designing and building solutions that matter.
Twitter: @sfrechette
LinkedIn: ca.linkedin.com/stephanefrechette
Blog: stephanefrechette.com
Email: stephanefrechette@ukubu.com
SQLSaturday Boston #364
Who Am I?
4. A temporal table is a table for which a PERIOD definition exists and which
contains system columns with a datatype of datetime2 into which the period
of validity is recorded by the system, and which has an associated history
table into which the system records all prior versions of each record with their
period of validity.
With a temporal table, the value of each record at any point in time can be
determined, rather than just the current value of each record. A temporal
table is also referred to as a system-versioned table
What is a Temporal Table?
5. Real data sources are dynamic
Historical data may be critical to business success
Traditional databases fail to provide required insights
Workarounds are…
Complex, expensive, limited, inflexible, inefficient
SQL Server 2016 makes life easy
No change in programming model
New Insights
Why Temporal
Time Travel Data Audit
Slowly Changing
Dimensions
Repair record-level
corruptions
6. No change in programming model New Insights
INSERT / BULK INSERT
UPDATE
DELETE
MERGE
DML SELECT * FROM temporal
Querying
How to start with temporal
CREATE temporal
TABLE PERIOD FOR
SYSTEM_TIME…
ALTER regular_table
TABLE ADD PERIOD…
DDL
FOR SYSTEM_TIME
AS OF
FROM..TO
BETWEEN..AND
CONTAINED IN
Temporal Querying
7. Temporal table (actual data)
Insert / Bulk Insert
* Old versions
Update */ Delete *
How system-time works?
History Table
8. Temporal table (actual data)
Temporal Queries *
(Time travel,etc.)
How system-time works?
History Table
Regular queries
(current data)
* Include Historical
Version
9. Querying Temporal Data
Expression Qualifying Rows
AS OF <date_time> SysStartTime < = date_time AND SysEndTime >
date_time
FROM <start_date_time> TO <end_date_time> SysStartTime < end_date_time AND SysEndTime >
start_date_time
BETWEEN <start_date_time> AND
<end_date_time>
SysStartTime < = end_date_time AND SysEndTime >
start_date_time
CONTAINED IN (<start_date_time>,
<end_date_time>
SysStartTime > = start_date_time AND SysEndTime
< = end_date_time
10. DepNum DepName MngrID From To
A001 Marketing 5 2005 2008
A002 Sales 2 2005 2007
A003 Consulting 6 2005 2006
A003 Consulting 10 2009 2012
DepNum DepName MngrID
A001 Marketing 5
A001 Marketing 6
A002 Sales 2
A002 Sales 5
A003 Consulting 6
A003 Consulting 10
DepNum DepName MngrID From To
A001 Marketing 6 2008 ∞
A002 Sales 5 2007 ∞
A001
A002
A003
period of validity current time
∞
∞
Department (current)
Department (history)
Department (current + history)
2005 2015
SELECT * FROM DepartmentSELECT * FROM Department FOR SYSTEM_TIME
BETWEEN '2006.01.01' AND '2007.01.01'
SELECT * FROM Department FOR SYSTEM_TIME
CONTAINED IN ('2007.01.01', '2009.01.01')
SELECT * FROM Department FOR SYSTEM_TIME
AS OF '2006.01.01'
Getting insights from temporal
A001
A002
A003
“Get actual row versions”AS OFBETWEEN..ANDCONTAINED IN
11. Provides correct information
about stored facts at any
point in time, or between 2
points in time.
There are two orthogonal
sets of scenarios with regards
to temporal data:
System(transaction)-time
Application-time
SELECT * FROM Person.BusinessEntityContact
FOR SYSTEM_TIME BETWEEN @Start AND @End
WHERE ContactTypeID = 17
Performance
Temporal database support - BETWEEN
12. • A temporal table must have a primary key defined
• History table cannot have constraints; primary key, foreign key, table or column constraints
• INSERT and UPDATE statements cannot reference the SYSTEM_TIME period columns
• TRUNCATE TABLE is not supported while SYSTEM_VERSIONING is ON
• Direct modification of the data in a history table is not permitted
• INSTEAD OF triggers not permitted on current and history table, AFTER triggers permitted only
on current table
• REPLICATION usage is limited, some objects/properties are not replicated
Limitations…
13. SELECT * FROM Department
FOR SYSTEM_TIME
AS OF '2010.01.01' Facts:
1. History is much bigger than actual data
2. Retained between 3 and 10 years
3. “Warm”: up to a few weeks/months
4. “Cold”: rarely queried
Solution:
History as a stretch table:
PeriodEnd < “Now - 6 months”
Azure SQL Database
Business Scenario…
14. MSDN Documentation
https://msdn.microsoft.com/en-CA/library/dn935015.aspx
Temporal in SQL Server 2016 (Video) – Channel 9
https://channel9.msdn.com/Shows/Data-Exposed/Temporal-in-SQL-Server-2016
SQL Server Pro – First Look at System-Versioned Temporal Tables…
http://sqlmag.com/sql-server/first-look-system-versioned-temporal-tables-part-1-creating-tables-and-modifying-data
Resources
Source: https://msdn.microsoft.com/en-us/library/dn935015(v=sql.130).aspx
A temporal table is a new type of table that provides correct information about stored facts at any point in time. Each temporal table consists of two tables actually, one for the current data and one for the historical data. The system automagically ensures that when the data changes in the table with the current data the previous values are stored in the historical table. Querying constructs are provided to hide this complexity from users. For more information, see Temporal Tables.
Introduction to Key Components and Concepts
What is a Temporal Table?
A temporal table is a table for which a PERIOD definition exists and which contains system columns with a datatype of datetime2 into which the period of validity is recorded by the system, and which has an associated history table into which the system records all prior versions of each record with their period of validity. With a temporal table, the value of each record at any point in time can be determined, rather than just the current value of each record. A temporal table is also referred to as a system-versioned table.
Why Temporal?
Real data sources are dynamic and more often than not business decisions rely on insights that analysts can get from data evolution. Use cases for temporal tables include:
Understanding business trends over time
Tracking data changes over time
Auditing all changes to data
Maintaining a slowly changing dimension for decision support applications
Recovering from accidental data changes and application errors
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
A temporal table is a new type of table that provides correct information about stored facts at any point in time. Each temporal table consists of two tables actually, one for the current data and one for the historical data. The system automagically ensures that when the data changes in the table with the current data the previous values are stored in the historical table. Querying constructs are provided to hide this complexity from users. For more information, see Temporal Tables.
Introduction to Key Components and Concepts
What is a Temporal Table?
A temporal table is a table for which a PERIOD definition exists and which contains system columns with a datatype of datetime2 into which the period of validity is recorded by the system, and which has an associated history table into which the system records all prior versions of each record with their period of validity. With a temporal table, the value of each record at any point in time can be determined, rather than just the current value of each record. A temporal table is also referred to as a system-versioned table.
Why Temporal?
Real data sources are dynamic and more often than not business decisions rely on insights that analysts can get from data evolution. Use cases for temporal tables include:
Understanding business trends over time
Tracking data changes over time
Auditing all changes to data
Maintaining a slowly changing dimension for decision support applications
Recovering from accidental data changes and application errors
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
How Does Temporal Work?
System-versioning for a table is implemented as a pair of tables, a current table and a history table. Within each of these tables, two additional datetime (datetime2 datatype) columns are used to define the period of validity for each record – a system start time (SysStartTime) column and a system end time (SysEndTime) column. The current table contains the current value for each record. The history table contains the each previous value for each record, if any, and the start time and end time for the period for which it was valid.
INSERTS: On an INSERT, the system sets the value for the SysStartTime column to the UTC time of the current transaction based on the system clock and assigns the value for the SysEndTime column to the maximum value of 9999-12-31 – this marks the record as open.
UPDATES: On an UPDATE, the system stores the previous value of the record in the history table and sets the value for the SysEndTime column to the UTC time of the current transaction based on the system clock. This marks the record as closed, with a period recorded for which the record was valid. In the current table, the record is updated with its new value and the system sets the value for the SysStartTime column to the UTC time for the transaction based on the system clock. The value for the updated record in the current table for the SysEndTime column remains the maximum value of 9999-12-31.
DELETES: On a DELETE, the system stores the previous value of the record in the history table and sets the value for the SysEndTime column to the UTC time of the current transaction based on the system clock. This marks this record as closed, with a period recorded for which the previous record was valid. In the current table, the record is removed. Queries of the current table will not return this value. Only queries that deal with history data return data for which a record is closed.
MERGE: On a MERGE, MERGE behaves as an INSERT, an UPDATE, or a DELETE based on the condition for each record.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
The SYSTEM_TIME period columns used to record the SysStartTime and SysEndTime values must be defined with a datatype of datetime2.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
The SYSTEM_TIME period columns used to record the SysStartTime and SysEndTime values must be defined with a datatype of datetime2.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
Returns a table with the values for all record versions that were active within the specified time range, regardless of whether they started being active before the <start_date_time> parameter value for the FROM argument or ceased being active after the <end_date_time> parameter value for the TO argument. Internally, a union is performed between the temporal table and its history table and the results are filtered to return the values for all row versions that were active at any time during the time range specified. Records that became active exactly on the lower boundary defined by the FROM endpoint are included and records that became active exactly on the upper boundary defined by the TO endpoint are also included.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
Returns a table with single record for each row containing the values that were actual (current) at the specified point in time in the past. Internally, a union is performed between the temporal table and its history table and the results are filtered to return the values in the row that was valid at the point in time specified by the <date_time> parameter. The value for a row is deemed valid if the system_start_time_column_name value is less than or equal to the <date_time> parameter value and the system_end_time_column_name value is greater than the <date_time> parameter value.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
Returns a table with the values for all record versions that were active within the specified time range, regardless of whether they started being active before the <start_date_time> parameter value for the FROM argument or ceased being active after the <end_date_time> parameter value for the TO argument. Internally, a union is performed between the temporal table and its history table and the results are filtered to return the values for all row versions that were active at any time during the time range specified. Records that became active exactly on the lower boundary defined by the FROM endpoint are included and records that became active exactly on the upper boundary defined by the TO endpoint are also included.
Source: https://msdn.microsoft.com/en-us/library/dn935015.aspx
A temporal table is a new type of table that provides correct information about stored facts at any point in time. Each temporal table consists of two tables actually, one for the current data and one for the historical data. The system automagically ensures that when the data changes in the table with the current data the previous values are stored in the historical table. Querying constructs are provided to hide this complexity from users. For more information, see Temporal Tables.
Introduction to Key Components and Concepts
What is a Temporal Table?
A temporal table is a table for which a PERIOD definition exists and which contains system columns with a datatype of datetime2 into which the period of validity is recorded by the system, and which has an associated history table into which the system records all prior versions of each record with their period of validity. With a temporal table, the value of each record at any point in time can be determined, rather than just the current value of each record. A temporal table is also referred to as a system-versioned table.
Why Temporal?
Real data sources are dynamic and more often than not business decisions rely on insights that analysts can get from data evolution. Use cases for temporal tables include:
Understanding business trends over time
Tracking data changes over time
Auditing all changes to data
Maintaining a slowly changing dimension for decision support applications
Recovering from accidental data changes and application errors
Source: http://channel9.msdn.com/Shows/Data-Exposed/Temporal-in-SQL-Server-2016
Example of using temporal tables with Azure SQL Database stretch tables.