Discovering and querying temporal data
•
0 likes•894 views
Did you know MariaDB Server is the only open source database to implement temporal tables per the SQL specification, allowing you to query data as it existed at a previous point in time? MariaDB Server 10.3 uses system-versioned tables and MariaDB Server 10.4 uses system- or application-versioned tables. Whether it is for reporting and analysis or fine-grained data recovery, temporal data and queries can change the way you think about and manage data. In this session, you’ll learn how this game-changing feature can be used to tackle problems that simply were not solvable before.
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Discovering and querying temporal data
Similar to Discovering and querying temporal data (20)
More from MariaDB plc
More from MariaDB plc (20)
Recently uploaded
Recently uploaded (20)
Discovering and querying temporal data
- 1. MARIADB TEMPORAL TABLES Roger Eisentrager Solutions Architect MariaDB Corporation 4pm Feb 26th -- West C
- 2. FOCUSING ON 1. What are Temporal Tables ? 2. Temporal Tables Owners Manual (basic commands) 3. Demo (4 parts) 4. Considerations 5. Question and Answer
- 4. Time ?
- 5. Temporal Tables ● A system-versioned temporal table is a type of user table. ○ This type of temporal table is referred to as a system-versioned temporal table because the period of validity for each row is managed by the system (i.e. database engine). ● Temporal means: “relating to time”. System Versioned Tables means: tracking/record versioning table changes. So, using both terms the feature is tracking table (data and structure changes) “as of” a point in time.
- 6. Why Temporal Tables ● System Version Tables can be used for ○ Data analysis (retrospective, trends etc.) ○ Forensic discovery ○ Legal requirements ( to store data for ‘N’ years ) ■ HIPAA, SOX, GDPR etc... ○ Point in Time Analysis (PITA) and Point in Time Recovery (PITR) ○ Anomaly Detection
- 7. What are Temporal Tables ● System Version Tables include timestamped versions of the data in a table. This allows to: ○ Track changes ○ Compare data based on a date and time of the event ○ Visualize the cycle of data and to create trends ○ Audit the change of data (not used for user auditing) ○ Saves space
- 9. Temporal Table commands - Create CREATE TABLE ALTER TABLE create table apple (x int) WITH SYSTEM VERSIONING; alter table apple ADD SYSTEM VERSIONING;
- 10. Temporal Table commands - Drop/Stop collection ALTER TABLE alter table apple DROP SYSTEM VERSIONING; DROP TABLE drop table apple;
- 11. Temporal Table commands - select SYSTEM_TIME AS OF TIMESTAMP ALL select * from tablea FOR SYSTEM_TIME AS OF TIMESTAMP '2019-03-01 10:00:00'; select * from tablea FOR SYSTEM_TIME ALL;
- 12. Demo
- 13. Demo ● There 4 main parts to the demo 1 - create a table and add versioning 2 - create a table and test truncate with versioning 3 - create a table and alter its structure (ddl) 4 - create a table with system version partitioning option
- 14. Demo Part 1 - Create Table with Versioning
- 15. Demo - Part 1 - Create Table w/Versioning MariaDB [test2]> create table apple (c1 int, c2 int); MariaDB [test2]> alter table apple ADD SYSTEM VERSIONING; MariaDB [test2]> insert into apple (c1,c2) values (1,1); MariaDB [test2]> insert into apple (c1,c2) values (2,2); MariaDB [test2]> insert into apple (c1,c2) values (3,3); MariaDB [test2]> insert into apple (c1,c2) values (4,4); MariaDB [test2]> insert into apple (c1,c2) values (5,5); MariaDB [test2]> select * from apple; +------+------+ | c1 | c2 | +------+------+ | 1 | 1 | | 2 | 2 | | 3 | 3 | | 4 | 4 | | 5 | 5 | +------+------+
- 16. Demo - Part 1 - Create Table w/Versioning MariaDB [test2]> select *, row_start, row_end from apple FOR SYSTEM_TIME ALL order by row_start; +------+------+----------------------------+----------------------------+ | c1 | c2 | row_start | row_end | +------+------+----------------------------+----------------------------+ | 1 | 1 | 2019-02-19 10:40:29.492757 | 2038-01-18 21:14:07.999999 | | 2 | 2 | 2019-02-19 10:40:29.497545 | 2038-01-18 21:14:07.999999 | | 3 | 3 | 2019-02-19 10:40:29.501682 | 2038-01-18 21:14:07.999999 | | 4 | 4 | 2019-02-19 10:40:29.506487 | 2038-01-18 21:14:07.999999 | | 5 | 5 | 2019-02-19 10:40:30.251711 | 2038-01-18 21:14:07.999999 | +------+------+----------------------------+----------------------------+
- 17. Demo - Part 1 - Create Table w/Versioning MariaDB [test2]> update apple set c1=9 WHERE c1=3; MariaDB [test2]> -- order by c1 MariaDB [test2]> select * from apple where c2=3 order by c1; +------+------+ | c1 | c2 | +------+------+ | 9 | 3 | +------+------+ MariaDB [test2]> select *, row_start, row_end from apple FOR SYSTEM_TIME ALL where c2=3 order by row_start; +-----+------+----------------------------+----------------------------+ | c1 | c2 | row_start | row_end | +-----+------+----------------------------+----------------------------+ | 3 | 3 | 2019-02-19 10:40:29.501682 | 2019-02-19 10:43:29.005184 | -> OLD | 9 | 3 | 2019-02-19 10:43:29.005184 | 2038-01-18 21:14:07.999999 | -> LIVE +-----+------+-----------------------------+----------------------------+
- 18. Demo Part 2 - Table Truncate with Versioning
- 19. Demo - Part 2 - Test Truncate -- test truncate MariaDB [test2]> truncate table apple; Query OK, 0 rows affected (0.047 sec) -- see results of truncate MariaDB [test2]> select * from apple; Empty set (0.001 sec) MariaDB [test2]> select * from apple for system_time all; Empty set (0.000 sec) -- insert some rows back in MariaDB [test2]> insert into apple (c1,c2) values (57,57); Query OK, 1 row affected (0.005 sec) MariaDB [test2]> insert into apple (c1,c2) values (58,58); Query OK, 1 row affected (0.003 sec) MariaDB [test2]> insert into apple (c1,c2) values (59,59); Query OK, 1 row affected (0.003 sec)
- 20. Demo - Part 2 - Test Truncate MariaDB [test2]> update apple set c1=77 WHERE c1=58; MariaDB [test2]> select * from apple where c2=58; +------+------+ | c1 | c2 | +------+------+ | 77 | 58 | +------+------+ MariaDB [test2]> select * from apple order by c1; +------+------+ | c1 | c2 | +------+------+ | 57 | 57 | | 59 | 59 | | 77 | 58 | +------+------+
- 21. Demo - Part 2 - Test Truncate MariaDB [test2]> select *, row_start, row_end from apple FOR SYSTEM_TIME ALL where c2=58 order by row_start; +------+------+----------------------------+----------------------------+ | c1 | c2 | row_start | row_end | +------+------+----------------------------+----------------------------+ | 58 | 58 | 2019-02-19 10:45:13.386396 | 2019-02-19 10:45:13.419903 | -> OLD | 77 | 58 | 2019-02-19 10:45:13.419903 | 2038-01-18 21:14:07.999999 | -> LIVE +------+------+----------------------------+----------------------------+ 2 rows in set (0.001 sec)
- 22. Demo Part 3 - Create Table Alter Structure (ddl)
- 23. Demo - Part 3 - Alter Table Structure MariaDB [test2]> create table apple (c1 int, c2 int); Query OK, 0 rows affected (0.045 sec) MariaDB [test2]> alter table apple ADD SYSTEM VERSIONING; Query OK, 0 rows affected (0.097 sec) Records: 0 Duplicates: 0 Warnings: 0 MariaDB [test2]> alter table apple add column (c10 int); ERROR 4119 (HY000): Not allowed for system-versioned `test2`.`apple`.
- 24. Demo - Part 3 - Alter Table Structure -- work around: MariaDB [test2]> alter table apple DROP SYSTEM VERSIONING; Query OK, 0 rows affected (0.099 sec) Records: 0 Duplicates: 0 Warnings: 0 MariaDB [test2]> alter table apple add column (c10 int); Query OK, 0 rows affected (0.016 sec) Records: 0 Duplicates: 0 Warnings: 0 MariaDB [test2]> alter table apple ADD SYSTEM VERSIONING; Query OK, 0 rows affected (0.093 sec) Records: 0 Duplicates: 0 Warnings: 0
- 25. Demo Part 4 - Table Versioning with Partitioning
- 26. Demo - Part 4 - Table Partitioning MariaDB [test2]> CREATE TABLE apple (c1 INT, c2 INT) -> WITH SYSTEM VERSIONING -> PARTITION BY SYSTEM_TIME INTERVAL 1 MONTH -> ( -> PARTITION m01 HISTORY, -> PARTITION m02 HISTORY, -> PARTITION m03 HISTORY, -> PARTITION m04 HISTORY, -> PARTITION m05 HISTORY, -> PARTITION m06 HISTORY, -> PARTITION m07 HISTORY, -> PARTITION m08 HISTORY, -> PARTITION m09 HISTORY, -> PARTITION m10 HISTORY, -> PARTITION m11 HISTORY, -> PARTITION m12 HISTORY, -> PARTITION pcur CURRENT );
- 27. Demo - Part 4 - Table Partitioning -- Using ---> INFORMATION_SCHEMA.PARTITIONS internal object +--------+--------+-----------+------------------+-----------------------+------------+---------------------+ | t_name | partnm | p_ord_pos | partition_method | partition_description | table_rows | create_time | +--------+-----------+--------+------------------+-----------------------+------------+---------------------+ | apple | m01 | 1 | SYSTEM_TIME | 2019-03-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m02 | 2 | SYSTEM_TIME | 2019-04-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m03 | 3 | SYSTEM_TIME | 2019-05-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m04 | 4 | SYSTEM_TIME | 2019-06-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m05 | 5 | SYSTEM_TIME | 2019-07-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m06 | 6 | SYSTEM_TIME | 2019-08-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m07 | 7 | SYSTEM_TIME | 2019-09-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m08 | 8 | SYSTEM_TIME | 2019-10-19 11:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m09 | 9 | SYSTEM_TIME | 2019-11-19 10:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m10 | 10 | SYSTEM_TIME | 2019-12-19 10:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m11 | 11 | SYSTEM_TIME | 2020-01-19 10:58:54 | 0 | 2019-02-19 10:58:54 | | apple | m12 | 12 | SYSTEM_TIME | 2020-02-19 10:58:54 | 0 | 2019-02-19 10:58:54 | | apple | pcur | 13 | SYSTEM_TIME | CURRENT | 0 | 2019-02-19 10:58:54 | +--------+-----------+-----------+--------------------+----------------------+-----------+---------------------+
- 28. Demo - Part 4 - Table Partitioning MariaDB [test2]> insert into apple (c1,c2) values (1,1); Query OK, 1 row affected (0.005 sec) MariaDB [test2]> insert into apple (c1,c2) values (2,2); Query OK, 1 row affected (0.002 sec) MariaDB [test2]> insert into apple (c1,c2) values (3,3); Query OK, 1 row affected (0.004 sec) MariaDB [test2]> select * from apple order by c1; +------+------+ | c1 | c2 | +------+------+ | 1 | 1 | | 2 | 2 | | 3 | 3 | +------+------+ 3 rows in set (0.001 sec)
- 29. Demo - Part 4 - Table Partitioning MariaDB [test2]> select *, row_start, row_end from apple FOR SYSTEM_TIME ALL order by row_start; +------+------+----------------------------+----------------------------+ | c1 | c2 | row_start | row_end | +------+------+----------------------------+----------------------------+ | 1 | 1 | 2019-02-19 11:06:27.016036 | 2038-01-18 21:14:07.999999 | | 2 | 2 | 2019-02-19 11:06:27.022636 | 2038-01-18 21:14:07.999999 | | 3 | 3 | 2019-02-19 11:06:27.026585 | 2038-01-18 21:14:07.999999 | +------+------+----------------------------+----------------------------+
- 30. Demo - Part 4 - Table Partitioning -- Using ---> INFORMATION_SCHEMA.PARTITIONS internal object +--------+-----------+-----------+------------------+-----------------------+------------+ | t_name | part_name | p_ord_pos | partition_method | partition_description | table_rows | +---- ----+-----------+-----------+------------------+-----------------------+------------+ | apple | m01 | 1 | SYSTEM_TIME | 2019-03-19 11:58:54 | 0 | | apple | m02 | 2 | SYSTEM_TIME | 2019-04-19 11:58:54 | 0 | | apple | m03 | 3 | SYSTEM_TIME | 2019-05-19 11:58:54 | 0 | | apple | m04 | 4 | SYSTEM_TIME | 2019-06-19 11:58:54 | 0 | | apple | m05 | 5 | SYSTEM_TIME | 2019-07-19 11:58:54 | 0 | | apple | m06 | 6 | SYSTEM_TIME | 2019-08-19 11:58:54 | 0 | | apple | m07 | 7 | SYSTEM_TIME | 2019-09-19 11:58:54 | 0 | | apple | m08 | 8 | SYSTEM_TIME | 2019-10-19 11:58:54 | 0 | | apple | m09 | 9 | SYSTEM_TIME | 2019-11-19 10:58:54 | 0 | | apple | m10 | 10 | SYSTEM_TIME | 2019-12-19 10:58:54 | 0 | | apple | m11 | 11 | SYSTEM_TIME | 2020-01-19 10:58:54 | 0 | | apple | m12 | 12 | SYSTEM_TIME | 2020-02-19 10:58:54 | 0 | | apple | pcur | 13 | SYSTEM_TIME | CURRENT | 3 | +--------+-----------+-----------+--------------------+----------------------+-----------+
- 31. Demo - Part 4 - Table Partitioning MariaDB [test2]> -- Lets alter the 2nd row MariaDB [test2]> update apple set c1=165 WHERE c1=2; MariaDB [test2]> select * from apple where c1=165; +------+------+ | c1 | c2 | +------+------+ | 165 | 2 | +------+------+ MariaDB [test2]> select *, row_start, row_end from apple FOR SYSTEM_TIME ALL where c2=2 order by row_start; +------+------+----------------------------+----------------------------+ | c1 | c2 | row_start | row_end | +------+------+----------------------------+----------------------------+ | 2 | 2 | 2019-02-19 11:06:27.022636 | 2019-02-19 11:07:46.447169 | -> OLD |165 | 2 | 2019-02-19 11:07:46.447169 | 2038-01-18 21:14:07.999999 | -> LIVE +------+------+----------------------------+----------------------------+
- 32. Demo - Part 4 - Table Partitioning -- Using ---> INFORMATION_SCHEMA.PARTITIONS internal object +--------+-----------+-----------+------------------+-----------------------+------------+ | t_name | part_name | p_ord_pos | partition_method | partition_description | table_rows | +---- ----+-----------+-----------+------------------+-----------------------+------------+ | apple | m01 | 1 | SYSTEM_TIME | 2019-03-19 11:58:54 | 1 | | apple | m02 | 2 | SYSTEM_TIME | 2019-04-19 11:58:54 | 0 | | apple | m03 | 3 | SYSTEM_TIME | 2019-05-19 11:58:54 | 0 | | apple | m04 | 4 | SYSTEM_TIME | 2019-06-19 11:58:54 | 0 | | apple | m05 | 5 | SYSTEM_TIME | 2019-07-19 11:58:54 | 0 | | apple | m06 | 6 | SYSTEM_TIME | 2019-08-19 11:58:54 | 0 | | apple | m07 | 7 | SYSTEM_TIME | 2019-09-19 11:58:54 | 0 | | apple | m08 | 8 | SYSTEM_TIME | 2019-10-19 11:58:54 | 0 | | apple | m09 | 9 | SYSTEM_TIME | 2019-11-19 10:58:54 | 0 | | apple | m10 | 10 | SYSTEM_TIME | 2019-12-19 10:58:54 | 0 | | apple | m11 | 11 | SYSTEM_TIME | 2020-01-19 10:58:54 | 0 | | apple | m12 | 12 | SYSTEM_TIME | 2020-02-19 10:58:54 | 0 | | apple | pcur | 13 | SYSTEM_TIME | CURRENT | 3 | +--------+-----------+-----------+--------------------+----------------------+-----------+
- 33. Considerations
- 34. Considerations ● Table can be altered to enable, disable or remove system versioned data ● Transparent to the application ● Temporal columns are generated columns, thus their values are generated and cannot be modified by the user ● Can add system versioning with all (or subset) of specific table columns
- 35. Considerations ● System versioned tables are queried using: ○ AS OF to select data “as of” a given point in time ○ BETWEEN .. AND to select data which has been visible in between two point in times ○ ALL to show current and all historical versions ● A new partitioning BY SYSTEM_TIME exists to partition data for: ○ historical data ○ currently valid data
- 36. Considerations ● Updates creates versioned record (ie - new row for every update) ● Temporal columns contain timestamps that indicate when a row was INSERTed, UPDATEd, or DELETEd. ● Truncate removes data and all version history - Why ? ● TIMESTAMP ● Primary Key ● Versioning can not be applied to generated (virtual and persistent) columns ● DIY (do it yourself or design it yourself) System/Hardware Performance ○ More memory and disk i/o pressure depending on activity level ○ More disk space
Editor's Notes
- Welcome to OpenWorks and the MariaDB Temporal tables session How is Everyone ? Learning a lot ? Explain what I do for MariaDB as a Solutions Architect and how I help clients. Introduce myself and give a little background.
- I will go thru the higher level concepts, BUT the best part is the demo.
- What are Temporal Tables are they Time Travel for data recovery ? Snap back in time ? Not exactly there are much more !
- A system-versioned temporal table is a type of user table designed to keep a full history of data changes and allow easy point in time analysis. This type of temporal table is referred to as a system-versioned temporal table because the period of validity for each row is managed by the system (i.e. database engine). Temporal means: “relating to time”. System Versioned Tables means: tracking/record versioning table changes. So, using both terms the feature is tracking table (data and ddl) “as of” a point in time.
- Data Analysis: The power of temporal querying becomes apparent in combination with views, especially in scenarios when you need to query complex database models including multiple Temporal Tables “as of” any point in time in the past. Who, when, and what broke ! Forensic discovery: The temporal tables provide an audit trail to determine when data was modified in the “parent” table. This helps to meet the requirements of regulatory compliance and to do data forensics when needed by tracking and auditing data changes over time. Simple example, why is “Bill” changing the price of beer at this one distribution center and then immediately changing it back ? Maybe it is not Bill, but someone using account ? Why is this happening only on Friday’s at 5pm ? Legal Requirements: Businesses are facing increased requirements to maintain historical data and even a history of all data changes. Some of these requirements are expressed in the form of government regulations where auditing and compliance inquiries require point in time analysis of data at any time in the past 10 years. Today’s government regulations place strict requirements on enterprises to audit the corporate information access details and produce reports detailing who has changed, or even seen, that information. Consider Health Insurance Portability and Accountability Act (HIPAA) regulations that require healthcare providers to deliver audit trails right down to the row and record. Or the Sarbanes-Oxley Act (SOX), for example, places a wide range of accounting regulations on public corporations. The new European Union General Data Protection Regulation (GDPR) has similar requirements. All kinds of industries – from finance and energy to food service and public works – have similar regulations. Point in Time Analysis (PITA) and Point in Time Recovery (PITR): Let's assume all the records in the accounts table were deleted by “someone”. Even after this deletion of data from this table, you will still have data in the enabled system versioned (ie - history) table, where you can recover your data without much hassle (PITR). Second point is you can then do an analysis for who (id) (if auditing is enabled), and when, this delete happened (PITA). Also, with PITR: Establishing a way of ‘undoing’ a data change on a table’s row without downtime in case a record is accidentally deleted or updated. Therefore, the previous version of the data can be retrieved from the history table and inserted back into the ‘parent’ table. – This helps when someone (or because of some application errors) accidentally deletes data and you want to revert to it or recover it. Explain, it protects from an user doing a fat-finger delete or update, and that also, is a big help. Anomaly Detection or “outlier” detection: An outlier is “an observation (or a set of observations) which appears to be inconsistent with the remainder of that set of data”. System Version/Temporal tables can be a great asset for helping finding or confirming that this indeed happening. For example, a business user come to you and relates that the number of claims spiked for a set of people names that seem to be repeating and for a specific range of claim amounts. After some investigation, not only is this confirmed, but a prediction is setup to determine when and what sort of claim this will happen again. It is suspect for fraud after several data points are confirmed using the system versioning of the data. Further analysis is ensued, and possible fraud is eliminated.
- A system-versioned temporal table is a type of user table designed to keep a full history of data changes and allow easy point in time analysis. This type of temporal table is referred to as a system-versioned temporal table because the period of validity for each row is managed by the system (i.e. database engine). Track Changes: track all “changes”. Changes = updates, inserts, and deletes. Compare Data based on date and time: Can compare data events based off a date and time -- The invisible columns of automatic versioning are ROW_START and ROW_END. They define the time period for which the version of the row was/is valid. Visualize Date/Trends: Able to see changes over time, like price changes, or inventory changes. Auditing: Can audit if a specific user or set of user making approved/non-approved changes to a specific data point. IF end user audit is needed, MariaDB has a different tool for that called: MariaDB Audit Plugin https://mariadb.com/kb/en/library/mariadb-audit-plugin/ https://mariadb.com/kb/en/library/mariadb-audit-plugin-log-settings/ Saves space: If data is stored in-line with base table, data will add up quickly over time. To track changes a trigger is used, and this is overhead expense in terms of CPU.
- This is really a general overview of the commands and options for using Temporal Tables
- 1 - Temporal Tables can be created : 1a - with the create table statement or 1b: After the base table is created NOTE: The CREATE TABLE syntax has been extended to permit creating a system-versioned table. To be system-versioned, according to SQL:2011, a table must have two generated columns, a period, and a special table option clause: CREATE TABLE t( x INT, start_timestamp TIMESTAMP(6) GENERATED ALWAYS AS ROW START, end_timestamp TIMESTAMP(6) GENERATED ALWAYS AS ROW END, PERIOD FOR SYSTEM_TIME(start_timestamp, end_timestamp) ) WITH SYSTEM VERSIONING;
- To query the historical data one uses the clause FOR SYSTEM_TIME directly after the table name; Additional options: AS OF TIMESTAMP - Example: select * from tablea FOR SYSTEM_TIME AS OF TIMESTAMP '2019-03-01 10:00:00'; AS OF is used to see the table as it was at a specific point in time in the past. BETWEEN start AND end will show all rows that were visible at any point between two specified points in time. It works inclusively, a row visible exactly at start or exactly at end will be shown too. FROM start TO end will also show all rows that were visible at any point between two specified points in time, including start. Additionally MariaDB implements a non-standard extension: ALL will show all rows, historical and current. See: https://mariadb.com/kb/en/library/system-versioned-tables/
- Point in-time Recovery
- select table_schema, table_name, table_type from information_schema.tables where table_type in ('SYSTEM VERSIONED');
- Create table w/2 columns add system versioning, then insert 5 rows of data.
- Point in-time Recovery
- Note: IS system versioning still enabled at this point ?
- In bold (c1=78) is the ACTIVE row, and c1=58 (row1) is history versioned.
- Point in-time Recovery
- Alter table system versioning in this example is at the table level, and not at the column level thus the error.
- Point in-time Recovery
- select table_name t_name , partition_name, partition_ordinal_position p_ord_pos, partition_method, partition_description, table_rows, create_time from INFORMATION_SCHEMA.PARTITIONS where table_schema in ('test2') and table_name in ('apple') ORDER BY table_name, partition_name, partition_description;
- Note that there is now 1 row in m01 partition, and 3 active rows in the active table. Describe IF all 3 active rows (currently in “pcur”) were deleted ? They too would move to m01 partition.
- Considerations are things to watch out for or to be aware of including both highlights of good features, and negative ones.
- Temporal columns are generated columns, thus their values are generated by MariaDB and cannot be modified by the user BUT a truncate will remove ALL data from the table including system versioned data. One can also add system versioning with all columns created explicitly or just a few of the columns in a table.
- Partitioning of data helps with performance and data management.
- 1 - UPDATES, do not remove the old row from the system table, but instead creates a NEW row in the table. 2 - Be aware of the truncate command on the table. A truncate will REMOVE (behind the covers this is a drop and recreate) all system data from the system table. NOTE: TRUNCATE TABLE is faster than DELETE, because it drops and re-creates a table (under the covers). 3 - One difference between TIMESTAMP and DATETIME in MariaDB, and that is timezone support. The MariaDB TIMESTAMP data type supports time zones, which the DATETIME data type does not. Thus, why TIMESTAMP data type is used for this feature in temporal tables. 4 - Primary Key needed ? In some of the documentation on the web, non-MariaDB, I ran across some sites that posted a primary key must be in place in order for system versioning to be possible on the table. In my testing, I found this not to be true. Having a primary key or not seemed to work the same. 5 - Versioning on generated columns (like a ltrim or a computation) is not supported. CREATE TABLE table1 (INT NOT NULL, VARCHAR(32), INT AS (a mod 10) VIRTUAL, VARCHAR(5) AS (left(b,5)) PERSISTENT); 6 - Design my own (or DIY) -- Without temporal tables, a trigger would need to implement with some procedural logic to handle some of these features, the overhead of implementing self-made versioned tables would be greater than using the built feature we are discussing. So what is the overhead of temporal tables to MariaDB and the supporting hardware ? It depends. :-)
- References: https://mariadb.com/resources/blog/now-with-system-versioned-tables-announcing-mariadb-server-10-3-second-beta/ https://mariadb.com/kb/en/library/system-versioned-tables/ https://mariadb.com/kb/en/library/system-versioned-tables/ https://mariadb.com/resources/blog/automatic-data-versioning-in-mariadb-server-10-3/ https://dzone.com/articles/automatic-data-versioning-in-mariadb-server-103 https://www.batimes.com/articles/5-business-problems-you-can-solve-using-sql-temporal-tables.html https://mariadb.com/resources/blog/on-databases-temporal-datatypes-and-trains/ http://www.dataarchitect.cloud/system-versioned-tables/ https://mariadb.com/kb/en/library/mariadb-audit-plugin/ https://mariadb.com/kb/en/library/mariadb-audit-plugin-log-settings/