SlideShare a Scribd company logo
AN AGILE MODELING TECHNIQUE USING THE SIXTH NORMAL FORM
     FOR STRUCTURALLY AND TEMPORALLY EVOLVING DATA
                      Lars Rönnbäck
                          [ER09]
“You can never step
     into the same river
     twice.”




2
DW     MDM    EDW    DW     TDWI   KTH/SU    CC    WWW   DW     ER09
2003   2004   2005   2006   2007            2008         2009
Entity
            Relationship
            Modeling



                           Sixth
                           Normal
                           Form




 Temporal
Databases
Customer                                      Gender
Store                                         CustomerClass
Purchase                                      HouseholdOwner
Item                                          VisitingFrequencyInterval
PriceList
Inventory




                                                 Customer_Address
                                                 Customer_Household
CustomerDateOfBirth                              Card_Customer
CustomerNumber
CustomerName          Attributes and ties
CustomerGender        come in four flavors,
                      historized or static
                      combined with knotted
select top 5 * from CU_Customer                 select top 5 * from GEN_Gender

CU_ID                                            GEN_ID           GEN_Gender
1                                                1                Male
2                                                2                Female
3
4
5




select top 5 * from CUDOB_CustomerDateOfBirth   select top 5 * from CUHH_Customer_Household

 CU_ID     CUDOB_CustomerDateOfBirth            CU_ID     HH_ID    HOW_ID    CUHH_FromDate
 1         1905-03-02                           1         1        1         2009-02-13
 2         1905-07-02                           1         895      0         2009-09-21
 3         1908-09-14                           2         2        1         2006-10-17
 4         1910-02-03                           3         3        1         2002-08-20
 5         1912-04-01                           4         4        1         1993-08-29
1   2                                  3

4                         5




        All previous versions of the
        schema are present and
        were never modified,
        allowing extensions to be
        made ”online”.
Joins all attributes and finds the
                                 attribute row with the latest
                                 FromDate earlier or on the given
                                 timepoint if historized




Joins all attributes and finds
the attribute row with the
latest FromDate if historized
YearOfBirth   Customers
                                      1950          20615
                                      1951          19282
                                      1952          20003
                                      1953          19782
                                      1954          19249




The query execution plan shows
that only two tables are touched
(the anchor and the selected
attribute) despite of the fact that
several others are joined into the
view we are using.
The query optimizer will remove table T from the            Support
execution plan of a query if the following two conditions   Microsoft SQL Server
are fulfilled:                                              Oracle
                                                            IBM DB2
 i.    no column from T is explicitly selected              PostgreSQL
 ii.   the number of rows in the returned data set is not   MariaDB (fork of MySQL)
       affected by the join with T                          Teradata (partial)
The scripts for setting up
                                                         the database, including all
                                                         views and functions, can be
                                                         automatically generated
                                                         from a compact XML
                                                         description.




                           Pseudo loading code given ”wide” source data:
                           o Check if there already is an associated surrogate key for
                              each natural key
                                o For unknown individuals
                                     o Create and associate surrogate keys
                                     o Directly insert data into all relevant tables
                                     (most tables including the anchor)
Data loading templates          o For known individuals
can be made in which                 o If this is a delta file, directly insert data into all
only the names of the                    relevant tables
                                     o If this is not a delta file, check if the value in
tables and the join with
                                         the source differs from the latest value in the
the natural key have to                  destination and insert if the data is new
be changed.                          (few tables excluding the anchor)
Ease of Modeling                           Simplified Maintenance
Simple concepts and notation               Ease of temporal querying
Historization by design                    Absence of null values
Iterative and incremental development      Reusability and automation
Reduced translation logic                  Asynchronous arrival of data

                   High Performance
                   High run-time performance
                   Efficient storage
                   Parallelized physical media access




www.anchormodeling.com

More Related Content

Similar to Anchor Modeling

Anchor Modeling ER09 Presentation
Anchor Modeling ER09 PresentationAnchor Modeling ER09 Presentation
Anchor Modeling ER09 Presentation
Roenbaeck
 
SQL Server 2008 Overview
SQL Server 2008 OverviewSQL Server 2008 Overview
SQL Server 2008 Overview
David Chou
 
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
Michael M David
 
SQLFire Webinar
SQLFire WebinarSQLFire Webinar
SQLFire Webinar
Carter Shanklin
 
D3 meetup (Backbone and D3)
D3 meetup (Backbone and D3)D3 meetup (Backbone and D3)
D3 meetup (Backbone and D3)
Alpine Data
 
Introducing U-SQL (SQLPASS 2016)
Introducing U-SQL (SQLPASS 2016)Introducing U-SQL (SQLPASS 2016)
Introducing U-SQL (SQLPASS 2016)
Michael Rys
 
Ef code first
Ef code firstEf code first
Ef code first
dotNETUserGroupDnipro
 
Meetup cassandra for_java_cql
Meetup cassandra for_java_cqlMeetup cassandra for_java_cql
Meetup cassandra for_java_cql
zznate
 
Cassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series ModelingCassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series Modeling
Vassilis Bekiaris
 
Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)
David McCarter
 
Sql tutorial
Sql tutorialSql tutorial
Sql tutorial
Axmed Mo.
 
Sql tutorial
Sql tutorialSql tutorial
Sql tutorial
togather111
 
EclipseCon 2021 NoSQL Endgame
EclipseCon 2021 NoSQL EndgameEclipseCon 2021 NoSQL Endgame
EclipseCon 2021 NoSQL Endgame
Thodoris Bais
 
cPanel now supports MySQL 8.0 - My Top Seven Features
cPanel now supports MySQL 8.0 - My Top Seven FeaturescPanel now supports MySQL 8.0 - My Top Seven Features
cPanel now supports MySQL 8.0 - My Top Seven Features
Dave Stokes
 
Ef code first
Ef code firstEf code first
Ef code first
ZealousysDev
 
Data Access Tech Ed India
Data Access   Tech Ed IndiaData Access   Tech Ed India
Data Access Tech Ed India
rsnarayanan
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Citus Data
 
NOSQL and Cassandra
NOSQL and CassandraNOSQL and Cassandra
NOSQL and Cassandra
rantav
 
Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)
David McCarter
 
U-SQL - Azure Data Lake Analytics for Developers
U-SQL - Azure Data Lake Analytics for DevelopersU-SQL - Azure Data Lake Analytics for Developers
U-SQL - Azure Data Lake Analytics for Developers
Michael Rys
 

Similar to Anchor Modeling (20)

Anchor Modeling ER09 Presentation
Anchor Modeling ER09 PresentationAnchor Modeling ER09 Presentation
Anchor Modeling ER09 Presentation
 
SQL Server 2008 Overview
SQL Server 2008 OverviewSQL Server 2008 Overview
SQL Server 2008 Overview
 
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
ANSI SQL Transparent Multipath Hierarchical Structured Data Processing for Re...
 
SQLFire Webinar
SQLFire WebinarSQLFire Webinar
SQLFire Webinar
 
D3 meetup (Backbone and D3)
D3 meetup (Backbone and D3)D3 meetup (Backbone and D3)
D3 meetup (Backbone and D3)
 
Introducing U-SQL (SQLPASS 2016)
Introducing U-SQL (SQLPASS 2016)Introducing U-SQL (SQLPASS 2016)
Introducing U-SQL (SQLPASS 2016)
 
Ef code first
Ef code firstEf code first
Ef code first
 
Meetup cassandra for_java_cql
Meetup cassandra for_java_cqlMeetup cassandra for_java_cql
Meetup cassandra for_java_cql
 
Cassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series ModelingCassandra Basics, Counters and Time Series Modeling
Cassandra Basics, Counters and Time Series Modeling
 
Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)
 
Sql tutorial
Sql tutorialSql tutorial
Sql tutorial
 
Sql tutorial
Sql tutorialSql tutorial
Sql tutorial
 
EclipseCon 2021 NoSQL Endgame
EclipseCon 2021 NoSQL EndgameEclipseCon 2021 NoSQL Endgame
EclipseCon 2021 NoSQL Endgame
 
cPanel now supports MySQL 8.0 - My Top Seven Features
cPanel now supports MySQL 8.0 - My Top Seven FeaturescPanel now supports MySQL 8.0 - My Top Seven Features
cPanel now supports MySQL 8.0 - My Top Seven Features
 
Ef code first
Ef code firstEf code first
Ef code first
 
Data Access Tech Ed India
Data Access   Tech Ed IndiaData Access   Tech Ed India
Data Access Tech Ed India
 
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
Data Modeling, Normalization, and De-Normalization | PostgresOpen 2019 | Dimi...
 
NOSQL and Cassandra
NOSQL and CassandraNOSQL and Cassandra
NOSQL and Cassandra
 
Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)Building nTier Applications with Entity Framework Services (Part 1)
Building nTier Applications with Entity Framework Services (Part 1)
 
U-SQL - Azure Data Lake Analytics for Developers
U-SQL - Azure Data Lake Analytics for DevelopersU-SQL - Azure Data Lake Analytics for Developers
U-SQL - Azure Data Lake Analytics for Developers
 

Recently uploaded

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
Dinusha Kumarasiri
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
Postman
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Hiike
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 

Recently uploaded (20)

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Azure API Management to expose backend services securely
Azure API Management to expose backend services securelyAzure API Management to expose backend services securely
Azure API Management to expose backend services securely
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
WeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation TechniquesWeTestAthens: Postman's AI & Automation Techniques
WeTestAthens: Postman's AI & Automation Techniques
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - HiikeSystem Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 

Anchor Modeling

  • 1. AN AGILE MODELING TECHNIQUE USING THE SIXTH NORMAL FORM FOR STRUCTURALLY AND TEMPORALLY EVOLVING DATA Lars Rönnbäck [ER09]
  • 2. “You can never step into the same river twice.” 2
  • 3. DW MDM EDW DW TDWI KTH/SU CC WWW DW ER09 2003 2004 2005 2006 2007 2008 2009
  • 4. Entity Relationship Modeling Sixth Normal Form Temporal Databases
  • 5. Customer Gender Store CustomerClass Purchase HouseholdOwner Item VisitingFrequencyInterval PriceList Inventory Customer_Address Customer_Household CustomerDateOfBirth Card_Customer CustomerNumber CustomerName Attributes and ties CustomerGender come in four flavors, historized or static combined with knotted
  • 6. select top 5 * from CU_Customer select top 5 * from GEN_Gender CU_ID GEN_ID GEN_Gender 1 1 Male 2 2 Female 3 4 5 select top 5 * from CUDOB_CustomerDateOfBirth select top 5 * from CUHH_Customer_Household CU_ID CUDOB_CustomerDateOfBirth CU_ID HH_ID HOW_ID CUHH_FromDate 1 1905-03-02 1 1 1 2009-02-13 2 1905-07-02 1 895 0 2009-09-21 3 1908-09-14 2 2 1 2006-10-17 4 1910-02-03 3 3 1 2002-08-20 5 1912-04-01 4 4 1 1993-08-29
  • 7. 1 2 3 4 5 All previous versions of the schema are present and were never modified, allowing extensions to be made ”online”.
  • 8. Joins all attributes and finds the attribute row with the latest FromDate earlier or on the given timepoint if historized Joins all attributes and finds the attribute row with the latest FromDate if historized
  • 9. YearOfBirth Customers 1950 20615 1951 19282 1952 20003 1953 19782 1954 19249 The query execution plan shows that only two tables are touched (the anchor and the selected attribute) despite of the fact that several others are joined into the view we are using.
  • 10. The query optimizer will remove table T from the Support execution plan of a query if the following two conditions Microsoft SQL Server are fulfilled: Oracle IBM DB2 i. no column from T is explicitly selected PostgreSQL ii. the number of rows in the returned data set is not MariaDB (fork of MySQL) affected by the join with T Teradata (partial)
  • 11. The scripts for setting up the database, including all views and functions, can be automatically generated from a compact XML description. Pseudo loading code given ”wide” source data: o Check if there already is an associated surrogate key for each natural key o For unknown individuals o Create and associate surrogate keys o Directly insert data into all relevant tables (most tables including the anchor) Data loading templates o For known individuals can be made in which o If this is a delta file, directly insert data into all only the names of the relevant tables o If this is not a delta file, check if the value in tables and the join with the source differs from the latest value in the the natural key have to destination and insert if the data is new be changed. (few tables excluding the anchor)
  • 12. Ease of Modeling Simplified Maintenance Simple concepts and notation Ease of temporal querying Historization by design Absence of null values Iterative and incremental development Reusability and automation Reduced translation logic Asynchronous arrival of data High Performance High run-time performance Efficient storage Parallelized physical media access www.anchormodeling.com