Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

TechEvent 2019: Whats new in biGENiUS; Robert Kranabether - Trivadis


Published on

Whats new in biGENiUS; Robert Kranabether - Trivadis TechEvent 2019

Published in: Technology
  • Be the first to comment

  • Be the first to like this

TechEvent 2019: Whats new in biGENiUS; Robert Kranabether - Trivadis

  1. 1. What‘s new in biGENiUS Robert Kranabether
  2. 2. Agenda • Application • New Data Flow Set Management • New Source Term Mapping Management • Custom Property Types • Target Objects • Migration Framework • Data Vault Generator • Load Control Management Application
  3. 3. Demo Data Flow Sets Source Term Mappings Custom Properties
  4. 4. Target Objects
  5. 5. Overview Generator Architecture Cleansing Core Data Mart Staging Runtime Config LCM Architecture consists of – DWH parts – DWH part groups – DWH componentsFactEntity Entity Fact Dimen- sion Entity Fact Dimen. Stage SQL Stage Flat File Table Loader View Action Table Loader View Action Table Loader View Table Loader View Table Loader View View View View Table Loader View Table Loader View
  6. 6. Applying a generator architecture to a model Model Product Sales SubCategory Category Generator Configuration Trivadis Best Practice generate • CA.Category_Cleansing • CA.Product_Cleansing • CA.Sales_Cleansing • CA.SubCategory_Cleansing • CA.CleansingActionLog • DW.Category • DW.Product • DW.SubCategory • DW.Sales • CA.Category_Level • CA.Product_Level • CA.Sales_Fact • CA.SubCategory_Level • DW.Category_MergeView • DW.Product _MergeView • DW.SubCat _MergeView • DW.Sales _MergeView • DM.Category • DM.Product • DM.SubCategory • DM.Sales • CA.Category_CA1 • CA.Category_Level_Loader • CA.Product_CA1 • CA.Product_Level_Loader • CA.Sales_CA1 • CA.Sales_Fact_Loader • CA.SubCategory_CA1 • CA.SubCategory_Level_Loader • DW.Category_Core_Loader • DW.Product _Core_Loader • DW.SubCat _Core_Loader • DW.Sales _Core_Loader • CA.CA_FinalizeCleasning • CA.CA_PrepareCleasning • DM.DM_Finalize_Core • DM.DM_Prepare_Core • DW.DW_Finalize_Core • DW.DW_Prepare_Core
  7. 7. Target Objects DWH object „Person“ Cleansing Core Mart CRM ERP Source System ResultSource Loader 1Source ResultLoader Action Source ResultLoader Action CA_EN_VI_SQL CA_EN_LO_SQL CA_EN_TA_SQL 1 2 Source Result Loader Source Loader 1 11 22 Staging ResultSource Loader ResultSource Loader 1 1 Usage=Source Type=View SourceLayer=CA Usage=Loader Type=Procedure DirectDependencies=CA_EN_AC3_SQL RelationshipDependencies=DW_EN_LO_SQL Usage=Result Type=Table Usage=Source Type=View Options=VirtualOnly Usage=Result Type=Table Usage=Source Type=View Options=UseSqlCode, BuildEmptySelect Usage=Loader Type=Procedure DataFlowDependencies=SA_ST_LO_SQL, SA_SC_LO_SQL Usage=Result Type=Table Usage=Action UsageType=Cleansing Type=Procedure Status=Activated DirectDependencies=CA_EN_LO_SQL RelationshipDependencies=CA_EN_LO_SQL Usage=Loader Type=Procedure DirectDependencies=DW_EN_LO_SQL SourceLayer=DW Usage=Source Type=View Usage=Loader Type=Procedure Usage=Result Type=Table
  8. 8. Target Objects in EM Viewer foreach(var targetObject in Instance.GetTargetObjects(dwhPartName)) { BuildCleanseView(targetObject); } foreach(var term in targetObject.BusinessKeys.Where(t => !t.IsForeignKey)) { WriteLine(" {0,1}{1,-50}", delimiter, ToSqlName("ca", term.ColumnName)); }
  9. 9. Migration Framework
  10. 10. So what‘s the point here? biGENiUS offers you a wide range of customization and extensibility Meanwhile, biGENiUS is constantly being developed over time. Project start Development - customized rules - templates - settings Release of new project version Increasing complexity over time Development - Adoption of standardizable customizations - Introduction of new features - Bug Fixes Release of biGENiUS/generator version biGENiUS development
  11. 11. Why do you need this? • You started some time ago using biGENiUS • You want to… • get rid of customizations that are now part of the standard • use the latest generator features e.g. Data Vault 2.0, … • benefit from the latest UI version e.g. improved data flow mappings, LCM Application
  12. 12. Which challenges do we have? • Migration of UI and Meta DB is relativly easy, but that ain't true for the generator configuration! • We have to ensure… • to migrate all relevant nuts and bolts • integrity of your modeled data • integrity of your already loaded data and operation of your production data warehouse
  13. 13. What do we provide? • Specialized meta database that supports your migration project • Extensive reporting for changes between legacy and latest meta db version • Identification of changes within customized and stock templates • Automated way to import enties, facts, custom properties and settings
  14. 14. What does the upgrade process look like? 1. Setup migration environment •Import legacy meta db and create a new (up to date) one •Create the DWH in the migration environment 2. Analyze •Generate migration report •Identify changes to the meta db and derive required manual adjustments •Identify changes to the templates and derive required adjustments •Generate migration script 3. Migration •Manual adjustments to new meta db •Execute the migration scripts to import data 4. Testing •Generate & deploy migrated DWH •Run initial & incremental loads and compare legacy and new DWH •Delta deployment to migrate legacy DWH
  15. 15. Data Vault Generator
  16. 16. Data Vault Dani Schnider explains: ▪ Areas: ▪ Raw Vault ▪ Business Vault ▪ Main Objects ▪ Hubs ▪ Links ▪ Satellites
  17. 17. BeerVault Architecture
  18. 18. Demo Data Vault Generator
  19. 19. Demo Load Control Management Application