CDMCentennium Datawarehouse MethodologyStructure – Model – Generate – Knowledge Partnership1Erik FransenAntoine StelmaOctober 6, 2011
Agenda2Centennium BI expertisehuisCDMStructure		Reference ArchitectureModel			Data VaultGenerate		Template Based DevelopmentKnowledge Partnership		Knowledge transfer, coachingCDM process
Centennium BI expertisehuisIndependent We provide our customers with the right skills, at the right time and the right wayConsultancy, ad-interim support, project implementation and training services, but primarily acts as a BI/DWH knowledge partner for its clientsWe take or share responsibility for the execution and management of BI and DWH projects and provide second or third line support for existing implementationsWe offer an extensive portfolio of courses and training services We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environmentswww.centennium.nl3
Centennium Datawarehouse MethodologyA methodology, including DWH-toolsBuild or migrate datawarehouses fast, with high qualityand low costAutomatic generation of datawarehouse based on descriptivemetadataCDM includes:Template Based DevelopmentBest practicesQuality control mechanismKnowledge partnershipBest Practices: Data Vault, Kimball, Reference ArchitectureQualitycontrol mechanism: extensive checklists anddocumentationKnowledge transfer by training, certificationandlearning on-the-job4
5Centennium Datawarehouse MethodologyKnowledge PartnershipModellingStructuringGenerating
Knowledge partnershipGoal:  Self-supporting, CDM-awarecustomersTraining options	BI and DWH fundamentalsCDM designer / TBD developer training	Data VaultfundamentalsData Vaultcertification(Genesee Academy)	DimensionalmodellingMore info on www.bi-opleidingen.nlOn-the-job coaching, learningbydoingCentennium supports it’scustomersby(management) consultancy, assessments, projects, training andsourcing6
Reference Architecture: Structuring7
Template Based Development:GeneratingTBD generatesall ETL and objects forRegistrationlayerPresentation/PublicationlayerRepositoryand scripts are free of chargeCreating the staginglayer is not part of TBD but canbeautomated per customer8
TBD databases9Repository (REP)Central Data Warehouse (CDW)Staging (STA)Publication (PUB)
TBD repository10Repository (REP)Central Data Warehouse (CDW)Staging (STA)Publication (PUB)
Functionality TBD repositorylogtables (1)Functionality: Registration of all ETL processesInsight in data logisticsHistoric profile of loadingprocessesGoal:TraceabilityMaintenance: localisationandidentification of all data/transactions (wrong andcorrect)Linkage to source systems11REPPUBSTACDW
Functionality TBD repositorytemplate tablesFunctionality: Central storage of template scripts forgeneratingtablesand ETL proceduresEasy promotion to other environments (OTAP)TransparantGoal: Version management of templatesObject (re-)creationand maintenance12REPPUBSTACDW
Functionality TBD repositorymeta data tablesFunctionality: DefinesregistrationandpublicationlayerContains object and ETL definitionsContains data logisticsGoal:Generating:Registrationlayer(Data Vaulttables)Publicationlayer (Star schemes) ETL processes via ‘stored procedures’Object creationand maintenance13REPPUBSTACDW
Functionality TBD repositorymeta data tablesTable ‘Repos_Table’: Contains object namesforStagingRegistrationPublicationTable is filledbyanapplication or Excel sheetTable ‘Repos_Mapping’: Contains mapping of staging, registration and presentationTable is filledbyanapplication or Excel sheet14REPPUBSTACDW
Registrationlayer15Repository (REP)Central Data Warehouse (CDW)Staging (STA)Publication (PUB)
RegistrationlayerScript ‘CDW 01a table hub’Generatesall hub tables in the CDW, as defined in repositoryEvery hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedHubs generated according to Data Vault Standards!16REPPUBSTACDW
RegistrationlayerScript ‘CDW 01a tabel hub’Generatesall hub tables in the CDW, as defined in repositoryEvery hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedHubs generated according to Data Vault Standards!17REPPUBSTACDW
RegistrationlayerScript ‘CDW 02a table sat’Generatesallsatellitetables in the CDW, as defined in repositoryEverysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedSatsgenerated according to Data Vault Standards!18REPPUBSTACDW
RegistrationlayerScript ‘CDW 02a tabel sat’Generatesallsatellitetables in the CDW, as defined in repositoryEverysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedSatsgenerated according to Data Vault Standards!19REPPUBSTACDW
RegistrationlayerScript‘CDW 03a table link’Generatesalllink tables in the CDW, as defined in repositoryEverylink is connected to two or more hubsVariables fromrepository are appliedOnce the links are generated, the repository is updatedLinks generated according to Data Vault Standards!20REPPUBSTACDW
RegistrationlayerScript‘CDW 03a table link’Generatesalllink tables in the CDW, as defined in repositoryEverylink is connected to two or more hubsVariables fromrepository are appliedOnce the links are generated, the repository is updatedLinks generated according to Data Vault Standards!21REPPUBSTACDW
RegistrationlayerScript‘CDW 04a table link satellite’Generatesalllink satellitetables in the CDW, as defined in repositoryVariables fromrepository are appliedOnce the link satellites are generated, the repository is updatedLink satellites are generated according to Data Vault Standards!22REPPUBSTACDW
RegistrationlayerScript‘CDW 04a table link satellite’Generatesalllink satellitetables in the CDW, as defined in repositoryVariables fromrepository are appliedOnce the link satellites are generated, the repository is updatedLink satellites are generated according to Data Vault Standards!23REPPUBSTACDW
Publicationlayer24Repository (REP)Central Data Warehouse (CDW)Staging (STA)Publication (PUB)
PublicationlayerdimensionsandfactsFedonlyanddirectlyfrom CDWConforms to Kimball’sstar scheme standardsOr anyother format Business rulescanbeappliedCurrentlyusing viewsBusiness rule editor in next releaseType SCD I, II, etc.Conformeddimensionswhenneeded25REPPUBSTACDW
CDM: processIncremental approachTimeboxes of 4-6 weeks26
CDM: typical increment90% Centennium 30% Customer70% Customer40% Centennium100% Customer10% Centennium100% CentenniumPhase 1Phase 2Phase 3Phase 4Increment31-12-20111-8-2011Supporting customerTraining and Coaching on-the-jobCDM-trainingData Vault CertificationTypical increment ranges from 2 to 6 months
Centennium role changes from LEAD to FOLLOW
Customer is fully CDM-aware at the end of the increment
Centennium continuessupportingcustomersthroughknowledge partnershipQ & A28
About centennium29
Centennium BI expertisehuis houses all the experts under one roof, hereby offering all knowledge and expertise to address the complex business intelligence issues facing our clients todayFacts and figures:Founded: 1998
45+ business intelligence consultantsCore values:Human Capital

#Centennium Presentation Data Vault Automation seminar #dvauto Oct 6th 2011

  • 1.
    CDMCentennium Datawarehouse MethodologyStructure– Model – Generate – Knowledge Partnership1Erik FransenAntoine StelmaOctober 6, 2011
  • 2.
    Agenda2Centennium BI expertisehuisCDMStructure ReferenceArchitectureModel Data VaultGenerate Template Based DevelopmentKnowledge Partnership Knowledge transfer, coachingCDM process
  • 3.
    Centennium BI expertisehuisIndependentWe provide our customers with the right skills, at the right time and the right wayConsultancy, ad-interim support, project implementation and training services, but primarily acts as a BI/DWH knowledge partner for its clientsWe take or share responsibility for the execution and management of BI and DWH projects and provide second or third line support for existing implementationsWe offer an extensive portfolio of courses and training services We provide our customers with the knowledge and practical insights required to be(come) self-sufficient in maintaining and expanding their BI-environmentswww.centennium.nl3
  • 4.
    Centennium Datawarehouse MethodologyAmethodology, including DWH-toolsBuild or migrate datawarehouses fast, with high qualityand low costAutomatic generation of datawarehouse based on descriptivemetadataCDM includes:Template Based DevelopmentBest practicesQuality control mechanismKnowledge partnershipBest Practices: Data Vault, Kimball, Reference ArchitectureQualitycontrol mechanism: extensive checklists anddocumentationKnowledge transfer by training, certificationandlearning on-the-job4
  • 5.
    5Centennium Datawarehouse MethodologyKnowledgePartnershipModellingStructuringGenerating
  • 6.
    Knowledge partnershipGoal: Self-supporting, CDM-awarecustomersTraining options BI and DWH fundamentalsCDM designer / TBD developer training Data VaultfundamentalsData Vaultcertification(Genesee Academy) DimensionalmodellingMore info on www.bi-opleidingen.nlOn-the-job coaching, learningbydoingCentennium supports it’scustomersby(management) consultancy, assessments, projects, training andsourcing6
  • 7.
  • 8.
    Template Based Development:GeneratingTBDgeneratesall ETL and objects forRegistrationlayerPresentation/PublicationlayerRepositoryand scripts are free of chargeCreating the staginglayer is not part of TBD but canbeautomated per customer8
  • 9.
    TBD databases9Repository (REP)CentralData Warehouse (CDW)Staging (STA)Publication (PUB)
  • 10.
    TBD repository10Repository (REP)CentralData Warehouse (CDW)Staging (STA)Publication (PUB)
  • 11.
    Functionality TBD repositorylogtables(1)Functionality: Registration of all ETL processesInsight in data logisticsHistoric profile of loadingprocessesGoal:TraceabilityMaintenance: localisationandidentification of all data/transactions (wrong andcorrect)Linkage to source systems11REPPUBSTACDW
  • 12.
    Functionality TBD repositorytemplatetablesFunctionality: Central storage of template scripts forgeneratingtablesand ETL proceduresEasy promotion to other environments (OTAP)TransparantGoal: Version management of templatesObject (re-)creationand maintenance12REPPUBSTACDW
  • 13.
    Functionality TBD repositorymetadata tablesFunctionality: DefinesregistrationandpublicationlayerContains object and ETL definitionsContains data logisticsGoal:Generating:Registrationlayer(Data Vaulttables)Publicationlayer (Star schemes) ETL processes via ‘stored procedures’Object creationand maintenance13REPPUBSTACDW
  • 14.
    Functionality TBD repositorymetadata tablesTable ‘Repos_Table’: Contains object namesforStagingRegistrationPublicationTable is filledbyanapplication or Excel sheetTable ‘Repos_Mapping’: Contains mapping of staging, registration and presentationTable is filledbyanapplication or Excel sheet14REPPUBSTACDW
  • 15.
    Registrationlayer15Repository (REP)Central DataWarehouse (CDW)Staging (STA)Publication (PUB)
  • 16.
    RegistrationlayerScript ‘CDW 01atable hub’Generatesall hub tables in the CDW, as defined in repositoryEvery hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedHubs generated according to Data Vault Standards!16REPPUBSTACDW
  • 17.
    RegistrationlayerScript ‘CDW 01atabel hub’Generatesall hub tables in the CDW, as defined in repositoryEvery hub contains the columns: id (primarykey), <afkorting>_key, meta_laad_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedHubs generated according to Data Vault Standards!17REPPUBSTACDW
  • 18.
    RegistrationlayerScript ‘CDW 02atable sat’Generatesallsatellitetables in the CDW, as defined in repositoryEverysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedSatsgenerated according to Data Vault Standards!18REPPUBSTACDW
  • 19.
    RegistrationlayerScript ‘CDW 02atabel sat’Generatesallsatellitetables in the CDW, as defined in repositoryEverysatellitecontains the the columns: h_id (foreignkey), meta_laad_dts, meta_laad_eind_dts, meta_bron en meta_audit_idVariables fromrepository are appliedOnce the hubs are generated, the repository is updatedSatsgenerated according to Data Vault Standards!19REPPUBSTACDW
  • 20.
    RegistrationlayerScript‘CDW 03a tablelink’Generatesalllink tables in the CDW, as defined in repositoryEverylink is connected to two or more hubsVariables fromrepository are appliedOnce the links are generated, the repository is updatedLinks generated according to Data Vault Standards!20REPPUBSTACDW
  • 21.
    RegistrationlayerScript‘CDW 03a tablelink’Generatesalllink tables in the CDW, as defined in repositoryEverylink is connected to two or more hubsVariables fromrepository are appliedOnce the links are generated, the repository is updatedLinks generated according to Data Vault Standards!21REPPUBSTACDW
  • 22.
    RegistrationlayerScript‘CDW 04a tablelink satellite’Generatesalllink satellitetables in the CDW, as defined in repositoryVariables fromrepository are appliedOnce the link satellites are generated, the repository is updatedLink satellites are generated according to Data Vault Standards!22REPPUBSTACDW
  • 23.
    RegistrationlayerScript‘CDW 04a tablelink satellite’Generatesalllink satellitetables in the CDW, as defined in repositoryVariables fromrepository are appliedOnce the link satellites are generated, the repository is updatedLink satellites are generated according to Data Vault Standards!23REPPUBSTACDW
  • 24.
    Publicationlayer24Repository (REP)Central DataWarehouse (CDW)Staging (STA)Publication (PUB)
  • 25.
    PublicationlayerdimensionsandfactsFedonlyanddirectlyfrom CDWConforms toKimball’sstar scheme standardsOr anyother format Business rulescanbeappliedCurrentlyusing viewsBusiness rule editor in next releaseType SCD I, II, etc.Conformeddimensionswhenneeded25REPPUBSTACDW
  • 26.
  • 27.
    CDM: typical increment90%Centennium 30% Customer70% Customer40% Centennium100% Customer10% Centennium100% CentenniumPhase 1Phase 2Phase 3Phase 4Increment31-12-20111-8-2011Supporting customerTraining and Coaching on-the-jobCDM-trainingData Vault CertificationTypical increment ranges from 2 to 6 months
  • 28.
    Centennium role changesfrom LEAD to FOLLOW
  • 29.
    Customer is fullyCDM-aware at the end of the increment
  • 30.
  • 31.
  • 32.
    Centennium BI expertisehuishouses all the experts under one roof, hereby offering all knowledge and expertise to address the complex business intelligence issues facing our clients todayFacts and figures:Founded: 1998
  • 33.
    45+ business intelligenceconsultantsCore values:Human Capital
  • 34.
  • 35.
  • 36.
  • 37.
    Vision based on“effective BI”Services overview:Consultancy
  • 38.
  • 39.
  • 40.
    EducationSome of ourclients: Woonbron, Albron, NZa, CAK, OBR, Vopak, several Dutch Municipalities, Aegon, Nutreco, TNO,Genzyme, Tata Steel, KPN, DELTA, IKEA, Accell, TomTom, KAS BANK, LeasePlan, Brabant Water
  • 41.
    Certification and IndustryRecognitionAll Centennium consultants are Certified BI Professionals (CBIP) awarded by TDWI. CBIP, backed by TDWI's reputation as the leader in BI education and research, makes a strong statement that we are among the leaders in the industry—up to date with BI technologies, knowledgeable about best practices and state-of-the-art solutions, and informed of emerging trendsAll Centennium Data Vault specialists and architects are certified on Data Vault modeling by Dan Linstedt, Data Warehouse industry thought leader and founder of the Data Vault modeling standard.Centennium is the business partner of Genesee Academy in organizing the Data Vault Modeling & Certification – Europe Course. We are accredited to perform the Genesee Academy Data Vault trainings.Centennium’s customer Nutreco was awarded the world’s first Certified Data Vault Data Model in 2009 for their corporate financial and procurement data warehouse model.
  • 42.
  • 43.
    Centennium BI expertisehuisLangeVoorhout 432514 EC 's-Gravenhage Telefoon 070 31 20 370Fax 070 31 20 371URL www.centennium.nlwww.bi-opleidingen.nl www.data-vault.nl