Implementing an Incremental ETL Process 
Ram Kedem
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Slowly Changing Dimension Transformation 
•The Slowly Changing Dimension transformation coordinates the updating and inserting of records in data warehouse dimension tables. 
•The Slowly Changing Dimension Wizard only supports connections to SQL Server. 
•The Slowly Changing Dimension transformation supports four types of changes: fixed attribute, changing attribute, historical attribute, and inferred member.
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute 
•Fixed attribute changes indicate the column value must not change. The Slowly Changing Dimension transformation detects changes and can direct the rows with changes to an output named Fixed Attribute Output.
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Inferred Dimension Members 
•Use the Inferred Dimension Members dialog box to specify options for using inferred members. Inferred members exist when a fact table references dimension members that are not yet loaded. 
•When data for the inferred member is loaded, you can update the existing record rather than create a new one. 
•If you choose to enable inferred members, you must select one of the two options that follow 
•All columns with a change type are null -Specify whether to enter null values in all columns with a change type in the new inferred member record. 
•Use a Boolean column to indicate whether the current record is an inferred member -Specify whether to use an existing Boolean column to indicate whether the current record is an inferred member.
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Fixed Attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute 
•Changing attribute changes overwrite existing records. 
•The Slowly Changing Dimension transformation directs these rows to an output named Changing Attributes Updates Output.
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Changing attribute
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute 
•Historical attribute changes create new records instead of updating existing ones. The only change that is permitted in an existing record is an update to a column that indicates whether the record is current or expired. 
•The Slowly Changing Dimension transformation directs these rows to two outputs: Historical Attribute Inserts Output and New Output
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Current Row
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date
Copyright 2014 © Ram Kedem. All rights reserved. Not to be reproduced without written consent 
ramkedem.com 
Historical Attribute with Start / End Date

SSIS Incremental ETL process

  • 1.
    Implementing an IncrementalETL Process Ram Kedem
  • 2.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Slowly Changing Dimension Transformation •The Slowly Changing Dimension transformation coordinates the updating and inserting of records in data warehouse dimension tables. •The Slowly Changing Dimension Wizard only supports connections to SQL Server. •The Slowly Changing Dimension transformation supports four types of changes: fixed attribute, changing attribute, historical attribute, and inferred member.
  • 3.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute •Fixed attribute changes indicate the column value must not change. The Slowly Changing Dimension transformation detects changes and can direct the rows with changes to an output named Fixed Attribute Output.
  • 4.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 5.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 6.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 7.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 8.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 9.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 10.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 11.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 12.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Inferred Dimension Members •Use the Inferred Dimension Members dialog box to specify options for using inferred members. Inferred members exist when a fact table references dimension members that are not yet loaded. •When data for the inferred member is loaded, you can update the existing record rather than create a new one. •If you choose to enable inferred members, you must select one of the two options that follow •All columns with a change type are null -Specify whether to enter null values in all columns with a change type in the new inferred member record. •Use a Boolean column to indicate whether the current record is an inferred member -Specify whether to use an existing Boolean column to indicate whether the current record is an inferred member.
  • 13.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 14.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 15.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Fixed Attribute
  • 16.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute •Changing attribute changes overwrite existing records. •The Slowly Changing Dimension transformation directs these rows to an output named Changing Attributes Updates Output.
  • 17.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 18.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 19.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 20.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 21.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 22.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 23.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 24.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 25.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Changing attribute
  • 26.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute •Historical attribute changes create new records instead of updating existing ones. The only change that is permitted in an existing record is an update to a column that indicates whether the record is current or expired. •The Slowly Changing Dimension transformation directs these rows to two outputs: Historical Attribute Inserts Output and New Output
  • 27.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 28.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 29.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 30.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 31.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 32.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 33.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 34.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 35.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 36.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 37.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 38.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Current Row
  • 39.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 40.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 41.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 42.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 43.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 44.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 45.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 46.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 47.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 48.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 49.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date
  • 50.
    Copyright 2014 ©Ram Kedem. All rights reserved. Not to be reproduced without written consent ramkedem.com Historical Attribute with Start / End Date