SlideShare a Scribd company logo
1 of 11
Capture Change and Apply it With
Change Data Capture & SSIS
Steve Wake
Sr. Business Intelligence Consultant
President, Denver SQL Server User Group
MAKING BUSINESS INTELLIGENT
www.pragmaticworks.com
What is Change Data Capture (CDC)?
• CDC first introduced in SQL Server 2008
• First available only in the Database Engine
• Must be enabled for the database(s) and
table(s) you want to track changes on
• SQL Server 2012 Integration Services (SSIS)
added components for CDC
What is Change Data Capture (CDC)?
Microsoft MSDN description:
Change data capture is designed to capture insert, update, and delete
activity applied to SQL Server tables, and to make the details of the
changes available in an easily consumed relational format. The change
tables used by change data capture contain columns that mirror the
column structure of a tracked source table, along with the metadata
needed to understand the changes that have occurred.
Change data capture is available only on the Enterprise, Developer, and
Evaluation editions of SQL Server.
Source: MSDN - http://msdn.microsoft.com/en-us/library/bb522489(v=sql.105).aspx
Why use CDC?
• Loading a Data Warehouse
– Changes from Source Only
– Near Real-Time
• Maintaining a Audit DB
• Maintaining a Change Log
CDC Key Concepts
• sys.sp_cdc_enable_db/sys.sp_cdc_disable_db
• sys.sp_cdc_enable_table/sys.sp_cdc_disable_table
• cdc Database Schema
• lsn = Log Sequence Number
• _$operation - 1=Delete, 2=Insert, 3=Update (before
image), 4=Update (after image)
• CDC SQL Agent Jobs (capture & cleanup)
CDC Limitations
Source: MSDN - http://msdn.microsoft.com/en-us/library/cc645593(v=sql.105).aspx
Type of Column Changes Captured in Change Tables Limitations
Sparse Columns Yes Does not support capturing changes when
using a columnset.
Computed Columns No Changes to computed columns are not
tracked. The column will appear in the
change table with the appropriate type,
but will have a value of NULL.
XML Yes Changes to individual XML elements are
not tracked.
Timestamp Yes The data type in the change table is
converted to binary.
BLOB data types Yes The previous image of the BLOB column is
stored only if the column itself is changed.
Performance Considerations
• Use different filegroups for CDC tables
• @supports_net_changes (non-clustered index added
to cdc table)
• Use @captured_column_list with
sys.sp_cdc_enable_table to limit columns tracked
• FMI – SQL CAT whitepaper:
http://msdn.microsoft.com/en-
us/library/dd266396(v=sql.100).aspx
SSIS 2012
• CDC Control Task
– manage LSN’s, handles errors and recovery
• CDC Source
– read CDC tables/metadata
• CDC Splitter
– splits based on _$operation
• Still requires Database/Table to be setup for CDC
Demo
• Setup CDC on database and table
• Managing CDC in SSMS
• Using SSIS 2012 Packages
Summary
• CDC added to SQL Server 2008
• Must be enabled manually, not on by default
• SSIS 2012 added support to more easily
handle CDC in SSIS packages
• CDC is a built-in way to track changes to SQL
databases/tables
Services
Speed development through training, and
rapid development services from
Pragmatic Works.
Products
BI products to convert to a Microsoft BI
platform and simplify development on the
platform.
Foundation
Helping those who do not have the means
to get into information technology and to
achieve their dreams.
See you soon…
Sales : sales@pragmaticworks.com
My Email: swake@pragmaticworks.com
My Blog: http://blog.wakebi.com
My Twitter: @stevewake
Office Number: 904-638-3805

More Related Content

What's hot

SQL Server Reporting Services 2008
SQL Server Reporting Services 2008SQL Server Reporting Services 2008
SQL Server Reporting Services 2008VishalJharwade
 
Ssis sql ssrs_ssas_sp_mdx_hb_li
Ssis sql ssrs_ssas_sp_mdx_hb_liSsis sql ssrs_ssas_sp_mdx_hb_li
Ssis sql ssrs_ssas_sp_mdx_hb_liHong-Bing Li
 
MSBI-SQL Server Reporting Services
MSBI-SQL Server Reporting ServicesMSBI-SQL Server Reporting Services
MSBI-SQL Server Reporting ServicesThejaswi shasthri
 
Juan ignacio mazza - cv (english)
Juan ignacio mazza - cv (english)Juan ignacio mazza - cv (english)
Juan ignacio mazza - cv (english)JuanMazza
 
Sql server 2008 r2 application and multi server management fa qs
Sql server 2008 r2 application and multi server management fa qsSql server 2008 r2 application and multi server management fa qs
Sql server 2008 r2 application and multi server management fa qsKlaudiia Jacome
 
New features of sql server 2005
New features of sql server 2005New features of sql server 2005
New features of sql server 2005Govind Raj
 
Whatsnew in-my sql-primary
Whatsnew in-my sql-primaryWhatsnew in-my sql-primary
Whatsnew in-my sql-primaryKaizenlogcom
 
Krishna Resume
Krishna ResumeKrishna Resume
Krishna Resumekrishna G
 
SQL SERVER 2008 R2 CTP
SQL SERVER 2008 R2 CTPSQL SERVER 2008 R2 CTP
SQL SERVER 2008 R2 CTPGovind S Yadav
 
Ssis ssas sps_mdx_hong_bingli
Ssis ssas sps_mdx_hong_bingliSsis ssas sps_mdx_hong_bingli
Ssis ssas sps_mdx_hong_bingliHong-Bing Li
 
MS Sql Server: Getting Started With Sql Server 2008
MS Sql Server: Getting Started With Sql Server 2008MS Sql Server: Getting Started With Sql Server 2008
MS Sql Server: Getting Started With Sql Server 2008DataminingTools Inc
 
Ssrs introduction session 1
Ssrs introduction session 1Ssrs introduction session 1
Ssrs introduction session 1Muthuvel P
 
Introduction of ssis
Introduction of ssisIntroduction of ssis
Introduction of ssisdeepakk073
 
SQL Server Reporting Services (SSRS) 101
 SQL Server Reporting Services (SSRS) 101 SQL Server Reporting Services (SSRS) 101
SQL Server Reporting Services (SSRS) 101Sparkhound Inc.
 
SQL Reporting Services
SQL Reporting ServicesSQL Reporting Services
SQL Reporting Servicesneha mittal
 
First Look to SSIS 2012
First Look to SSIS 2012First Look to SSIS 2012
First Look to SSIS 2012Pedro Perfeito
 
SSRS 2008 R2
SSRS 2008 R2SSRS 2008 R2
SSRS 2008 R2tomerl
 

What's hot (20)

SQL Server Reporting Services 2008
SQL Server Reporting Services 2008SQL Server Reporting Services 2008
SQL Server Reporting Services 2008
 
Ssis sql ssrs_ssas_sp_mdx_hb_li
Ssis sql ssrs_ssas_sp_mdx_hb_liSsis sql ssrs_ssas_sp_mdx_hb_li
Ssis sql ssrs_ssas_sp_mdx_hb_li
 
MSBI-SQL Server Reporting Services
MSBI-SQL Server Reporting ServicesMSBI-SQL Server Reporting Services
MSBI-SQL Server Reporting Services
 
Juan ignacio mazza - cv (english)
Juan ignacio mazza - cv (english)Juan ignacio mazza - cv (english)
Juan ignacio mazza - cv (english)
 
DWMH Projects
DWMH ProjectsDWMH Projects
DWMH Projects
 
Sql server 2008 r2 application and multi server management fa qs
Sql server 2008 r2 application and multi server management fa qsSql server 2008 r2 application and multi server management fa qs
Sql server 2008 r2 application and multi server management fa qs
 
New features of sql server 2005
New features of sql server 2005New features of sql server 2005
New features of sql server 2005
 
Whatsnew in-my sql-primary
Whatsnew in-my sql-primaryWhatsnew in-my sql-primary
Whatsnew in-my sql-primary
 
Krishna Resume
Krishna ResumeKrishna Resume
Krishna Resume
 
SQL SERVER 2008 R2 CTP
SQL SERVER 2008 R2 CTPSQL SERVER 2008 R2 CTP
SQL SERVER 2008 R2 CTP
 
Ssis ssas sps_mdx_hong_bingli
Ssis ssas sps_mdx_hong_bingliSsis ssas sps_mdx_hong_bingli
Ssis ssas sps_mdx_hong_bingli
 
MS Sql Server: Getting Started With Sql Server 2008
MS Sql Server: Getting Started With Sql Server 2008MS Sql Server: Getting Started With Sql Server 2008
MS Sql Server: Getting Started With Sql Server 2008
 
Integrating SSRS with SharePoint
Integrating SSRS with SharePointIntegrating SSRS with SharePoint
Integrating SSRS with SharePoint
 
Ssrs introduction session 1
Ssrs introduction session 1Ssrs introduction session 1
Ssrs introduction session 1
 
Ssrs 2008 R2 webinar
Ssrs 2008 R2   webinarSsrs 2008 R2   webinar
Ssrs 2008 R2 webinar
 
Introduction of ssis
Introduction of ssisIntroduction of ssis
Introduction of ssis
 
SQL Server Reporting Services (SSRS) 101
 SQL Server Reporting Services (SSRS) 101 SQL Server Reporting Services (SSRS) 101
SQL Server Reporting Services (SSRS) 101
 
SQL Reporting Services
SQL Reporting ServicesSQL Reporting Services
SQL Reporting Services
 
First Look to SSIS 2012
First Look to SSIS 2012First Look to SSIS 2012
First Look to SSIS 2012
 
SSRS 2008 R2
SSRS 2008 R2SSRS 2008 R2
SSRS 2008 R2
 

Similar to Capture Change and Apply It!

Sql Server 2008 Enhancements
Sql Server 2008 EnhancementsSql Server 2008 Enhancements
Sql Server 2008 Enhancementskobico10
 
SQLServerDays2012_SSIS_CDC
SQLServerDays2012_SSIS_CDCSQLServerDays2012_SSIS_CDC
SQLServerDays2012_SSIS_CDCKoenVerbeeck
 
Resume_Navneet_Formatted
Resume_Navneet_FormattedResume_Navneet_Formatted
Resume_Navneet_FormattedNavneet Tiwari
 
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012Andrew Brust
 
Introduction to change data capture in sql server 2008 tech republic
Introduction to change data capture in sql server 2008   tech republicIntroduction to change data capture in sql server 2008   tech republic
Introduction to change data capture in sql server 2008 tech republicKaing Menglieng
 
Getting Started with MySQL II
Getting Started with MySQL IIGetting Started with MySQL II
Getting Started with MySQL IISankhya_Analytics
 
Continuous DB migration based on carbon5 framework
Continuous DB migration based on carbon5 frameworkContinuous DB migration based on carbon5 framework
Continuous DB migration based on carbon5 frameworkb0ris_1
 
Борис Трофимов. Continuous Database migration-это просто!
Борис Трофимов. Continuous Database migration-это просто!Борис Трофимов. Continuous Database migration-это просто!
Борис Трофимов. Continuous Database migration-это просто!Volha Banadyseva
 
Whats New Sql Server 2008 R2 Cw
Whats New Sql Server 2008 R2 CwWhats New Sql Server 2008 R2 Cw
Whats New Sql Server 2008 R2 CwEduardo Castro
 
Whats New Sql Server 2008 R2
Whats New Sql Server 2008 R2Whats New Sql Server 2008 R2
Whats New Sql Server 2008 R2Eduardo Castro
 
New features of sql server 2016 bi features
New features of sql server 2016 bi featuresNew features of sql server 2016 bi features
New features of sql server 2016 bi featuresChris Testa-O'Neill
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresJitendra Singh
 
Modernizing SQL Server the Right Way
Modernizing SQL Server the Right WayModernizing SQL Server the Right Way
Modernizing SQL Server the Right WayJuan Fabian
 
Balamurugan msbi cv
Balamurugan msbi cvBalamurugan msbi cv
Balamurugan msbi cvbala murugan
 
Mds cdc implementation
Mds cdc implementationMds cdc implementation
Mds cdc implementationSainatth Wagh
 

Similar to Capture Change and Apply It! (20)

Sql Server 2008 Enhancements
Sql Server 2008 EnhancementsSql Server 2008 Enhancements
Sql Server 2008 Enhancements
 
CDC
CDCCDC
CDC
 
SQLServerDays2012_SSIS_CDC
SQLServerDays2012_SSIS_CDCSQLServerDays2012_SSIS_CDC
SQLServerDays2012_SSIS_CDC
 
Resume_Navneet_Formatted
Resume_Navneet_FormattedResume_Navneet_Formatted
Resume_Navneet_Formatted
 
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012SQL Server Workshop for Developers - Visual Studio Live! NY 2012
SQL Server Workshop for Developers - Visual Studio Live! NY 2012
 
Introduction to change data capture in sql server 2008 tech republic
Introduction to change data capture in sql server 2008   tech republicIntroduction to change data capture in sql server 2008   tech republic
Introduction to change data capture in sql server 2008 tech republic
 
Getting Started with MySQL II
Getting Started with MySQL IIGetting Started with MySQL II
Getting Started with MySQL II
 
Continuous DB migration based on carbon5 framework
Continuous DB migration based on carbon5 frameworkContinuous DB migration based on carbon5 framework
Continuous DB migration based on carbon5 framework
 
Борис Трофимов. Continuous Database migration-это просто!
Борис Трофимов. Continuous Database migration-это просто!Борис Трофимов. Continuous Database migration-это просто!
Борис Трофимов. Continuous Database migration-это просто!
 
Whats New Sql Server 2008 R2 Cw
Whats New Sql Server 2008 R2 CwWhats New Sql Server 2008 R2 Cw
Whats New Sql Server 2008 R2 Cw
 
Whats New Sql Server 2008 R2
Whats New Sql Server 2008 R2Whats New Sql Server 2008 R2
Whats New Sql Server 2008 R2
 
Manageability Enhancements of SQL Server 2012
Manageability Enhancements of SQL Server 2012Manageability Enhancements of SQL Server 2012
Manageability Enhancements of SQL Server 2012
 
New features of sql server 2016 bi features
New features of sql server 2016 bi featuresNew features of sql server 2016 bi features
New features of sql server 2016 bi features
 
Vijaya_Lakshmi_Resume1
Vijaya_Lakshmi_Resume1Vijaya_Lakshmi_Resume1
Vijaya_Lakshmi_Resume1
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and Underscores
 
Modernizing SQL Server the Right Way
Modernizing SQL Server the Right WayModernizing SQL Server the Right Way
Modernizing SQL Server the Right Way
 
Balamurugan msbi cv
Balamurugan msbi cvBalamurugan msbi cv
Balamurugan msbi cv
 
Mds cdc implementation
Mds cdc implementationMds cdc implementation
Mds cdc implementation
 
Readme
ReadmeReadme
Readme
 
resume mumbai
resume mumbairesume mumbai
resume mumbai
 

Recently uploaded

Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Patryk Bandurski
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhisoniya singh
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksSoftradix Technologies
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 

Recently uploaded (20)

Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
Integration and Automation in Practice: CI/CD in Mule Integration and Automat...
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | DelhiFULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
FULL ENJOY 🔝 8264348440 🔝 Call Girls in Diplomatic Enclave | Delhi
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
The transition to renewables in India.pdf
The transition to renewables in India.pdfThe transition to renewables in India.pdf
The transition to renewables in India.pdf
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Benefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other FrameworksBenefits Of Flutter Compared To Other Frameworks
Benefits Of Flutter Compared To Other Frameworks
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 

Capture Change and Apply It!

  • 1. Capture Change and Apply it With Change Data Capture & SSIS Steve Wake Sr. Business Intelligence Consultant President, Denver SQL Server User Group MAKING BUSINESS INTELLIGENT www.pragmaticworks.com
  • 2. What is Change Data Capture (CDC)? • CDC first introduced in SQL Server 2008 • First available only in the Database Engine • Must be enabled for the database(s) and table(s) you want to track changes on • SQL Server 2012 Integration Services (SSIS) added components for CDC
  • 3. What is Change Data Capture (CDC)? Microsoft MSDN description: Change data capture is designed to capture insert, update, and delete activity applied to SQL Server tables, and to make the details of the changes available in an easily consumed relational format. The change tables used by change data capture contain columns that mirror the column structure of a tracked source table, along with the metadata needed to understand the changes that have occurred. Change data capture is available only on the Enterprise, Developer, and Evaluation editions of SQL Server. Source: MSDN - http://msdn.microsoft.com/en-us/library/bb522489(v=sql.105).aspx
  • 4. Why use CDC? • Loading a Data Warehouse – Changes from Source Only – Near Real-Time • Maintaining a Audit DB • Maintaining a Change Log
  • 5. CDC Key Concepts • sys.sp_cdc_enable_db/sys.sp_cdc_disable_db • sys.sp_cdc_enable_table/sys.sp_cdc_disable_table • cdc Database Schema • lsn = Log Sequence Number • _$operation - 1=Delete, 2=Insert, 3=Update (before image), 4=Update (after image) • CDC SQL Agent Jobs (capture & cleanup)
  • 6. CDC Limitations Source: MSDN - http://msdn.microsoft.com/en-us/library/cc645593(v=sql.105).aspx Type of Column Changes Captured in Change Tables Limitations Sparse Columns Yes Does not support capturing changes when using a columnset. Computed Columns No Changes to computed columns are not tracked. The column will appear in the change table with the appropriate type, but will have a value of NULL. XML Yes Changes to individual XML elements are not tracked. Timestamp Yes The data type in the change table is converted to binary. BLOB data types Yes The previous image of the BLOB column is stored only if the column itself is changed.
  • 7. Performance Considerations • Use different filegroups for CDC tables • @supports_net_changes (non-clustered index added to cdc table) • Use @captured_column_list with sys.sp_cdc_enable_table to limit columns tracked • FMI – SQL CAT whitepaper: http://msdn.microsoft.com/en- us/library/dd266396(v=sql.100).aspx
  • 8. SSIS 2012 • CDC Control Task – manage LSN’s, handles errors and recovery • CDC Source – read CDC tables/metadata • CDC Splitter – splits based on _$operation • Still requires Database/Table to be setup for CDC
  • 9. Demo • Setup CDC on database and table • Managing CDC in SSMS • Using SSIS 2012 Packages
  • 10. Summary • CDC added to SQL Server 2008 • Must be enabled manually, not on by default • SSIS 2012 added support to more easily handle CDC in SSIS packages • CDC is a built-in way to track changes to SQL databases/tables
  • 11. Services Speed development through training, and rapid development services from Pragmatic Works. Products BI products to convert to a Microsoft BI platform and simplify development on the platform. Foundation Helping those who do not have the means to get into information technology and to achieve their dreams. See you soon… Sales : sales@pragmaticworks.com My Email: swake@pragmaticworks.com My Blog: http://blog.wakebi.com My Twitter: @stevewake Office Number: 904-638-3805