There are many ways of moving data from source databases into your target system. Historically, the Extract, Transform, and Load (ETL) has been the method for moving data effectively between databases and analytics platforms. But ETL is no longer necessarily the right solution for the modern data-movement needs. Back in the early days of data movement, extraction on a monthly, weekly, even daily, basis could be enough. Today, analytics often needs to be executed the same day, sometimes even within seconds of the original data being generated. Replication, where information is moved in near real-time, provides a more efficient, and quicker, data movement solution. AGENDA - How ETL operates - How Change Data Capture (CDC) operates - Replication operation - Comparing data movement cadence - The data impedance mismatch

1. Real-time replication vs ETL - How analytics requires new
technologies designed for it, not '70s technology

2. Topics
In today’s webinar, we will discuss:
• What is ETL?
• What is CDC?
• What is replication?
• Differences

3. Extract, Transform, and Load (ETL)
SELECT *
FROM XXX
Transform
(Datatypes,
structure,
Embedded types)
INSERT INTO
YYY

4. ETL Latency
Every
Hour, or
Day, or
Week
Large Blocks

5. ETL Incremental
Track primary key, load PKey from Last
Track existing timestamp, extract from last TS
Modify structure to add timestamp
or change tag

6. ETL Parameters
Advantages
• Loads large blocks of data efficiently
• Allows for complex and detailed transformation
• Allows for full data extraction
• Can be easy to implement
• Efficient data loading for some target environments
Disadvantages
• Can be time consuming
• Places extreme load on source database
• Transformation can take excessive time
• Bulk load can create data ingestion performance
problems on the target database
• Incremental loading difficult or complex
– Either timestamp based
– Data/structure changes

7. Change Data Capture (CDC)
INSERT,
UPDATE,
DELETE
Trigger or
Background Process
Modified Rows
Copy to
Target

8. CDC Latency
LoadCDC
Generation
Row
Extraction

9. CDC Parameters
Advantages
• Easy method of identify or extracting data
• Requires no schema modifications to the source
database
• Medium latency
• Medium data loading cadence
Disadvantages
• Requires database level support
– Either custom CDC or Triggers
• Requires additional database and table space
• Implies overhead on database (both for generation
and extraction)
• Extraction latency can increase as data volumes
increase
• Requires source access to DB
• No or few transformations

10. Replication
DBMS
Logs
Master Replicator:
Extractor
Slave Replicator:
Applier

11. Replication Latency
Transaction
Batch

12. Replication Parameters
Advantages
• Low latency replication
• No source database access
• No source database load
• No source database changes
Disadvantages
• Limited transformation and combination of data
• Per-transaction loads slower
– Mitigated by batching or parallel apply

13. Comparison
Feature ETL CDC Replication
Data Rate Low, periodic Periodic High, real-time
Transformations All data and modifications
possible before load
Very low complexity only Low complexity only
Combination Complex combinations
possible
Very limited Very limited
Source DB Load Impact High to Very High Medium Low
Incremental Support Difficult, or requires DDL
changes
Fully supported Fully supported

14. Example of modern Heterogeneous Deployments
Replicator
Warehouse
Per User
Per Database
Audit

15. Conclusions
• ETL
– Efficient and simple system
– Slow, and not usable for modern deployments
– Incremental Complex
• CDC
– Capable with database load
– Lower latency
– DB dependent
• Replication
– Very low latency/High performance
– Limited transformations and combinations

16. Next Steps
• If you are interested in knowing more about Tungsten Replicator and would like to try it out for
yourself, please contact our sales team who will be able to take you through the details and
setup a POC – sales@continuent.com
• Read the documentation at http://docs.continuent.com/tungsten-replicator-5.2/index.html
• Subscribe to our Tungsten University YouTube channel! http://tinyurl.com/TungstenUni

17. For more information, contact us:
Eric Stone
COO
eric.stone@continuent.com
MC Brown
VP Products
mc.brown@continuent.com
Chris Parker
Director, Professional Services EMEA & APAC
chris.parker@continuent.com
Eero Teerikorpi
Founder, CEO
eero.teerikorpi@continuent.com
+1 (408) 431-3305