1. A successful data migration requires meeting quality criteria such as agreed stakeholder impact, reliable execution, a controlled process, and being auditable.
2. Data migration is represented as a workstream in a transition program including activities such as data analysis, data quality improvement, and data mapping.
3. Data migration is typically done through a series of incremental iterations consisting of standard activities such as data analysis, data mapping, and migration testing.
2. 2
A successful Data Migration is defined by a set of quality
criteria
2
A seasoned method balances
attention to the criteria
• Agreed stakeholder impact – customer, organization & third parties
• Reliable and correct execution – supported by a transparent validation process
• Controlled migration process – with well defined decision points
• Auditable – process is fully reconstructable
• Right cost and duration – adjusted to risk appetite
3. 3
Data Migration is represented as a Workstream in a
transition program
3
Analysis Design, Build & Component Test Integration Test
Data
Quality
Commercial
Preparation
Target Application
Landscape
Work Processes and
Instructions
Data
Migration
Execute
PilotDress
Rehearsal
Business Process Tests
Realization of target application
landscape
Configuration of Target Applications
(i.e. products, …)
Migration
Iterations
Commercial Migration Process
Client Communication (i.e. letters)
Customer
panel
Customer
Impact analysis
Data
Analysis
Data Quality
Improvement
Functional Migration Process
Data Mapping
Design of Migration Components
Migration System (Components)
Work Process Definition and
Description
Work Instructions (i.e. Aris)
Migration
Tests
Ist/Soll
Gap
Analysis
Product/Chann
el matrix
1 2 3 4
Data Quality
Improvement
New objects in
Target Applications
(Commercial Go
Live) Pilot
Test of target landscape for
new Objects
Dry-Run of
Work
Processes
Data Migration
Strategy (incl.
Acceptance
Criteria)
4. 4
Data Migration done Agile: series of incremental
migration iterations
4
Migratie
strategy
Migration Tranche 1
Migration Tranche 2
Migration Tranche N
Close out
Number and duration of iterations depends on:
• Desired level of automation
• Available IT capacity
• Available capacity for manual migration processes
• acceptabele customer timpact & impact on own organization
Gap analysis
Execution
(optional)
Data analysis
5. 5
Migration Iterations consist of a set of standard activities
5
Objectives, scope &
acceptance criteria
Data Analysis
Data Cleansing
Data Mapping
Design, Build, Test
Extract, Transform, Load, Validate
Migration test
Business process test
Execution
(0, 1 or n tranches)
Design, Build, Test
Configuration and adaptations of tagret applications
• test cycles (2)
• Dress Rehearsal
• Internal Tranche
• Live Migration
• Data Ware House
6. 6
Migration activities result in standard migration
deliverables (with dependencies)
6
Migration
Strategy
Program
Objectives
& Scope
Data
Analysis
(source)
Production
Data
Functional
Migration
Process
Component
Design
Technical
Migration
Process
Migration
System
Component
Design
Business
Process
Test
Target
Application
Landscape
Clean
Production
Data
Dress
Reheasal
Run
Book
Migration
System/Int.
Test
Design
Execution
Governance
Migration
Test Plan
Readiness
Assessment
Migration
Test Report
Data
Mapping
Execution
Status/
Evaluation
Report
Work
Instructions
Ist/Soll
Gap
Analysis
Commercial
Migration
Approach
7. 7
Data Migration Strategy defines the outlines of the migration
7
Subjects to be addressed
Migrating Objects (incl. volumes)
Source & Target Systems
Scope (i.e. active objects vs historic data)
Stake Holders
Acceptance Criteria
Migration Approach
• big bang vs iterations, slicing criteria,
• On-/Off-Line/Real time/Batch
• Timing – critical migration windows vs. synchronization source/target
• Load approach (database, application, service)
• Test approach (Business Process Tests)
• Manual/Automated
• Pipeline issues
• Rollback / fallback
Validation Approach (i.e. confidence checks)
Customer Impact, systems availability, communication
Data Quality (required vs desirable)
8. 8
Data Analysis provide insight in production data volumes,
relations and frequency distribution
8
Data Analysis ReportSQL QueriesExcel Pivot Table
The truth is in the data
9. 9
Data Migration designed according to Extract Transform Load
Validate pattern
9
Source
Systems
Landing
Extracted
Files
Staging
Conversion Engine
Transform Load
Target
Systems
Extracted
Files
Validation Tool
Validation
Database
Validation Tool
Validation tool shares platform with Conversion Engine
Both the engine and the validation tool are based on extraction files from
Source and Target
Mapping is conducted independently from conversion engine to ensure
proper validation
Control Model
The control model defines where the checkpoints in the migration process
are created and the metrics
Every time data flows from one stage to the other, a report is generated to
show counts, sums and cross connections
Each table contains source, timestamp and where
applicable reference to transformation rules
Counts and Sums are calculated at each point where
data is handed over to next stage
10. 10
First Test Cycle serves as proof of concept and learning
curve, subsequent Test Cycles iron out defects
10
Cycle #1 Cycle #2 Cycle #3
Proof of concept Test completion
• Goal 1st test cycle is a complete run to proof the concept
• Duration needed for sub-sequent cycles can be derived
• Both full migration execution as well as business processes in
target in scope
• Goal of subsequent cycles is to complete test execution until defects are
removed
• Every cycle will be based on a new release of migration engine (except the last
one which must be based on a stable version)
• Duration of a cycle is not expected to decrease significant
Set-up preconditions
Execute migration
Conduct BP Testing
Wrap-up results
Fix defects
11. 11
Successful Data Migration execution relies on five
deliverables
11
Five Deliverables for Data Migration Execution Stage:
1. Governance process and structure with representation of key stake holders
2. Central control room capable of directing the migration process
3. Run book with full details of data migration activities (who, when, where and dependencies)
4. Proven migration system and process – passed successful Dress Rehearsal
5. Readiness assessment artifacts
12. 12
Successful Data Migration requires skills and experience
12
DataDogs have Data in their DNA
20 Highly trained, skilled and experienced professionals
More than 15 successful migration projects
Vincent Wormer
Ruud Kapteijn Manuel Noechel
Jacco de Gooijer
Datadogs is the right partner for a
successful Data Migration