Data migrations can look like deceivingly straightforward efforts when in fact they are risky undertakings. This presentation shares what many data practitioners and project managers see as the #1 success factor for data migration projects. Become a data migration hero by paying attention to this simple yet highly effective recommendation.
3. by Roque Daudt, BCompSC, ITIL, CRM
rdaudt@yahoo.com
The #1 Success Factor for Data
Migration Projects
Data Migration
4. Target Audience
Who needs to know it?
Project managers
..who want to ensure that data migration
projects are adequately resourced and
that deliverables are produced within the
approved budget and schedule
Project sponsors
..who want to know in advance the risks and
costs of data migration projects
BAs, SMEs, other project members
..who will work together with the data
migration team
Data teams
..who may be new to data migration projects
5. Problem Definition
What is the issue with data migration projects?
About 40% of the
projects overrun or are
cancelled 1
Data quality issues disable or severely cripple
application functionality. Compliance issues.
When there is an
overrun it nearly
always involves both
time and costs 2
QUALITY
Simply put, they fail too often
1, 2 Bloor Research, 2011
6. Problem definition
Data loss
Impact on quality of analytics services
Impact on business processes
Extended downtime
Lost revenue
Missed business timelines
Impact on clients and partners
Non-compliance
What is the impact on business?
7. Root cause
Data migration projects fail because they are poorly planned
Limited
understanding about
the nature, complexity
and risks associated
to data migration
Limited understanding about data quality
Lack of awareness
about the role and
importance of
collaboration and
governance
Root causes of poor planning
Data migration myths
8. Common attributes of a failed data migration project
Thin on expertise
Underfunded
Underresourced
Inadequate approach
Little business/IT collaboration
Unrealistic timelines
High risk
Poor governance
It starts too late!
9. Data migration mythology
Top myth
Misconceptions about data migrations are also
fueled by some myths
Chief among them is the understanding that the
work on data migration should only start when
the larger project is wrapping up
This is simply not true. A tremendous amount of
data migration-related work can and should be
done as soon as the broader project starts
10. An early start is the #1 success
factor for data migration efforts
Let’s explore ahead how it helps to decrease risk
and increase the chances of success
11. Benefits of an early start
Learn about the business and the application
The data team needs to acquire a good understanding of the
business and the application in order to develop and run a
successful data migration
Early access to SMEs, BAs, Architects, the legacy
application and existing documentation provide the data
team with enough knowledge and time to create and refine
the data migration approach
12. Benefits of an early start
Get familiar with the source data model
Detailed requirements will be fleshed out later on but the
domain of business data that will be migrated is usually known
at the start of most projects
Early access to the data model of the legacy application
provides the data team with an opportunity to map it to the
business data at a high level. This will be key input when
the time comes to map the source to the target data model.
13. Benefits of an early start
Design the user migration
In many projects, users will also be migrated to the new application/platform.
The user management frameworks are likely different between the legacy and
the new platforms
Very likely, the data team knows none of them and knows nothing about the
support that exists to it in the persistence layer
An early opportunity to review them will help the data team to develop the
approach to user migration. In some cases, it may also allow for early design
14. Benefits of an early start
Run technology survey and analysis
The diversity of database products and persistence
technologies is almost endless. Once the data team is put in
contact with the source data custodians, it can start to survey
and analyze the technology landscape
Information gathered by this work will be a key input to
develop the approach for data extract and data movement
15. Benefits of an early start
Get a handle on size
When it comes to data, size matters
Data profiling can be easily done with a productivity tool
such as MS Excel when the data source is, say, little more
than a few thousand Accounts. Things get increasingly more
challenging as the data volumes increase
Insights on data volumes will be key input for the
development of the overall data migration approach,
including the need to procure and acquire tools
16. Benefits of an early start
Review the sharing mechanisms
Data is always owned by one or more application users, teams,
groups, etc. It must often be shared with others
More often than not, the data sharing-related requirements
don't change much when an application is re-developed or
moved to a new platform
The sharing mechanisms are platform-specific, though. For
this reason, an early opportunity to review these
mechanisms will greatly help the data team to build support
for the data sharing requirements in the data migration
17. Benefits of an early start
Establish governance
Most enterprises have a slew of policies, standards and practices
that govern data availability, access, movement, storage, usability,
integrity and security
These constraints must be followed and taken into consideration
when the approach for the data migration is developed
Early review and structuring of the governance aspects of the data
migration will ensure compliance and will allow the data team to
timely identify and address potential roadblocks
18. Benefits of an early start
Review the toolbox
At times the data team will be required to work with a specific set of
ETL tools; those made available by the client. These tools may not
be well known by the team
To early identify such constraints allows the data migration team
to bring awareness to the project manager about the need to
build a ramp-up time in the project.
19. Benefits of an early start
Build a better application
The process of developing requirements for a new application or a re-
platform initiative is complex.
One valuable tool is the analysis of the data model of the legacy application.
By getting familiar with it early in the project, the data migration team can
provide rich insights to the BAs about data domains, relationships and
business rules that were missed or not clearly articulated while running
requirements development.
20. Benefits of an early start
Summary
the data team is given
the time and tools to
develop and refine the
approach for the data
migration
the data team gets an
opportunity to validate
several aspects of the
approach and confirm
that governance will be
in place
the overall development process is improved by the
insights that the data team can share with other project
members about the data source
when the time comes to design the
data migration in details the data team
is already familiar with the target and
source platforms, the tools, the security
and sharing models, the source data
model
all these aspects significanty improve the chances that the
data migration efforts will produce their deliverables in
time, on budget and with better quality.
21. And by starting it early, for
whatever was missed, there is
still time to deal with it!
23. Why we created this presentation
I have been involved with data-related projects a number of times in my career. Many of
them were data migration projects. This gave me the opportunity to make my share of
mistakes and learn a bit from them. In particular, to learn that data migrations can look
like deceivingly straightforward efforts when in fact they are risky undertakings.
The risks carried by data migration are nothing new, though. Data practitioners have
been vocal about it for quite a while. However, for whatever reason, too often this
message fails to reach the ears of those in position to make the decisions that will make
or break such projects.
This presentation is a lighthearted, visual-oriented attempt to share the same message,
with hopes that it can reach a wider audience and foster some conversation. If you find
value in it and know someone who could benefit from these notes, feel free to pass it
along.
24. A couple of good references
Seek and you shall find. There is no shortage of good material about data and data
migration out there. Here are some that I bumped into while preparing this deck.
http://www.oracle.com/technetwork/middleware/oedq/successful-data-migration-wp-
1555708.pdf
http://dataagilitygroup.com/blog/most-data-migration-projects-bust-both-budgets-and-
schedules/
http://www.oracle.com/us/products/middleware/data-integration/enterprise-data-
quality/data-migration-wp-2345281.pdf
https://premier-international.com/the-impact-of-a-failed-data-migration/
https://www.dataqualitypro.com/
25. Let’s keep the conversation going!
Does it all make sense to you? Yes? No? Why not? Different experience? Similar
experience? No experience? Like the colors of the deck? Hate them? Are you Mike the
Sql Guy? Let us know. Sharing is caring. It is all for the greater good.
This material was originally published to LinkedIn where you can find me at
www.linkedin.com/in/rdaudt
Good, old email still works
rdaudt@yahoo.com
26. We appreciated the contributions received
Thank you very much to the fine folks that helped in one way or another to put this
deck together and..