Your SlideShare is downloading. ×
Migrating data: How to reduce risk
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Migrating data: How to reduce risk

5,380
views

Published on

A recommended methodology for reducing the risks involved in data migration.

A recommended methodology for reducing the risks involved in data migration.

Published in: Technology

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,380
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
226
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Migrating data: Howto reduce riskwww.etlsolutions.com
  • 2. Data migration: A risk!
  • 3. A real-life example of the risks involvedWe were called in to rescue a project in which tools hadnot been used:• The vendor hand-wrote the migration to target system• Rewritten three times due to: • Structural limitations • Performance (twice)• Bugs in code delayed the migration• Bugs in code caused incorrect data, making it difficult to test actual migration logic
  • 4. Recommended best practice Business ownership: transparency, control, engagement Use of toolsets Use of a project management methodology A team with skills and experience in data migration Use standards wherever applicable Use of a data migration methodology
  • 5. Recommended team leader objectives Identify and mitigate risks Ensure confidence in the project team Appropriate resourcing Keep the project on track:  Within budget  On time Reduce the overall cost and effort needed
  • 6. Overview of our recommended data migration process Business engagement Scoping Core Legacy migration decommissioning Assessments
  • 7. Embed best practice within the process• Migration methodology: an in-depth process for each stage of the data migration, bringing in specialists at key times and working to a structured plan and documents• Project management methodology: We use PRINCE2 to ensure that the migration is well managed to a common standard• ISO standards: We use 15288,12207,and 8000 to ensure the methodologies have a core standard structure• Migration software: The core migration steps should be carried out using a well established tool• Migration specialists: Specialists should be available for the duration of the project
  • 8. Data migration process: Assessments Staff competency assessment Project assessment• Helps to identify gaps in the • A strategic review of the existing skill base proposed project, examining:• Shows where training may be • Plans required • Processes• Reduces reliance on external • Workflows experts • Data targets• Grows the confidence of the • Systems project team • Identifies and mitigates risks• Provides individuals with greater and issues before they occur clarity about their roles• Our proprietary Data Migration Competency Framework is designed specifically to support data migration teams
  • 9. Data migration process: Scoping Project scoping Technical scoping• A detailed, tactical examination • A detailed examination of the of the proposed project, project’s technical structure: including: • Available models • Stakeholders and deliverables • Available software expected • Data volume and quality • Budgets • Identifies potential technical • Deadlines issues before they occur • Communication plans • Available experts: business domain, systems, migration experts• Helps with project planning to mitigate risk• Provides business leaders with a clear view of the project plan
  • 10. Data migration methodology Iterative agile development is used throughout Project scoping Core migration Configuration Landscape analysis Data assurance Migration design Migration developmentRequirements analysis Data discovery Data review Testing design Testing development Data modelling Data cleansing Execution Review Legacy decommissioning
  • 11. Landscape analysis in more detail• Landscape analysis encompasses the systematic process of identifying all source and target systems that may be involved in the data migration• Gain an overview of the source and target systems• Key tasks: • Understand how each system works • Understand how the data within each system is structured • Model the systems • Model links and interactions between systems • Model data structures
  • 12. Data assurance in more detail• Data assurance puts measures in place to ensure that all Data assurance questions: information used within the • What data migration and data migration is handled profiling tools are available? accurately • Are there key areas of• Data quality is a key element, weakness in data? along with data cleansing • Are rules for the data quality where required (attribute and relational) within• Planning is required for the the source already documented? retirement of data, for deletion • Which governing rules have to or for storage due to industry be applied to the data? regulations • Will all data be migrated?• Key tasks: • Data review • Data cleansing
  • 13. The key data assurance tasks Data review Data cleansing• Profiling is carried out to identify • Define the cleansing rules which areas of the data that may not will be carried out manually and be of sufficient quality to meet those which will be automated: business requirements • The manual cleansing will be• Data quality definitions define typically be carried out prior to the quality that must be attained migration for elements, attributes and • The automated cleansing will be relationships within the source carried out as a first step of the migration or indeed may also be system; these definitions will be able to be carried out prior to used during the profiling to migration verify if the data adheres to the • Data verification is focused on the rules defined checking of data being available,• The system retirement plan accessible, in the correct format defines which data will be and complete moved from the old system to • Data impact analysis is carried the new and what is no longer out to evaluate the effect on other required elements and systems
  • 14. What happens when data assurance is omitted? • Multinational 3-way multi-company venture • US vendor migrating data • UK project management • No migration management • Lack of resource • No verification or quality rules • No profiling carried out • Ad-hoc testing • Security delays of 6 months • Testing time increased by 3 months • 1 month delay for invalid characters • Additional dry-run required due to issues encountered
  • 15. Core migration in more detail• Use a structured approach with methodologies to guide the process• Establish clear structures and boundaries: a methodology will help with this• Ensure tool-based execution: we use our own commercially available software, Transformation Manager• Carry out tool-based testing• Create an organisation-focused go live: scoping the project at an early stage can assist in disseminating the information to stakeholders around timings and resource required to achieve the goals
  • 16. A core migration design
  • 17. Our offerings for data management Transformation Data Manager data migration movement packaged software services Support, training and mentoring services
  • 18. Why Transformation Manager?For the user:  Everything under one roof  Greater control and transparency  Identify and test against errors iteratively  Greater understanding of the transformation requirement  Automatically document  Re-use and change management  Uses domain specific terminology in the mapping
  • 19. Why Transformation Manager?For the business:  Reduces cost and effort  Reduces risk in the project  Delivers higher quality and reduces error  Increases control and transparency in the development  Single product  Reduces time to market
  • 20. Contact us for more information: Karl Glenn, Business Development Director kg@etlsolutions.com +44 (0) 1912 894040 Read more on our website: www.etlsolutions.com Raising data management standardswww.etlsolutions.com www.etlsolutions.com Images from Free Digital Photos freedigitalphotos.net