Data Migration: Aiming for a goal beyond moving data.
that know how the pipes work within the company. They will secure the success of
    transforming data in the right shape ...
In order to avoid this risk the key is on the analysis needed to define the criteria on the data
            to be migrate...
The key turning point from Neoris point of view; is not choosing an MDM strategy but where do you start
and how – in a pra...
department to a high-level officer in the company in order to gain an active sponsorship. Any less
    and the initiative ...
The following aspects should be defined regarding data quality:

               - Develop metrics. Identify feasible metri...
Upcoming SlideShare
Loading in …5

Data Migration: Aiming for a goal beyond moving data.


Published on

Whether you are planning to implement or migrate to a new ERP platform; or just considering to consolidate your information within your organization as a building block for further data management initiatives; do not minimize the importance of migrating data because ....

Published in: Technology
  • Be the first to comment

Data Migration: Aiming for a goal beyond moving data.

  1. 1. Data Migration: Aiming for a goal beyond moving data. Written by Jorge Zamora Whether you are planning to implement or migrate to a new ERP platform; or just considering to consolidate your information within your organization as a building block for further data management initiatives; do not minimize the importance of migrating data because is a key task in order to achieve implementation goals. Furthermore, it is a golden opportunity for empowering data and turn it into an strategic asset for your organization. Data Migration Challenges Often, when it’s talked about data migration, it is pictured as a set of activities executed by a group of technical and business savvies, with the specific task of extracting, transforming and loading legacy data carefully prepared by the business; into a new system. It is a ‘one shoot’ task to enable a new platform. In this type of projects, mainly ERP implementations, the main goal pursued is that the platform covers the right functionality to support the business, and the data should ‘be there’ in the system in order to start using it, as a follow-up goal associated to the main focus. In summary; the functionality changes and the data should follow. Within this context, many organizations have already tackled data migration successfully, since they have implemented or changed into a new ERP: It’s up and running on the agreed due date, with the most critical data up-to-date. Sure, data structures have some glitches but its doing what it’s suppose to be doing, supporting the business. On the way to achieve migration success, they may have faced many challenges, among others: • Data migration seen as a technical effort. Many times organizations realize too late that the main efforts associated to migration data are not technology related. One may say that this is obvious, but this point have more angles than just a business definition for right formatting mask to the right fields in order to have the data clean and ready for the new platform. When the business define the new ‘blueprints’ on how he wants to operate on the new platform, the related data should follow. In other words, what the business is doing as he defines the ‘new blueprints’ is shifting the main course on the way things work, into a different one. And the business data structures should change accordingly. What is a business data structure? It can be defined as a business entity model that is composed by a set of metadata associated to a business transaction: Invoice, Purchase Order, Shipment Order, AP Open Items, for example. A new definition on tax retentions may change the way an invoice is composed. Or changes associated to new practices brought by the platform related to materials measure units master, may change the way a purchase order is placed. These are a couple of examples on changes in the business data structures that can be triggered by the new business ‘blueprint’ designs. These may be perceived as small changes, but if you multiply it by the quantity of live data records a company may have, the effort for implementing this change can grow exponentially. Hundreds of thousands of open items or purchasing orders may need to be transformed into a different structure. For all these transformations a very strong support business network needs to be put in place. This group of people needs to be formed by experts from the different business areas
  2. 2. that know how the pipes work within the company. They will secure the success of transforming data in the right shape and form according to the ‘blueprint’ definitions specified. If these definitions are not applied to business data structures for all possible scenarios in the right way, the data will not ‘fit’ into the new system and the data migration will be delayed, or end in failure. That’s why this analytical task is much more critical that actually defining and coding the right mechanisms to use for data loading. • Data migration borders get blurry. When the design and ‘blueprints’ phase ends and implementation tasks begin, sometimes the data migration team and support network get more than they bargain for. Data is the result of a company adaptation. Each time an organization agrees to sell, buy, promote, distribute or reimburse with different practices to different actors, a new business scenario is born. These scenarios are the main input for analysis in the design phase in order to be enhanced or changed. When scenarios are over-looked in this analysis, one of the events in which these are detected is on the first runs of data loading testing. At high level, sometimes it seems normal to have errors in this step, since this is of course, the goal of these test runs: To locate specific cases in which certain extraction / transformation rules or loading processes failed to work. However, the recurrence and impact of errors can indicate that the problem it’s not just a flawed interpretation of certain rule but a group or set of transactions and records that are associated to a scenario that was not analyzed and defined as part of the ‘blueprint’. When this happens, these errors are presumed to be intrinsically associated to the data migration processes. Even tough, the migration team may have the resources to solve this type of issues from the information point of view; the way to solve the scenario may not be correctly aligned to the business model as a whole. That’s why it is important to step back, involve the right people, and understand the type of errors presented on the first runs of the migration process attacking the root of the problem not the symptom, starting from the top, that is, the validation of the business scenario reviewing actual cases with live data, and decide on how the model will work. Consequently, the task triggered by this definition will fix the data migration issue. • Unclear rules for transactional data migration causing a ‘domino’ effect. One of the problems that have the greatest risk associated in a data migration process is the lack of clear boundaries rules for the transactional data to be migrated. This type of data refers to open transactions that need to be moved into the new platform or system in order to continue its lifecycle, they are produced on daily basis and contributes to the cyclical performance evaluation of the company, for example AP open items or open work orders. When the business analysis is flawed and hence, the rules delivered to the migration team for building the sets of this transactional data; a post go-live mayhem of missing data will happen and the business operation at all levels will suffer: deltas of transactional data will be required causing a domino effect on all data associated to it: Master data required to migrate this new delta and calculated data delivered by the business itself to complement it. A very hard effort will be required to keep the business in a safe place without jeopardizing its performance.
  3. 3. In order to avoid this risk the key is on the analysis needed to define the criteria on the data to be migrated. This may be a straightforward decision in some cases, and not very much so in others depending on the complexity and culture of the company. Bottom-line; when key users, sponsors and stakeholders have doubts on what set of data should be carried on the new platform and what data should be left behind; or the rules are not very clear, a warning need to be raised by the migration team or the technical leader. When these rules on live data are delivered to the migration team, it is critical to test them against real data in order to measure and identify what is being left behind and the set being migrated. Figures on both sets of raw data should be delivered to the business in order for them to see the consequences of the decision taken, so they understand the implications. This exercise will diminish the risk of missing data. • The hidden enemy. One of the toughest challenges on data migration efforts that in the long run reflects the apparent failure or the success of this type of projects is data quality. The key word on the past sentence is ‘apparent’. Why? Because Data Quality, in a rigorous approach, it really has nothing to do with a migration project. It has to do with transformation but not with quality. In a prefect world, quality of data should be taken for granted in a data migration project, because people should know how to transact in their IT platforms and be precise with the data that it creates, updates or modifies. But in this reality, its normal that a company faces data quality challenges, as it is normal to associate the success of a migration project with the quality of the data achieved while migrated to a new platform. Involving the business networks in well-monitored data quality tasks and measure data quality at the beginning, middle and end of the project are two of the main countermeasures that can be taken to secure data quality, while migrating data. Thinking ahead If a minute is taken to analyze the problems described above, we will find one fact in common: A lack of practical understanding on the way the organization relays on data in order to function. This lack of understanding is the cause of inaccuracies on the analysis and related decisions. How can we enhance the data understanding within a company in order to make the migration process smoother? Furthermore, how can re-position the way a company thinks about its data in order to understand it better and generate value from it? According to Aberdeen Group Research - Ninety-five (95) percent of Best-in-Class (BIC) companies have seen an improvement in on-time completion of data migration projects, upon implementing an MDM 1 strategy - It its clear that this is the way to go, but when facing an imminent migration project how do you ‘squeeze in’ and MDM initiative with all of its implications of people, procedures and technology? 1 Aberdeen Group Research - Master Data Management: Strategies and Tools to Improve Data Migration – April 2007
  4. 4. The key turning point from Neoris point of view; is not choosing an MDM strategy but where do you start and how – in a practical way - in order to achieve results soon enough so the migration project you’re facing can run smoothly. In Neoris experience the first step to take towards this MDM direction is has to do with the company organization itself, implementing a Data Governance initiative aiming to control and understanding of the company’s data. This strategy will prepare the business for a task in which data and information knowledge is critical for success. In the pragmatic sense, Data Governance can be defined as the practice of putting in place all accountabilities, policies, communications and processes associated to manage the life cycle of data within the organization, from its creation to its archiving and eventual deletion. The fist step in a Data Governance effort is defining a framework that will work as a guide for implementing it within the organization. This framework will help organize and align the company in the way its thinks about Data Governance and how it will communicate internally about its concepts. This framework should include 1) scope and rules of engagement; this is the mission of the initiative, objectives, data metadata definitions, and controls; 2) people and organizational entities, like stakeholders, organizational positions, owners and stewards, and 3) Processes aligned to goals and objectives. All of these elements are very important to define a clear and usable framework in order to begin working towards a data governance practice. Once the mission and objectives are clear, the following three elements should be considered as critical in defining this framework, while facing a ‘data migration’ initiative: Sponsorship, Ownership & Stewardship. One of the key elements in a data governance - initiative is its correct sponsorship and the empowerment that it provides. This kind of initiative is not an effort that should be sponsored solely by an IT department; it is an initiative that needs to be proposed by either a business transformation or strategic planning
  5. 5. department to a high-level officer in the company in order to gain an active sponsorship. Any less and the initiative could fail to bring everyone needed on board since this initiative could implicate adjustments in the current organization structure. Ownership, on the other hand, must be carefully analyzed since this could be in fact, one of the most complex tasks associated to this effort. Each owner is accountable for the conceptual understanding of an entity and the rules and exceptions that fulfill business requirements. Owners of different data entities (i.e. Customer, Vendor) need to be appointed taking in consideration the level of data dispersion of the entity within the company, business policies, current role of the proposed owner and mainly, his knowledge and experience on the entity. In some cases the organization structure itself provides a challenge for this type of appointments: Different divisions or lines of business have different sets of customers, and thus, ownership can be tricky to assign. The suggested path in this case is to assign the most experienced resource as owner and define a committee or support network including collaborators from the different lines of business in order to have a holistic view of the entity. Stewardship complements ownership. Data stewards act as a link between IT and the business. Their role is ‘hand-on’ over the data. They also accept accountability for data definition, data lifecycle specification, and information quality levels for specific data entities. Stewardship involves taking responsibility for data elements for their end-to-end usage across the enterprise. Data stewards become the “public face” for data and among their main responsibilities are data standards definitions, metadata definitions, data quality rules, data ownership conflicts, data security and retention. They also play a central role in the management of information across the organization and in assuring its usefulness and alignment to business requirements. Metadata Definitions. “Data about data” is the most common definition of metadata. The ‘data’ - about the data of the company is critical to understand it. It’s a set of definitions that help to clarify the business meaning of each information entity above any specific system data model and should be structured in a way that it could be exported, queried and consolidated without losing its validity. In consequence, pursuing metadata for the company’s data will set the record straight on the definitions that make sense for the business and will became a reference for understanding its data. How we identify a customer? How many addresses a vendor may have? How it’s the unique identifier of a material formed? All these questions will be answered by this reference. A critical element of the metadata definition effort is standards and rules definitions that may be applicable to each information entity. For example, ‘A customer from the retail division will only have one billing address’ or ‘An invoice will have at least one tax retention’. These rules are very important because are the core of the data management policies to be applied through the data governance processes. It may sound as a big challenge to develop company data’s metadata, but again a pragmatic approach should be chosen; focusing on the main entities that interact within the business (customer, material, vendor) and the main transactions that they do with the business (purchase order, invoice, sale order), using the scope of the migration effort in hand as a compass on which metadata should be tackled first. • Data Quality Metrics & Procedures. As mentioned earlier, data quality is really a challenge companies face not just while migrating data, but on daily basis. Customers with wrong addresses, products with the wrong specifications, vendors with wrong bank accounts, all of them are problems organizations may have recurrently, that have a direct impact of operation costs.
  6. 6. The following aspects should be defined regarding data quality: - Develop metrics. Identify feasible metrics regarding critical data entities: Products, Vendor, Work Orders, among others. For this, an initial assessment on quality of data is required, in order to identify the state in which they are on, and the next proposed and achievable goal. - Develop procedures for collection and maintenance of data. Definitions of clear procedures for gathering of data and maintain it, its type (proactive or reactive) and recurrence will provide to the company the actual tools in order to achieve the metrics proposed. - Identify control points and procedures. Throughout the value chain of the company, data is created, updated or deleted. These events can help to identify control points for monitoring data quality, either to fix any problem identified or secure a transaction with a third party (customer, vendor). - Implement!. Set target dates, empower stewards, make the metrics, procedures, control points available to the company and communicate them within the company. In summary … Targeting Data Governance focusing on ownership, metadata definitions and data quality while the early stages of a data migration related project such an ERP implementation, can provide a solid base for project success. Also, bear in mind that the real turnover is achieved when following trough this governance initiative adopting it as part of the ‘business as usual’, nevertheless the platform being implemented, project at hand or changes in your business operations. Thinking ahead will set the foundation for understanding and managing your data beyond any technology or IT solution.