EMC Documentum Compliance Manager Migration Revisited — What a Difference Two Years Makes


Published on

Momentum 2006 Presentation

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • EMC Documentum Compliance Manager Migration Revisited — What a Difference Two Years Makes

    1. 1. Allergan: EMC Documentum Compliance Manager Migration Revisited — What a Difference Two Years Makes David Mc Donagh Project Manager, EDMS Allergan, Inc. Jes Wills Migrations Practice Manager Blue Fish Development Group
    2. 2. Agenda <ul><li>Overview of Allergan </li></ul><ul><li>Allergan & Documentum (a long history) </li></ul><ul><li>The DCM 5 Project (Project Snow Goose/CORAL) </li></ul><ul><li>Blue Fish Overview </li></ul><ul><li>Project Snow Goose Migration </li></ul>
    3. 3. Allergan, Inc. – Overview <ul><li>Global specialty pharmaceutical & medical device company, headquartered in Irvine, CA </li></ul><ul><li>Neuro Science (BOTOX ® & BOTOX ® Cosmetic) </li></ul><ul><li>Eye Care ( Treatments for glaucoma, ocular allergy and infections, retinal disease and dry eyes ) </li></ul><ul><li>Dermatology ( Treatments for various skin diseases including Acne and Psoriasis ) </li></ul><ul><li>Allergan - Inamed (Facial Asthetics, Breast Augmentation & Reconstruction, Obesity Products) </li></ul>
    4. 4. Allergan, Inc. – Overview <ul><li>$2.3B annual sales (2005) </li></ul><ul><li>5,000+ employees globally </li></ul><ul><li>Manufacturing: USA (Texas), Ireland, Brazil </li></ul><ul><li>R&D: USA (California) & UK </li></ul>
    5. 5. Allergan, Inc. – Centralized I.S. <ul><li>Global operations supported from I.S. based in Irvine, CA </li></ul><ul><li>Centralized I.S., little infrastructure outside of Irvine </li></ul><ul><li>Single Documentum server for manufacturing applications </li></ul><ul><li>Advantages / Disadvantages </li></ul>
    6. 6. Allergan’s History with Documentum <ul><li>Documentum purchased initially in 1996. </li></ul><ul><li>Start: WorkSpace-based (pre-EDMS98) apps </li></ul><ul><li>R&D applications: </li></ul><ul><ul><li>DCM 4.3 (March 2003) </li></ul></ul><ul><ul><li>Publishing (Core Dossier) </li></ul></ul><ul><li>Web applications </li></ul><ul><ul><li>Web Publisher (Inter & Intranet) </li></ul></ul><ul><li>Manufacturing applications </li></ul><ul><ul><li>3 Different Workspace applications </li></ul></ul>
    7. 7. DCM 5 – Beta Program <ul><li>December 2003 – March 2004 </li></ul><ul><li>Tier 1 participant </li></ul><ul><li>What We Said at Momentum 2004 : </li></ul><ul><ul><li>Great to get experience with the software early </li></ul></ul><ul><ul><li>Software was not as ‘buggy’ as we expected </li></ul></ul><ul><ul><li>Beta program was short (2 months – not enough time to really work with the software) </li></ul></ul><ul><ul><li>Significant time commitment was required </li></ul></ul><ul><ul><li>DCM software was refreshed in the middle </li></ul></ul><ul><ul><li>PDFAqua not available until 2 nd month </li></ul></ul><ul><ul><li>Learning curve was steep & Little documentation available </li></ul></ul>
    8. 8. Project Snow Goose <ul><li>Project Snow Goose: “A North American bird that undergoes a long, difficult migration” </li></ul><ul><li>Aim: </li></ul><ul><ul><li>Convert 4 separate apps, organizations and processes into a single DCM application. </li></ul></ul><ul><ul><li>End Users: Global Manufacturing & Global I.S. </li></ul></ul><ul><li>Progress: </li></ul><ul><ul><li>Prior to 2005: </li></ul></ul><ul><ul><ul><li>Wait for R&D to complete DCM 4.3 project </li></ul></ul></ul><ul><ul><ul><li>Merge document types in EDMS 98 apps (Merge over 50 different types  7 document types) </li></ul></ul></ul><ul><ul><ul><li>Define Requirements based on streamlined business processes & DCM 4.3 knowledge </li></ul></ul></ul>
    9. 9. CORAL <ul><li>CO ntrolled R epository for AL lergan Documents </li></ul><ul><li>Project Snow Goose Core Team </li></ul><ul><ul><li>6 full-time Allergan staff members on project </li></ul></ul><ul><ul><li>Focused consulting assistance </li></ul></ul><ul><ul><li>Extremely committed user organization </li></ul></ul><ul><li>Use consultants only to complement existing staff </li></ul><ul><ul><li>Defined ownership/ maintains internal control </li></ul></ul><ul><ul><li>Knowledge transfer at end of project CRITICAL </li></ul></ul><ul><ul><li>Especially technical expertise </li></ul></ul><ul><li>Target completion July & August 2005 (2 distinct go-lives) </li></ul><ul><li>Migration critical to success </li></ul>
    10. 10. CORAL – Design/Development <ul><li>Timeline restrictions => Rollout DCM 5.2.5. sp1 (build 132c) July 2005 </li></ul><ul><li>Issues: </li></ul><ul><li>Java Method Server lockup (if more than a few (4-5) people were promoting).  Installed Media Server as part of the SP1 &quot;C&quot; build to process the promotion and notification events.  HP/UX required additional time to get fix. </li></ul><ul><li>Serial workflow functionality was only 25% developed.  Modified Quickflow to get required functionality. </li></ul><ul><li>17 reviewer/approver limit - Never fixed in 5.2.5 sp1 </li></ul><ul><li>dcmCheckin method used for custom check ins did not work </li></ul><ul><li>dcm_change_cd_properties audit event was not logged.  Fixed in SP1 &quot;FCS&quot; build. </li></ul><ul><li>Could not demote a document.  Fixed in SP1 &quot;FCS&quot; build. </li></ul><ul><li>Could only assign the document owner as the periodic review owner and all class coordinators would receive the notification.  We customized periodic review. </li></ul><ul><li>Search did not work as expected.  Customized simple and advanced search. </li></ul><ul><li>Future Implementations: </li></ul><ul><li>Implement a STABLE “.x” product. </li></ul>
    11. 11. Migration - Requirements <ul><li>1) Migrate ALL REQUIRED data </li></ul><ul><li>2) Migration from a Network Filestore (Global I.S. documentation) </li></ul><ul><li>3) Migrate from 3 different docbases (Manufacturing documentation) </li></ul><ul><li>4) New standardised cabinet/folder structure in CORAL </li></ul><ul><li>5) Property mappings </li></ul><ul><li>6) Docbase migration: Complete migration within a “frozen” window of 5 days – NO docbase activity guaranteed </li></ul><ul><li>7) Validatable </li></ul><ul><li>8) Reusable </li></ul>
    12. 12. Migration – The REALITY (1) <ul><li>How “clean” is the data you own? </li></ul><ul><ul><li>Clean/consistent data required for migration tools </li></ul></ul><ul><ul><li>Do you really understand the scope of the data cleanup </li></ul></ul><ul><ul><li>Mapping of data from varying locations to consistent CORAL location </li></ul></ul><ul><ul><li>Focussed team reviewing and managing data </li></ul></ul><ul><ul><li>Inconsistent data = Initial docbase migration Failure! </li></ul></ul><ul><ul><li>I.S. Network filestore migration – Target July 2005 – Achieved </li></ul></ul><ul><ul><li>Manufacturing Docbase Migration – Target August 2005 – Not Achieved </li></ul></ul>
    13. 13. Migration – The REALITY (2) <ul><li>New docbase Migration plan: </li></ul><ul><ul><li>Timelines agreed with business </li></ul></ul><ul><ul><li>Focus on data cleanup </li></ul></ul><ul><ul><li>Maintenance of data in clean state </li></ul></ul><ul><ul><li>Validate </li></ul></ul><ul><ul><li>Dress Rehearsals </li></ul></ul><ul><ul><li>Manufacturing Docbase Migration – New Target January 2006 – Achieved </li></ul></ul>
    14. 14. Project Snow Goose Blue Fish Pedigree <ul><li>What Blue Fish brought to the engagement … </li></ul>
    15. 15. Project Snow Goose What we said at Momentum 2005 … <ul><li>Organization Challenges Addressed by: </li></ul><ul><ul><li>Forming a global migration delivery team </li></ul></ul><ul><ul><li>Assigning team leads on local level </li></ul></ul><ul><ul><ul><li>Give each decision authority </li></ul></ul></ul><ul><ul><ul><li>All team leads had a good command of English </li></ul></ul></ul><ul><ul><li>Executing each migration at least three times </li></ul></ul><ul><ul><li>Business users auditing the results of each migration </li></ul></ul><ul><ul><li>Adopting an approach based on the Blue Fish Migration Methodology and product suite </li></ul></ul>
    16. 16. Project Snow Goose The Blue Fish Framework <ul><ul><li>Identify the data to be migrated </li></ul></ul><ul><ul><li>Extract the data from its current repository/file store </li></ul></ul><ul><ul><li>Transport the data from its current location to the new location </li></ul></ul><ul><ul><li>Transform the data into the new structure </li></ul></ul><ul><ul><li>Load the data into the new repository </li></ul></ul><ul><ul><li>Validate that the data was transformed and loaded correctly </li></ul></ul>
    17. 17. Project Snow Goose The Blue Fish Approach <ul><ul><li>Work with business users to define migration rules for each kind of document </li></ul></ul><ul><ul><li>Automate the conversion of business rules into XML </li></ul></ul><ul><ul><li>Leverage Blue Fish DIXI (Documentum Import/eXport Interface) </li></ul></ul><ul><ul><li>Allow non-technical business users to validate documents at each step of migration </li></ul></ul><ul><ul><li>Introduce an enrichment step to allow users and/or administrators to perform meta-data enrichment </li></ul></ul><ul><ul><li>Automate the process to allow repeatable test migrations </li></ul></ul>
    18. 18. Project Snow Goose Theory and Practice <ul><li>How we used the framework and methodology to deliver the Project Snow Goose Migration … </li></ul>
    19. 19. Project Snow Goose Migration Specification Matrix <ul><li>“ Work with business users to define migration rules for each kind of document” </li></ul><ul><li>The entire migration was broken apart into separate lots of documents, where each lot was a collection of documents that were all handled in the same way. </li></ul><ul><li>The behavior of each lot was specified as a single line in a spreadsheet. Other tabs in the spreadsheet defined attribute mappings, lifecycle information and the like. </li></ul>
    20. 20. Project Snow Goose Migration Specification Matrix Document Group Source Docbase Content Location
    21. 21. Project Snow Goose Migration Specification Matrix <ul><li>“ Automate the conversion of the business rules” </li></ul><ul><li>The completed Specification Matrix is automatically transformed into XML and fed directly into the system </li></ul>
    22. 22. Project Snow Goose Re-architecture <ul><li>Project Snow Goose involved not only a migration, but also an information re-architecture. </li></ul><ul><li>This re-architecture involved systematically determining the target object-type, attributes, lifecycle, content and location of a migrated object, based on the object’s previous incarnation. While the source repositories held a significant amount of information, this needed to be validated for consistency and then made accessible to the migration system. </li></ul><ul><li>Many facets of this re-architecture were complex. </li></ul>
    23. 23. Project Snow Goose Examples of complexity – Assembly Rules SOP#1, 2.0, Obsolete SOPs Released SOPs SOP#1, 2.0, Current SOP#1, 3.0, Current SOP#1, 4.0, Current Master SOPs SOP#1, 4.0, Current SOP#1, 3.0, Retired SOP#1, 4.0, Current + SOP#1, 3.0, Retired + SOP#1, 2.0, Obsolete +
    24. 24. Project Snow Goose Examples of complexity – Naming Conventions A naming convention describes how to break apart names into their meaningful elements. An example of a naming convention specification is: type – pla_state – unit_size – batch_size – destination – formula The elements of the name are used in other components of the migration, for example to determine the name of the destination folder in the target repository. An example target folder specification would be specified as: formula – 0525 – batch_size An example of a name that conforms to the convention, and its target folder: ointment – production – 30mg – 600 – waco – F245 F245 – 0525 - 600
    25. 25. Project Snow Goose Reduce Complexity <ul><li>To manage complexity, and to allow the system to develop as it needed to, we adopted a model where we could develop individual components to handle a single aspect of the migration processing. </li></ul><ul><li>Each component addressed a single element of the process. </li></ul><ul><li>We then arranged these components as a ‘pipeline’, where the processing steps required for each migration lot could be executed or skipped as necessary. </li></ul>
    26. 26. Project Snow Goose Pipeline Operations <ul><li>Folder Naming Convention </li></ul><ul><li>Detect Duplicates </li></ul><ul><li>Select Lifecycle </li></ul><ul><li>Attribute Mapping </li></ul><ul><li>Cascade Attributes </li></ul><ul><li>Set Owner Name </li></ul><ul><li>Check Effective Date </li></ul><ul><li>Strip Unwanted Content </li></ul><ul><li>Post-export Selection </li></ul><ul><li>Detect Split Trees </li></ul><ul><li>Process Annotations </li></ul><ul><li>Document Name Convention </li></ul><ul><li>Set Lifecycle State </li></ul><ul><li>Perform Document Assembly </li></ul><ul><li>Remove ‘do not migrate’ objects </li></ul><ul><li>Set Paper Size </li></ul><ul><li>Set PLA State </li></ul><ul><li>Set Retired Date </li></ul><ul><li>Set Target Path </li></ul><ul><li>Separate Multi Lots </li></ul><ul><li>Set Site </li></ul>
    27. 27. Project Snow Goose Pipeline Schematic Start Component #n OK? Component #n+1 YES NO Fail
    28. 28. Project Snow Goose Inspection <ul><li>“ Allow non-technical business users to validate documents at each step of migration” </li></ul><ul><li>“ Introduce an enrichment step to allow users and/or administrators to perform meta-data enrichment“ </li></ul><ul><li>We re-cast the enrichment step into an inspection step – this allowed users and/or administrators to inspect the migration information before importing it into the target repository. </li></ul>
    29. 29. Project Snow Goose Inspection Application Source Object Information Migrated Object Information Reviewer Notes
    30. 30. Project Snow Goose Pre-import Validation <ul><li>Before attempting to import the objects, check for errors such as: </li></ul><ul><li>Invalid users </li></ul><ul><li>Unexpected file formats </li></ul><ul><li>Missing target folders </li></ul><ul><li>Missing ‘effective’ dates </li></ul><ul><li>Unwanted native content </li></ul>
    31. 31. Project Snow Goose Validation <ul><li>“ Allow non-technical business users to validate documents at each step of migration” </li></ul>Source Object Information Migrated Object Information List of Migrated Objects
    32. 32. Project Snow Goose Overall Process Flow <ul><li>“ Leverage Blue Fish DIXI (Documentum Import/eXport Interface)” </li></ul>Start DIXI Export Inspection Pipeline DIXI Import DIXI Validation Validation End
    33. 33. Project Snow Goose Scheduler <ul><li>“ Automate the process to allow repeatable test migrations” </li></ul><ul><li>Allows the Migration Engineers to control the what/when/where </li></ul><ul><li>Allows the use of multiple machines to reduce overall processing time </li></ul><ul><li>Simple to use, provides visual status feedback </li></ul>
    34. 34. Project Snow Goose Scheduler Completed Tasks Active Tasks Pending Tasks
    35. 35. Project Snow Goose Deployment Staged Local Content CORAL Repository
    36. 36. Project Snow Goose Data Cleanup Management and Other Tools <ul><li>In addition to the migration system, we also developed other tools and utilities to help with the migration effort. </li></ul><ul><li>Source Data Cleanup tools </li></ul><ul><li>Batch Management Spreadsheet </li></ul><ul><li>Daily monitoring of data hygiene </li></ul>
    37. 37. Project Snow Goose Results <ul><li>Migration Dress Rehearsals paid off </li></ul><ul><li>Focused Team - Teamwork Triumphs </li></ul><ul><li>Automation reduces error </li></ul><ul><li>Validation reduces anxiety </li></ul><ul><li>Data Cleanup a key to success! </li></ul>