14 tips for planning a ecm content migration to share point


Published on

Many organizations that have deployed SharePoint 2010 or 2013 also have some type of legacy ECM system in place.

However, SharePoint also has rich ECM capabilities with many innovative features not available in other ECM systems. As a result, we have found that many companies that use SharePoint wish to consolidate their IT infrastructure, by migrating content from older ECM systems into SharePoint.

While we think this is a great idea, such a migration requires a good deal of planning and design as well as specialized tools to automate the migration.

This slide deck provide detailed best practice guidance on how to plan for a migration of content from nearly any ECM system into MS SharePoint 2010 or 2013.

Published in: Technology
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • At Hershey Technologies, we’ve been providing IT solutions for more than 20 years. We have deep, end-to-end SharePoint consulting and development expertise, and we specialize in document capture, document management and workflow solutions!

    We’re also resellers for a number of major SharePoint ISVs such as Nintex, AvePoint, and Vizit. We’re also a Diamond partner with Kofax.
  • I’d like to note that today’s webinar is not going to be a “product pitch”. The content today will focus on how to plan a migration of content from another ECM system to SharePoint. Although I will reference a few software tools that can automate your migration, in my experience the success of most migrations to SharePoint has more to do with how SharePoint is architected and how the migration process was planned than it has to do with the tools used to perform the migration.

    I’d like to make today’s presentation more interactive than a typical webinar. So please chime in with any questions using the “chat” window. The content today assume that you have some familiarity with SharePoint. However, if you would like me to explain further about specific SharePoint features and how they might be relevant to your migration plans, please let us know through the chat window.
  • Before we get started, I want to ensure our terms are clear.

    In the SharePoint community, the term “migration” can have different meanings.

    Often people use it to refer to an upgrade of SharePoint version X to version Y. However, for this presentation, I am referring to migration of content (documents and metadata) from some other ECM (non-SharePoint) platform into SharePoint
  • There has been a great deal of consolidation in the ECM market. Many of the larger vendors have purchased smaller competitors. In many cases, the new owner’s primary goal is not to continue innovating on the new product, but to push customers to move to main product line. A classic example of this is EMC, who owns Documentum and ApplicationXtender. Customers on the AX platform tell us how there are no longer new features added, and EMC wants them to move to Documentum.

    But if you are an AX shop and you already have a mature SharePoint installation, its far more cost effective to migrate to SharePoint then Documentum.
  • Over the last few years we have seen increased interest from our clients in migrating content from what I’ll call “legacy” ECM products to SharePoint.

    So if you are in this webinar, I presume that you already are using SharePoint or that your company is already planning to use SharePoint. The purpose of this presentation is focus on “how” to plan your migration, not “why” you should migrate to SharePoint.

    That being said, I’ll review some of the key reasons why so many companies are deciding to migrate ECM content to SharePoint…
  • Because of SharePoint’s enormous success, many ECM vendors have struggled to keep up. Some products that were previously strong players are being formally deprecated (e.g. ECM’s ApplicationXtender).

    Some products have huge annual maintenance costs

    Once an organization already has a mature SharePoint environment, it’s often a no-brainer IT decision to consolidate content from legacy ECM products into SharePoint.

    Besides reducing annual maintenance costs, by consolidating number of systems that IT needs to support, internal IT staffing costs can be reduced as well.
  • Many legacy ECM products lack taxonomy design features that are core to SharePoint, so the existing taxonomy in the legacy ECM system may have been based not on ‘best practice’ but on what features were available. SharePoint features like Content types, site columns, managed metadata, BCS and Lookup columns allow us to design taxonomies that are far more elegant and functional than most other ECM products.

    Many legacy ECM products either don’t have Records Management features or support RM as a separate ‘bolt on’ product, which may or may not be well integrated with the core ECM solution.

    In cases when you wish to customize your ECM solution, SharePoint offers far more flexibility than any other ECM product… either through native SharePoint features, custom development or the huge market of Off-the-shelf 3rd party solutions.
  • Content Types in SharePoint let us group related metadata columns into a re-usable template. This allows us to de-couple the metadata schema from document libraries, and gives us flexibility that many other ECM products don’t have.

    A content type can be re-used in multiple document libraries and conversely, a single document library can store documents of different content types.
  • Sometimes a legacy ECM system may have a small number of very large repositories that ideally should be re-factored into multiple SharePoint libraries. Some document management systems might not allow a unified search across multiple repositories, and so documents get grouped into a single large repository to simplify the end user search experience.

    This approach often forces systems to use document level security models which may reduce the scalability of the system. There may be business requirements to separate documents into various document libraries, by department, business unit or some other metadata attribute.

    Although SharePoint also supports item-level permissions, using item-level permissions can limit scalability. So if your current system is using item level permissions, consider refactoring the system into multiple repositories to avoid use of item level permissions.

    When your ECM solution in SharePoint is used for archive (read-only) purposes only, SharePoint can be support millions of documents in a single document library. But if your documents are also used in collaboration (read-write) scenarios or you have workflow processes on your documents, you need to architect your SharePoint taxonomy to keep the number of documents per library to much smaller numbers, which may require creating more libraries that you have in your legacy ECM system.
  • Many ECM document repositories have a metadata choice field named “Document Type”, which is typically a drop down list (choice) field of possible document types. This design can be migrated directly to SharePoint. However, sometimes documents should be treated differently based on the “document type”. For example, say we have repository for “contracts”, with a field named “contract type” containing options for: NDA, MSA, Maintenance Agreement, Partnership agreement, etc. Perhaps, the MSA contracts should have a different retention period than other contracts, and maybe Partnership Agreements require a different approval process than the others.

    In this case, we can define a new “Contract” content type in SharePoint which contains any other contract metadata fields (except for document type). Then we can create additional content types for NDA, MSA, Maintenance Agreement, Partnership agreement – each of these will inherit from the Contract content type, so they all share the same metadata. But we can define unique retention policies, workflow processes and templates for each one.
  • Many ECM document repositories have a metadata choice field named “Document Type”, which is typically a drop down list (choice) field of possible document types. This design can be migrated directly to SharePoint. However, sometimes documents should be treated differently based on the “document type”. For example, say we have repository for “contracts”, with a field named “contract type” containing options for: NDA, MSA, Maintenance Agreement, Partnership agreement, etc. Perhaps, the MSA contracts should have a different retention period than other contracts, and maybe the MSAs require a custom approval process than the others.

    In this case, we can define a new “Contract” content type in SharePoint which contains any other contract metadata fields (except for document type). Then we can create additional content types for NDA, MSA, Maintenance Agreement, Partnership agreement – each of these will inherit from the Contract content type, so they all share the same metadata. But we can define unique retention policies, workflow processes and templates for each one.

    We can now these content types in the same document library or multiple document libraries, and the documents will adhere to the retention rules and workflow processes defined by their content type.
  • All legacy document management systems support “choice fields”, where metadata values are constrained to a list of values, typically presented to users as a drop down list. Of course, SharePoint supports Choice fields as well. But SharePoint has several other options, which could provide long term benefits for certain use cases:
    Managed Metadata: If the options in a choice field need to be re-used in many different use cases and across many SharePoint sites and site collections, or if the options in a choice field are hierarchical in nature, then consider defining columns using Managed Metadata.
    Lookup columns: behave similar to a choice field, except that the list of choices are sourced from a SharePoint list that is in the same site as the document library. Lookup columns are great when you want to enable end users to dynamically add, remove or change choices available for that field, but the choices don’t need to be re-used across multiple sites.
    BCS (External Data): behave similar to Lookup columns, except that the values are sourced from an external database, rather than a list in SharePoint. Columns based on external data can be re-used in multiple sites and site collections.

    Note that all of these options complicate the migration process. So these options should not be considered lightly. Remember that your migration is a one time project, but your taxonomy lasts forever. So weigh whether the long time benefit of these features outweighs the short term costs to use these features.
  • Sometime users have created multiple repositories that have the same logical fields. Some ECM systems don’t support column re-use, and therefore certain columns may have been “manually copied” or re-created in multiple repositories. If users were not very careful, columns that should have shared the identical name were given slightly different names in each repository.

    By creating “Site Columns” instead of “list columns”, we can make sure that columns are properly re-used to create a consistent environment. Mistakes happen! But there’s no reason not to correct those mistakes before starting your migration to SharePoint.
  • Sometimes however, the opposite problem can occur. Where multiple repositories each contain a field with the same name, event though the field has a different purpose within each repository. The most common example of this is the “Document Type” field. Every repository has a field with this name, but the available ‘doc type’ choices in each are different in each case.

    This leaves us with two options:
    Leave the naming convention as is, and create new list level choice columns in each document library, and copy the same values to each library. However, this means we sacrifice all of the re-usability benefits of SharePoint site columns.
    Rename the columns to be more specific and create them as site level columns, which allows us to re-use the identical field settings in multiple libraries. This approach is a vastly improved design!
  • This information is important to know for a few reasons –
    It may impact your choice of migration tools and the approach used for the migration
    Depending on the file types, you’ll need to make sure that you have integrated viewers and editors for various file formats with SharePoint
  • This information is needed to help architect your SharePoint farm:
    Number of site collections and content databases
    Create one per repository
    Put all repositories in one site collection
    Group certain repositories in one site collection and others in a different site collection
    Warrant an additional Physical SQL server (or multiple SQL servers)
    Storage (SAN, NAS, IOPS)
  • Don’t assume that the current capture system will (or should) work exactly the same way with SharePoint as it does with the legacy ECM system. In many cases, you will be able to keep the identical capture process, and simply swap out an “export connector” for the legacy system with a new connector for SharePoint… but this isn’t always the case. Depending on your SharePoint taxonomy, your capture system may or may not have the features needed to support all of the relevant SharePoint features (e.g. columns types, such as Managed Metadata, BCS and Lookup columns).

    If you are using Choice fields, Managed Metadata, Lookup or BCS columns in SharePoint, think about how you will sync these terms with your document capture system, For example, Kofax Capture allows users to reference Managed Metadata termsets from SharePoint while they are validating documents. So the Kofax admins don’t need to worry about syncing the SharePoint terms with Kofax.

    Make sure you inventory all of the channels that can deliver content to your legacy ECM system, and validate that each capture sub-system is compatible with SharePoint.

    Be on the lookout for any specialized or custom document indexing processes, such as “One to many indexing” where a single document may contain many related metadata records.
  • When possible, avoid using item level security in SharePoint, especially for larger libraries or libraries that will have a great deal of unique ACLs, as this can inhibit scalability. SharePoint supports up to 50,000 unique ACLs per library. If your current repository is using item level security, consider refactoring it. Note that SharePoint’s security model supports very granular permission configurations. Be sure to document your legacy ECM system’s security model, so that SharePoint can be configured to in the same manner. Optionally, you may find that the current security model no longer reflects your business’ requirements.

    Use the migration as a good time to re-architect your security if needed.
  • Make sure you understand and document exactly how users interact with your current ECM system. Site down and watch how users really do their job on a daily basis. This is especially important for any users that use your ECM system on a daily basis to get their primary job done.
  • What do users do with the documents after they find and open them?

    If its more than read-only, consideration must be given to the file types (PDF, Office, TIF, etc.). While SharePoint treats office documents as a 1st class citizen, enabling all edit processes easily… when it comes to other file types, any but read-only operations may be a challenge unless you have integrated specialized components to support the editing operations that you require. Hershey recommends use of XenDocs Search and Vizit Pro to provide a user experience in SharePoint that closely resembles other legacy ECM products.

  • Be sure to inventory any workflow processes in use in your legacy ECM system. If it has any workflow components, these workflow need to be re-created in SharePoint. While SharePoint Designer is a great tool to build simple workflows, we typically recommend Nintex Workflow for its powerful, and easily workflow design features combined with reporting and metrics.
  • Also, be sure to inventory any integration points with your legacy ECM system. Perhaps your Accounting or ERP application has links to documents in your legacy ECM system. You may have custom reports or dashboards that pull metadata from your current system, or maybe the metadata from your current ECM system is exported to other applications to drive some other process. Be sure to document any integration points so that these can be re-created in SharePoint.
  • There are two major features that most document management systems have:
    -intuitive metadata filtering/query builder
    -search results displayed in a tabular grid

    SharePoint’s native metadata filtering displays resulting a grid, but it doesn’t support column types like single line of text, and it doesn’t scale well in document libraries with more than 5000 documents.

    SharePoint Search provides the advanced search web part, but this can be challenging to configure and formats results like a search engine, rather than in a tabular display.
  • XenDocs Search provides a user experience that is similar to virtually all other document management systems, including intuitive metadata filtering, tabular search results, document sorting, document previews, etc.

    XenDocs allows SharePoint users to have users the “best of both worlds” – all of the web content management and document collaboration features AND a great solution for managing transactional document.
  • There are many migration tools for SharePoint available. Depending on your source ECM system you may find an off-the-shelf migration product that meets all of your needs for a low cost. In many cases, some level of customization may be needed. Features to look for include:
    Field mapping, in case columns in SharePoint are different from the legacy system.
    Ability to migrate sub-sets of documents in filtered chunks
    Ability to Start/Pause/Stop/Resume a migration
    Extremely robust auditing of every document that is successfully migrated or that encounters an error. This is critical to troubleshooting any migration problems. This data should be stored in a database, not exported to a text file.
    File Conversions (especially to PDF, very common!)
    Support for advanced SharePoint column types (e.g. MMS, Lookup, BCS). However, there are work-arounds in SharePoint for transforming data from text columns to other column types, so if the migration tool doesn’t support this, its not a huge problem. Either way, you need to know what column types are supported.
  • Most content migration tools fall into one of the following three models:

    A Flat fee for the software per machine. Note that most migration tools do not install on your SharePoint server. You may be able to install the software on multiple machines to migrate multiple content sources in parallel

    Subscription based – charge a fee per month or annually

    Size based – charge a fee per GB of content

    Annual Support – many migration tools do not charge an annual support fee as it is assumed most migrations shall be completed in less than 1 year. However, so migration tools can be used for ongoing projects.
  • Our XenDocs ECM Migration toolset is comprised of two major components:
    Export modules, which exports batches of content from your legacy ECM systems, such as ApplicationXtender, FileNet, File Magic, Liberty, Sire, LaserFiche, etc.
    File Importer for SharePoint which imports these batches into the appropriate sites, libraries, content types and columns. The File Importer can also perform image processing on your documents during the migration, such as OCR, convert to searchable PDF, auto-rotate upside down pages, etc. This tool can also be used after your migration project to integrate multi-function scanners, fax servers or other line of business applications with SharePoint on an ongoing basis.

    If you have a migration project in mind we’d be happy to provide you with a personalized demonstration.

  • 14 tips for planning a ecm content migration to share point

    1. 1. 14 Tips for Planning an ECM content migration to SharePoint
    2. 2. Agenda • Overview of SharePoint ECM • Why migrate content from legacy ECM systems into SharePoint? • 14 Planning tips for ECM content migration to SharePoint
    3. 3. About Tom Castiglia • Principal / Director of SharePoint Consulting Services • Started with HersheyTech in 1998 • Active in the SharePoint community • Speaker at various SharePoint Saturday conferences • President of the San Diego SharePoint User Group • Contact Me • Twitter: @TomCastiglia • tom@hersheytech.com
    4. 4. Why Migrate to SharePoint?
    5. 5. Common Legacy ECM Applications EMC - Documentum • D5/D6/D7 • ApplicationXtender Hyland • OnBase • Liberty Fortis • Fortis • File Magic FileBound LaserFiche Hershey Technologies • XenDocs Content Server Xerox • DocuShare Canon • ImageWare IBM – • FileNet • Content Manager OpenText Perceptive Software • ImageNow
    6. 6. Why SharePoint for ECM? Powerful taxonomy features Managed metadata Site content types and columns Business Connectivity Services Lookup columns Document Sets Powerful search engine Workflow Versioning Check-In / Checkout Records Management eDiscovery and Holds Site Mailboxes Content Organizer Rules Metadata Navigation / Filtering Partner Eco-system IT Consolidation
    7. 7. Why Migrate Content to SharePoint? • Many legacy ECM products have been deprecated • ECM ApplicationXtender is being phase out. EMC is pushing clients to upgrade to Documentum D6/D7 which is much more expensive • File Magic from Westbrook Technologies was discontinued in 2006 to favor a new product called Fortis. • Many other vendors simply have stopped innovating their products • Consolidate IT infrastructure • Reduce maintenance costs • If IT staff is already maintaining a mature SharePoint farm, do they really need to also maintain a separate document management system?
    8. 8. Why Migrate Content to SharePoint? • Take advantage of improved taxonomy features in SharePoint • Content Types • Site Columns • Managed Metadata • BCS • Lookup columns • Take advantage of Records Management features in SharePoint • Take advantage of SharePoint’s flexibility in general • SharePoint is more extensible and customizable than any other ECM product • SharePoint has largest market of 3rd party solutions to provide additional features
    9. 9. SharePoint Migration Planning Tips
    10. 10. Tip # 1 – Consider splitting one legacy repository into multiple SharePoint libraries Eliminate the need for item level permissions Reduce number of documents per library (improves performance) SharePoint and XenDocs allows unified search across multiple libraries
    11. 11. Tip # 1 – Take advantage of SharePoint Content types Vendor PO # Invoice # Division Alpha, LLC 3456617 74584 ACSC Bravo, Inc 3456633 88363 ACMO Charlie Co. 3456641 56546454 ACSC Alpha, LLC 3456648 74584 ACSC Delta Signs 3456652 675676 ACTX Echo Ink 3456661 INV-324454 ACTX Bravo, Inc. 3456670 456546464 ACMO Vendor PO # Invoice # Alpha, LLC 3456617 74584 Charlie Co. 3456641 56546454 Alpha, LLC 3456648 74589 ACSC Invoices Vendor PO # Invoice # Bravo, Inc 3456633 88363 Bravo, Inc. 3456670 456546464 Vendor PO # Invoice # Delta Signs 3456652 675676 Echo Ink 3456661 INV-324454 ACMO Invoices ACTX Invoices Legacy Repository SharePoint Libraries
    12. 12. Tip # 2 – Consider converting legacy “document types” into SharePoint Content Types. Content Types Inheritance Workflow Processes Retention Policies Custom templates Legacy Document Types
    13. 13. Tip # 2 – Consider converting legacy “document types” into SharePoint Content Types. Document Contract NDA Retention: 5 Years Workflow: none MSA Retention: 7 Years Workflow: Approval Fields PartyName ContactNumber EffectiveDate ExpirationDate Fields Name Title
    14. 14. Tip # 3 – Consider converting “choice” fields into Managed Metadata, Lookup or External data columns Managed Metadata Lookup External Data (BCS) Hierarchical YES NO NO Reference other columns NO YES YES Scope Farm Site Collection Enterprise
    15. 15. Tip # 4 – Improve column name consistency Legacy ECM System Repository Field NameColumn Auto Policies PolicyNumber Medical Policies PolicyNo PolicyNumber Policy No Policy# SharePoint Site Policy Number Home Policies Policy#
    16. 16. Tip # 5 – Fix Improper Column Re-Use Repository Field Name DocType Values Policies DocType Application SharePoint Site Column Renewal Cancellation Legacy ECM System “Invoice DocType” Invoices DocType PO Invoice “Policy DocType” Non-PO Invoice Expense Report Check Request Claims DocType Accident Report Estimate Doc Type “Claim DocType”
    17. 17. Tip # 6 – Understand file types in legacy ECM system Scanned Documents PDF • Searchable • Image-Only TIF • Multi-Page (1 file per document) • Single-Page (multiple files per document) MS Office Word Excel PowerPoint Engineering Auto-Cad • dwg Email msg eml Multimedia Audio • mp3 • wmv • wav Video • avi • mpeg • mp4
    18. 18. Tip # 7 – Understand metrics in legacy ECM system Total doc count in each repository Average page count per document (or total page count per repository) Average and peak file size per repository Growth rates per document type or repository (# new docs/month) Frequency of document retrievals per day per repository
    19. 19. Tip # 8 – Understand document capture use cases Do users validate new documents after they have been exported to the document management system? How do we sync terms between the ECM system and the document capture system? Does the current document capture system support exporting documents and metadata to SharePoint? How is content received? (dedicated scanners? fax? file import? MFPs? email?) Are there any specialized indexing scenarios (e.g. One to Many data)
    20. 20. Tip # 9 – Understand ECM security model By Repository Document Level Security
    21. 21. Tip # 10 – Understand how documents are found How many docs are returned in typical query? Save queries for re-use? Search metadata vs. full-text (or both) What business condition cause users to search for a document (e.g. customer service inquiry, vendor payment, etc.) How many times per day is repository queried? How many users frequently search for documents?
    22. 22. Tip # 10 – What do users do with documents? Read-Only • View • Print • Email Edit Metadata Annotate • Sticky notes • redactions • shapes • lines • Stamps Edit pages • Rotate • Re-Order • Clean-up • Insert • Delete
    23. 23. Tip # 11 – Understand any workflow processes related to the documents Ad-Hoc routing Formalized workflow
    24. 24. Tip # 12 – Understand any application integration with current ECM system Are there any Line of Business apps that have links to documents in current ECM system? Are there any custom reports created on data stored in current ECM system? Is data from current ECM system ever exported to other systems?
    25. 25. Tip # 13 – Consider some 3rd party ECM add-ons Metadata-based search is less intuitive SharePoint formats Search results like a “search engine”, not a DMS. SharePoint metadata filtering does not scale for large libraries
    26. 26. Tip #13 – XenDocs ECM Search With Vizit™ Intuitively build precise document queries Sortable search results Image previews for scanned images
    27. 27. Tip # 14 – Understand migration tool options • Existing 3rd party tools vs Custom tools • Field mapping (different field names between source system and SharePoint) • Filters to allow migrations to be performed in chunks • E.g. Only migrate documents where DocType=‘Expense Report’ and DocDate>’02/13/2009’ • Start/pause/stop/resume – • not needed for smaller repositories, but critical for larger ones • Ability to audit status of every record • File Conversions • Single Page TIF to Multi-Page TIF • TIF to PDF • Image only PDF to Searchable PDF • PDF to PDF/A
    28. 28. Tip # 14 – Migration tool pricing • Pricing Models for SharePoint Content Migration Tools • Per PC or server • Subscription based (monthly or annual) • Size based (Per GB of content)
    29. 29. XenDocs Migration Tool
    30. 30. In closing • If you need assistance in planning an ECM content migration , note that we can offer 2 hours of complementary consulting services to assist with this or to provide a demo of various migration tools. • Upcoming webinars • Tomorrow at 9:30am – Using Kapow for web content migration • Sept. 24th on AP Invoice Automation