Successfully reported this slideshow.
Your SlideShare is downloading. ×

Tips and tricks for complex migrations to SharePoint Online

Loading in …3

Check these out next

1 of 47 Ad

More Related Content

Slideshows for you (20)

Similar to Tips and tricks for complex migrations to SharePoint Online (20)


Recently uploaded (20)

Tips and tricks for complex migrations to SharePoint Online

  1. 1. Tips and tricks for complex migrations to SharePoint Online Office 365 and SharePoint Connect – Community Edition, October 2019
  2. 2. Are you considering, preparing or executing a large-scale migration to Office 365?
  3. 3. Expect to run into some problems along the way
  4. 4. This session aims to help you with tips and tricks to increase your project’s success rate
  5. 5. Are you in the right session? • Expect a mix of technical and tactical topics that will help you to design and execute your large-scale migration project • Large-scale migrations as in Terabytes and Petabytes • We’re talking documents, file shares and SharePoint, not Exchange • Some live-demo’s and walk-throughs • Not really a business session • Up-tempo due to time constraints (45 minutes).
  6. 6. Agenda • Introductions and objectives • Typical migration challenges • Migration means rationalization • Moving from classic to modern • Choosing the right migration tool • Bulk migrations using pipelines • Maximize automation for large-scale
  7. 7. A brief personal introduction • Working in technology since 1999 • Architect for Wortell (via ETTU) since March 2015 • Previously: Pink Elephant, PinkRoccade, Getronics, Ordina & KPN Consulting • Crazy about technology that connects people • Focus on Business & IT Alignment:  Governance & security  Use of Best practices  Certified Microsoft Office 365 technology expert  Addicted to large-scale migrations  Twitter: @awdhaan
  8. 8. Organizations have built up large collections of “Dark data” over the years…
  9. 9. Typical challenges for large-scale migrations • Organizations with multiple divisions and offices • File shares existing for a long time, supporting various processes • Also SharePoint Server environments with multiple site collections • Variety in file structures and authorizations • Difficult to track responsible owner (if any) • Decisions on what to do with the data (migrate, dispose, quarantine or …) • Protecting sensitive data (PII, financial or even medical) • Designing a futureproof target architecture in the cloud • Remapping data from source • The sheer volume of data can result in longer processing time.
  10. 10. What about business continuity?
  11. 11. Typical project approach for large-scale migrations* Discover • Identify stakeholders and requirements • Prepare tooling and access to data sources • Conduct stakeholder interviews • Start scanning data sources • …. Analyze Rationalize Design Deploy • Implement target architecture • Deploy migration solution • Perform test migrations for technical validity and to gain performance metrics • Plan and estimate the migration • … Migrate • Perform continuous migrations • Adjust and optimize • Resolve known issues and investigate new • Progress reporting • … • Analyze scan results • Review functional and security requirements • Determine potential target locations • Define business rules to classify data • … • Classify data via defined business rules • Identify migration and relocation scenarios • Estimate migration throughput • Determine ordering and priority • … • Envision future state • Design target architecture • Determine governance guidelines and security baseline • Prepare migration and onboarding process • … * Technical perspective only. Parallel activities commonly include adoption and training
  12. 12. Engage your stakeholders to set and keep things in motion
  13. 13. To the Cloud and beyond Archive, dispose or … Application or HMUD? File types Ownership Age Sensitivity Rationalizing your data is hard work
  14. 14. Tools you can use to support inventory and rationalization Tool Source Usage NTFS Permissions Reporter File share Analyze permission structures Microsoft Inventory Manager File share, SharePoint, Box, OneDrive, Google Drive Fasttrack tool: Inventory and issue reporting (use, file type and aging for file share data) Jam Software Tree Size Pro File share Analyze data (storage) structures and aging SharePoint Modernization Scanner SharePoint Provides insights on readiness and issues for modernizing classic sites SharePoint Migration Assessment Tool SharePoint Identify issues for SharePoint Server 2013 source environments Veritas Data Insights File share Inventory and analysis. Can extract presumable ownership
  15. 15. NTFS Permissions Reporter can help to determine permission structures
  16. 16. Demo: Microsoft Inventory Manager
  17. 17. Your target architecture should be futureproof • Leverage the goodness of the Microsoft Cloud:  Better control of data lifecycle using retention policies and labels  Increased protection of sensitive data using DLP and sensitivity labels a.o.  Guided provisioning and flexibility through templating  Governance for policies & standards, operations and innovation • Easier to identify ownership (person, team, division, org) • Flexible site/team structure to accommodate for changes in the organization (Hub sites FTW) • …
  18. 18. Mapping source and targets • Avoid granular permissions, follow the Office 365 Group structure and split into multiple groups/teams if required • Use sensitivity labels for item-level protection • Look for data and permission patterns • Design a naming convention for sites/teams and then design it again after validation with your stakeholders • Only migrate active data • Keep it simple. File share Finance division Teams Corporate Finance Admin Contracts Reports Archive Incoming
  19. 19. Choosing between migration tools Microsoft SharePoint Migration tool: • Basic set of migration features • Support for one direction: on-premises-cloud • Basic PowerShell module • Currently lacking support for mapping, migration of specific content • Suitable for basic migrations • Easily scalable • Free to use • Improving with each new update. ShareGate: • Lot’s of features for migrations and reporting • Rich PowerShell module • On-premises-on-premises, on-premises-cloud, cloud-on-premises, cloud-cloud • Map and migrate specific data • Copy, flatten and merge site objects • Suitable for complex migrations • Export data to file system • Migrate term sets, web parts and other artefacts • Metadata enrichment during migrations • Reasonable pricing and excellent support service • Improving with each new update.
  20. 20. Performance characteristics when using the SharePoint API for migration (Microsoft) Type of metadata Examples Maximum Light ISO files, video files 2 TB/day Medium List items, Office files (~1.5MB) 1 TB/day Heavy List items with custom columns, small files (~50kb) 250 GB /day Rules of thumb: • Large file size migrates faster than smaller ones • Files migrate faster than objects and list items • Actual throughput depends on local infrastructure, network speeds etc.
  21. 21. A single ShareGate:Desktop instance typically migrates 5-15 GB/hour with peaks up to 35 GB • Sharegate is optimized to run with 4 cores (64 concurrent threads) • Also consider pre- and postmigration activities:  SharePoint site/Team provisioning (fair)  Applying configuration (depends on complexity)  Assigning Owner and Membership (fast)  Reporting and logging (fast) • Network dependencies:  1 GB via 5/1 Mbps ADSL line takes 16 minutes  1 GB via 1 GBPS line takes 8 seconds • Number simultaneous import jobs to each separate Content Database (contact Microsoft to pre-provision content databases for > 1TB migrations) • Provisioning sites and Teams in bulk prior to starting data migrations.
  22. 22. Improving migration performance in ShareGate:Desktop • ShareGate:Desktop 12.0 can now register as an Azure application • This allows ShareGate to identify to Office 365 that it is running operations on your tenant through an access token • This will reduce throttling on tasks between Sharegate and Office 365 • Also grants ShareGate rights to use certain Microsoft functionality, such as the Graph API • Global Administrator role required for consent of the permissions.
  23. 23. Azure Active Directory App registration
  24. 24. Modernizing classic SharePoint sites • Converting classic artefacts to modern experiences • Why? Duh! Modern is simple, responsive and benefits from ongoing Microsoft investments • Some basic principles:  When moving to the cloud, only migrate data  When already in the Cloud, modernize what’s classic • SharePoint Patterns and Practices (PnP) are your best friends:  Guidance:  Toolkit: • Start with gaining insights on modernization readiness and issues.
  25. 25. The Modernization Scanner is awesome • Only for SharePoint Online • Use via GUI or command line • Scope to entire tenant, csv or individual sites • Configure specific scans or perform a full scan • Various authentication options • Detailed reports in Excel and csv.
  26. 26. Demo: Modernization Scanner
  27. 27. Modernize by connecting to an Office 365 group • Not supported for classic publishing portals • Upon “Groupify”:  A new Office 365 group is created, and connected to the site A new modern home page is created and set as home page  Group owners are now the site collection admins  Group owners are added to the site's Owners group  Group members are added to the site's Members group • Options to perform “Groupification”:  Self-service on the site (“Connect to new Office 365 Group”)  Via PnP PowerShell: Add-PnPOffice365GroupToSite.
  28. 28. Demo: Modernizing by Groupification
  29. 29. Options for modernizing SharePoint pages PnP PowerShell conversion: • Great for centrally managed scenarios • Superfast and easy to deploy • Integrates into a migration pipeline • Mapping files for web parts and layouts • Built-in logging. PnP Page Transformation UI solution: • Great for self-service scenarios • Fully deploy using a provided script • Best for non-customized environments • Feedback option • Enable or disable on a per site basis.
  30. 30. Demos: SharePoint page conversion
  31. 31. Main script Create a migration pipeline with all required functions {f} {f} {f} {f} {f} csv input Logging, reporting <xml> <xml> mapping file(s)
  32. 32. Designing a migration pipeline • Design each function with the expected input variables • Securely cache source and destination credentials (encrypted xml, Windows Credential Manager) • Accommodate for nesting of functions (e.g. copy-data in sub sites) • Handle errors if e.g. artefacts already exist • Functions are called through a “main” script and fed by input files (to-do, mappings, configs).
  33. 33. Demo: Anatomy of a migration pipeline
  34. 34. Complexity increases when the numbers go up • IT does not have the full insights on ownership, and they can’t make decisions for data at scale. And they shouldn’t anyway! • Bigger organizations have multiple divisions each with different rhythms. Planning migrations becomes difficult at scale • Complexity of merging or splitting data and other transformations • Multiple sources: File shares, SharePoint • Processing time of all repeating tasks are multiplied by the amount of data: e.g. 1 min. x 20.000 migrations already takes ~ 14 days.
  35. 35. For large-scale, even common actions take time • Creating and continuously updating inventories • Typical actions like creating mappings or input files • Logging success and failure • Scripting solutions for known issues • Cleaning and customizing • Liaising with data stewards to resolve data issues • …
  36. 36. Automate to epic proportions using a Migration Factory • Build and (incrementally) refresh data source inventories • Provide self-service to the business to:  Determine and claim ownership  Help resolve data related issues  Enrich inventories with sensitivity info  Prepare and approve decisions on what to do with the data  Choose an appropriate migration scenario  Allow for opt-in on scheduled migration batches  (Dis)approve migration outcome  Lock source environment and display banners • Be flexible in supporting different business requirements* • Scale up for increasing demand • Optimize the process of solving problems • Realtime reporting on progress and issues.
  37. 37. Supporting multiple migration scenarios • Each migration scenario may contain multiple steps to:  Provision SharePoint Online sites or Teams (PnP)  Apply specific configurations (Site Design, PnP)  Migrate data (ShareGate:Desktop)  Perform checks and compares  Notify data stewards to approve migration outcome  Report output of each step back to the central database • PowerShell for a high degree of flexibility and extensibility • Standardized by function approach and shared elements • Use a repo for source control.
  38. 38. High level design of a Migration Factory Migration database Migration Administrator Stakeholders SharePoint 2013 SharePoint 2010 Filers Inventory clientInventory client Migration portal C#js Migration servers PS SharePoint Online Microsoft Teams Reporting Migration AIP-Scanner Disposition Inventory client Decision n Data Owner(s) Data Steward(s)
  39. 39. Walkthrough: Migration Factory portal
  40. 40. Key takeaways of this session • Complexity increases with increasing amounts of data • The business owns the data, support them to rationalize and facilitate them to migrate their own stuff • large-scale is all about optimizing the process • Current technology allow for a high degree of automation • Keep things simple when the numbers go up. Move first, adjust later! The nitty gritty stuff will slow you down • The PnP Community is simply awesome!

Editor's Notes

  • Gartner defines dark data as the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships and direct monetizing). Similar to dark matter in physics, dark data often comprises most organizations’ universe of information assets. Thus, organizations often retain dark data for compliance purposes only. Storing and securing data typically incurs more expense (and sometimes greater risk) than value.
  • PII - Personally identifiable information
    PHI - Protected Health Information
  • Plan frequent update sessions and keep them short and sweet;
    In case of issues that require stakeholder approval, provide options and advice
    Share progress reports and elaborate during update sessions.
  • Multiple scan (inventory client) and migration servers
    Single central database containing all (meta)data