Office365 / SharePoint:
A File Migration Framework
Victoria Office 365 Users Group
January 2016
Gerry Brimacombe, Lightlever Systems Inc.
Servers
(M: K: etc)
DropBox and
other sources
Personal Shares Copy / move
What are we trying to accomplish?
People work
here Instead
So What’s the BIG DEAL?
• EASY: Move or copy files to SharePoint!
• Is it that EASY?
• Files from where? Servers, local drives, DropBox, legacy
intranet, email, websites?....
• All files or just some?
• Just files, or other content?
• Are file names compatible? Or desirable (e.g. spaces and other
conventions)?
• Are folders too deep? Are we even using folders? [insert two
hour discussion on metadata vs. folders here]
• What about changing people’s behaviour?
File Migration – NON TECHNICAL ISSUES
• Sheer dumb Volume!
• Messy / deep folder structures.
• People are busy
• Resistance to Change
• Network speeds
• Setting Metadata
File Migration – TECHNICAL ISSUES
• Invalid characters in filenames
• Undesirable Spaces in files names
• Path or Filename too long
• Blocked file types (*.json, *.soap, etc)
• Limitations of OneDrive for Business (5k
files per library, 20k total)
• File size (SP2013 default 250MB; max
2GB)
• Modified dates not migrated
Max Folder
Length
250
Max Filename
Length
256
(128 for SharePoint 2013)
Max Folder+File
Length
250
Undesirable
Characters
#
&
*
/
{
|
?

}
~
+
<
>
%
`
...
..
Migration Plan
Approve
Plan
Capture
File
Listings
Select
Subset of
Files
Copy to
Staging
Clean Up
in Staging
Move to
Share-
Point
Set
Servers
Read-only
Add
Metadata
File Migration Tips
• This will take time!
• Must involve the users or becomes GIGO
• Create Staging area to allow users to “clean-up”
without impacting day-to-day
• Note the date you copied to staging
• Delete old versions
• Create newer, better, flatter folder structure
• Standardize names
• Move files in from a number of locations
• Create document libraries, with permissions!
• You can often “net use” to map a drive to SharePoint document library, then
xcopy or robocopy work well
• Lock down servers, but don’t delete so historical data is there (comfy feeling).
Setting Metadata
• Expectations Management: this will be onerous, but
important!
• Do not set any metadata as MANDATORY prior to file load.
• If using folders with Metadata, can you set defaults by
folder? (Column Default Value Settings)
• After loading files, use Quick Edit (datasheet view) to set lots
of data.
• Change Metadata to Mandatory as required/desired
Servers
(M: K: etc)
DropBox and
other sources
Personal Shares Copy / move
Remember This?
People work
here Instead
Servers
(M: K: etc)
DropBox and
other sources
Staging
(temporary)
Personal Shares move
Clean-up

copy
Here’s one Way
Check/Set
Metadata

Migration Tools(Not an exhaustive list, by any means)
Tool Pros Cons
EasyFix Tool to rename files Simple
Free
Fairly limited
Drag and Drop in Browser Super Easy
Free
Can’t drop folders
Lose modified dates
Drag and Drop using Windows
Explorer
Easy
Free
Lose modified dates
Hard to capture errors (recopy)
100 files at a time
Lightlever Excel-based tool Pretty easy
Free
Can capture Log
Can rename files
Lose modified dates
Can be confusing
Challenges with WebDAV connections
ShareGate Lots of power and functionality
Quite User Friendly
Retains modification dates
Annual license (US$4k/year/user)
AvePoint, Metalogix, Metavis,
etc.
Super powerful
Many features
Incremental migrations
etc.
Very Expensive
Can be complex to set up
Features we don’t need (migration workflows,
etc)
www.lightlever.ca
gerry@lightlever.ca
BUSINESS ANALYSIS
 Requirements gathering
 Business case for cloud IT
 Improve business processes
 Evaluate technology
SHAREPOINT SOLUTIONS
 Corporate portals
 Document management
 Custom lists and libraries
 Electronic forms
 SharePoint Training
PROJECT MANAGEMENT
 Develop project plans
 Run projects efficiently
 Build project management office
 Project Management Coaching
Other Content Migration Examples
• Contact List
• Copy from AD
• Copy/Paste in SharePoint
• Or Drag and Drop in Outlook
• Policy Manual or Employee Handbook
• Word to Wiki in 1 hour
• See how to blog post
Demo of Migration Tool
• Command prompt file dump
• Set variables and transformations
• Filter and Sort
• Select files / folders to keep
• Produce and run command files
File Migration Using Excel Tool
• Command Line Directory Listing: Paste into
Excel file migration tool
• Tool strips out invalided characters (including
spaces if you want)
• Changes to CamelCase, where spaces present.
• Shows file and path length
• Can filter records by date, size, type,
folder, etc.
• Generate Commands
• Put commands in Batch file, and run
References
• https://support.office.com/en-ca/article/invalid-characters-in-file-or-
folder-names-or-invalid-file-types-in-onedrive-for-business-
64883a5d-228e-48f5-b3d2-eb39e07630fa
• https://support.office.com/en-ca/article/Types-of-files-that-cannot-
be-added-to-a-list-or-library-30be234d-e551-4c2a-8de8-f8546ffbf5b3
• http://sharepoint-community.net/profiles/blogs/sharepoint-
comparison-matrix-for-3rd-party-migration-tools
• http://lightlever.ca/convert-document-to-sharepoint-wiki/

A SharePoint File Migration Framework

  • 1.
    Office365 / SharePoint: AFile Migration Framework Victoria Office 365 Users Group January 2016 Gerry Brimacombe, Lightlever Systems Inc.
  • 2.
    Servers (M: K: etc) DropBoxand other sources Personal Shares Copy / move What are we trying to accomplish? People work here Instead
  • 3.
    So What’s theBIG DEAL? • EASY: Move or copy files to SharePoint! • Is it that EASY? • Files from where? Servers, local drives, DropBox, legacy intranet, email, websites?.... • All files or just some? • Just files, or other content? • Are file names compatible? Or desirable (e.g. spaces and other conventions)? • Are folders too deep? Are we even using folders? [insert two hour discussion on metadata vs. folders here] • What about changing people’s behaviour?
  • 4.
    File Migration –NON TECHNICAL ISSUES • Sheer dumb Volume! • Messy / deep folder structures. • People are busy • Resistance to Change • Network speeds • Setting Metadata
  • 5.
    File Migration –TECHNICAL ISSUES • Invalid characters in filenames • Undesirable Spaces in files names • Path or Filename too long • Blocked file types (*.json, *.soap, etc) • Limitations of OneDrive for Business (5k files per library, 20k total) • File size (SP2013 default 250MB; max 2GB) • Modified dates not migrated Max Folder Length 250 Max Filename Length 256 (128 for SharePoint 2013) Max Folder+File Length 250 Undesirable Characters # & * / { | ? } ~ + < > % ` ... ..
  • 6.
    Migration Plan Approve Plan Capture File Listings Select Subset of Files Copyto Staging Clean Up in Staging Move to Share- Point Set Servers Read-only Add Metadata
  • 7.
    File Migration Tips •This will take time! • Must involve the users or becomes GIGO • Create Staging area to allow users to “clean-up” without impacting day-to-day • Note the date you copied to staging • Delete old versions • Create newer, better, flatter folder structure • Standardize names • Move files in from a number of locations • Create document libraries, with permissions! • You can often “net use” to map a drive to SharePoint document library, then xcopy or robocopy work well • Lock down servers, but don’t delete so historical data is there (comfy feeling).
  • 8.
    Setting Metadata • ExpectationsManagement: this will be onerous, but important! • Do not set any metadata as MANDATORY prior to file load. • If using folders with Metadata, can you set defaults by folder? (Column Default Value Settings) • After loading files, use Quick Edit (datasheet view) to set lots of data. • Change Metadata to Mandatory as required/desired
  • 9.
    Servers (M: K: etc) DropBoxand other sources Personal Shares Copy / move Remember This? People work here Instead
  • 10.
    Servers (M: K: etc) DropBoxand other sources Staging (temporary) Personal Shares move Clean-up  copy Here’s one Way Check/Set Metadata 
  • 11.
    Migration Tools(Not anexhaustive list, by any means) Tool Pros Cons EasyFix Tool to rename files Simple Free Fairly limited Drag and Drop in Browser Super Easy Free Can’t drop folders Lose modified dates Drag and Drop using Windows Explorer Easy Free Lose modified dates Hard to capture errors (recopy) 100 files at a time Lightlever Excel-based tool Pretty easy Free Can capture Log Can rename files Lose modified dates Can be confusing Challenges with WebDAV connections ShareGate Lots of power and functionality Quite User Friendly Retains modification dates Annual license (US$4k/year/user) AvePoint, Metalogix, Metavis, etc. Super powerful Many features Incremental migrations etc. Very Expensive Can be complex to set up Features we don’t need (migration workflows, etc)
  • 12.
    www.lightlever.ca gerry@lightlever.ca BUSINESS ANALYSIS  Requirementsgathering  Business case for cloud IT  Improve business processes  Evaluate technology SHAREPOINT SOLUTIONS  Corporate portals  Document management  Custom lists and libraries  Electronic forms  SharePoint Training PROJECT MANAGEMENT  Develop project plans  Run projects efficiently  Build project management office  Project Management Coaching
  • 13.
    Other Content MigrationExamples • Contact List • Copy from AD • Copy/Paste in SharePoint • Or Drag and Drop in Outlook • Policy Manual or Employee Handbook • Word to Wiki in 1 hour • See how to blog post
  • 14.
    Demo of MigrationTool • Command prompt file dump • Set variables and transformations • Filter and Sort • Select files / folders to keep • Produce and run command files
  • 15.
    File Migration UsingExcel Tool • Command Line Directory Listing: Paste into Excel file migration tool • Tool strips out invalided characters (including spaces if you want) • Changes to CamelCase, where spaces present. • Shows file and path length • Can filter records by date, size, type, folder, etc. • Generate Commands • Put commands in Batch file, and run
  • 16.

Editor's Notes

  • #8 May or may not use “staging” area – we did to give them more control Copy to staging / Move to SharePoint Servers are untouched, just readonly, at the end which gives people comfort level We named Data Steward to assist with this, in their own areas
  • #10 Contacts – from AD export, then cut-paste Can also use Outlook Drag-Drop Handbook – blog post on how to do this in an hour.
  • #15 Contacts – from AD export, then cut-paste Can also use Outlook Drag-Drop Handbook – blog post on how to do this in an hour.