A presentation by John Holliday, President of SharePoint Architects, Inc. at the 2011 European Best Practices Conference in London, April, 2011 on best practices for developing Records Management solutions on the SharePoint platform.
DEV212 SharePoint 2010 Records Management Development
1. Clarity. Direction. Confidence.
SHAREPOINT 2010
RECORDS MANAGEMENT
DEVELOPMENT
John F. Holliday
SharePoint Architects, Inc.
www.SharePointArchitects.us
BEST PRACTICES CONFERENCE SHAREPOINT
2. About Me
• Books
• Professional SharePoint 2007 Development
• Professional SharePoint 2007 Records
Management Development
• Activities/Projects
• SharePoint Server MVP, Developer, Consultant
• CAML.NET Framework/CAML.NET IntelliSense
• SharePoint Developer Network (@SPDEVNET)
• SharePoint Heroes (sharepointheroes.com)
BEST PRACTICES CONFERENCE SHAREPOINT
3. Agenda
• Evolution of Records Management
Development in SharePoint
• Understand key RM features
• Explore RM-Focused Solution Strategies
• Integrating Managed Metadata Services
• Enabling PowerShell Admin/Deployment
BEST PRACTICES CONFERENCE SHAREPOINT
4. Key Challenges for RM in MOSS
• Declaring Records
• How to identify documents as records
• Protecting Records
• How to control what happens to “records”
• Developing File Plans
• How to identify records
• How to specify record lifecycles
• Technology agnostic?
BEST PRACTICES CONFERENCE SHAREPOINT
5. Key Features for RM in 2010
• Document ID Service
• Content Type Syndication
• Managed Metadata and Keywords
• Content Organizer for Routing
• “In Place” Record Declaration
• Multi-Stage Retention Policies
BEST PRACTICES CONFERENCE SHAREPOINT
6. Document Identifiers: Main Points
Every document can have a unique ID
• Ability to reference documents by ID
• Ability to move docs without breaking links
• Scoped to Site Collection (DOCID feature)
• Adds “Document ID” field to documents
• Need to checkout/checkin existing docs to get an ID
Example DOCID reference:
http://<site collection url>/_layouts/DocIdRedir.aspx?ID=<doc id>
BEST PRACTICES CONFERENCE SHAREPOINT
7. Using Document IDs for
Records Management
• Must ensure DOCID feature is activated
everywhere records are referenced
• Content Type Hub Site
• Collaboration Sites
• Records Center Sites
BEST PRACTICES CONFERENCE SHAREPOINT
8. Content Type Syndication: Main Points
• Available throughout the Farm
• Can be used for content organizer rules
• Can specify “publishing” rules
• Steps:
• Provision a site to be an Enterprise Metadata Hub Site
• Best Practice: Use a dedicated site collection.
• Assign the site collection to the MMS
• All content types created in the hub are now available for use in
other site collections
BEST PRACTICES CONFERENCE SHAREPOINT
9. Managed Metadata: Main Points
• Support for multiple languages
• Can set default values for terms
• Can add synonyms for terms
• Can create reusable term sets
BEST PRACTICES CONFERENCE SHAREPOINT
10. Content Organizer: Main Points
• Metadata-driven Routing
• Automatically handles incoming records
(no need for custom router)
• Target destination determined from metadata
(configured using rules)
• Support for file plan hierarchy
• Can target subfolders of destination library
• Automatically applies policies associated with target
location
BEST PRACTICES CONFERENCE SHAREPOINT
11. Content Organizer: Value Proposition
• Advantages
• More control over document routing
• No need for custom routers
• Still supports custom routers
• New “Rule Manager” user group
• Disadvantages
• Rules must be managed – requires skill
• Rules must be updated if content types change
BEST PRACTICES CONFERENCE SHAREPOINT
12. Content Organizer: Observations
• Dealing with the Classification Issue
• Records Managers like Taxonomies
• End Users like Folksonomies
• Managed Metadata to the rescue
• Building a “record declaration pipeline”
• “Funnel” documents towards proper declaration using
“progressive classification”
BEST PRACTICES CONFERENCE SHAREPOINT
13. “Progressive Classification”
• CONTOSO Healthcare
• Incoming patient interviews with embedded images
• High volume requires preliminary classification by a human
• Easy to misclassify => needs review by certified radiologist before
final declaration as official record
• Solution
• Interviews sent to drop-off library in dedicated non-
records center site with preliminary CO rules driven by
managed metadata
• Pre-classified interviews routed to specialized Records
Centers with CO rules driven by approval workflow.
BEST PRACTICES CONFERENCE SHAREPOINT
14. Email Records Management
• Designed for Exchange 2010
• Integrated Email Archiving
• Integrated Email Retention
• Integrated Email Discovery
• Can use Content Organizer for Email
• Avoid inconsistency/complexity
• Avoid bottlenecks/bandwidth issues
• Recommended only for Records Center
BEST PRACTICES CONFERENCE SHAREPOINT
15. Enabling Email Processing
• From PowerShell
• Enable-SPFeature –id EmailRouting
–url “<site url>”
• Creates a separate list to receive emails
• Assigns a unique email address
• Creates a built-in content type for emails
• “E-mail Submission”
• Pre-defined fields common to emails
BEST PRACTICES CONFERENCE SHAREPOINT
16. Multi-Stage Retention
• Missing from SP2007
• Enables modeling of real-world scenarios
• Stages are “activated” based on conditions
• Stages can recur automatically until the
„next‟ stage is activated
• Example:
• Perform annual review of invoices (recurring),
then purge after 3 years
BEST PRACTICES CONFERENCE SHAREPOINT
17. Multi-Stage Retention: Main Points
• Define retention based on any date field
• Stages activated by event triggers
• Stages support limited recurrence
• Support for custom formulas & actions
BEST PRACTICES CONFERENCE SHAREPOINT
18. Multi-Stage Retention: How it Works
• Basic Architecture
• What happens when a document „expires‟?
• When does SharePoint check for expiration?
Get-SPTimerJob ExpirationProcessing | Start-SPTimerJob
• Alternatives?
• Workflow to monitor document lifecycle?
• Timer job to check document lifecycle?
• => SharePoint is more efficient
BEST PRACTICES CONFERENCE SHAREPOINT
19. Legal Holds: Main Points
• Tied to Search
• New “Search and Add Holds” Page
• Use search criteria to locate documents
• Use search results to perform actions
• Examples:
• copy to a new location
• mark as read only
• Can be applied to any type of content
• Can have multiple holds for a site
• Use Excel to get holds report
BEST PRACTICES CONFERENCE SHAREPOINT
21. Two Solution Strategies
• In-Place
• Documents are declared as records in place.
• Documents are “locked down”.
• Repository-Based
• Documents are moved to a central repository.
• Document metadata is evaluated.
• Documents are routed to library/folder.
• Information policy (retention) is applied.
BEST PRACTICES CONFERENCE SHAREPOINT
22. In-Place Record Declaration:
Main Points
• Ability to declare (not manage) a record without
having to move it to a records center
• Can apply protections (lock down)
• Can apply custom processing (workflow)
• Some limitations:
• Cannot take advantage of features only found in the
Records Center site:
• Cannot assign in-place records to file plan folders
• Cannot inherit rules + permissions based on
file plan folder
BEST PRACTICES CONFERENCE SHAREPOINT
23. Pros/Cons of In-Place Approach
• Pro
• Lower cost (Records Center not required)
• Easier to configure (feature activation)
• Can apply records-only retention
• Con
• Cannot apply location-based rules
• Harder to administer
• Records are interspersed with non-records
BEST PRACTICES CONFERENCE SHAREPOINT
24. Pros/Cons of Repository Approach
• Pro
• Easier to administer
• Records can be segregated
• Records can be moved to cheaper storage
• Supports complex file plans
• Location-based policies can be inherited
• Metadata-driven rules for routing
• Con
• Harder to configure
BEST PRACTICES CONFERENCE SHAREPOINT
25. Hybrid Solution Strategy
• Maintain distinction between “record declaration”
and “records management”
• Custom event handler when records are
declared/undeclared
• => Start “classification” workflow when declared
• => Move to repository when approved
• Leave hyperlink in its place
BEST PRACTICES CONFERENCE SHAREPOINT
26. IRecordDeclarationHandler
public class RecordDeclarationHandler : IRecordDeclarationHandler
{
// Records the record declaration event to an external database.
RecordOperationResult
IRecordDeclarationHandler.OnDeclare(SPListItem item)
{
HandleDeclaration(item, true);
// continue with default processing
return RecordOperationResult.ContinueRecordProcessing;
}
}
BEST PRACTICES CONFERENCE SHAREPOINT
27. Registering the Record Handler
using Microsoft.Office.RecordsManagement.RecordsRepository;
public static void RegisterDeclarationHandler(
this SPSite site, Type handler)
{
Records.RegisterCustomCodeForRecordDeclaration(
site,handler.Assembly.FullName,
Handler.FullName);
}
….
SPSite site = new SPSite(“…”);
site.RegisterDeclarationHandler(typeof(myHandler));
BEST PRACTICES CONFERENCE SHAREPOINT
30. RM-Focused Solutions: Rule #1
Avoid Semantic Disconnect with RM Pros
• Must understand their language:
• “File Plan”
• “Retention Schedule”
• “Classification”
• “Record Series”
BEST PRACTICES CONFERENCE SHAREPOINT
31. RM-Focused Solutions: Rule #2
Target RM-Specific Document Lifecycles
and Administration Requirements
• Must consider long-term evolution
• Separate web application for records
• Content isolation
• Separate site collections for primary classifications
• Quotas; document volume; security
• Separate document libraries for record series
• Managed metadata
BEST PRACTICES CONFERENCE SHAREPOINT
32. RM-Focused Solutions: Rule #3
Incorporate Managed Metadata into Overall
Development Plan
• Maintain consistency between MMS taxonomy
and Content Organizer rules
• Important for Records Managers, who are fixated on
classification terms
• Important for traceability (“management evidence”)
long after the records have been routed
BEST PRACTICES CONFERENCE SHAREPOINT
33. Empowering Administrators
Want to empower Admins to operate at a higher
level than raw configuration steps.
Records Manager: “I need a place for a new primary
classification, with 10 record series and about 2 dozen
terms.”
Admin: “Do you want a new site collection?”
Records Manager: “Huh?”
BEST PRACTICES CONFERENCE SHAREPOINT
36. Records Management Setup
Basic Steps:
1. Create a Specialized Web Application
2. Create Site Collections (1 RC per Classification)
3. Create Columns (add custom metadata)
4. Create (and syndicate) Content Types
5. Create Libraries and Folders (1 DL per Series)
6. Create Information Policies (Retention)
7. Create Content Organizer Rules
BEST PRACTICES CONFERENCE SHAREPOINT
37. Records Management
Web Application
Key Points:
• Create a separate web app just for RM.
• Can assign a unique application pool.
• Enables process-level tracking of users/processes.
• Enables common URL for multiple records centers.
• Can fine-tune bandwidth specific to RM.
BEST PRACTICES CONFERENCE SHAREPOINT
38. DEMO
PowerShell: New-RMApplication
BEST PRACTICES CONFERENCE SHAREPOINT
39. “Primary Classification”
= “Records Center”
1. Create a Separate Records Center Site Collection for
each Primary Classification
“Case Files”
“Organizational Files”
“Program Files”
“Administrative Files”
2. Associate each Records Center with its own content
database
• Easier to archive by primary classification
• Restrict maximum number of sites to 1
BEST PRACTICES CONFERENCE SHAREPOINT
40. DEMO
PowerShell: New-RMClassification
BEST PRACTICES CONFERENCE SHAREPOINT
41. “Record Series” = “Document Library”
1. Create a Document Library for each Record Series
within a Classification
• Hierarchy of folders/sub-folders as needed
2. Create a Matching Content Type for each Record Series
• Serves as the primary mechanism for capturing critical metadata
• Required for setting up CO rules
BEST PRACTICES CONFERENCE SHAREPOINT
42. DEMO
PowerShell: New-RMRecordSeries
BEST PRACTICES CONFERENCE SHAREPOINT
43. Summary
New features in SharePoint 2010 further enhance
the RM story by making it easier to organize
content and manage metadata.
The convergence of traditional RM with emerging
ECM technologies requires an RM-focused
solution development strategy.
In-Place records management means richer and
more intuitive UX for records managers and end
users, but increases the need for sound
information architecture.
BEST PRACTICES CONFERENCE SHAREPOINT
44. Clarity. Direction. Confidence.
FOR MORE INFO
Visit my blog: www.JohnHolliday.net
Upcoming Book: “Managing Official Records Using
SharePoint 2010: A Guide for Developers and Administrators”
BEST PRACTICES CONFERENCE SHAREPOINT
Editor's Notes
Document identifiers are scoped to the site collection, and are enabled by activating the “docid” feature. This can be done either from the Site Settings page or by using a PowerShell command:PS> Enable-SPFeature id docid –url <site collection url>Enabling this feature adds a “Document ID” field to every document. For existing documents, you have to perform an action on the document (such as checkin or checkout) to have an identifier assigned. For new documents, the identifier is assigned on upload.The DocIdRedir.aspx page works even if the document is moved to a different library. It also works for document sets.
Add Managed Metadata column to a content type.Use the centralized Term Store to create and manage Term Sets.Example: Continents/Countries/States/Cities.Can create synonyms for terms: JAX=JacksonvilleSupports publish and subscribe through Content Type SyndicationExample:>> Setup a "hub" site collection to hold global content types>> Add site columns and content types based on them>> Configure the Managed Metadata service.Central Admin..Manage Service Applications...Managed Metadata Service....Properties.....Content Type hub: <site collection url>.....Report syndication import errors = checked....Consumes content types from the Content Type Gallery at...=checked>> Go to hub site collection>> Edit any content type(new menu item, "Manage Publishing...")select "Publish" and click OK>> force timer jobs to run.Central Admin..Monitoring...Timer Jobs -> Review Job Definitions....Content Type Hub job => "Run Now"....Content Type Subscriber job => "Run Now">> create a new site collection.site Settings=> see published content types
General purpose routing mechanism- Activate the Content Organizer feature- Only applies to "Document" content type or derivatives>> adds 2 menu items to Site Administration: Content Organizer Settings :: Turn routing on and offContent Organizer Rules:: Setup rules for routing documents>> scoped to Web>> creates a "Drop Off" library>> all uploads are placed into the drop off library (if configured)>> content organizer rules are applied by a timer job==> Good for use as a staging area for new documents until all required metadata has been added>> when a rule is added, content types are added to the droppoff library>> workflow can be attached to documents before they are routed, based on conditions (fulfilled by workflow activities, for example)>> approval can be applied before routing>> checkin/checkout of documents to modify metadata (documents are routed as soon as conditions are met, whether checked in or not)>> (optional) automatically notifies Rule Managers if documents remain after all rules have been applied. Can specify how long before notification is sent (documents still in library after N days)
Expiration Processing Timer Job-------------------------------Runs weekly by default.
RecordDeclarationHandler is a custom handler for record declaration and undeclarationevents. Whenever a record is declared or undeclared, this code appends an event to the system log.
Creating a dedicated web application for RM enables process-level tracking of which processes and users are touching the records store. This is important “management evidence” that can help an organization deal with compliance challenges.
Organizational Files – used to document relationships between departmentsAdministrative Files – used for internal administration and operationsProgram Files – general activities and programsSeparate content DB means simpler implementation of cyclical backup strategies:Archive primary classification and empty the repository once per year/quarter