SharePoint Large List Support

1. Gold Sponsors Bronze SponsorsSilver Sponsors
Large List Support
List Throttling
Rick Rosato & Benjamin Wong

2. Agenda
• Large List support
• What this means? (Throttling/Threshold/Limits)
• Strategies for managing large lists in SharePoint
• Not covering
• Office 365
• Amount of content
• Size considerations
• Content Type management
• Archiving
• Storage strategies
• If you have any of these issue, come see me afterwards!

3. What usually happens?
• Using SharePoint happily
• Upload or add to the list
• Listview throws an error, you have insufficient
privileges to view the page content.
• You panic, your world is imploding
• You call the helpdesk
• They scramble to solve the issue

4. SharePoint 2013 Limits
• Lists and Libraries – limits (SharePoint 2013)
• 50 million (items)
• 2 GB (item or attachment)
• Threshold Limit is 5,000, why 5,000?
• Minimize database contention (row-level locking)
• When we do get errors, what’s going to happen?

5. These things stop if I exceed the
threshold?
• List issues
• Datasheet view breaks -
permission error
• Add/remove/update a list
column/content type
• Create/remove an index
• Manage files which have
no checked-in version
• Non-indexed recursive
queries
• Cross list query
• Lookup columns that
enforce relationship
behaviour
• Deleting a list/site
• Save list as template with
data
• Showing totals in list views
• Enable/disable
attachments in a list
• Folder issues
• Delete/copy/rename
• Queries that filter on non-
indexed col
• Setting item-level security
• Open with Explorer

6. Behind the scenes

7. Resource throttles and limits
Threshold or Limit Default value Description
List View Threshold 5,000
(warning at 3,000)
Specifies the maximum number of list or library items that a database operation,
such as a query, can process at one time. Operations that exceed this limit are
blocked.
Unique permissions limit 50,000
(breaking the inheritance is
counted as 1 unique permission)
Specifies the maximum number of unique permissions allowed for a list or library.
Row size limit per list
(internal rows)
6 (8,000 bytes)
(related to column
Specifies the maximum number of table rows internal to the database used for a
list or library item. To accommodate wide lists with many columns, each item is
wrapped over several internal table rows,.
Continue…

8. Resource throttles and limits (cont..)
Threshold or Limit Default value Description
List View Lookup Threshold 12 (joins) Specifies the maximum number of join operations, such as those based on lookup,
Person/Group, or workflow status columns.
List View Threshold size for
auditors and administrators
20,000 Specifies the maximum number of list or library items that a database operation, such as a query,
can process at one time when performed by an auditor or administrator with appropriate
permissions. This setting works in conjunction with Allow Object Model Override.
Allow Object Model
Override
Y Specifies whether or not developers can perform database operations, such as queries, that
request an override of the List View Threshold to the higher limit specified by the List View
Threshold for auditors and administrators.
Daily time window None Specifies a time period during which resource thresholds and limits are ignored. An administrator
can configure a time window during "off-peak" hours in 15 minute increments and up to 24
hours, such as, 6:00 PM to 10:00 PM or 1:30 AM to 5:15 AM.

9. Ways to manage large lists and
libraries
• Create SharePoint Indexed Columns
• Create Filtered Views Based on Column Indexes
• Organizing Items into Folders
• Controlling the display of data on the page
• Using offline synchronisation
• Using Search to find list or library items
• Using indexed and fallback queries
• Using External Data

10. Create SharePoint indexed Columns
• Help improve performance, up to 20 columns
• Only add to those you need, does add overhead
Supported Column Types Unsupported Column Types
Single line of text Multiple lines of text
Choice (single value) Choice (multi-valued)
Number Calculated
Currency Hyperlink or Picture
Date and Time Custom Columns
Lookup (single value) Lookup (multi-valued)
Person or Group (single value) Person or Group (multi-valued)
Yes/No External data External data
Managed Metadata

11. Create Filtered Views Based on
Column Indexes
• First filter MUST be indexed (to return less than List
View Threshold)
• Other filtered columns not countered (except for devs
using compound indexes)
• Multi-filter with AND operator to limit results
• Recycle Bin may skew results set
• Some operations might still be blocked (requires
entire list)
• adding or deleting an index
• creating a sort in the view definition;
• displaying a column total
• adding, updating, or deleting calculated fields.

12. Organising Items into Folders
• Folders - are an internal index
• Subfolders count as an item (but not items in that
subfolder).
• Good quick strategy for fixing List View Threshold
issues.
• Moving between folders is difficult

13. Controlling the display of data on
the page
• Setting an item limit on a page
• Display items in batches of the specified size (for
paging),
• Limit the total number of items returned to the
specified amount
• Using the Preview Pane view style
• Using the Group By view style
• Using RSS (turn on Site Collection and List)
• Configure List RSS - Based on modified date – limit
results
• Set the maximum number of days to include

14. Using offline synchronisation
• Access 2013
• Local copy, synchronises when on online (upto 2GB)
• 50,000 items (configurable)
• Outlook 2013
• Contacts
• Tasks
• Calendar
• Discussion List
• Documents
• Creates a PST file, synchronises when on online.

15. Using Search to find list or library
items
• It own indexing mechanisms, not subject to the
List View Threshold
• Results can be scoped narrow or widen as needed
• Search is really fast when querying the list/library
• Search on list/library attributes and use refiners to
limit results

16. Metadata Navigation Feature and
using indexed and fallback queries
• Metadata Navigation and Filtering can assist with
large lists, via query optimisation and best
selection of queries.
• If there’s no best query, Metadata Nav & Filtering
will construct and execute a fallback query –
limited results returned.
• Intended to return some useful results when
original query is blocked due to large list
throttling.
• Not 100% reliable

17. Using External Data
• Q. Should it be in SharePoint in the first place?
• Using BCS – provides standardized interfaces to
external data sources
• External List usually relates to transactional data –
LoB, ERP.
• Not bound be list throttling limits, it has it’s own
configuration.

18. Demos
• Create a simple or compound index
• Create a filtered view based on indexed column
• Metadata navigation
• Central Admin - Resource Throttling
• Qualitem Solution
• Large List administration
• Content Rollup support for large list
• Paging and Search

19. Don’ts
• Lookup columns, don’t exceed 12 per list
• What classifies as a Lookup column?
• Name (linked to document/edit with menu)
• Edit (link to edit item)
• Type (icon linked to document)
• Standard lookup columns
• Managed metadata (single and multiple-value)
• People and group columns (single and multiple-value
people and group columns
• Created by , Modified by ( people )
• Workflow Status

20. If you get stuck!
• Buy our tools to help you manage your lists –
Qualitem
• Increase the List View Threshold temporarily via
CA
• Administrators can run queries on large lists
• Enable a daily time window for large queries in CA
• Call me – Rick Rosato – 04 1111 7270
(rick@qualitem.com)

21. Reference
• http://office.microsoft.com/en-in/sharepoint-
server-help/manage-lists-and-libraries-with-
many-items-HA102771361.aspx#_Toc346115854
• http://technet.microsoft.com/en-
us/library/cc262813(v=office.14).aspx
• Rick Rosato (rick@qualitem.com)
& A