This document discusses techniques for accessing and querying list data in SharePoint. It covers: - Lists are the primary data storage mechanism in SharePoint and store data in rows and columns like Excel sheets rather than database tables. - Items can be accessed and updated via the SPListItem object. Fields are accessed by internal name and queries use CAML. - CAML supports filtering, sorting, paging, and joining lists. Large lists have throttling that can be configured. - The REST API provides a way to access lists from remote applications using OData and returns data in JSON/AtomPub format. - LINQ to SharePoint generates CAML queries from LINQ expressions for strongly typed

1. Techniques to Access
List Data
Rob Windsor
rwindsor@portalsolutions.net
@robwindsor

2. Lists
• The data storage mechanism in SharePoint
 Virtually all content stored in lists
• Provisioning engine handles list creation and schema management
• SharePoint provides interface to add, edit, delete and view items
• Lists support:
 Multiple views (for sorting, filtering, grouping)
 Simple validation
 Content approval
 Item versioning
• Data stored in rows and columns
 More like Excel worksheets than database tables

3. Accessing and Updating List Items
• Field values accessed via SPListItem indexer
• Generally accessed by internal name
• Weakly typed
var web = SPContext.Current.Web; var web = SPContext.Current.Web;
var list = web.Lists.TryGetList("Products"); var list = web.Lists.TryGetList("Products");
if (list == null) return; if (list == null) return;
foreach (SPListItem item in list.Items) foreach (SPListItem item in list.Items)
{ {
Console.WriteLine("{0} ({1})", var price = Convert.ToDouble(
item["Title"], item["UnitPrice"]);
item["UnitPrice"]); price *= 1.1;
} item["UnitPrice"] = price;
item.Update();
}

4. Field Names
• Fields have three names
 Display name
 The name displayed to the end user
 Can be changed
 Internal name
 Set when field is created
 Does not change
 Non-alphanumeric chars escaped (e.g. Unit_x0020_Price)
 Find using Server Explorer
 Static name
 Used by custom field types

5. CAML Queries
• Query list items using XML syntax
• SPQuery
 Query single list
 Returns SPLitsItemCollection
• SPSiteDataQuery
 Query multiple lists across site collection
 Returns DataSet
• Use QueryThrottleMode to enable/disable throttling
 Enabled by default

6. CAML Queries
var query = new SPQuery();
query.ViewFields =
"<FieldRef Name='Title' />" +
"<FieldRef Name='UnitPrice' />";
query.Query =
"<OrderBy>" +
" <FieldRef Name='Title' />" +
"</OrderBy>" +
"<Where>" +
" <And>" +
" <Gt>" +
" <FieldRef Name='UnitsInStock' />" +
" <Value Type='Integer'>0</Value>" +
" </Gt>" +
" <Eq>" +
" <FieldRef Name='Category' LookupId='True' />" +
" <Value Type='Lookup'>1</Value>" +
" </Eq>" +
" </And>" +
"</Where>";
var items = list.GetItems(query);

7. SPQuery.ViewAttributes
• Scope
 Default = Show files and folders of specific folder
 FilesOnly = Show files of specific folder
 Recursive = Shows files in all folders
 RecursiveAll = Show files and folders in all folders
• Moderation
 HideUnapproved = Don’t show draft items
 Contributor = Show draft items for current user only
 Moderator = Show draft items
• Syntax
 Query.ViewAttributes = "Scope='Recursive'"

8. “All we can do is decide what to do with
the time tools that is are given us”
Image from The Lord of the Rings: The Fellowship of the Ring by New
Line Cinema

9. Large List Throttling
• Lists can store millions of items
• Retrieval of items is throttled
 Max 5,000 items
 Max 20,000 for Owners
 No max for local admin
• Can override limits in code
 QueryThrottleMode
• Can configure time window
where limits are ignored
 For reporting, list aggregation, etc
 Done during off-peak hours

10. Paging CAML Queries
• Seriously convoluted for all but the simplest cases
• Paging managed by SPListItemCollectionPosition
 Only supports Prev / Next paging
 Cannot jump to a specific page
• PagingInfo
 String representing the parameters used to get the next page of
items
 Parts
 Paged=TRUE – the list is paged
 PagedPrev=TRUE – retrieve the previous page
 p_ID=n – list item id of the first or last value in the set last retrieved
 p_<Field Name>=v - value of first or last value of the order by field in the set
last retrieved
 PageFirstRow – the position of the last item in the last set retrieved

11. CAML Queries
var list = SPContext.Current.Web.Lists["Announcements"];
var query = new SPQuery();
query.RowLimit = 10;
query.Query = "<OrderBy Override="TRUE">" +
"<FieldRef Name="FileLeafRef" /></OrderBy>";
do
{
var items = list.GetItems(query);
// work with items
query.ListItemCollectionPosition =
items.ListItemCollectionPosition;
} while (query.ListItemCollectionPosition != null);

12. List Relationships

13. Joins in CAML Queries
• SPQuery supports joins
 SPSiteDataQuery does not
• Joins can only be done on lookup columns
• Key properties
 J oins: Define joins to other lists
 ProjectedFields: Fields being projected from the foreign list
• Only inner and left outer joins are supported

14. SPQuery with Join Example
var query = new SPQuery { RowLimit = 5 };
query.ProjectedFields =
"<Field Name='CategoryTitle' Type='Lookup' List='productCategory' ShowField='Title' />";
query.ViewFields =
"<FieldRef Name='Title' />" +
"<FieldRef Name='UnitPrice' />" +
"<FieldRef Name='UnitsInStock' />" +
"<FieldRef Name='CategoryTitle' />";
query.Query =
"<Where>" +
" <And>" +
" <Gt>" +
" <FieldRef Name='UnitsInStock' />" +
" <Value Type='Integer'>0</Value>" +
" </Gt>" +
" <Eq>" +
" <FieldRef Name='CategoryTitle' />" +
" <Value Type='Text'>Condiments</Value>" +
" </Eq>" +
" </And>" +
"</Where>";
query.Joins =
"<Join Type='INNER' ListAlias='productCategory'>" +
" <Eq>" +
" <FieldRef Name='Category' RefType='ID' />" +
" <FieldRef List='productCategory' Name='ID' />" +
" </Eq>" +
"</Join>";
return list.GetItems(query);

15. Querying Multiple Lists with SPSiteDataQuery
• Core properties the same as SPQuery
• Cannot query lists outside site collection
• Key additional properties
 Lists
 Defines which lists will be included in query
 Use ServerTemplate for OOB content types
 Use BaseType plus filter for ID in where clause for custom content
types
 Webs
 Defines the scope of the query
 SiteCollection
– Query all matching lists in site collection
 Recursive:
– Query all matching lists in current Web and it’s children

16. SPSiteDataQuery Example
var query = new SPSiteDataQuery { RowLimit = 50 };
query.ViewFields =
"<FieldRef Name='Title' />" +
"<FieldRef Name='DueDate' />" +
"<FieldRef Name='AssignedTo' />";
query.Query =
"<Where>" +
" <And>" +
" <Geq>" +
" <FieldRef Name='DueDate' />" +
" <Value Type='DateTime'><Today /></Value>" +
" </Geq>" +
" <Leq>" +
" <FieldRef Name='DueDate' />" +
" <Value Type='DateTime'><Today OffsetDays='5' /></Value>" +
" </Leq>" +
" </And>" +
"</Where>";
query.Lists =
"<Lists ServerTemplate='107' />"; <!– Task Lists -->
query.Webs =
"<Webs Scope='SiteCollection' />";
return web.GetSiteData(query);

17. DEMO
List access using Server
Object Model

18. REST APIs
• Generally used by remote applications
• Uses Open Data (OData) Protocol
 Also known as: Astoria, ADO.NET Data Services, WCF Data Services
 Data centric (REST) Web service
 Data returned in AtomPub or JSON format
 Metadata is available
 Allows creation of service proxy
 Strongly-typed access to data
 Requests for collections may not return all items
 Max 1,000 items returned per request
 Need to check for continuation

19. REST APIs
• URLs map to SharePoint resources
 Example: ListData.svc/Products(2)/Category maps to the category
for the product with id == 2
• Protocol commands
 $filter={simple predicate}
 $expand={Entity}
 $orderby={property}
 $skip=n
 $top=n
 $metadata

20. REST APIs (OData)

21. Consuming Data via REST APIs

22. Consuming Data via REST APIs
var context = new DemoDataContext(new Uri(
"http://localhost/sites/demo/_vti_bin/ListData.svc"));
context.Credentials = CredentialCache.DefaultCredentials;
var query = from detail in context.OrderDetails
where detail.UnitPrice < 100
select detail;
var queryCount = query.Count();
var details = new List<OrderDetailsItem>();
var dsquery = query as DataServiceQuery<OrderDetailsItem>;
var response = dsquery.Execute() as QueryOperationResponse<OrderDetailsItem>;
details.AddRange(response);
var token = response.GetContinuation();
while (token != null)
{
response = context.Execute(token);
details.AddRange(response);
token = response.GetContinuation();
}

23. Consuming Data via REST APIs
var url = "http://localhost/sites/demo/_vti_bin/ListData.svc";
var context = new DemoProxy.DemoDataContext(new Uri(url));
context.Credentials = System.Net.CredentialCache.DefaultCredentials;
var products = from product in context.Products
where product.Category.Title == "Condiments" &&
product.UnitsInStock > 0
select product;
foreach (var product in products)
{
product.UnitsInStock += 1;
context.UpdateObject(product); // don’t forget this
}
context.SaveChanges();

24. DEMO
Using the REST APIs

25. LINQ to SharePoint
• LINQ provider generates CAML for query
 Requires reference to Microsoft.SharePoint.Linq.dll
• Only available to server object model
• Queries are strongly-typed
• Entities created using SPMetal command-line utility
 Similar to SqlMetal in LINQ to SQL
 Located in <System root>bin
 Basic usage
 spmetal /web:<site url> /code:<file name>
 Code generation can be customized via parameters XML file
 http://msdn.microsoft.com/en-us/library/ee535056.aspx

26. Consuming Data with LINQ to SharePoint
var url = "http://localhost/sites/demo";
var context = new SPNorthwind.SPNorthwindDataContext(url);
// context.Log = Console.Out;
var products = from product in context.Products
where product.Category.Title == "Condiments" &&
product.UnitsInStock > 0
select product;
foreach (var product in products)
{
product.UnitsInStock += 1;
}
context.SubmitChanges();

27. DEMO
LINQ to SharePoint