Improve and Understand your SharePoint Online Performance - serge luca patrick guimonet

1. SharePoint Online Performance
Patrick Guimonet
Serge Luca

2. Hi I’m Serge Luca !
Brussels, Belgium http://www.shareql.com 8 x SharePoint MVP
Works with SharePoint
since 2002 @sergeluca
https://sergeluca.
wordpress.com

3. Hi I’m Patrick Guimonet !
Paris, France http://abalon.fr
5 x SharePoint MVP
2 x Office 365 MVP
Microsoft Regional
Director @patricg
https://yos365.
wordpress.com

4. Agenda
• Part 1 – From End User
to SharePoint Online
• Get technical understanding on how
you access data and services in Office
365.
• Be able to use the first level tools to
get some metrics on your access.
• Know how to investigate further
• Part 2 – Within SharePoint
Online
• SharePoint Online Performance
• How to diagnostic
• What is causing perf issues in
SharePoint Online (top 5)

5. Understanding the way to your data…
Microsoft perspective
Source: MVA Course – Office 365 Performance management

6. Internet Service
providers
networks
Access to Office 365, from a customer perspective
Server
workloads
Rich client
applications
Content
delivery
network (CDN)
Microsoft network +
edge nodes peering
and routing
Customer
Internal
network
Internet
egress
point
Public
DNS
Office 365
endpoint

7. Growing networks to cloud-scale
FY09 FY10 FY11 FY12 FY13 FY14 FY15
Network Device Count Growth
Geo-Redundant
Service/Application Design
• All nodes active, all nodes
stateless
Top 3 Most Connected
Networks in the World
• Peer in more than 23 countries
with over 2000 ISP’s globally
DC-to-Internet Backbone
• Multiple Terabits,
• Over 50 Points of Presence globally
• Global backbone connecting MS Datacenter to the
Internet
DC-to-DC Backbone
• Multiple Terabits of Capacity
• Dark fiber based DC-DC backbone to enable high
bandwidth between Datacenters
Dark Fiber
• Tens of thousands of Route Miles of owned Dark
Fiber Backbone
• Million+ 10G DWDM Route Miles of capacity
deployed
Cache Node
• Hosting Services collocated at User location
(metro)
Edge Nodes
• Multiple Terabits of Edge Interconnect capacity
• Directly connected to more than 2000 networks
with over 4,000 connections
Decoupled DCs • Separation of CPU’s Storage, SQL Services
IT Capacity Unit = STAMP • DC Capacity Unit or Workload Appliance
Microsoft

8. Rule 1: The power of the backbone!
Demo 1: Data transfer between Azure
and Office 365

9. Direct versus Azure upload (6 MB)
• From my PC in Stockholm to an EMEA tenant
• From an Azure VM in the North Europe region
5 s to7 s to 4 s

10. Direct versus Azure upload (10 MB)
• From my PC in Stockholm to an EMEA tenant
• From an Azure VM in the North Europe region
5 s to14 s to 5 s

11. Direct versus Azure upload (42 MB)
• From my PC in Stockholm to an
EMEA tenant
• From an Azure VM in the EMEA
region
5 s to51 s to 16 s

12. Rule 2: Know your Office 365 Datacenter and the fastest
way to it
• http://azurespeedtest.azurewebsites.net/
• http://www.azurespeed.com/
• Test which Microsoft Datacenters are the
closest to you
ping <your name>.sharepoint.com
psping -n 20 <your name>.sharepoint.com:443
tracert -4 <your name>.sharepoint.com

13. Microsoft’s network is one of the three largest in the world
1 million+ servers • 100+ Datacenters in over 40 countries • 1,500 network agreements and 50 Internet connections
*Operated by 21Vianet BRK 2161

14. From a legal perspective…
http://www.microsoft.com/online/legal/v2/
en-us/MOS_PTC_Geo_Boundaries.htm

15. Office 365 Microsoft Edge is live in 30 locations
http://www.priasoft.com/solutions/office_365_endpoint_testing_tool/
BRK 2161

16. Internet Network peering locations
Site data is published at
http://www.peeringdb.com/view.php?asn=8075
Some cities have multiple peering points
Peering locations may be on-net or off-net
Peering may involve physical connection
and/or routing advertisements
Data as of July 2014 is subject to change
Brisbane Australia
Melbourne Australia
Perth Australia
Sydney Australia
Vienna Austria
Luxembourg Luxembourg
Sao Paulo Brazil
Montreal Canada
Toronto Canada
Prague Czechoslovakia
Paris France
Frankfurt Germany
Hong Kong Hong Kong
Dublin Ireland
Milan Italy
Turin Italy
Tokyo Japan
Seoul Korea
Kuala Lumpur Malaysia
Amsterdam Netherlands
Auckland New Zealand
Wellington New Zealand
Moscow Russia
Singapore Singapore
Stockholm Sweden
Zurich Switzerland
Taipei Taiwan
London UK
Ashburn USA
Atlanta USA
Boston USA
Chicago USA
Dallas USA
Denver USA
Honolulu USA
Las Vegas USA
Los Angeles USA
Miami USA
New York USA
Palo Alto USA
San Jose USA
Seattle USA
BRK 2161

17. Azure CDN
This topic lists current Microsoft Azure CDN point of presence (POP) locations.
https://azure.microsoft.com/en-us/documentation/articles/cdn-pop-locations/

18. Demo 2: Measure RTT to Office 365

19. Measure Internal and External RTT (TCP Ping here)
Source: How to measure the Network Round Trip Time to Office 365 by Paul Collinge
Internal should be less than 100 ms, ideally 50 ms
EMEA to EMEA should be less than 150 ms
Outlook in cached mode can cope well with worse RTT
But for SharePoint 350 to 400 ms is the tipping between good and bad performances
aka Edge or Egress point

20. Client connectivity to Office 365
From support.office.com/en-us/article/Client-connectivity...
The SharePoint way
The Exchange way

21. Rule 3: Check REORGANIZED TOP 10 tips for optimising
and troubleshooting Office 365 Network Connectivity
• 6. TCP Window Scaling
• 7. TCP Idle time settings
• 8. TCP Max Segment size
(Should be >= 1300)
• 9. Selective Acknowledgement
• 10. Application Level
troubleshooting
• 1. Latency/Round Trip Time
(RTT)
• 2. DNS Geo location
• 3. DNS performance
• 4. Proxy Scalability
• 5. Proxy Authentication
Top 10 tips for optimizing and troubleshooting your Office
365 network connectivity by Paul Collinge

22. Demo 3: Perform network diagnostics
with Microsoft Office 365 Client
Performance Analyzer (OCPA)

23. OPCA from Stockholmsmässan Wifi
Microsoft
Network

24. OPCA from a North Europe based VM

25. 0 5 10 15 20 25
France - Near Paris - @Home
France - Nantes - Wifi Hotel
France - Nantes - 4G
Spain - Barcelona - Wifi Conf
Canada - Montréal - MS Office Wifi
Canada - Vancouver - Hotel Wifi
US - Bellevue - Hotel Wifi
US - Redmond - MS Open Wifi
Sweden - Stockholm - Wifi
Azure VM - Europe North
Azure VM - US West
Network HOPS to Office 365
Should be <= 25
Some OPCA results (1/2)
DNS Résolution Time Network HOPS to Office 365 Network Latency ( HTTP Ping) to Office 365 Http latency to CDN Client download bandwith (Kbps)
Should be <= 25 <= 25 <=1000 <=1000 >= 1024
France - Near Paris - @Home 3 15 222 76 19476
France - Nantes - Wifi Hotel 185 17 323 125 1983
France - Nantes - 4G 11 19 613 332 15903
Spain - Barcelona - Wifi Conf 57 16 388 196 590
Canada - Montréal - MS Office Wifi 454 20 588 321 788
Canada - Vancouver - Hotel Wifi 2 16 647 118 33456
US - Bellevue - Hotel Wifi 303 14 694 173 8529
US - Redmond - MS Open Wifi 110 16 705 76 56567
Sweden - Stockholm - Wifi 2 13 454 46 10199
Azure VM - Europe North 25 9 69 5 81331
Azure VM - US West 201NAN 638 17 125982

26. Some OPCA results (2/2)
0 50 100 150 200 250 300 350 400 450 500
France - Near Paris - @Home
France - Nantes - Wifi Hotel
France - Nantes - 4G
Spain - Barcelona - Wifi Conf
Canada - Montréal - MS Office Wifi
Canada - Vancouver - Hotel Wifi
US - Bellevue - Hotel Wifi
US - Redmond - MS Open Wifi
Sweden - Stockholm - Wifi
Azure VM - Europe North
Azure VM - US West
DNS Résolution Time to my european tenant
Should be <= 25
0 20000 40000 60000 80000 100000 120000 140000
France - Near Paris - @Home
France - Nantes - Wifi Hotel
France - Nantes - 4G
Spain - Barcelona - Wifi Conf
Canada - Montréal - MS Office Wifi
Canada - Vancouver - Hotel Wifi
US - Bellevue - Hotel Wifi
US - Redmond - MS Open Wifi
Sweden - Stockholm - Wifi
Azure VM - Europe North
Azure VM - US West
Client download bandwith (Kbps) >= 1024

27. 1.2) Master the tools
• IP standard tools:
• Ping, traceroute, nslookup, …
• PSPing is better
• Dedicated tools:
• Office 365 Client Performance
Analyzer
• AzureSpeedTest
• Fast Track Network Analysis (EMEA)
(need java)
• Network capture tools
• Netmon 3.4 is the last version.
• Message Analyzer is the
replacement for Network Monitor
• WireShark, TCPDump, …
• Application level tools
• HTTPWatch
• Fiddler
• IE and Edge F12
• Chrome DevTools
Rule 4: If you should use only one tool, use OPCA

28. Quick summary of IT Pro job
Use the fastest
way to the
backbone
First check with
OPCA
Investigate with
other tools

29. Microsoft Best Practices
And in SharePoint Online ?

30. SPRequestDuration X-SharePointHealthScore
Use F12
Check the following SharePoint Http Response Headers :
Check Files load time
Create an OOB Site Collection as a baseline

31. SPRequestDuration
How much work the server is
doing to serve the page
X-SharePointHealthScore
Health of the server that your
SharePoint instance is running
on.
0 = more healthy
10 = less healthy

32. Demo 4
F12 + SharePoint Http performance
Headers

33. Top 5 performance issues
Top 1
Navigation

34. Structural
Navigation
Managed
Navigation
Search based
Navigation
Navigation

35. Structural Managed Search based
-Slower +Faster +The fastest
++Easy to configure +Easy to maintain -Requires Customization of the master
page
+Security trimmed -Not security trimmed +Security trimmed
+Automatically updated -Not automatically updated +Automatically updated
-depends on cache -Cannot be ordered
+can cache navigation in client storage

36. Demo 5
Navigation & Performance

37. Top 5 performance causes
Top 1
Navigation
Top 2
Content Rollup

38. Content Rollup
From Slow to Fast
Content By Query web part
Content By Search web part
Static content
Content By Search web part Tip :
Use the Group caching option
Refreshes every 15 minutes
Only AD groups

39. Content By Search Web Part Group
Caching Option

40. A fast alternative is to use static content
If content seldom changes
Common Links, depts,…

41. Demo 6
Content Rollup & Performance

42. Top 5 performance issues
Top 1
Navigation
Top 2
Content Rollup
Top 3
Large Files

43. Larges Files
Javascript
• CDN
• Minifying
Videos
• Office 365
Video Portal
(Azure Media
Services)
Pictures
• Picture Lazy
loading
• Sprites
• Image
Rendition

44. Image Lazy loading : how to

45. Images : sprites
composite image that
contains many smaller images
ex: spcommon.png (in
…layouts/15/images/spcommon.png)
Benefit : downloads only 1 picture and caches it
Use CSS to display on a particular part
https://css-tricks.com/css-sprites/

46. Image rendition
1 (big) image stored in SharePoint
Several variations are rendered
SharePoint automatically sizes the image
You can also crop
Super important for mobiles
Don’t have to visually resize the picture
The server sends the smaller version

47. Demo 7
Images Rendition

48. Top 5 performance issues
Top 1
Navigation
Top 2
Content Rollup
Top 3
Large Files
Top 4
Many Requests
to Server

49. Top 5 performance issues
Top 1
Navigation
Top 2
Content Rollup
Top 3
Large Files
Top 4
Many Requests
to Server
Top 5
A lot of Web
Parts

50. Front end Caching in SPO :
a huge problem for MS

51. Use Azure Redis Cache
Use the Azure Redis cache
Ex to store session state data like
to SharePoint context
By default the SharePointContext
class requires ASP.Net session
state
In Proc mode ->not ok if several
ASP.Net servers
More details here
http://tinyurl.com/osst9xj

53. SPO does user throttling
• Limits number of concurrent calls
99% throttling : custom code
• Error 429 (« Too many requests »)
• Error 503 (« service not available »)
O365 Migration API (Preview)
Throttling Pattern
• Reduce number operation /request
• Reduce number of Requests
• Incremental Back Off

54. O365 PnP Throttling pattern
(incremental backoff)

55. Predictive capacity:forecasting
Forecasts are
for groups of
farms in a
zone
Requests
Time

56. Predictive capacity: managing farms
0
10
20
30
40
50
60
70
80
90
100
Average front end CPU load
%CPU
Monday Tuesday Wednesday Thursday Friday
0% to 40 % - Operating zone
40% to 85 % - Buffer
85 to 100 % - Danger Zone
Front ends in content farms run at
40% CPU
As load increases, we
add capacity

57. Solution :
rollout to all
users in
waves
Don’t do performance testings in SPO
You will be
throttled


58. Thank You ! Questions ?

59. Documentation
Support.office.com
Network planning and performance tuning for Office 365
On MSDN
Network planning and performance tuning for Office 365

60. Sources (Part 1)
• MVA Course – Office 365
Performance management
• Office 365 Client Performance
Analyzer
• Microsoft Azure Speed Test
• Top 10 Tips for Optimising &
Troubleshooting your Office 365
Network Connectivity from Paul
Collinge
• MS Ignite 2015 – BRK4121 -
Office 365 Network Performance
Troubleshooting