A set of mindsets, methodologies and tips for troubleshooting in SharePoint and Office 365

1. Ian Campbell
Troublefree Troubleshooting
@i4n0fd00m

4. Event
• Session Schedules are not printed for all attendees, find these next to the doors of
the session rooms or on the SPS Events site:
http://www.spsevents.org/city/Johannesburg/
• Please evaluate the sessions you attended – feedback is very important to us, and
yes, we do read them.
• Stay for the prize giving at the end of the day
• Say hi and have a chat to the sponsors and speakers – after all, they made this
possible
• Take a selfie with a speaker and upload it to Twitter with the #SPSJHB hashtag as
well as tagging the speaker, we will have a competition for these at the end of the
day
• Most tweets on #SPSJHB tweets by single person also gets a prize
• Learning through sharing is a motto for us, share your learnings on Twitter /
Facebook or LinkedIn – remember to use the event hashtag as well

5. Event
• Session Schedules are not printed for all attendees, find these next to the doors of
the session rooms or on the SPS Events site:
http://www.spsevents.org/city/Johannesburg/
• Please evaluate the sessions you attended – feedback is very important to us, and
yes, we do read them.
• Stay for the prize giving at the end of the day
• Say hi and have a chat to the sponsors and speakers – after all, they made this
possible
• Take a selfie with a speaker and upload it to Twitter with the #SPSJHB hashtag as
well as tagging the speaker, we will have a competition for these at the end of the
day
• Most tweets on #SPSJHB tweets by single person also gets a prize
• Learning through sharing is a motto for us, share your learnings on Twitter /
Facebook or LinkedIn – remember to use the event hashtag as well

6. Troubleless Troubleshooting
• Define
• Replicate
• Research
• Hypothesis
• Test
• Deploy
• Document
Troubleshooting methodology 101

7. Troubleless Troubleshooting
• Desktop support background.
• Senior SharePoint Consultant.
Why can I talk about this?

8. Troubleless Troubleshooting
• Desktop support background.
• Senior SharePoint Consultant.
• Passion for problem solving.
• I think quite abstractly.
Why can I talk about this?

9. Troubleless Troubleshooting
• Desktop support background.
• Senior SharePoint Consultant.
• Passion for problem solving.
• I think quite abstractly.
• I work a lot with SharePoint
adjacent apps, which has led to
some interesting problems to solve
Why can I talk about this?

10. Troubleless Troubleshooting
This presentation is linear.
Troubleshooting is NOT linear.
Disclaimer

11. Philosophical Razors
A principle or rule of thumb that allows
one to eliminate ("shave off") unlikely
explanations for a phenomenon, or
avoid unnecessary actions.

12. Define the problem

13. Philosophical Razors
A principle or rule of thumb that allows one to eliminate
("shave off") unlikely explanations for a phenomenon, or
avoid unnecessary actions.
•Hanlon's razor: Never attribute to
malice that which can be
adequately explained by stupidity.
•(or Ignorance)

14. Define the problem
Example
Servers had lost power the night before.
SharePoint was not available
IT supplier had tried to fix SharePoint.
Initial Problem Definition :
SharePoint is ‘down’

15. Define the problem
What are the symptoms or effects of the problem?
Did it ever work?
When did it stop working?

16. Define the problem
What Problem does this point to?

17. Define the problem
Once we have defined
the problem, we can
start looking for the
cause

18. Define the problem
What Problem does this point to?
After confirming that SharePoint could not be reached
normally, and doing some initial log and event checking,
I wanted to check some things with PowerShell.
Imagine my surprise when SharePoint was 100%
available through PowerShell.

19. Define the problem
What Problem does this point to?
This pointed to a problem with IIS or page serving, not
SharePoint as a whole.

20. Define the problem
What Problem does this point to?
‘SharePoint is down’ turns out to be an effect.

21. Define the problem
What Problem does this point to?
‘SharePoint is down’ turns out to be an effect.
So we redefine the problem…

22. Define the problem
What Problem does this point to?
‘SharePoint is down’ turns out to be an effect.
So we redefine the problem…
‘IIS is not serving SharePoint pages’

23. Define the problem
What went wrong?
• What is the problem?

24. Define the problem
What went wrong?
• What is the problem?
• Does it come with error messages?

25. Define the problem
What went wrong?
• What is the problem?
• Does it come with error messages?
• Does it come with logs?

26. Define the problem
What went wrong?
• What is the problem?
• Does it come with error messages?
• Does it come with logs?
• Symptom or Root cause?

27. Define the problem
What went wrong?
• What is the problem?
• Does it come with error messages?
• Does it come with logs?
• Symptom or Root cause?
• Did it ever work

28. Define the problem
What went wrong?
• What is the problem?
• Does it come with error messages?
• Does it come with logs?
• Symptom or Root cause?
• Did it ever work? When did it stop working?

29. Replicate the problem

30. Replicate the problem
Can I perform a set of steps to make the problem re-appear in the
same environment?
Can I do it again?

31. Replicate the problem
Can I do the same in a different environment.
Can I do it again?

32. Replicate the problem
Can I do the same with a different user.
Can I do it again?

33. Research the problem

34. Philosophical Razors
A principle or rule of thumb that allows one to eliminate
("shave off") unlikely explanations for a phenomenon, or
avoid unnecessary actions.
•Hitchens's razor: "What can be
asserted without evidence can be
dismissed without evidence.

35. Research the problem
Inside the system
Assumptions

36. Research the problem
Inside the system
Assumptions
are key to troubleshooting you problem.

37. Research the problem
Inside the system
Assumptions
are key to troubleshooting you problem.
The only thing you do with an assumption is to test it.
If it fails, put it aside.
If it passes, it is now fact.

38. Research the problem
Establish Facts
Was anything in the system changed recently?

39. Research the problem
Establish Facts
Was anything in the system changed recently?
Was anything anywhere near the system changed
recently?

40. Research the problem
Establish Facts
Was anything in the system changed recently?
Was anything anywhere near the system changed
recently?
What do the logs say?

41. Research the problem
Establish Facts
Was anything in the system changed recently?
Was anything anywhere near the system changed
recently?
What do the logs say?
What do the logs say?

42. Research the problem
Inside the system
So now we were looking at something that would only
present after a reboot, that would cause issues with IIS

43. Research the problem
Inside the system
We looked at recent updates installed, which might
have caused issues after the ‘reboot’ of the servers
going down

44. Research the problem
Inside the system
So we trawled the list and went to the internet.

45. Research the problem
Inside the system
Now, all the logs and server access we had are not
available to O365 users.

46. Research the problem
Inside the system
Microsoft Support and Recovery Assistant for Office 365
Microsoft 365 Service health status
Search the audit log in the Security & Compliance Center
Fiddler

47. Research the problem
Inside the system
Microsoft Support and Recovery Assistant for Office 365
Outlook and O365 user app related.
Useful for fixing issues that originate on a users pc.
Microsoft 365 Service health status
Search the audit log in the Security & Compliance Center
Fiddler

48. Research the problem
Inside the system
Microsoft Support and Recovery Assistant for Office 365
Microsoft 365 Service health status
O365 service health, degraded services
If Search is degraded, that might be why your managed
property isn’t working.
Search the audit log in the Security & Compliance Center
Fiddler

49. Research the problem
Inside the system
Microsoft Support and Recovery Assistant for Office 365
Microsoft 365 Service health status
Search the audit log in the Security & Compliance Center
User/login related
Helps identify account relates issues
Fiddler

50. Research the problem
Inside the system
Microsoft Support and Recovery Assistant for Office 365
Microsoft 365 Service health status
Search the audit log in the Security & Compliance Center
Fiddler
Web Debugger
Logs all HTTP(s) traffic between your browser and the web

51. Research the problem
On the web

52. Research the problem
On the web
Search

53. Research the problem
On the web
Search

54. Research the problem
On the web
Copy paste findings from research.

55. Research the problem
On the web
Copy paste findings from research.
But remove GUIDs and Tenant specific details

56. Research the problem
On the web
Prioritise results from Microsoft resources
TechNet Blogs - Microsoft
Microsoft forums
Microsoft Support

57. Research the problem
On the web

58. Research the problem
On the web

59. Research the problem
On the web

60. Research the problem
On the web

61. Research the problem
On the web
Reword your search

62. Research the problem
On the web
Post your question yourself
TechNet Blogs - Microsoft
Microsoft forums
Microsoft Support

63. Research the problem
Example
In this case, searching the names of recently updated
apps with ‘SharePoint’ revealed the culprit to be none
other than…

64. Research the problem

65. Research the problem
…With added APM

66. Research the problem
Application Performance Monitoring
The APM agent uses the .Net framework 2.0 app
pool.
It was a known issue.
…With added APM

67. Hypothesis

68. Philosophical Razors
A principle or rule of thumb that allows one to eliminate
("shave off") unlikely explanations for a phenomenon, or
avoid unnecessary actions.
•Occam's razor: Simpler explanations
are more likely to be correct; avoid
unnecessary or improbable
assumptions.

69. Occam’s Razor
Put simply…

70. Beware…
Occasionally…
A duck…

71. Beware…
Is a Rabbit

72. Beware…
Is a Rabbit
Adjust your
hypothesis…

73. Hypothesis
Inside the system
We alerted the Client and IT supplier that we were
going to uninstall the MMA.

74. Hypothesis
Inside the system
In a perfect world, we would have had a non production
environment to test this in, but the IT supplier had not
deployed MMA to the Dev server.

75. Test

76. Test
Well, I uploaded the new template,
activated the feature, adjusted the
workflow, and republished as the
system account.
What bit worked?
Dunno…

77. Test
Change ONE variable at a time.

78. Test
Allow time for Propagation

79. Test
Allow time for Propagation
On Premises servers give control over timer jobs.
O365 does not.
Sometimes, waiting overnight can let the solution work.

80. Test
Example
So we uninstalled MMA and Voila!
Central Admin was up again.

81. Test
Example
Sites were up again

82. Test
Example
Our solution was, um, non-functional.

83. Philosophical Razors
A principle or rule of thumb that allows one to eliminate
("shave off") unlikely explanations for a phenomenon, or
avoid unnecessary actions.
•Hume's razor: "If the cause,
assigned for any effect, be not
sufficient to produce it, we must
either reject that cause, or add to it
such qualities as will give it a just
proportion to the effect.

84. Test
Example
Seems that the IT supplier’s attempts to fix SharePoint
before bringing us on board had had some
consequences to our Nintex heavy solution.
Workflows were not firing reliably, Jsignature had quit,
and other minor issues.

85. Test
Example
It is important to test all points of a system you have
fixed, as issues can appear in areas you weren’t
specifically working on.

86. Test
Example
Although, from the IT suppliers perspective, we had
fixed their mistake, we now knew that we were dealing
with 2 problems.
The one we had fixed,
and whatever the IT supplier had done.

87. Deploy

88. Deploy
Usually, this would be taking the fix/solution from Dev into Prod

89. Deploy
Usually, this would be taking the fix/solution from Dev into Prod
We requested that the IT supplier redeploy MMA with the
NOAPM=1 switch.
Then we tested again.

90. Deploy
It worked.

91. Deploy
And then we started working on the rest of the
problem

92. Document

93. Document
Why do we document?

94. Document
Why do we document?There might be a handover.
You might go on leave or
get sick before completing
or deploying.
You might have to escalate
the issue, internally, or
externally, perhaps to
Microsoft.

95. Document
Why do we document?
Handover
There might be a
reapplication needed.
Multiple servers or tenants
might need the same fix.
This issue might appear
again elsewhere

96. Document
Why do we document?
Handover
Reapplication
You can build a Knowledgebase
A collection of correctly
documented fix steps can be
useful. In 2 years time, will you
remember the exact fix if you
come across that problem
again?

97. Document
Why do we document?
Handover
Reapplication
Knowledgebase
Avoid duplication of effort.
Did I request a reindex yet?
Better check the documentation
I‘m making as I go.

98. Document
Why do we document?
Handover
Reapplication
Knowledgebase
Effort
CYA.
Cover your (bottom)
As a consultant, some clients
want detailed information,
sometimes months later when
they’re asked to pay.

99. Document
Why do we document?
Handover
Reapplication
Knowledgebase
Effort
CYA

100. Document
Why do we document?
Handover
Reapplication
Knowledgebase
Effort
CYA

101. Document
When do we document?

102. Document
When do we document?
Throughout the process!

103. In Conclusion
• Define
• Replicate
• Research
• Hypothesis
• Test
• Deploy
• Document
Troubleshooting methodology 101

104. In Conclusion
Troubleshooting
Is
Not
Linear