Monitoring allows organizations to be proactive by detecting problems, scaling infrastructure automatically, and reacting faster to issues. It provides visibility into what is affecting systems and can enable automatic fixes. When errors occurred at an organization, monitoring systems alerted engineers within 12 minutes, and automatically resolved the original issue and deployed a new configuration, rapidly resolving the problems. Effective monitoring provides automatic alerting, limits business impact from issues, and can facilitate proactive infrastructure fixes and scalability.

1. Monitoring a
Microsoft Dynamics CRM
Infrastructure
Stéphane Dorrekens
RealDolmen
EXTREMECRM
2014 BARCELONA

2. • Monitoring allows
– knowing what hits you
– Proactivity
– Faster Reaction Time
• And also
– Automatic fixes
– Automatic performance scalability
Users began seeing these errors on affected services at 11:02 a.m.,
and at that time our internal monitoring alerted Google’s Site Reliability Team.
Engineers were still debugging 12 minutes later when the same system,
having automatically cleared the original error, generated a new correct configuration at 11:14
and began sending it; errors subsided rapidly starting at this time.
Google Official Blog, 26th January, 2014

3. Introduction
o Health, Performance and E2E Monitoring
o Microsoft Dynamics CRM Events
o Events and Services Monitoring Demo
o Microsoft Dynamics CRM
Performance Counters
o Performance Counters Monitoring and
Automatic Scalability Demo
o Microsoft
System Center Operations Manager
o Q&A
Agenda

4. Health, Performance and End 2 End Monitoring

5. Health Monitoring
– Detect problems after
they occurred
– Automatic alerting
– Allows faster recovery
– Limits/Remove business
impact
Performance Monitoring
- Detect Degradation
- Automatic alerting
- Preemptive
fix/scalability
End to End Monitoring
- Not (crm) server based
- Detect health and
performance issues
- Does not gives causes

6. MSDxCRM Events

7. MSDxCRM Events
• +600 Events in the Application Branch
• Also Generate events in the System Branch (ie:
7000, 7034, 7036 for service start, fail)
• Email Router events can be
activated separately (KB2862024)
in a MSCRMEmailLog Branch

8. Application Events Sources
MSCRM Event Source Role
MSCRMAsyncService Asynchronous Service
MSCRMAsyncService$maintenance Asynchronous Service
MSCRMCallout n/a
MSCRMDeletionService Asynchronous Service
MSCRMDeployment Deployment Web Service
MSCRMEmail E-mail Router
MSCRMKeyArchiveManager Asynchronous Service
MSCRMKeyGenerator Asynchronous Service
MSCRMKeyService Asynchronous Service
MSCRMLocatorService Discovery Web Service
MSCRMMonitoringRuntime All
MSCRMMonitoringServerRole All
MSCRMMonitoringService All
MSCRMMonitoringTest All
MSCRMPerfCouters All
MSCRMPlatform Web Application Server
MSCRMReporting Microsoft Dynamics CRM Reporting Extensions
MSCRMReportingDataConnector Microsoft Dynamics CRM Reporting Extensions
MSCRMSandboxClient Organization Web Service
MSCRMSandboxService Sandbox Processing Service
MSCRMSandboxWorker Sandbox Processing Service
MSCRMTracing All
MSCRMUnzipService Web Application Server
MSCRMVssWriter Deployment Tools
MSCRMWebService Organization Web Service
Used to detect
expired digital
certificates

9. MSDxCRM Events
22 17
53
24 19
6
18
118
24
2
34
7 10
1 8 4
118
6
1
107
5
Events List

10. Some Critical Events

11. Some Critical Events
• Platform
– 16940, 17205, 17206 (Authentication Failed)
• KeyGenerator
– 18951, 18955, 18956 (crashed)
• Async
– 17409 (crashed)
• Sandbox
– 20240, 20244 (not started, crashed)

12. Services Monitoring

13. Services Monitoring
• Alert
– Available trough System/Service Control Manager Branch
– Will triggers once per status change.
– Event 7000 (Failed to start)
– Event 7034 (crashed, corrective actions failed)
• http://technet.microsoft.com/en-us/library/dd349427(v=ws.10).aspx
– A service failing will most of the time triggers an internal MSCRM
Event and then a Service Monitoring Event
• Ex : 17409 -> 7034
• State
– Available trough Service Control Api
– Will triggers every time until status change

14. Services Names to monitor
– W3SVC
• World Wide Web Publishing Service
– MSSQLSERVER
• SQL Server
– MSCRMSandboxService
• Microsoft Dynamic CRM Sandbox Processing Service
– ReportServer
• SQL Server Reporting Services
– MSCRMAsyncService
• Microsoft Dynamic CRM Asynchronous Processing Service
– MSCRMAsyncService$maintenance
• Microsoft Dynamic CRM Asynchronous Processing Service (maintenance)
– MSCRMMonitoringService
• Microsoft Dynamic CRM Monitoring Service (2013 only)
– MSCRMUnzipService
• Microsoft Dynamic CRM Unzip Service
– MSCRMVssWriterService
• Microsoft Dynamic CRM VSS Writer (2013 only)
– Wsearch
• Windows Search
External dependencies (AD, DNS, Exchange, ADFS, …)

15. Events Tools

16. Events Tools
– Event viewer (eventvwr)
• Live display
• Filtering
• Export
• Basic Alerting
– Task Scheduler (schtasks)
• On alert trigger
– Powershell
• Get-WinEvent cmdlet (replaces getEventLog)
• Live or read exported data
• Xpath can be extracted from Event Viewer or Task Scheduler filter
• Get-Service cmdled (get Service State)
• Automation is done via Task Scheduler (schtasks)

17. Alerting
Events and Services
Demo
17

18. MSDxCRM Performance counters

19. MSDxCRM Performance counters
• +950 Counters in CRM Branches
– (25% of all counters in a Full CRM Server)
• Other noteworthy counters in :
– Processor, W3SVC, .NET CLR, ASP .NET, SQLServer,
LogicalDisk, Memory

20. Performance counters & Roles
Counter Path Role
CRM Async Service Asynchronous Service
CRM Authentication Web Application Server
CRM ConfigDB Deployment Web Service
CRM Discovery Discovery Web Service
CRM LocatorService Discovery Web Service
CRM OutlookSync Web Application Server
CRM Platform Web Application Server
CRM Router Service E-mail Router
CRM Sandbox Client Sandbox Processing Service
CRM Sandbox Host Sandbox Processing Service
CRM Server Web Application Server

21. Performance counters Categories
862
10
1
6
4
8
23
27
18
TOTAL

22. Async Performance counters
14
22
14
14
14
14
14
14
14
14
14
19
9
14
14
14
14
14
14
48
14
14
14
15
14
14
14
14
22
14
14
14
14
15
14
22
14
14
14
14
14
14
14
14
14
14
14
14
15
14
14
15
42
15
15
TOTAL
Counters List

23. Counters Guidelines

24. Counters Guidelines
• Always in relation to a timeframe (ie: every x sec)
• Threshold needs to be adjusted to every
implementation, timeframe.
• Performance
– is monitored via
• Outstanding, waiting state (too much operations)
• Average time spent (operation too slow)
– Event 17972 (Db Query > 10 sec)
• Health
– Usually better monitored via Events
– Is monitored via
– Failed counters

25. Typical Counters
Operations Completed
Operations Completion Throughput
Operations Executing
Operations Failed
Operations Failed With Retry
Operations Outstanding
Rate of Operations Failed With Exception
Rate of Operations Failed With Retry
Average time spent in operation
Average time spent in waiting state
Average time spent in throttled state
Operations Waiting on I/O
Operations Throttled
Operations Resumed Prematurely
Health
Performance
Health
Performance

26. Performance counters Tools

27. Performance counters Tools
– Typeperf
• http://technet.microsoft.com/en-us/library/bb490960.aspx
– Performance monitor (perfmon*)
• Live display
• Filtering
• Scheduled collection
• Alerting or generic tasks (Trigger is only above level)
• Can be saved as template
– Powershell
• Get-Counter cmdlet to Get Live data
• Export-Counter cmdlet to save data collection points
• Import-Counter to import saved data collection points
• Xpath can be extracted from Perfmon filter
• Alerting or more complex actions like automatic scalability
*Use mmc /add snapin to save counters configuration

28. Alerting and automatic scalability
PerformanceCounters
Demo
28

29. MSCRM 2013
Full Server
Monitoring &
Spare Server
Stress Server
Email
Demo Setup

30. System Center Operations Manager

31. System Center Operations Manager
– Management Pack currently exists only for CRM 2011 (new
services not monitored, events)
– Pack is designed for SCOM 2007 but can be imported in
SCOM 2012R2
– Contains both health and performance monitoring
– Performance monitoring is not enabled by default as the
thresholds needs to be defined for each implementations

32. Priorities and Escalations

33. Priorities and Escalations
• Alerting is not enough
• For every alert, you need a specific priority (ie:
P1,P2,P3,..)
• For most alerts
(ie: all P1,P2; less P3,P4)
you need escalation and
resolutions procedures

34. Escalations Procedures
Monitor Target Category Enabled Monitored Health States
Production
Non
Production
Microsoft Dynamics CRM Asynchronous Processing Service (maintenance)
Microsoft Dynamics CRM
2011 Asynchronous
Processing Service
Availability Health Yes Automatic Service
P3 P4
Microsoft Dynamics CRM Asynchronous Processing Service
Microsoft Dynamics CRM
2011 Asynchronous
Processing Service
Availability Health Yes Automatic Service
P2 P3
E-mail Router Service
Microsoft Dynamics CRM
2011 E-mail Router
Availability Health Yes Automatic Service
P2 P3
Indexing Service
Microsoft Dynamics CRM
2011 Help Server
Availability Health Yes Automatic Service
P3 P4
World Wide Web Publishing Service
Microsoft Dynamics CRM
2011 IIS Dependent Server
Availability Health Yes Automatic Service
P1 P2
The SQL Server Reporting Services: MSSQLSERVER
Microsoft Dynamics CRM
2011 Reporting Extensions
Availability Health Yes Automatic Service
P3 P4
Microsoft Dynamics CRM Sandbox Processing Service
Microsoft Dynamics CRM
2011 Sandbox Processing
Service
Availability Health Yes Automatic Service
P4 P5
File Server Resource Manager service is not running
Microsoft Dynamics CRM
2011 Web Application
Server
Availability Health Yes Automatic Service
P4 P5
Microsoft Dynamics CRM Unzip Service is not running
Microsoft Dynamics CRM
2011 Web Application
Server
Availability Health Yes Automatic Service
P3 P4

36. • Not everything makes sense to monitor
• Uses (mainly) Events and Services State
for health monitoring
• Use (mainly) Performance counters
for performance monitoring
• Use Traces for incident analysis
• Monitoring is more than alerting, it can triggers
auto-remediation or scalability.
• Alerting is not enough, you need
escalation procedures as well.

37. Stéphane Dorrekens, RealDolmen
Stephane.Dorrekens@realdolmen.com
@stephanedujour
http://blog.dorrekens.com
Please remember to fill out your session evaluation survey online!
The link to the survey was emailed to you, or go to:
http://www.extremecrm.com/2014Barcelona/2014BarcelonaSurvey.aspx
Complete prior to the closing session to be included in today’s drawing!

38. Survey: http://www.extremecrm.com/2014Barcelona/2014BarcelonaSurvey.aspx
THANK YOU to all of our eXtremeCRM 2014 Barcelona Sponsors!
CORPORATEGOLD SILVER
BRONZE
EXHIBITORS