2. Module Overview
• Monitoring Exchange Server 2010
• Maintaining Exchange Server 2010
• Troubleshooting Exchange Server 2010
3. Lesson 1: Monitoring Exchange Server 2010
• Why Is Performance Monitoring Important?
• Tools for Monitoring Exchange Server
• Collecting Performance Data for the Exchange Server
• Collecting Performance Data for the Mailbox Server
• Collecting Performance Data for the Hub Transport and
Edge Transport Servers
• Collecting Performance Data for the Client Access Server
• Using the Collected Performance Data
4. Why Is Performance Monitoring Important?
Performance monitoring can help you:
• Identify performance issues
• Identify growth trends to improve plans for
upgrades
• Measure performance against service level
agreements
• Identify security issues and denial-of-service
attacks
5. Tools for Monitoring Exchange Server
The following tools can help you monitor system
health:
• System Center Operations Manager or System
Center Essentials (with Exchange Server 2010
Management Pack)
• Third-party monitoring products
• Performance and Reliability Monitor
6. Collecting Performance Data for the Exchange Server
Suggested performance counters for all Exchange Server
roles:
Processor: System:
• _Total% Processor • Processor Queue
Time Length
• _Total% User Time
• _Total% Privilege
Time Memory:
• Available Mbytes
• Pool Paged Bytes
MSExchange ADAccess
Domain Controllers: • Transition Pages
• LDAP Read Time Repurposed/sec
• LDAP Search Time • Page Reads/sec
• LDAP Searches time out • Pages/sec
per minute
• Long running LDAP • Page Input/sec
operations/min • Pages Output/sec
7. Collecting Performance Data for the Mailbox Server
Suggested performance counters for all Mailbox server roles:
LogicalDisk: MSExchangeIS:
• Avg. Disk sec/Read • RPC Requests
• Ave. Disk sec/Write • RPC Averaged Latency
• Avg. Disk sec/Transfer • RPC Operations/sec
• RPC Num Slow Packets
• RPC Average Latency
MSExchangeIS Mailbox:
• Messages Queued for
Submission MSExchangeIS Public:
• Messages Queued for
Submission
8. Collecting Performance Data for the Hub Transport
and Edge Transport Servers
Suggested performance counters for the Hub Transport and
Edge Transport server roles:
LogicalDisk: MSExchangeTransport
Queues:
• Avg. Disk sec/Read
• Aggregate Delivery Queue
• Avg. Disk sec/Write Length (All Queues)
• Avg. Disk Queue Length • Active Report Delivery
Queue Length
• Active Mailbox Delivery
MSExchange Database Queue Length
==> Instances:
• Retry Mailbox Delivery
• Log Generation Queue Length
Checkpoint Depth
• Unreachable Queue Length
• Version buckets
allocated • Largest Delivery Queue
Length
• Log Record Stalls/sec
• Poison Queue Length
9. Collecting Performance Data for the Client
Access Server
Suggested performance counters for the Client Access server
role:
LogicalDisk: MSExchange OWA:
• Avg. Disk sec/Read • Average Response Time
• Ave. Disk sec/Write • Average Search Time
ASP.NET: MSExchange ActiveSync:
• Application Restarts • Average Request Time
• Worker Process Restarts
MSExchangeFS OAB:
• Requests Current
• Download Task Queued
• Request Wait Time
MSExchange Availability
ASP.NET Applications: Service:
• Requests in Application • Average Time to Process
Queue a Free Busy Request
RPC/HTTP Proxy:
• Number of failed back-end connection attempts per second
10. Using the Collected Performance Data
To use the collected performance data:
Create a baseline:
• Monitor performance for a full business cycle
• Note any peaks or troughs in the data
Set warning and error level thresholds
Review growth trends regularly to:
• Adjust threshold
• Adjust server configuration
11. Lesson 2: Maintaining Exchange Server 2010
• Discussion: What Is Change Management?
• Considerations for Managing Change
• Process for Deploying Exchange Software Updates
• Determining the Need for Hardware Upgrades
12. Discussion: What Is Change Management?
• How does your organization address change management?
• Are there some situations where change management is
more important than others?
• What are the benefits of having a formal change
management process?
• Are there situations in which you cannot follow the normal
change process?
13. Considerations for Managing Change
• Define a process and use it consistently
• Support the change management process
• Use a process model like MOF to describe a system
life cycle
14. Process for Deploying Exchange Software Updates
Exchange Server is updated with rollup packages and service
packs
When updating Exchange Server, consider the following:
• During update, Exchange Server connects to CRLs
• Update process may update customized Logon.aspx
file for Outlook Web App
• Apply updates to the Internet-facing Client Access
servers first
• Exchange Services are stopped during the update
process
• DAG members should be updated by following a
specific procedure
15. Determining the Need for Hardware Upgrades
• Investigate causes of poor user experiences
• Monitor processor counters
• Monitor memory counters
• Monitor disk counters
• Monitor network counters
• Scale out instead of scale up
17. Developing a Troubleshooting Methodology
1 Clearly define the problem
2 Define the problem scope
Gather information related to the problem
• Turn up logging
3 • Review logs
• Try to reproduce the problem
4 List the probable cause of the problem
5 Rank the cause by probability and define solutions
6 Rank solutions by ease of resolution and impact to complete
Try the most probable and easily implemented resolution until
7 the problem is resolved
8 Reduce logging to normal
9 Document resolution and root cause for future reference
18. Troubleshooting Tools
All Exchange servers:
• Best Practices Analyzer
• Performance Troubleshooter
• Network Monitor
• Performance and Reliability Monitor
• Test cmdlets
• Microsoft Remote Connectivity
Analyzer
Hub Transport and Edge
Transport servers:
• Mail Flow Troubleshooter
• Message tracking
• Queue Viewer
• Routing Log Viewer
• Telnet
20. Discussion: Troubleshooting Client Access Servers
Outlook users no longer can connect to the system. What
process can you use to troubleshoot the problem?
23. Lab Scenario
You are the messaging administrator at A. Datum
Corporation. You need to configure basic monitoring by
using the Performance and Reliability Monitor. You also
must troubleshoot issues with a mailbox database and a
Client Access server.
24. Lab Review
• Was the Exchange Best Practices Analyzer helpful in
troubleshooting the database error? When might using
Exchange Best Practices Analyzer be a better fit?
• Why do you need to run IISReset after reconfiguring
Outlook Web App?
25. Module Review and Takeaways
• Review Questions
• Common Issues and Troubleshooting Tips
• Real-World Issues and Scenarios
• Best Practices
Editor's Notes
Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B Presentation: 70 minutes Lab: 60 minutes After completing this module, students will be able to: Monitor the Microsoft® Exchange Server 2010 messaging system Maintain the Exchange Server 2010 messaging system Troubleshoot problems with the Exchange servers Required materials To teach this module, you need the Microsoft Office PowerPoint® file 10135B_11.ppt. Important: It is recommended that you use PowerPoint 2002 or a newer version to display the slides for this course. If you use PowerPoint Viewer or an earlier version of PowerPoint, all the features of the slides might not be displayed correctly. Preparation tasks To prepare for this module: Read all of the materials for this module. Work through the Module Review and Takeaways section, and determine how you will use this section to reinforce student learning and promote knowledge transfer to on-the-job performance.
Course 10135B Module 11: Maintaining Microsoft Exchange Server 2010
Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Although it might be obvious that performance monitoring is important, discuss why monitoring is important and why students should apply what they learn from this module to their Exchange Server 2010 environments. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss the main tools used for monitoring system health. Most enterprise environments already have a performance-monitoring and alerting system in place. In cases where a monitoring solution does not exist, Microsoft System Center Operations Manager 2007 or System Center Essentials (with the Exchange Server 2010 management pack) provide an easily deployable Exchange monitoring solution. Enterprise-class monitoring solutions also allow you to customize the data you want to collect, which can be helpful when tracking down specific problems or when default monitoring sets do not collect the appropriate data. You can use the Performance and Reliability Monitor to collect performance data when an enterprise monitoring solution is not available or is undesirable. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss each of the listed counters, why they are important to trend, and any suggested values that may indicate a problem. References Exchange Server 2007 Help: Monitoring Common Counters Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss each of the listed counters, why they are important to trend, and any suggested values that may indicate a problem. Emphasize that these lists are not exhaustive, but rather are a suggested beginning. There are many other performance counters that may be beneficial to trend, depending on the messaging environment. Question: If any of these performance counters measured outside its normal range, what is the most likely cause? Answer: Slow client response will cause most of the mailbox performance counter data to be outside the normal range, whether the client is Microsoft Office Outlook® Live or the full Microsoft Office Outlook client. References Exchange Server 2007 Help: Monitoring Mailbox Servers Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss each of the listed counters, why they are important to trend, and any suggested values that may indicate a problem. Emphasize that these lists are not exhaustive, but rather a suggested beginning. There are many other performance counters that may be beneficial to trend, depending on the messaging environment. Question: If any of these performance counters measured outside its normal range, what is the most likely cause? Answer: Slow email delivery will result in many of the transport counters being outside the normal range. References Exchange Server 2007 Help: Monitoring Hub Transport Servers Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss each of the listed counters, why they are important to trend, and any suggested values that may indicate a problem. Emphasize that these lists are not exhaustive, but rather a suggested beginning. There are many other performance counters that may be beneficial to trend, depending on the messaging environment. Question: If any of these performance counters measured outside its normal range, what is the most likely cause? Answer: Most of the measurements that are outside the normal range are the result of a slow response from Outlook Live, Outlook clients, Internet Message Access Protocol (IMAP) and Post Office Protocol (POP) clients, Exchange Web Services, or the Autodiscover service. References Exchange Server 2007 Help: Monitoring Client Access Servers Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discuss how to create a monitoring baseline by reviewing monitoring data over a full business cycle. A business cycle will vary for each business, but should include the busy and slow times. For some businesses, the busy times might correlate with the end-of-month accounting-close process or periods with notably high sales figures. Gathering a broad data set will provide enough data to determine the appropriate operating thresholds. Once you set thresholds are set and enable monitoring, a periodic review is important so that you can adjust the servers or thresholds to ensure proper monitoring. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discussion time : 15 minutes Conduct a group discussion in which students describe change management, and list the importance and benefits of having a clearly documented process. The students can provide real-life experiences or come up with experiences for using a process. Have the students describe change management, and then list the benefits of having a clearly documented process, perhaps by describing some real-life scenarios in which having a documented process benefited them. Question: How does your organization address change management? Answer: Answers will vary. Some organizations have a formal change management process, but these typically are larger organizations. Students from smaller organizations may not have a formal process. Question: Are there some situations where change management is more important than others? Answer: Change management is important in all situations, to prevent unintended consequences. However, for those changes that are likely to impact many users or high-profile users, change management is even more critical. Changes to mission-critical software, such as a messaging system, also tend to be more critical than changes to noncritical software, such as software for a backup server. Question: What are the benefits of having a formal change management process? Answer: Benefits include: Other organizational stakeholders are aware of changes, and can gauge the impact on their systems and staff. Multiple changes are coordinated to ensure that they do not conflict. Formalizing the change process ensures that it is consistent so mistakes are not made. Change management provides additional reviews, and allows time for additional planning, if required. Changes without a formal review often are thought out poorly, and organizations do not consider every alternative. As an IT professional, using the change management process can help deflect blame in situations where there are problems during a change. You can improve recovery times from change problems by including a formal back-out plan as part of the change management process. Question: Are there situations in which you cannot follow the normal change process? Answer: Yes, there are emergency situations in which services are broken and you cannot follow the full change management process. However, there should be an emergency change process in place to handle those situations. For example, if a critical service is down, it is not realistic to document and approve a detailed plan to solve the problem. The first priority is repairing the failed service. However, the changes made when repairing the service should be documented and evaluated afterwards to ensure there will be no negative effects on other services. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Key message: Discuss the considerations for managing change in a production messaging environment. The process for change management varies widely between organizations. Ask your students how their organizations handle change management. You then can suggest how introducing a formal change management process can benefit them and their organizations. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Key message: Describe how Exchange Server is updated with r ollup packages and s ervice p acks . Explain what is a rollup package and what is service pack. Also, emphasize that both update types are cumulative. After that , discuss considerations for updating Exchange servers, especially if the y are in a cluster. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Key message: Describe how to determine the need for hardware upgrades. Exchange Server 2010 makes more efficient use of hardware than Microsoft Exchange Server 2003 and Microsoft Exchange Server 2007. Students may have less need to upgrade hardware than in the past. In particular, gains have been made in reducing disk activity. Disk capacity is one of the most commonly needed hardware upgrades. Also, discuss scale out approach instead of scale up. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Key message: There are many troubleshooting methods, which vary depending upon the type of problem that you are resolving. Having a repeatable process is important so that you can quickly resolve problems. Discuss the troubleshooting process and ask for input from the students on what processes work for them. Question: Why is it important to have a methodology for troubleshooting? Answer: Answers will vary. Troubleshooting a problem quickly and efficiently requires an organized process. Students from smaller organizations may not have a formal process, however learning to be efficient will help them. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Key message: The students must be familiar with the built-in troubleshooting tools. Discuss each of the tools that you can use for troubleshooting problems and when you use them. Solicit responses from the students on other tools that they could use. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discussion time : 15 minutes The goal of this discussion is to have the students come up with ideas about what to do in the case of a database or mailbox issue. Ask the class the discussion question. Write their answers on a whiteboard, and reference them during the rest of the lesson. Question: A database has gone offline. What process can you use to troubleshoot the problem? Answer: Answers may vary. The following is one suggested answer: Identify which databases have the problem. Review logs, and run the Database Troubleshooter tool. Review the probable causes of the problem. Rank causes by probability, and review possible solutions. Rank solutions by ease of resolution and impact to complete. Try the most probable and easily implemented resolutions until you resolve the problem. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discussion time : 15 minutes The goal of this discussion is to have the students come up with ideas about what to do if a client-access issue occurs. Ask the class the discussion question. Write their answers on a whiteboard, and reference them during the rest of the lesson. Question: Outlook users no longer can connect to the system. What process can you use to troubleshoot the problem? Answer: Answers may vary. The following is one suggested answer: Identify which users are experiencing the problem, and when the problem began. Review logs for any involved Client Access servers. Run the Exchange Best Practices Analyzer. Review the problem’s probable causes. Rank causes by probability, and review possible solutions. Rank solutions by ease of resolution and impact to complete. Try the most probable and easily implemented resolution until you resolve the problem. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Discussion time : 15 minutes The goal of this discussion is to have the students develop ideas about how to handle a message-transport issue. Ask the class the discussion question. Write their answers on a whiteboard, and reference them during the rest of the lesson. Question: Users are reporting non-deliverable and slow-to-deliver outbound email. What process can you use to troubleshoot the problem? Answer: Answers may vary. The following is one suggested answer: Identify which users are experiencing the problem and when the problem started. Use the Mail Flow Troubleshooter, message tracking system, Queue Viewer, Routing Log Viewer, and Telnet to pinpoint the problem Review the probable causes of the problem. Rank causes by probability, and review possible solutions. Rank solutions by ease of resolution and impact to complete. Try the most probable and easily implemented resolution until you resolve the problem. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
In this lab, students will monitor and troubleshoot Exchange Server 2010. Exercise 1 Inputs: Students will be provided with information on the Exchange Server components that they need to monitor. Outputs: Students will have implemented a monitoring plan for the most important counters for each Exchange Server role. Exercise 2 Inputs: Students will be provided with a scenario that describes a database failure. They also will be required to run a script that creates the problem (or the virtual machine will need to be preconfigured with the issue). Outputs: Students will use the troubleshooting methodology that this module describes to identify the cause of the database availability issue and resolve the problem. They will verify that their solution addressed the situation. Exercise 3 Inputs: Students will be provided with a scenario that describes a client access failure. They also will be required to run a script that creates the problem (or the virtual machine will need to be preconfigured with the issue). Outputs: Students will use the troubleshooting methodology that this module describes to identify the cause of the client access issue, and will resolve the problem. They will verify that their solution addressed the situation. Before the students begin the lab, read the scenario associated with each exercise to the class. This will reinforce the broad issue that the students are troubleshooting, and will help to facilitate the lab discussion at the module’s end. Remind students to complete the discussion questions after the last lab exercise. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Use the questions on the slide to guide the debriefing after students complete the lab exercises. Question: Was the Exchange Best Practices Analyzer helpful in troubleshooting the database error? When might using Exchange Best Practices Analyzer be a better fit? Answer: Exchange Best Practices Analyzer did not help you identify database errors. The Best Practices Analyzer is best used when troubleshooting intermittent errors, configuration errors, and proactively to ensure best practices are being applied. Question: Why do you need to run IISReset after reconfiguring Outlook Web App? Answer: After making the configuration change, the Exchange Management Console instructs you to restart Microsoft Internet Information Services (IIS) so that the new configuration options can be applied. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
Review Questions Question: Users are reporting issues with sending email to a remote domain. You need to determine and resolve the problem. What should you do? Answer: Use the Mail Flow Troubleshooter and the Queue Viewer to review the queued messages and the status of the queues. Question: Recent organizational growth has resulted in two issues. It has caused several memory thresholds to exceed recommended limits, as well as the average read-latency threshold for the logical disk that stores the page file. What issue should you address first? Answer: First, add memory to the server. When there is not enough available memory, memory is paged out to the page file, which can lead to an increased amount of input/output (I/O) on the disk that stores the page file. Question: After reviewing the trend information retrieved from the monitoring system, you noticed that the processor usage for one of the four Mailbox servers is higher than average. What should you do? Answer: Determine which processes are using the additional processor time, and check for changes in mailbox usage on the servers. To solve the problem you may be able to move mailboxes to other Mailbox servers or add additional processing capabilities to the current server. Common Issues and Troubleshooting Tips Point the students to possible troubleshooting tips for the issues that this section presents. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B Issue Troubleshooting tip Outbound email messages are queuing on the Hub Transport server. Always start with the most common problem causes, such as network connectivity and domain name system (DNS) name resolution. Multiple sources are simultaneously reporting different problems. Gather as much information as possible about each of the reported problems. Although there might be multiple issues, it is likely that you will find a connection between the multiple reported problems. Users are reporting slowness or other subjective problems. As always, take each report seriously and try to gather as much objective information about the problem as possible. Only then will you reach a suitable and objective solution.
Real-World Issues and Scenarios Question: A company has recently experienced growth because of a popular new product. The company has had numerous Mail server outages and downtime due to undocumented changes. What should the company invest in to ensure that it can support continued growth? Answer: To control downtime and constant changes that are required to keep the company growing, the company should adopt a change management process. Question: A database has gone offline, and the organization needs to troubleshoot the problem. A number of impatient users have mailboxes stored in the offline database. What is the best way to address the situation? Answer: Follow a proven troubleshooting technique. Stressful situations make it even more important to stick to a proven methodology. Question: An Exchange Server service pack was recently released, and the company has decided to deploy it. What should you do before scheduling the deployment? Answer: Thoroughly test and document the deployment and server backup. Testing should include functionality and compatibility testing with the company’s systems. Best Practices Help the students understand the best practices that this section presents. Ask students to consider these best practices in the context of their own business situations. Follow the same steps each time you troubleshoot a problem. This way you get into a habit of making good decisions and finding the answers quickly. Be diligent about separating facts about the issue from feelings or other subjective information. A single person’s subjective observation could cause you to troubleshoot the wrong problem and delay resolution of the actual issue. Ask a lot of questions about the problem before starting to troubleshoot. If you have not properly defined the problem, you cannot properly target your troubleshooting steps. Module 11: Maintaining Microsoft Exchange Server 2010 Course 10135B
