1. Exchange Server 2013 improves mailbox server availability by consolidating previously separate roles onto single mailbox servers and replicating mailbox databases across multiple servers in a database availability group (DAG). 2. The mailbox transport service was redesigned with separate submission and delivery components to improve mail flow, and transport is now located on mailbox servers rather than front-end servers. 3. Exchange 2013 provides increased mailbox sizes up to 100GB, modernized public folders based on the mailbox model, and improved search capabilities with a new search foundation.

1. Exchange Server 2013
Architecture Deep Dive, Part 2
Scott Schnoll
Principal Technical Writer
Microsoft Corporation

2. Agenda
Mailbox Server Role
Availability Improvements

4. Mailbox Server Role
Server that hosts components that process, render and store Exchange
data
Includes components previously found in separate roles
Connectivity to a mailbox is always provided by the protocol instance on
the server hosting the active database copy

5. Database Availability Group
Collection of servers that form a unit of high availability
Boundary for replication and *over
DAG members can be in different sites
Can have a maximum of 16 Mailbox servers

6. Mailbox Server Role Changes
Managed Store and IOPS reductions
Transport-related Changes
Larger mailbox support
Modern public folders
New search infrastructure

7. Covered in Database and Store Changes

9. Transport-related Changes
Transport on Mailbox server is comprised of three services:
• Microsoft Exchange Transport - Stateful and handles SMTP mail flow
for the organization and performs content inspection
• Microsoft Exchange Mailbox Transport Delivery - Receives mail from
the Transport service and deliveries to the mailbox database
• Microsoft Exchange Mailbox Transport Submission - Takes mail from
the mailbox databases and submits to the Transport service

10. Transport-related Changes
Transport has the following responsibilities
• Receives all inbound mail to the organization
• Submits all outbound mail from the organization
• Handles all internal message processing such as transport
rules, content filtering, and antivirus
• Performs mail flow routing
• Queue messages
• Supports SMTP extensibility

11. Transport Service Architecture
SMTP from FET or the Mailbox SMTP to FET or Mailbox
Transport service on other servers Transport service on other
servers
Transport Service
SMTP
Receive
Protocol
Agents
Categorizer
Routing SMTP Send
Agents
Submission Delivery
Queue Queue
Pickup/Repla
y Directory
SMTP from Mailbox Transport SMTP to Mailbox Transport
Submission service Delivery service

12. Mailbox Transport Service Architecture
SMTP from Transport SMTP to Transport
Service Service
Mailbox Transport
SMTP Receive service SMTP Send
Hub Selector
(Router)
Store Driver
Deliver
Mailbox Store Driver Submit
Deliver
Mailbox
Agents Mailbox Submit
Assistants Agents
Mailbox Transport Delivery Mailbox Transport Submission
MAPI MAPI
Mailbox Database

13. Mailbox Transport
Two separate services to handle mail submissions (from the store) and
mail delivery (from the Transport service)
Mailbox Assistant and Store Driver combined
Leverages SMTP (encrypted) for communication with the Transport
component and TCP465 for inbound traffic
Leverages local RPC for delivery to store
Is stateless and does not have a persistent storage mechanism

14. Mailbox Transport
When receiving a message, the Mailbox Transport component can either
deliver the message or not deliver the message
If non-delivery is chosen, then the Mailbox Transport component must
provide a response back to Transport
• Retry delivery
• Generate an NDR
• Reroute the message

16. Larger Mailbox Support
Large Mailbox Size is 100 GB+ Time Items Mailbox Size
Aggregate Mailbox = Primary Mailbox + Archive Mailbox + Recoverable Items
1 Day 150 11 MB
1 Month 3300 242 MB
1-2 years of mail (minimum) 1 Year 39000 2.8 GB
Increase IW productivity
Eliminate or reduce PST files
2 Years 78000 5.6 GB
Eliminate or reduce third-party archive solutions 4 Years 156000 11.2 GB
OST size control with Outlook 2013

18. Modern Public Folders
Public folders based on the mailbox architecture
Single-master model Privat
Public logon
Public logon
 Hierarchy is stored in a PF mailbox (one writeable) e
logon
 Content can be broken up and placed in multiple mailboxes
 The hierarchy folder points to the target content mailbox CAS2013
Hierarchy
Mailbox Content
Because it’s a mailbox, it’s in a mailbox database…thus, MBX
Mailbox
 High availability achieved through continuous replication
MBX MBX
 No separate replication mechanism 2013 2013 2013
Similar administrative features to current PFs
 No end-user changes

19. Modern Public Folders
1 - User connects to their home Public Folder
mailbox first, which should be located near their
primary mailbox.
2- Folder contents live in one specific mailbox for
that folder. All content operations are redirected
to the mailbox for that folder
3 – Folder hierarchy changes are intercepted and
written to writeable copy of Public Folder
hierarchy
4 – All Public Folder mailboxes listen for hierarchy
changes and update similar to Outlook clients
5 - When a Public Folder mailbox gets full, move
some folders to a new mailbox

21. Search Foundation
Significant improvements in query and indexing performance
Significant reduction in number of times an item is indexed

22. Search Foundation Primer
Incoming Documents Incoming Queries
CTS IMS
Word Content Content
Filter Query Parse
Break XForm XForm
“CTS Flow” MARS “IMS Flow”
Writer
Core
Catalog

23. Exchange Search Infrastructure
Transport
Transport CTS
Mailbox
Store ExSearch Index Node
CTS
Mailbox
Passive
D Id
DB Idx
B x
Log Log
Read Content

25. Availability Improvements
Managed Availability
Transport Availability Improvements

26. Managed Availability
All core Exchange functionality for a given mailbox is served by the
Mailbox server where that mailbox’s database is currently active
Mailbox access fails over when a database fails over
Protocols shift to the server hosting the active database copy
Managed Availability
Internal monitoring and high availability are tied together and can be used
to detect and recover from problems as they occur and are discovered
Best copy selection now includes health of services when selecting best
copy (best copy and server selection)

28. Exchange Monitoring History
Exchange historically has relied on external applications like System
Center Operations Manager for monitoring
SCOM uses rules to collect data and performs actions after the data is
collected
Type of Monitoring Exchange 2010
Service not running Health manifest event rule
Performance counter Health manifest counter rule
Exchange events Health manifest event rule
Non-Exchange events Health manifest event rule
Scripts, cmdlets Health manifest script rule
Test cmdlets Test Cmdlets

29. Service Health Landscape
Component based monitoring does not tell the story
Exchange Online service experience resulted in the need to change our
approach to monitoring
Scale drives automation
Exchange Server 2013 reflects these learnings

30. Learnings
External monitoring solution required significant development
investment for the service
Investments did not accrue to on-premises Exchange 2010 product
High availability components separate from the monitoring and
recovery infrastructure affect end-to-end service availability goals
Scalability issues with 1000s of servers and databases
Monitoring solution had separate failure modes
Database health is not a true indicator of end-to-end service
Existing monitoring solution was focused on system and components, not on end user experience

31. Exchange 2013 Monitoring
Recovery oriented
Bringing the Monitoring based Protect experience
learnings from on the end user’s through recovery
the service to experience oriented computing
the enterprise

32. Cloud Trained
5+ Years of Directly Operating the Service
Since 2007, the Exchange Team has been operating a cloud version of Exchange
Knowledge Is Put Back Into the Product
Engineers are on-call for service related issues
Drives accountability for awareness and motivates team toward auto-recovery
Scale, Auto-Deployment, Optics, High Availability are key tenets
Decentralized complex processing
Rollouts don’t require extra configuration

33. User Focused
If you can’t measure it, you cannot manage it
Customer Touch Point Framework
Availability - Can I access the service?
Latency - How is my experience?
Errors - Am I able to accomplish what I want?

34. Recovery Oriented
Key principle: Stuff breaks, but the experience does not

35. Recovery Oriented
NLB CAS-1 DAG
MBX-1
—OWA send
OWA DB1 DB2 —OWA failure
—OWA fast recovery
—OWA verified as healthy
MBX-2 —OWA send
OWA DB1 DB2 —OWA failure
CAS-2 —OWA fast recovery
—Failover server’s databases
MBX-3 —OWA verified as healthy
—Server becomes “good”
OWA DB1 DB2 failover target (again)

36. Managed Availability Framework
ESCALATE
“take human Exchange
driven action”
Server
Managed
Availability
CHECK MONITOR RECOVER
“state of the “restore service or
world” prevent failure”
NOTIFY
Managed Availability

37. Probe Engine
PROBES
Measure customer’s perception of service
Typically synthetic end-to-end client transactions
PROBE
CHECKS
Measure actual customer traffic and become aware when they
CHECK
are experiencing issues
Typically implemented as performance counters where
thresholds can be set to detect spikes in customer failures
NOTIFY NOTIFY
Take action immediately based on a critical event
Typically exceptions or conditions that can be detected without
a large sample set

38. Monitors
MONITORS
Query the data collected by the probes and determine if an action
needs to occur based on a rule set
ESCALATE
“take human driven
action”
Depending on the rule, a monitor can escalate or initiate a
responder
Monitors can be
Healthy, Degraded, Unhealthy, Repairing, Disabled, or
Unavailable
Defines the time from failure that a responder is executed
MONITOR
“state of the world”

39. Responders
RESPONDER
A “plug-in” that executes a response to an alert generated by a
monitor
ESCALATE
“take human driven
action”
Built-in sequencing mechanism to control recovery actions
Several Types of Responders
Restart Responder – Terminates and restarts service
Reset AppPool Responder – Cycles IIS application pool
Failover Responder – Takes a MBX server out of service
Bugcheck Responder – Initiates a bugcheck of the server
RECOVER Offline Responder- Takes a protocol on a server out of service
“restore service or
prevent failure” Online Responder – Places a machine back into service
Escalate Responder – Escalates an issue
Specialized Component Responders

40. Reporting
Reporting is structured into four health groups:
Customer Touch Points – components which effect the real time, customer facing interactions
(OWA, OLK, Mobile, UM, etc.)
Service Components – components without direct real time, customer interactions (MRS, OABGen)
Server Components – physical resources of the physical server (disk space, memory, network)
Dependency Availability – server’s ability to call out to dependencies (AD, DNS, etc.)
Health groups are exposed in SCOM

41. Health Sets
A health set is a group of monitors, probes and responders that
determine whether a component within the system is healthy
The health of the set will be evaluated by a “worst of” evaluation of the
monitors in the health set
For example, OWA has these types of health sets…

42. Health Sets
CTP Proxy
Health Set Health Set
OWA.Proxy
OWA Protocol
Health Set
OWA.Protocol

43. Viewing Health Set Data
Get-ServerHealth is used to see raw health data for a server or set of
servers
Get-HealthReport operates on raw health data and provides snapshot
report
Get-MonitoringItemIdentity can be used to determine the
probes, monitors and responders that are associated with a given health
set

44. Overrides
Admins can alter the thresholds and parameters used by the
probes, monitors and responders
Enables emergency actions
Enables fine tuning of thresholds specific to the environment
Can be deployed for specific servers or for the entire environment
Server related overrides are stored in the registry
Global overrides are stored in Active Directory
Can be set for a specified duration or to apply to a specific version of the
server
Are not immediately implemented; Exchange Health Service only reads
configuration every 10 minutes (and global changes depend on AD
replication)
Wildcards are not supported (cannot override entire health set in one
activity)

45. Overrides
See what overrides have been set
Get-ServerMonitoringOverride –Server <Server>
Get-GlobalMonitoringOverride
Create an override
Add-ServerMonitoringOverride <HealthSet><Name> -Server <Server> -ItemType <Monitor,Probe,Responder> [-
Duration <Time> -ApplyVersion <Version>] -PropertyName <Property> -PropertyValue <Value>
Add-GlobalMonitoringOverride <HealthSet><Name> -ItemType <Monitor,Probe,Responder> [-Duration <Time> -
ApplyVersion <Version>] -PropertyName <Property> -PropertyValue <Value>
Remove Override
Remove-ServerMonitoringOverride
Remove-GlobalMonitoringOverride

47. Transport Availability Improvements
Every message is redundantly persisted before its receipt is
acknowledged to the sender
Delivered messages are kept redundant in transport, similar to active messages
Every DAG represents a transport HA boundary and owns its HA
implementation
If you have a stretched DAG, you also have transport site resilience
Resubmits due to transport DB loss or MDB *over are fully automatic and do not require any manual
involvement

48. Shadow Redundancy Improvements
Same fundamental concept as in Exchange 2010, with new
implementation in Exchange 2013
All mail is made redundant on a another server
Shadow messages are queued until Primary server successfully delivers the mail
Shadow server regularly heartbeats Primary server for status on the primary copy
On Primary server failure, Shadow server self-promotes itself as the Primary and delivers mail

49. Shadow Redundancy Improvements
Guaranteed Redundancy
New transport configuration – RejectOnShadowFailure ensures that no message is acknowledged
and accepted unless a shadow copy was first created
Messages are made redundant on other servers within a DAG or another site
Messages are tried for a configurable amount of time before giving up and rejecting the message

50. Safety Net
Introduced in Office 365 to redundantly store all mail for a configured time
span to protect against mailbox irrecoverable failures
SafetyNet retains data for a set period of time, regardless of whether the
message has been successfully replicated to all database copies or
delivered to final destination
Consolidates, improves and replaces Transport Dumpster
Processes replay requests from “primary” or “shadow” SafetyNet for
lossy mailbox failovers

51. Summary
Exchange Server 2013 uses Building Blocks to facilitate deployments
at all scales – from self-hosted, small organizations to Office 365
Exchange Server 2013 provides you with an architecture that is
Flexible, Scalable, and Simpler and helps customers reduce costs

52. Questions?
Scott Schnoll
Principal Technical Writer
scott.schnoll@microsoft.com
http://aka.ms/schnoll
schnoll