How Microsoft IT audits the network access

1. How Microsoft protects its Network Remus Rusanu High Volume Real Time Contiguous ETL and Audit

2. Agenda Network Access Protection NAP Audit as implemented by Microsoft IT Service Broker in 5 slides High Availability, Scale Out and Real Time Demo Similar Projects Q&A

3. Network Risks Highly connected Distributed data Mobile workers Remote access Web services Wireless Mobile smart devices

4. Network Access Protection Policy ValidationEvaluates company security policies and determines compliant computers (“healthy”) vs. non-compliant ones (“unhealthy”) Network RestrictionRestricts network access based on computer “health” RemediationApplies necessary updates for non-compliant computers to become compliant, “healthy”. Once healthy, the network restrictions are lifted Ongoing ComplianceChanges to the company’s security policy or to the computers compliance trigger a new evaluation of network restrictions Health Agents Windows Security Health Agent, SCCM, IPSec, Wireless, VPN, Forefront, DHCP, BitLocker

5. NAP Overview

6. NAP Modes Reporting Mode Backend receives metrics, no client impact Capture/analyze daily statistics of unhealthy vs. healthy clients Estimate impact to user base if enforcement enabled Deferred Enforcement Mode No network restrictions during the deferment period End users receive notifications when non-compliant Helpdesk contacted by end users in regards to notifications Enforced Mode Non compliant systems are quarantined. Productivity affected during quarantine. Health certificate required to access other NAP enable clients/servers

7. NAP Audit Network Protection Server logging: Text files SQL: exec dbo.ReportEvent @event; @event is an XML. Correlated by a session-id: Network access request (session start) Request Accepted/Request Denied Accounting information (for VPN every 10 min) The Health status is part of the second packet Status of each SHA on the computer: OS updates, firewall, anti-virus etc

8. NAP Reporting Aggregate all NAP audit events into a DW Allow analysis of Compliant/Non-compliant status and evolution Reasons for non-compliance Most frequent causes of computer quarantine Efficiency of automatic remediation Forensic analysis of computers and users activity

9. Processing NAP Audit Events Service Broker Delivery XML Shredding Transactional Replication 47 geo-distributed NPS Servers Local ReportEvent Mirrored Publication Mirroring allows for Maintenance Downtimes Mirrored Routes

10. A Crash Course on Service Broker Message based communication between SQL Server instances SEND is a T-SQL verb to send a message SEND ON CONVERSATION @handle (‘Hello, World’); RECEIVE is a T-SQL verb to receive messages Conversations are message exchange sessions Durable, persisted in the database Long lived, can be reused for days, years BEGIN CONVERSATION starts a conversation END CONVERSATION ends a conversation Any message belongs to exactly one conversation Order of delivery is guaranteed within a conversation

11. A Service Broker Application

12. The small print: all the Broker Objects Service An addressable Broker destination. Think mailing address. Message Types, Contracts Formalize the messages a Service can accept. Think COM Interfaces. Queues Where a Service keeps its messages until they are Received. Think mailbox. Remote Service Bindings Associate a targeted service with an identity (certificate) ‘when you send to service Foo, encrypt the data with certificate Bar’ Routes Specify the physical location of a Service. Think Post Master. Endpoints Configure the communication protocol to be used TCP listener port Authentication and authorization Encryption scheme Allows two SQL Server instances to connect

13. The Nugget: Activation Attach a stored procedure to a Service Broker Queue Will run when there are messages in the queue Will run a stored procedure inside SQL Server No external connection required Fully contained within the database No external process No msdb configuration No SQL Agent requirement Magically tunes itself to the load Launches new procedure instances as needed WAITFOR (RECEIVE …) is internationally LIFO When load is reduced, procedures timeout and exit Transactional semantics Will launch after a server shutdown and restart Will launch after a mirroring failover Will launch after a cluster failover Will launch after an attach or a restore The Server can crash and burn the procedure will launch when your DR procedure is complete

14. Local Availability: SQL Express If the NPS Server is running, the SQL Express is likely running too Express is light on resource usage Single CPU 1 GB RAM buffer pool 4Gb (10GB in R2) DB size Transact-SQL programming Cheap to distribute to hundreds of sites

15. Reliable Delivery: Service Broker SEND is a local transaction Never affected by the target availability Guarantees Exactly Once In Order delivery Handles retries Target downtime Connection problems can be resolved day, months even years after occurred without data loss Security can traverse domains NTLM/Kerberos Certificates Authentication, Authorization, Encryption handled at SQL endpoint configuration level

16. Scale Out: Service Broker Hundreds and thousands of peers EdConhandles +1500 data sources Abstracts physical location with ROUTEs Server relocation Heterogeneous SQL 2005/SQL 2008 Rolling upgrade of the deployed servers Available on all editions including Express High Throughput Spikes can be delivered at +6000 msgs/sec Highly optimized code path to insert into target

17. Process XML: XPath and Activation Service Broker Internal Activation readers launched when messages arrive Self-tuning reader count MAX_QUEUE_READERS No pulling! XML payload projected into columns XPath XQuery Automatic processing batching RECEIVE TOP 1000 creates a 1000 size batch to process Correlation awareness NPS packets 1 (Start) and 2/3 (Accept/Reject) processed by the same reader Original order is preserved during processing

18. DW: Transactional Replication Isolate the XML shredding from reporting Different indexes for processing vs. reporting Processing server delete data after 10 days DW retains 1 year of data (~1.5 TB) Transactional Replication Preserves order of operations Preserves transaction boundaries Easy to deploy and manage between few peers Supports mirrored publishers

19. Availability: Mirroring Activation processing is entirely DB contained No msdb jobs, no master dependencies Transactional consistent Automatically starts up on new host after failover Service Broker Routing is mirroring aware CREATE ROUTE … WITH ADDRESS = ‘tcp://principalname’,MIRROR_ADDRESS = ‘tcp://mirorrname’; Will instantly follow a failover Mirroring allows for maintenance to occur Apply CU and SP Apply OS patches

20. DEMO

21. Similar Projects Real Time Analytics with SQL Server 2008 R2 StreamInsight Silverlight media content delivery metrics nbcolympics.com, March Madness Real Time metrics with R2 StreamInsight Trends and analysis in DW Aggregated with Service Broker Processed with Activation SSIS for upload into DW

22. Silverlight Metrics Collection WCF to reportUsage Metrics StreamInsight Real Time Service Broker Local SEND Silverlight media player Activation Processing SSIS Extraction into OLAP DW

23. Critical for Performance Reuse Broker conversations Each SEND on its own conversation: ~15 writes into 6 tables (for a full round-trip) SEND on an existing conversation: 2 writes on 2 tables RECEIVE cannot batch process messages on distinct conversations

24. Gotchas Mirroring support for DB master key sp_control_dbmasterkey_password Allows Service Broker to open the database master key on the new principal, after a failover Mirroring and Service Broker routes If the mirroring session is suspended, rotes must be modified Replication and mirroring Only publisher can be mirrored Principal and Mirror must share the same distributor –PublisherFailoverPartner parameter added to the Log Reader agent Replication and SQL 2008 Upgrade rollout Publisher version must be less than Distributor version SQL Express is the have-not of monitoring No Data Collection Sets support

25. Acknowledgements Tom Baker, Senior SE Systems Engineer Roger Doherty, Senior Technical Evangelist

26. Q&A slideshare.net/rusanu @rusanu