Your SlideShare is downloading. ×
0
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Real-time Event Processing using LINQ for Logs & Traces
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Real-time Event Processing using LINQ for Logs & Traces

416

Published on

Dealing with real time events is hard, but required to gain visibility in the clouds; perform low-latency monitoring (seconds) or troubleshoot large number of machines at the same time. LINQ for …

Dealing with real time events is hard, but required to gain visibility in the clouds; perform low-latency monitoring (seconds) or troubleshoot large number of machines at the same time. LINQ for Traces (http://tx.codeplex.com/) provides a powerful way to deal with standing queries over real-time data feeds, such as Event Tracing for Windows (ETW) sessions or performance counters, as well as ad-hoc queries over past history in the form of trace and log files. This session will take you through the discovery and exploration of TX by building a real-time event analysis solution for web services. The presentation is based on building the actual monitoring event pipeline for Xbox Live Services.

Published in: Software, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
416
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Real-time Event Processing using LINQ for Logs & Traces (TX) Gert Drapers (#DataDude) Principle Software Design Engineer
  • 2. Agenda •Rx (Reactive Extensions) •Tx (LINQ to Logs and Traces) •Case Study Subtitel
  • 3. TX (LINQ to Logs and Traces) •http://tx.codeplex.com/ •TX allows you use LINQ directly on raw event sources: • Ad-hoc query on past history in trace and log files • Standing queries on real-time feeds, such as Event Tracing for Windows (ETW) sessions
  • 4. Builds on Reactive Extensions (RX.Net) http://rx.codeplex.com/
  • 5. Reactive Essential Interfaces namespace System { // Producers (.NET events, WinRT Events, Sensors, APM method etc.) public interface IObservable<out T> { IDisposable Subscribe(IObserver<T> observer); } // Events stream consumers public interface IObserver<in T> { void OnNext(T value); void OnError(Exception error); void OnCompleted(); } }
  • 6. Event processing at all scales with Reactive ExtensionsBart de Smet Thrusday 17 april 13:15 - 14:30
  • 7. Demo Using TX with LINQPad
  • 8. Event Sources • Tx.Windows • Event Tracing for Windows (ETW) • Performance Counters - interop with the "Performance Data Helper" (PDH) native API • Event Logs (.evtx) • IIS W3C text log files • Tx.SqlServer • SQL Server Extended Events (XEvent) is the tracing technology used by the SQL-Server engine. • Extensibility
  • 9. Type Generation
  • 10. Playback •Structured Mode • Lose analogy is that events of given type are like a "Table“ • playback.GetObservable< T> is similar to dataContext.GetTable<T> in LINQ to SQL • More precisely, IObservable is a sequence (i.e. order matters) • This is the same distinction as IEnumerable vs.. SQL Table •Timeline Mode • This mode is useful to ask questions that are about the timeline of all events, regardless of type
  • 11. Playback Features •Multiplexed sequences •Hiding the heterogeneity of the event sources •Same API for real-time and past history •Single-Read to answer multiple queries from file(s) •Occurrence Time Scheduler
  • 12. Demo HttpSpew
  • 13. Case Study XblWatch •Objective of XblWatch: • Replace existing rule based SCOM pack monitoring of web service • Provide near-realtime monitoring • Create Synthetic Performance Counters for success and failure rates per API and service • Publish Synthetic Performance Counters to integrate with existing monitoring and alerting system
  • 14. XblWatch/er xblwatcher.prod.live XblWatcher.exe XblWatchCmd.exe http://*:80 tcp://*:9000 Perf Counters Site/Service/API Machine 1 XblWatch.exe http.sys ETW trace Perf Counters Machine/Service/API Machine 2 XblWatch.exe http.sys ETW trace Perf Counters Machine/Service/API Machine 3 XblWatch.exe http.sys ETW trace Perf Counter Machine/Service/API Machine N XblWatch.exe http.sys ETW trace Perf Counters Machine/Service/API
  • 15. Demo XblWatch
  • 16. Resources •Tx (LINQ to Logs and Traces) •(Rx) Reactive Extensions •Event processing at all scales with Reactive Extensions Bart de Smet Thrusday 17 april 13:15 - 14:30
  • 17. Laat ons weten wat u vindt van deze sessie! Vul de evaluatie in via www.techdaysapp.nl en maak kans op een van de 20 prijzen*. Prijswinnaars worden bekend gemaakt via Twitter (#TechDaysNL). Gebruik hiervoor de code op uw badge. Let us know how you feel about this session! Give your feedback via www.techdaysapp.nl and possibly win one of the 20 prices*. Winners will be announced via Twitter (#TechDaysNL). Use your personal code on your badge. * Over de uitslag kan niet worden gecorrespondeerd, prijzen zijn voorbeelden – All results are final, prices are examples

×