Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cashing in on logging and exception data


Published on

Logging and Exception handling is one of the easiest tools to use when debugging; but how can you take those massive logs, thousands of errors and effortlessly use them to build a better product? This presentation share our developers team's lesson-learned to expedite releases and fix app issues faster. It discuss best practices that will help your dev team build a culture of logging such as: what to log, how to log it, and how to proactively put it to use.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Cashing in on logging and exception data

  1. 1. Cashing In on Logging and Exception Data Jason Taylor CTO, Stackify
  2. 2. “Logging is hard” -No one, ever.
  3. 3. 3 Logging: not hard, but not perfect The Easy • Libraries: log4net, nLog, Elmah, Enterprise Library • Very easy to integrate and use • Great for debug The Not Easy • Production log file access • Volume of data * Number of servers • Rotation and retention • Flat and hard to query • Getting enough context • Proactively consuming / monitoring
  4. 4. 4 Today’s topics Log All The Things Building a “culture of logging” to always have relevant, contextual logs that don’t add overhead. Make the most of your logging framework. Work Smarter, Not Harder Using proper tooling to consolidate & aggregate all of your logging to be available to everyone that needs it, quickly consume it, and proactively put it to use.
  5. 5. 5 What things? • Timestamp • Thread ID • Transaction ID • Server • Environment • User Data • Post Data • Cookies • Querystring • Etc, etc If you’re going to take the time to log something, log enough to give you a play-by-play picture of events. When? • For every important business transaction
  6. 6. Start Simple – handle all exceptions Pros Cons Elmah -Easy -Lots of storage possibilities -Web page to view errors -Limited email & notification possibilities -Unhandled only -Advanced usage requires a lot of configuration. -If MVC, don’t forget to grab Elmah.MVC -Don’t leave your errors web page exposed in producton! Global.asax (Application_Error) -Use the framework of your choice -Captures all exceptions -Fully customizable -You are responsible for returning state back to the method that called (e.g. a controller action) or redirecting Try/Catch blocks -Try/Catch/Finally generally perceived as a good pattern -You will catch 100% of the exceptions that you “catch” -You will miss 100% of the exceptions that you don’t “catch” -Can be “error prone” if you are only catching a certain type that doesn’t happen -Time consuming to “retro fit” into an app Exception Filters (MVC) -Like using global.asax, very customizable -You have to implement everything, including errors that happen globally, how to log, etc
  7. 7. Adding Context Treat logs like “debug when you can’t attach” • Use verbosity levels appropriately – allows you to “turn up” and “turn down” the amount of logging as necessary • Identity key processes and functions that need to be audited and logged • Capture the key steps, measure success / failure • Capture contextual data that helps you solve an issue or answer a question (i.e. logged in user, Http Request data) Don’t forget about logging in your client code • Choose a library that is smart enough to bridge server and client code while keeping context
  8. 8. 8 Adding Context - Basic Bad Slightly Better
  9. 9. 9 Adding Context - Advanced • Logged the input parameters • Logged the object created in my DbContext • Had to use a custom appender for log4net that knows how to serialize the objects Alternatively, would have to serialize to a string: log.Debug(string.Format("Creating a foo: {0}",JsonConvert.SerializeObject(foo)));
  10. 10. Adding Context – Power User!! Wouldn’t it be nice to: • Always capture the user who generated the log or exception? • Always capture basic request details? This would allow us to have deeper, more meaningful data to search through our logs.
  11. 11. Adding Context – Power User!! Context Properties (log4net) <log4net> <root> <level value="ALL" /> <appender-ref ref="StackifyAppender" /> </root> <appender name="StackifyAppender" type="StackifyLib.log4net.StackifyAppender, StackifyLib.log4net"> <logicalThreadContextKeys>User,Request</logicalThreadContextKeys> </appender> </log4net> </configuration>
  12. 12. Adding Context – Power User!! void MvcApplication_AuthenticateRequest(object sender, EventArgs e) { try { log4net.LogicalThreadContext.Properties["User"] = User; } catch (Exception ex) { log.Error(ex); } }
  13. 13. The Results: Rich Logs and Errors
  14. 14. Tips & Tricks • Use your verbosity levels • Make sure your logging framework is asynchronous • Log heavily is core / shared services & other high fan-in areas • Always log the “who” • Use patterns like AOP to write enty / exit logging once, apply everywhere • Make it a part of your development culture (don’t just write the logs – USE them)
  15. 15. Working Smarter At Stackify, we set out to make our logging and error monitoring smarter… …it became a core part of our product offering.
  16. 16. Working Smarter Logs & Errors Tell (Some of) the Story • Logs as “breadcrumb trail” are good for: • Capturing errors in raw form • Capturing key events (depending on logging level) • Piecing together the story (if you logged it) on one server at a time • Error monitoring is good for: • “After the fact” notification • Seeing just the error in question • Pointing you back at log files for more info
  17. 17. Working Smarter The Trouble With Your Current Logs: • Working with all those log files is tedious and time consuming • No cross-server / cross-app correlation • No trends / telemetry • File retention & rotation • Intermixed with Exception details
  18. 18. Working Smarter The Trouble With Your Errors: • Logged errors usually are no more than exception and stack trace • No cross-server/cross-app tracking for errors • Missing/hard to get request details • No trends / telemetry • Need intelligent notifications • In Error Monitoring apps, disconnected from Logs!
  19. 19. 19 Indexing: • Search across apps and servers w/o managing log files • Fully indexed “Context” details • Real-time log tailing • Narrow scope by time range Telemetry • Visualize trends in logging and exceptions • Across all levels of granularity • Contextual telemetry – for any app or server, by default Alerts • Notify when new or “first chance” exceptions occur • Notify on error regression • Notify when error rates are abnormal • Notify on any log query we can think of Engage • Provide access to those who can help solve issues • Give everyone the same picture, with no extra effort Architecture • Single platform solution • Relate exceptions to relevant logs for maximum context • Contribute to picture of “application health”
  20. 20. Live demo Or…. How good is the conference WI-FI?
  21. 21. Search! Quickly filter! Log tailing!
  22. 22. Context! Full error details and telemtry Full application health details
  23. 23. 23 Full search Error Rates Error Details
  24. 24. 24
  25. 25. 25
  26. 26. Benefits of an integrated logging / error platform • Get real telemetry, know how well your app behaves from release to release • In context to overall performance • Use in QA environments, look for new exceptions and abnormal rates before it is released to prod • Proactively hunt down bugs & improve your codebase • Know about an issue before your customers tell you • Quicker, more efficient debugging via logs