Functional and non functional application logging


Published on

This presentation will help you understand the importance of logging in applications. Every project is encountered with this aspect, functional or non-functional.This will become more and more important with current innovations: mobile (~offline) applications, SOA-architectures, Cloud integration.

An overview of the Java Logging-ecospace is discussed.

Eventually, the results are processed and analyzed with facilitating tools.

What is logging?
Why logging?
Who uses logs and what are they used for?
What needs to be logged?
How to log in Java
Log processing and analysis

Published in: Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Functional and non functional application logging

  1. 1.
  3. 3. About ME<br />Sander De Vos<br />Software Engineer<br />RealDolmen 2008<br />Past projects<br />MaTMa (M*)<br />E-Metro (FOD-ECO)<br />SP10B (FOD-FIN)<br />MIK (FOD-MOB)<br />
  4. 4. QuotesFrom The Field<br />“Whoneeds logs? Isuredon’t!”<br />“Debugger is better! Sysoutworkstoo!”<br />“Isn’tthatwhatinfrastructure does?”<br />“It slows down disk and memory!”<br />“It isn’t in the spec!”<br />“We don’tknowwhatto log – we’lljust log something!”<br />“We don’tknowhow – we’lljust do itsomehow!”<br />
  5. 5. Agenda<br />What is logging<br />Definition andChallenges<br />Whylogging<br />Whouses logs<br />What logs are usedfor<br />Functional or non-functional<br />Whatto log<br />Whatwill (not) help you<br />Log messagedecomposed<br />Guidelines<br />How to log in Java<br />Logging in production<br />Log processing and analysis<br />
  6. 6. What is logging<br />
  7. 7. Definition<br />“Logs are the recordings of one or more events occurring on information systems.”<br />“Logs can be organized based on the program, day, severity, host, or a number of other categories.“<br />“Logs may be referred to as log files, audit logs, or audit trails.<br />Logs also cover alerts, alarms and event records.“<br />
  8. 8. Log challenges<br />Decentralization<br />Cloud<br />SOA<br />Volatility<br />Multiple tiersandlayers<br />Archival<br />Accessability<br />Absence of critical information<br />Random log formats<br />
  9. 9. WHY logging<br />
  10. 10. Whouses logs<br />Managed Services<br />Security Teams<br />Monitor<br />Detect<br />Investigate<br />Track<br />Analyze<br />Auditors<br />Developers<br />Debugging<br />Customer<br />
  11. 11. What logs are usedfor<br />Debugging andForensics<br />Fault monitoring<br />Performance monitoring<br />Troubleshooting<br />Feature usage<br />Security / Incident detection<br />Regulatoryand standards compliance<br />
  12. 12. Functional – non-functionallogging<br />Functional<br />Audit<br />Governance<br />Security<br />Non-functional<br />Faults, Errors, Exceptions<br />Execution context<br />Performance<br />Component usage<br />
  13. 13. Putting italltogether<br />Functional Logs<br />CustomerAudit<br />Always on<br />Requirements<br />Scope known<br />Years<br />‘Clutter, yuck, slow’<br />Non-Functional Logs<br />System operatorDeveloper<br />Sometimes on<br />Errors, Exceptions, Debug<br />Scope notknown<br />Hours, Days<br />‘Mightcome in handy’<br />
  14. 14. WHAT to log<br />
  15. 15. Keep in mind<br />”Whatwouldhelp YOU at 3 AM in the morning? “<br />Image: Ambro /<br />
  16. 16. Whatwillhelp you<br />Exact notice of whathappened<br />When<br />Where<br />How<br />Who<br />Clearconcise context information<br />Analysis possible without application<br />Manual<br />Semi-automated<br />Automated<br />Remotelycollectible files<br />Low footprint<br />Proven reliableandauthentic<br />
  17. 17. Whatwill help youexamples<br />2011-05-31 08:46:11,308 +0200<br />INFO [] <br />ACCESS OF ProfileServiceImpl.findProfilesBY<br />[login: some_username];<br />Password: [PROTECTED];<br />Authenticated: true;<br />Details: RemoteIpAddress:;<br />SessionId: 8XyrNkLWv7FdXS…<br />Granted Authorities:ROLE_USER,ROLE…<br />
  18. 18. Whatwon’thelp you<br />No details about the record<br />No context information<br />Inconsistent format or bad use<br />Non-correlatable<br />Subjective/interpretativemessage<br />Too muchuseless log records<br />Loggingtoone single big file<br />
  19. 19. Whatwon’t help you – example#1<br />log4j: setFile called: ./logs/be/mobistar/network-status/log_functional.log, false<br />log4j: setFileended<br />tis nendefault<br />tis nendefault<br />Activations : 15<br />Deactivations : 0<br />In straigt to resource filter<br />
  20. 20. Whatwon’t help you – example#2<br />INFO [dao.ProductDao] [] - FOUND 40<br />INFO [web.listeners.PhaseLogger] [] - +RENDER_RESPONSE<br />INFO [service.ProductFamilyService] [] - product in family is true<br />INFO [.dao.ProfileDao] [] - [OBJECTS] find profiles<br />
  21. 21. log messagedecomposed<br />
  22. 22. “A log record shouldbeunderstandableby a human, andeasily machine processable.”<br />
  23. 23. LoggingGuidelines: Priorities<br />Fatal<br />Application crash&burn<br />Error<br />Exceptions<br />Monitoring team alert<br />Warn<br />Monitor health, performance<br />Info<br />Informative Context<br />Audit trail<br />Debug<br />Extensive Context<br />Development, Testing<br />Troubleshooting<br />
  24. 24. LoggingGuidelines: Exceptions<br />Handling<br />Recover<br />Log the details ifcritical<br />Not Handling, Translating<br />Translate<br />Handling logic ifnotpropagating<br />Not Handling<br />Propagate<br />Someoneelsewill handle (and log!)<br />Chooseexceptionsto log<br />Defendfromstacktrace chaos<br />Choosewhatto log foreachexception<br />Catching block has all context details!<br />
  25. 25. LoggingGuidelines: Correlation<br />1 transaction = * log records<br />Correlationnecessary<br />Executiontrail<br />Shared context variable<br />Shared per transaction<br />Reuse of context variables<br />Nested Diagnostic Context<br />Map Diagnostic Context<br />Onlyonecorrelation<br />Use a context uniquevariable<br />Correlate transactions<br />
  26. 26. How to log iN<br />
  27. 27. How loggingframeworkswork<br />
  28. 28. Java Logging Frameworks<br />Implementation<br />System.out – System.err<br />Redirect output<br />Java UtilLogging<br />Log4j<br />Logback<br />Abstraction<br />CommonsLogging<br />SLF4J<br />
  29. 29. Java UtilLogging<br />Java 1.4+<br />Based on Log4J<br />Less features out of the box<br />Console<br />File<br />Stream<br />
  30. 30. Log4j<br />First framework<br />Introductionof Logger, Appender, Level<br />Hiearchical Loggers<br />Configurableappenders, e.g.<br />MailAppender<br />FileAppender<br />SMTPAppender<br />SocketAppender<br />SyslogAppender<br />Leader quitandstartedlogback<br />
  31. 31. CommonsLogging<br />Apache Project<br />API Bridge<br />API-calls forward to<br />Log4j if on CP<br />Falls back to JUL<br />Plagued<br />Double Configuration<br />Runtime dependency check<br />ClasspathScanning<br />Discontinued<br />v1.1.1 dates from end 2007<br />
  32. 32. Slf4j - Logback<br />Twolibrariestorulethemall<br />Slf4j<br />Comparablewithcommons-logging<br />Parameter parsing<br />Framework bridges<br />Much more powerful<br />Logback<br />Evolved log4j<br />Implements Slf4j<br />“Picking up where log4j leaves off”<br />Testing<br />Filters on-need-base<br />
  33. 33. SLF4J Binding<br />
  34. 34. SLF4J Bridging<br />
  35. 35. Keepingit performant<br />String concatenation<br />LOGGER.debug("Hello " + name);<br />Protected block<br />if (logger.isDebugEnabled()) {LOGGER.debug("Hello " + name);}<br />Slf4j<br />LOGGER.debug("Hello {}", name);<br />
  36. 36. Context Variables In Practice<br />NDC.push(correlationId);<br />app.logMe("Hello world!");<br />NDC.pop();<br />Use %x in Pattern<br />ConversionPattern= %x %d %m %n<br />correlationId 2011-06-21 09:20:41 [INFO] Hello world!<br />
  37. 37. Specialized Loggers<br />Logger for class<br />Fine-tune log<br />Executiontrace<br />How distinguish<br />Security<br />Performance <br />Business<br />Specialized Loggers<br />Priorities<br />Output handlers<br />Groupedlogging<br />
  38. 38. Logging In Production<br />Onlynecessarylogging<br />Situation -> turn on debuglogging<br />No restart: DynamicReloading<br />‘Watch’ file for changes<br />Log file management<br />Rolling file<br />Seperation of concerns<br />Clustering<br />Different systems generatemessages<br />Synchronization of timestamps<br />Correlation on host-specific files<br />
  39. 39. My Opinion<br />Importance of loggingwillgrow<br />Mobile / offline applications<br />Cloud integration<br />System interaction<br />Logging standards enterprisewide<br />Standardization of logging effort<br />Inexpensiveexercise<br />Draft a standard<br />Build adapters / patterns<br />Enforce standards<br />
  40. 40. Log Processing and analysis<br />
  41. 41. Log Analysis<br />Bang foryourbuck<br />XpoLog<br /><br />Licensed<br />Faster<br />Splunk<br /><br />Free Edition<br />‘Slow’ AJAX feel<br />Nice graphics<br />Modular (<br />Tuningyourpattern is important?<br />
  42. 42. Demo<br />
  43. 43.
  44. 44. For more information:<br />visit our website WWW.REALDOLMEN.COM<br />Follow us on:<br />Selected presentations are available on:<br />Thank You<br />Or scan this QR code with your Smartphone to immediately go to the website<br />