Application logging for fun and profit


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Application logging for fun and profit

  1. 1. ©2010 Improving Enterprises, Inc. Logging For Fun and Profit Jane Prusakova Jane.Prusakova@ImprovingEnter Improving Enterprises 2013
  2. 2. ©2010 Improving Enterprises, Inc. Why How-to Using the results
  3. 3. ©2010 Improving Enterprises, Inc. Why logging? Enquiring minds want to know! Avoid programming-by-coincidence Trace what the users do See effects of Integration Multithreading High load
  4. 4. ©2010 Improving Enterprises, Inc. When to use logging? Greenfield development Brownfield development what came before the effect of changes Maintenance and troubleshooting trace bugs Bonus Evolve apps based on users behavior
  5. 5. ©2010 Improving Enterprises, Inc. Logging [vs Debugging] Set up once, always ready to use Minimum disruption to the flow Works locally and in the cloud Faster data collection Collected data is persisted
  6. 6. ©2010 Improving Enterprises, Inc. How-to Using the results
  7. 7. ©2010 Improving Enterprises, Inc. Yes, I want to setup logging! Pick a framework Proper configuration Consider ways to parse logs Logging will change with the application
  8. 8. ©2010 Improving Enterprises, Inc. Log4J SLF4J Apache MIT license Logging framework Logging facade Configuration, log levels and categories, rotate log files, thread-safe logging. Support: online documentation, tutorials, online forums.
  9. 9. ©2010 Improving Enterprises, Inc. Where to write logs? Console File system DB Queue mechanisms Combination
  10. 10. ©2010 Improving Enterprises, Inc. Database Files Easy to parse, hard to evolve Very hard to parse, easy to change Uses DB connections Requires FS access Can cause app crash, loss of logs, slow down app Can fill up space
  11. 11. ©2010 Improving Enterprises, Inc. Configuring Log4J Java properties-style XML Programmatically Hard-coded Takes preference
  12. 12. ©2010 Improving Enterprises, Inc. Log Message What to log with the message Importance level When Where in the code Data being processed Thread information
  13. 13. ©2010 Improving Enterprises, Inc. Example layout log4j.appender.stdout.layout.ConversionPattern= %d %5p [%t] (%F:%L) - %m%n - Date - Log Level - Thread name - File and line (slow to retrieve)
  14. 14. ©2010 Improving Enterprises, Inc. Enough and but not too much Logging should support in code correctness readability performance Consider making logs easier to use
  15. 15. ©2010 Improving Enterprises, Inc. Best Practices Relate each message to generating point Code Data Format messages for readability Use .ToString on objects
  16. 16. ©2010 Improving Enterprises, Inc. More Best Practices Input and output Errors and exceptions Transitions Integration points Thread rendezvous
  17. 17. ©2010 Improving Enterprises, Inc. Log input and output public int calculateCost(String user, Flight flight) { log.debug(“calculateCost: user “ + user + “: “ + flight); int cost = … ; … // cost calculation log.debug((“calculateCost: cost for “ + user + “ on “ + flight + “: “ + cost); return cost; }
  18. 18. ©2010 Improving Enterprises, Inc. Log errors and exceptions // do dangerous work result = DoDangerousWork(user, flight); } catch (Exception e) { log.error(“methodName: exception in DoDangerousWork for user “ + user + “ on “ + flight, e); // recover or re-throw }
  19. 19. ©2010 Improving Enterprises, Inc. More Log destination, parameters, outcomes DB calls SOAP calls Wait time Thread joins Available memory Memory-intensive operations Intermediate calculation results For complicated calculations
  20. 20. ©2010 Improving Enterprises, Inc. Practices to avoid Little information … // step 1“did something”); … // ste 2“did more work”); foreach (…) { … // useful work“working hard”); }
  21. 21. ©2010 Improving Enterprises, Inc. Watch for logging-caused bugs Side effects “: did something”); Errors and exceptions;
  22. 22. ©2010 Improving Enterprises, Inc. More practices to avoid DRY principle applies
  23. 23. ©2010 Improving Enterprises, Inc. Using the results
  24. 24. ©2010 Improving Enterprises, Inc. You’ve got logs! Rotate logs: by size, by time Retention policy 2-level storage Provide access - read only! Transfer and analysis Warning: large datasets
  25. 25. ©2010 Improving Enterprises, Inc. Security and performance Moving and processing can impact performance Absolutely no sensitive data
  26. 26. ©2010 Improving Enterprises, Inc. Learning from the logs Never rely on eye-balling WYS is not WYG Aggregate data Distinguish common and rare
  27. 27. ©2010 Improving Enterprises, Inc. Tools Process datasets: Line by line grep, Perl, awk, cut, tail, head Load entire file to eyeball vi, notepad++ Better way to eyeball more, less
  28. 28. ©2010 Improving Enterprises, Inc. Learning from the logs > grep userName MyApplication.log grep methodName MyApplication.log grep threadName MyApplication.log tail -2000 VeryImportant.log | grep Exception grep FATAL MyApplication.log
  29. 29. ©2010 Improving Enterprises, Inc. Querying logs using Awk GNU Awk create histograms Wait times Number of calls Exceptions per time period or per user Compare and relate events Exception stacks traces Outcomes by caller or thread
  30. 30. ©2010 Improving Enterprises, Inc. Logging For Fun and Profit Jane Prusakova Jane.Prusakova@ImprovingEnter Improving Enterprises 2013