Why and how to test logging - DevOps Showcase North - Feb 2016 - Matthew Skelton

2,256 views

Published on

Modern log aggregation & search tools provide significant new capabilities for teams building, testing, and running software systems. By treating logging as a core system component, and using techniques such as unique event IDs, transaction tracing, and structured log output, we gain rich insights into application behaviour and health. This talk explains why it is valuable to test aspects of logging and how to do this with modern log aggregation tooling.

Published in: Software
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,256
On SlideShare
0
From Embeds
0
Number of Embeds
1,024
Actions
Shares
0
Downloads
19
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Why and how to test logging - DevOps Showcase North - Feb 2016 - Matthew Skelton

  1. 1. Why and how to test logging DevOps Showcase North / Testing Showcase North Manchester, UK - Unicom Thursday 11th February 2015 Matthew Skelton Skelton Thatcher Consulting
  2. 2. confession: I am a big fan of logging
  3. 3. use logging as a channel/vector to make distributed systems more testable
  4. 4. transforming technology and teams high impact expertise Cloud, Agile, DevOps
  5. 5. Why we should test logging How to test logging effectively
  6. 6. exceptional situations edge cases metrics analytics ‘audits’ … @evanphx
  7. 7. logging as an execution trace
  8. 8. reduce time-to-detect increase team engagement increase configurability enhance DevOps collaboration #operability
  9. 9. (event sourcing) (structured logging) (CQRS)
  10. 10. using logging mainly for errors inconsistent use of logging “logging slows things down”
  11. 11. “logging pollutes my domain model” “logging is just for Ops people” log aggregation only in Production
  12. 12. logging assumed to be free ($0) to implement logs not available to Tester or Developers
  13. 13. distinct event IDs transaction tracing instrumentation log aggregation & search
  14. 14. cross-component visibility increased test coverage programmatic assertions deep insights
  15. 15. stories for testing logging
  16. 16. BasketItemAdded Search: BasketItem
  17. 17. Continuous event IDs
  18. 18. How many distinct event types (state transitions) in your application?
  19. 19. represent distinct states
  20. 20. enum Human-readable sets: unique values, sparse, immutable C#, Java, Python, node (Ruby, PHP, …)
  21. 21. public enum EventID { // Badly-initialised logging data NotSet = 0, // An unrecognised event has occurred UnexpectedError = 10000, ApplicationStarted = 20000, ApplicationShutdownNoticeReceived = 20001, PageGenerationStarted = 30000, PageGenerationCompleted = 30001, MessageQueued = 40000, MessagePeeked = 40001, BasketItemAdded = 60001, BasketItemRemoved = 60002, CreditCardDetailsSubmitted = 70001, // ... }
  22. 22. Technical Domain public enum EventID { // Badly-initialised logging data NotSet = 0, // An unrecognised event has occurred UnexpectedError = 10000, ApplicationStarted = 20000, ApplicationShutdownNoticeReceived = 20001, PageGenerationStarted = 30000, PageGenerationCompleted = 30001, MessageQueued = 40000, MessagePeeked = 40001, BasketItemAdded = 60001, BasketItemRemoved = 60002, CreditCardDetailsSubmitted = 70001, // ... }
  23. 23. BasketItemAdded = 60001
  24. 24. BasketItemAdded = 60001 BasketItemRemoved = 60002
  25. 25. BasketItemAdded = 60001 BasketItemRemoved = 60002
  26. 26. represent distinct states
  27. 27. OrderSvc_BasketItemAdded
  28. 28. Monolith to microservices: debugger does not have the full view
  29. 29. Even with remote debugger, it’s boring to attach and detach
  30. 30. Storage I/O Worker Job Queue Upload
  31. 31. Transaction tracing
  32. 32. ‘Unique-ish’ identifier for each request Passed through downstream layers
  33. 33. Unique-ish ID
  34. 34. Testing logging
  35. 35. use logging as a channel/vector to make distributed systems more testable
  36. 36. track a parcel online
  37. 37. search by event ID
  38. 38. transaction trace
  39. 39. Web App Doc store Audit service Msg queue
  40. 40. Transactional sub-system Log aggregation sub-system
  41. 41. An action here… …results in log entry here
  42. 42. trace events in a distributed, asynchronous system
  43. 43. stories for testing logging
  44. 44. AuditRecordCreated Search: AuditRecordCreated
  45. 45. AuditRecordCreated Search: AuditRecordCreated
  46. 46. Feature: User trades stocks Scenario: User requests a sell before close of trading Given I have 100 shares of MSFT stock And I have 150 shares of APPL stock And the time is before close of trading When I ask to sell 20 shares of MSFT stock Then I should have 80 shares of MSFT stock And I should have 150 shares of APPL stock And a sell order for 20 shares of MSFT stock should have been executed http://martinfowler.com/bliki/GivenWhenThen.html - Pete Hodgson
  47. 47. Given I run a scenario as a Lawyer And I create a document [And I wait 5 seconds] When I search the logs API Then I should find a recent entry for “AuditRecordCreated”
  48. 48. Recap
  49. 49. use logging as a channel/vector to make distributed systems more testable
  50. 50. unique-ish event type IDs transaction tracing log aggregation & search Stories: expected search results
  51. 51. cross-component visibility increased test coverage programmatic assertions deep insights
  52. 52. trace events in a distributed, asynchronous system
  53. 53. ‘Structured Logging’ TW: “Adopt” (May 2015) https://www.thoughtworks.com/radar/techniques/structured-logging http://gregoryszorc.com/ .NET: http://serilog.net/ Java: https://github.com/fluent/fluent-logger-java
  54. 54. More Ditch the Debugger and Use Log Analysis Instead Matthew Skelton https://blog.logentries.com/2015/07/ditch- the-debugger-and-use-log-analysis-instead/
  55. 55. Thank you http://skeltonthatcher.com/ enquiries@skeltonthatcher.com @SkeltonThatcher +44 (0)20 8242 4103 @matthewpskelton

×