1.
Why and how to test logging
DevOps Showcase North / Testing Showcase North
Manchester, UK - Unicom
Thursday 11th February 2015
Matthew Skelton
Skelton Thatcher Consulting

2.
confession:
I am a big fan of logging

3.
use logging as a
channel/vector to
make distributed
systems more testable

4.
transforming technology
and teams
high impact expertise
Cloud, Agile, DevOps

5.
Why we should test logging
How to test logging effectively

6.
exceptional situations
edge cases
metrics
analytics
‘audits’
…
@evanphx

7.
logging as an execution trace

8.
reduce time-to-detect
increase team engagement
increase configurability
enhance DevOps collaboration
#operability

9.
(event sourcing)
(structured logging)
(CQRS)

10.
using logging mainly for errors
inconsistent use of logging
“logging slows things down”

11.
“logging pollutes my domain model”
“logging is just for Ops people”
log aggregation only in Production

12.
logging assumed to be free
($0) to implement
logs not available to Tester or
Developers

13.
distinct event IDs
transaction tracing
instrumentation
log aggregation & search

14.
cross-component visibility
increased test coverage
programmatic assertions
deep insights

15.
stories for testing logging

16.
BasketItemAdded
Search: BasketItem

17.
Continuous event IDs

18.
How many distinct event
types (state transitions) in
your application?

19.
represent distinct states

20.
enum
Human-readable sets:
unique values, sparse, immutable
C#, Java, Python, node
(Ruby, PHP, …)

21.
public enum EventID
{
// Badly-initialised logging data
NotSet = 0,
// An unrecognised event has occurred
UnexpectedError = 10000,
ApplicationStarted = 20000,
ApplicationShutdownNoticeReceived = 20001,
PageGenerationStarted = 30000,
PageGenerationCompleted = 30001,
MessageQueued = 40000,
MessagePeeked = 40001,
BasketItemAdded = 60001,
BasketItemRemoved = 60002,
CreditCardDetailsSubmitted = 70001,
// ...
}

22.
Technical
Domain
public enum EventID
{
// Badly-initialised logging data
NotSet = 0,
// An unrecognised event has occurred
UnexpectedError = 10000,
ApplicationStarted = 20000,
ApplicationShutdownNoticeReceived = 20001,
PageGenerationStarted = 30000,
PageGenerationCompleted = 30001,
MessageQueued = 40000,
MessagePeeked = 40001,
BasketItemAdded = 60001,
BasketItemRemoved = 60002,
CreditCardDetailsSubmitted = 70001,
// ...
}

23.
BasketItemAdded = 60001

24.
BasketItemAdded = 60001
BasketItemRemoved = 60002

25.
BasketItemAdded = 60001
BasketItemRemoved = 60002

26.
represent distinct states

27.
OrderSvc_BasketItemAdded

28.
Monolith to microservices:
debugger does not have the full view

29.
Even with remote debugger, it’s
boring to attach and detach

30.
Storage I/O
Worker Job
Queue
Upload

31.
Transaction tracing

32.
‘Unique-ish’ identifier for each request
Passed through downstream layers

33.
Unique-ish ID

34.
Testing logging

35.
use logging as a
channel/vector to
make distributed
systems more testable

36.
track a parcel online

37.
search by
event ID

38.
transaction
trace

39.
Web App
Doc store
Audit service
Msg queue

40.
Transactional
sub-system
Log aggregation
sub-system

41.
An action here… …results in log entry here

42.
trace events in a
distributed,
asynchronous system

43.
stories for testing logging

44.
AuditRecordCreated
Search: AuditRecordCreated

45.
AuditRecordCreated
Search: AuditRecordCreated

46.
Feature: User trades stocks
Scenario: User requests a sell before close of trading
Given I have 100 shares of MSFT stock
And I have 150 shares of APPL stock
And the time is before close of trading
When I ask to sell 20 shares of MSFT stock
Then I should have 80 shares of MSFT stock
And I should have 150 shares of APPL stock
And a sell order for 20 shares of MSFT stock should have been executed
http://martinfowler.com/bliki/GivenWhenThen.html - Pete Hodgson

47.
Given I run a scenario as a Lawyer
And I create a document
[And I wait 5 seconds]
When I search the logs API
Then I should find a recent entry
for “AuditRecordCreated”

48.
Recap

49.
use logging as a
channel/vector to
make distributed
systems more testable

50.
unique-ish event type IDs
transaction tracing
log aggregation & search
Stories: expected search results

51.
cross-component visibility
increased test coverage
programmatic assertions
deep insights

52.
trace events in a
distributed,
asynchronous system

53.
‘Structured Logging’
TW: “Adopt” (May 2015)
https://www.thoughtworks.com/radar/techniques/structured-logging
http://gregoryszorc.com/
.NET: http://serilog.net/
Java: https://github.com/fluent/fluent-logger-java

54.
More
Ditch the Debugger and Use Log
Analysis Instead
Matthew Skelton
https://blog.logentries.com/2015/07/ditch-
the-debugger-and-use-log-analysis-instead/

55.
Thank you
http://skeltonthatcher.com/
enquiries@skeltonthatcher.com
@SkeltonThatcher
+44 (0)20 8242 4103
@matthewpskelton