Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Logstage - zero-cost-tructured-logging

270 views

Published on

Everybody needs logs. Humans and machines. Logs should be structured, json and text format. Library should me modular, provide full customisation of router, rendering policies, sinks, dynamic client context. What if we could provide a such logging library with compile-time structure and context extraction which satisfies both? Let’s dive into LogStage and try to implement own asynchronous sink for logs.

About the author:
Maksym Ratoshniuk: Maksym is interested in Big Data, distributed and scalable Architecture Design and Functional Programming.

logstage: https://izumi.7mind.io/latest/release/doc/logstage/index.html

Published in: Software
  • Login to see the comments

  • Be the first to like this

Logstage - zero-cost-tructured-logging

  1. 1. LogStage - zero cost structured logging Maksym Ratoshniuk, 7mind.io
  2. 2. Yet another logger?
  3. 3. What is a structured logging?
  4. 4. Common use cases: ● Tracing ● Liveness of your system ● Metrics ○ DB Calls ○ Http flow ○ Payment processing ○ Scheduling (jobs) ● Analytics ○ User behaviour ○ A/B Testing
  5. 5. Challenges
  6. 6. 1. Broken SOLID and lot’s of magic
  7. 7. Fluentd ?
  8. 8. Scala-logging
  9. 9. Finatra
  10. 10. 2. Production unreadiness ● Asynchronous sinks ● File rotation ● Json rendering ● User context (like MDC) ● No performance hotspots
  11. 11. 3. Jar Hell
  12. 12. 3. Singletons and jar hell
  13. 13. We always write code...
  14. 14. … that is always structured
  15. 15. Logging always should be easy to read... class ExampleService(log: IzLogger) { val justAnArg = "example" val justAList = List[Any](10, "green", "bottles") log.trace(s"Argument: $justAnArg, another arg: $justAList") log.info(s"Named expression: ${Random.nextInt() -> "random number"}") log.warn(s"Invisible: ${Random.nextInt() -> "random number" -> null}") val ctxLog = log("userId" -> "user@google.com", "company" -> "acme") val delta = Random.nextInt(1000) ctxLog.info(s"Processing time: $delta") }
  16. 16. … and easier to deal with class ExampleService(logger: IzLogger) { val justAnArg = "example" val justAList = List[Any]( 10, "green", "bottles" ) logger.trace( s"Argument: $justAnArg, another arg: $justAList" ) }
  17. 17. Features
  18. 18. Macro-based structuring and context extraction ● Argument names, types, ordering ● Static information (file, line, class, function) ● Static part of our message - message template (interpolation context)
  19. 19. Aliasing for references
  20. 20. Dynamic context & method granularity
  21. 21. Reference configuration
  22. 22. SLF4J backend A drop-in replacement for Logback, route your legacy logs also
  23. 23. Out of box provisioning - Console sink - File sink - Asynchronous sink (single worker thread at the moment) - String and Json rendering
  24. 24. Effectful adapters for ZIO, Cats, Monix
  25. 25. Automatic structure identifiers
  26. 26. Template as an identifier If you have a structured DB as a storage, you can querying your logs with the same structure
  27. 27. Let’s dive into coding
  28. 28. Almost no dependencies Clean, neat, no singletons (except slf4j interop), which may impact on working of systems with isolated classloaders
  29. 29. Modular Implement you own sink, router, policy, rotation
  30. 30. DI ready Two separate settings set. Isn’t cool? 😍
  31. 31. DIStage out of box
  32. 32. Comparison with popular frameworks Logstage Scribe Airframe Logback + SLF4j Scala Logging Structured ✅ ❌ ❌ ❌ ❌ SOLID rules ✅ ❌ ❌ ❌ ❌ No singletons ✅ ❌ ❌ ❌ ❌ Modularity ✅ ❌ ❌ ❌ ❌ Asynchronous ✅ ✅ ✅ ✅ ✅ DI-readiness ✅ ❌ ❌ ❌ ❌ Colourful ✅ ❌ ✅ ✅ ❌ Dynamic Context ✅ ❌ ❌ ✅ ✅ File rotation ✅ ✅ ✅ ✅ ✅
  33. 33. Plans to work with ● Rethinking of rendering policy ● Better configuration ● Integrations with Logback, Azure, ElasticSearch, Kafka ● Profiling and optimization performance
  34. 34. Welcome to contribution
  35. 35. Thank you for listening! https://github.com/7mind https://ratoshniuk.github.io/ https://github.com/ratoshniuk/scalaua-2019

×