Tools for Effective Logging in Microservices Architectures

17/28/17© 2017 Loggly Inc. Confidential and ProprietaryMORE DETAILS ON OUR BLOG: loggly.com/logging-microservices/
Tools and Techniques
for Logging Microservices
Presented by:

Microservices defined
A microservice is an application consisting of multiple small,
independent, intercommunicating components. Each component
(known as a service) is modular, reusable, and self-contained and
communicates with other services using language-agnostic
protocols.
As more businesses turn towards microservices as a way of
handling large workloads in a distributed and scalable way,
software developers need effective logging methods.

Example:
Microservices in an e-commerce application
Each task is a small, self-contained unit that communicates with other units through APIs.
NGINX
web
server
Customer
MySQL
server
Stores
customer
& order data
Handles requests
from public
Internet
Multiple PHP modules for
Order
processing
Validating payment
with third-party
processor
Generating
HTML
£ ¥ $ € <html>…
</html>

Microservices in a nutshell
Difference between traditional monolithic applications and microservices
An instance of a service can be created, stopped,
restarted, and destroyed at any time without
impacting other services. Any logging functionality
implemented can’t rely on the service persisting
for any period of time.
Microservices
are
stateless

Microservices
are
distributed
For logging related data from two completely
independent platforms. To log effectively, we need
a way to correlate events across the
infrastructure.

Microservices
are
independent
A stack trace in a monolithic Java application, will
bring you straight to the source of a problem. A
stack trace in a service will only bring you to the
start of that service, not the entry point of the
microservice itself.

Techniques for logging microservices
When logging microservices, you have to consider that logs
are coming from several different services
Two ways to
approach logging:
Logging from within each service
Logging from the infrastructure

Logging from individual services
• Add your logging framework into the
service itself
• Possible to use different logging
frameworks (log4php, Monolog) in
different services
• Append information to each log
event to help identify where the
event took place (e.g. service name)
Logging
framework
Logging
framework
Microservice architecture
Service Service
Log destination

Logging from individual services
Pluses:
• Affords maximum flexibility
to developers
Minuses:
• Creates redundancy
• Adds complexity
• Makes it difficult to change
logging behavior across
multiple services

Logging
framework
Logging
framework
Microservice architecture
Service Service
Log destination
Centralized logging service
• Services forward logs to a logging
service
• Services still need a way to generate
logs
• Logging service is responsible for
processing, storing, or sending logs
to a centralized location
Logging from a central service

Docker example of logging to a centralized service
Example: The Loggly Docker image
provided by SendGrid Labs.
• Image creates container running
rsyslog daemon that listens for
incoming syslog messages
• Incoming events are immediately
forwarded to centralized log
management system like Loggly
• Multiple logging containers can run
simultaneously for load balancing and
redundancy
Directory Loggly
Docker
App
Container 1
Directory
Docker
App
Container 2
Directory

Docker examples: Logging from a central service
Alternative: gather logs from the infrastructure itself
stderr stdout
Docker
App
Container 1
Directory
Docker
App
Container 2
Directory
Active Docker containers print log events to stderr and stdout
Docker logging driver
Docker logging driver detects output generated on these streams, captures it,
and forwards it through the configured driver. You can then configure a syslog
driver that forwards log events from your containers directly to Loggly.
Configured
driver
Alternative: Utilize tools like logspout
Loggly
Docker
App
Container 1
Directory
Docker
App
Container 2
Directory
logspout
Logspout is another Docker-based solution
that runs as its own service. It attaches to
other containers on the same host and routes
events to a destination such as Loggly.
logspout
The benefit of tools like logspout is
their ease of use: Simply deploy
them alongside your other services
and they immediately go to work.

Tips for logging microservices

Your containerized applications can generate application-
related content such as classes, methods, and stack
traces, but will they tell you where in your architecture the
event was generated?
Simple solution: Append a unique identifier, such as the
name of the container, to newly generated logs.
Without this, logs are completely unaware of your
microservice architecture.
Store
enough
data
1.

When dealing with distributed services, you need a way of
tracing a series of events back to the source.
• Attaching a unique identifier, such as a request ID for a
web server, to each request as it enters the
microservice
• By filtering on identifier, you can easily follow trail of
events from the entry point all the way to the error
Correlate
events
2.

In a microservice architecture, new code can be deployed
in a matter of seconds. You need to be ready to identify
the cause of any deployment failure and understand the
state of the architecture.
3.
Log
your
deployment

When a service goes down, you’ll want to know
immediately. A log management service can generate
alerts by filtering log events based on a pattern, such as a
large number of exceptions in a short amount of time.
4.
Set
alerts

Putting It All Together
What this process ultimately allows us to do is to generate microservice logs that are unique,
traceable, and contain enough data to act on quickly and effectively. To create these logs:
Correlate
events from each
microservice to
create an overall
view of the request
Append
a unique identifier
to requests as
they enter the
architecture
Identify
the service that
generates events
Generate
events specific to
each microservice
as the request
enters and leaves
the service

Putting It All Together
Traces a single request across the architecturerequest ID
service ID Uniquely identifies the service that generated the event
The result is a series of log events that contain (at a minimum):
Unique content associated with the event, such as messages,
severity levels, class and method names, and stack traces

So you have microservices. Now what?
Centralize all of your log data and see what matters fast.
Try Loggly for free!
Loggly is the world’s most popular cloud-based, enterprise-class log management service, serving more
than 10,000 customers including one-third of the Fortune 500. The Loggly service integrates into the
engineering processes of teams employing continuous deployment and DevOps practices to reduce MTTR,
improve service quality, accelerate innovation, and make better use of valuable development resources.
www.loggly.com/trial

About the author
Andre Newman is a software developer and writer on all things
tech. With over eight years of experience in Java, .NET, C++, and
Python, Andre has developed software for remote systems
management, online retail, and real-time web applications. He
currently manages 8-bit Buddhism, a blog that blends technology
with spirituality.

Tools for Effective Logging in Microservices Architectures

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Tools for Effective Logging in Microservices Architectures

Similar to Tools for Effective Logging in Microservices Architectures (20)

More from SolarWinds Loggly

More from SolarWinds Loggly (10)

Recently uploaded

Recently uploaded (20)

Tools for Effective Logging in Microservices Architectures