SlideShare a Scribd company logo
1 of 20
Download to read offline
MY
MY
LOGS
LOGS
HAVE SOMETHING TO TELL YOU 🦉
HAVE SOMETHING TO TELL YOU 🦉
2
WHAT ARE LOGS?
WHAT ARE LOGS?
Troubleshooting, Auditing, Profiling, Statistics
Troubleshooting, Auditing, Profiling, Statistics
Logging ≠ Observability ≠ Output
Logging ≠ Observability ≠ Output
3
WHAT ARE LOGS?
WHAT ARE LOGS?
Troubleshooting, Auditing, Profiling, Statistics
Troubleshooting, Auditing, Profiling, Statistics
Logging ≠ Observability ≠ Output
Logging ≠ Observability ≠ Output
Logging
Logging
Output meant for postmortem troubleshooting.
Output meant for postmortem troubleshooting.
4
THE BASICS
THE BASICS
5
THE BASICS
THE BASICS
import logging
logger = logging.getLogger(__name__)
user = "cooper"
town = "Twin Peaks"
logger.info("%s entering %s", user, town,
extra={"user": {"name": user}})
6
Automatic Extra Data
Automatic Extra Data
log = LoggerAdapter(
logger, extra={"user": {"name": user}}
)
logger.info("%s entering %s", user, town)
7
Global Extra Data
Global Extra Data
# utils/logs.py
local = threading.local()
def set_user(user: dict) -> None:
local.user = user
class ExtraDataFilter(logging.Filter):
"""
Adds global extra data to all log records
"""
def filter(self, record):
if user := getattr(local, "user", None):
record.user = user
8
try:
pour_coffee(cup)
except Exception:
logger.exception("Something strange occurred with %s", cup)
9
BEST PRACTISES
BEST PRACTISES
10
BEST PRACTISES
BEST PRACTISES
Be careful of Sensitive Information
Log relevant events *
Consider the reader. *
Let the logging library construct the message *
11
BEST PRACTISES
BEST PRACTISES
LOG RELEVANT EVENTS
LOG RELEVANT EVENTS
12
BEST PRACTISES
BEST PRACTISES
LOG RELEVANT EVENTS
LOG RELEVANT EVENTS
Log on changes
Log on strangeness
Business events and decisions
Network calls
Decision rationale
HTTP world: POST PUT DELETE.
HTTP world: POST PUT DELETE.
13
BEST PRACTISES
BEST PRACTISES
CONSIDER THE READER
CONSIDER THE READER
14
BEST PRACTISES
BEST PRACTISES
CONSIDER THE READER
CONSIDER THE READER
Logs are read in multitues.
Logs are read in multitues.
Keep them Short.
Keep them Self-contained.
Keep them Consistent.
Add the necessary data.
15
BEST PRACTISES
BEST PRACTISES
FORMATTING
FORMATTING
Do let the logging library construct the message.
Do let the logging library construct the message.
Do use time in seconds, unless otherwise specified.
Do use time in seconds, unless otherwise specified.
logger.info("%s entering %s", user, town)
16
BEST PRACTISES
BEST PRACTISES
FORMATTING
FORMATTING
Do add global contextual information, e.g. tracing IDs.
Do add global contextual information, e.g. tracing IDs.
Do rename and filter the default
Do rename and filter the default extra
extra data for
data for
better readability.
better readability.
Do define your
Do define your extra
extra schema,
schema,
with
with !
!
Elastic Common Schema
Elastic Common Schema
17
BEST PRACTISES
BEST PRACTISES
ANALYSIS
ANALYSIS
18
BEST PRACTISES
BEST PRACTISES
ANALYSIS
ANALYSIS
ELK stack with ElasticSearch and Kibana
ELK stack with ElasticSearch and Kibana
= Fantastic. ✨🚀
= Fantastic. ✨🚀
Sentry / LogRocket.
Sentry / LogRocket.
19
LIBRARIES
LIBRARIES
python-logstash-formatter
ecs-logging-python
loguru
structlog
20
21

More Related Content

Similar to Pycon SE 2022 LT - Python Logging Best Practises

Similar to Pycon SE 2022 LT - Python Logging Best Practises (20)

Logging
LoggingLogging
Logging
 
LOGGING FOR FUN, AND PROFIT
LOGGING FOR FUN, AND PROFITLOGGING FOR FUN, AND PROFIT
LOGGING FOR FUN, AND PROFIT
 
Log management principle and usage
Log management principle and usageLog management principle and usage
Log management principle and usage
 
Un-broken Logging - Operability.io 2015 - Matthew Skelton
Un-broken Logging - Operability.io 2015 - Matthew SkeltonUn-broken Logging - Operability.io 2015 - Matthew Skelton
Un-broken Logging - Operability.io 2015 - Matthew Skelton
 
Un-broken logging - the foundation of software operability - Operability.io -...
Un-broken logging - the foundation of software operability - Operability.io -...Un-broken logging - the foundation of software operability - Operability.io -...
Un-broken logging - the foundation of software operability - Operability.io -...
 
TDC 2015 - POA - Trilha PHP - Shit Happens
TDC 2015 - POA - Trilha PHP - Shit HappensTDC 2015 - POA - Trilha PHP - Shit Happens
TDC 2015 - POA - Trilha PHP - Shit Happens
 
Why you should be using structured logs
Why you should be using structured logsWhy you should be using structured logs
Why you should be using structured logs
 
Un-broken Logging - TechnologyUG - Leeds - Matthew Skelton
Un-broken Logging - TechnologyUG - Leeds - Matthew SkeltonUn-broken Logging - TechnologyUG - Leeds - Matthew Skelton
Un-broken Logging - TechnologyUG - Leeds - Matthew Skelton
 
Application Logging With Logstash
Application Logging With LogstashApplication Logging With Logstash
Application Logging With Logstash
 
Playground 11022017 user_monitoring
Playground 11022017 user_monitoringPlayground 11022017 user_monitoring
Playground 11022017 user_monitoring
 
Log monitoring with Logstash and Icinga
Log monitoring with Logstash and IcingaLog monitoring with Logstash and Icinga
Log monitoring with Logstash and Icinga
 
Software Engineering - RS4
Software Engineering - RS4Software Engineering - RS4
Software Engineering - RS4
 
OC Big Data Monthly Meetup #5 - Session 2 - Sumo Logic
OC Big Data Monthly Meetup #5 - Session 2 - Sumo LogicOC Big Data Monthly Meetup #5 - Session 2 - Sumo Logic
OC Big Data Monthly Meetup #5 - Session 2 - Sumo Logic
 
Application Logging in the 21st century - 2014.key
Application Logging in the 21st century - 2014.keyApplication Logging in the 21st century - 2014.key
Application Logging in the 21st century - 2014.key
 
Functional and non functional application logging
Functional and non functional application loggingFunctional and non functional application logging
Functional and non functional application logging
 
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
[db tech showcase Tokyo 2017] C23: Lessons from SQLite4 by SQLite.org - Richa...
 
Neo4j Network security
Neo4j Network securityNeo4j Network security
Neo4j Network security
 
Application Logging for large systems
Application Logging for large systemsApplication Logging for large systems
Application Logging for large systems
 
Introducing Neo4j 3.1: New Security and Clustering Architecture
Introducing Neo4j 3.1: New Security and Clustering Architecture Introducing Neo4j 3.1: New Security and Clustering Architecture
Introducing Neo4j 3.1: New Security and Clustering Architecture
 
Security threat analysis points for enterprise with oss
Security threat analysis points for enterprise with ossSecurity threat analysis points for enterprise with oss
Security threat analysis points for enterprise with oss
 

Recently uploaded

Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
UXDXConf
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Speed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in MinutesSpeed Wins: From Kafka to APIs in Minutes
Speed Wins: From Kafka to APIs in Minutes
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 

Pycon SE 2022 LT - Python Logging Best Practises