Python Debugging Techniques

2013-10-21
Who am I
Tuomas Suutari
Software Developer at Anders Inno
8 years Python experience
Anders Inno
Software Company from Turk...
Outline
Basic techniques
Debuggers
Profiling

2013-10-21
Basic techniques
Print statements
Logging

2013-10-21
Print statements
●

●

●

The simplest way to see
what's happening
Sometimes also the fastest
and easiest, so don't
overth...
Logging
●

●

●

●

Python has powerful and
flexible logging module
Like print statements with
more context information
Us...
Logging
●

Log output can be easily
controlled
•

•

●

Outputting to stdout, file,
network socket, ...
Filtering based on...
Debuggers
PDB
Winpdb

2013-10-21
PDB
●

The Python Debugger

●

In standard library

●

Console interface

●

Launch script with PDB or
jump into debugger ...
PDB
●

Basic commands
•

c, cont: continue running

•

s, step: step into function

•

p, print: print value of variable/e...
Winpdb
●

●

●

A Platform Independent
Python Debugger
Run Python script from the
GUI or embed to script
Attaching to embe...
Profiling
cProfile
Guppy

2013-10-21
cProfile
●

●

For profiling time
consumption of parts of the
program

python -m cProfile -s cumulative 

Visualizers for ...
Guppy
●

●

For profiling memory
consumption of parts of the
program

# Injecting Guppy memory dumping

Data structures in...
Guppy
●

Objects can be tracked
down by connecting a
monitor to running program

# Allow monitor to connect to the
# progr...
Thank you!
Questions?

2013-10-21
Upcoming SlideShare
Loading in …5
×

Python debugging techniques

937 views

Published on

Even though Python allows many ways to easily debug and profile your code, it is not uncommon to see people overusing simple print statements for this. The presentation will provide an overview of the most common basic debugging techniques that every Python programmer should know. Additionally, for debugging speed or memory problems, couple profilers are presented. Outline:

Basic techniques (print statements, logging)
Debuggers (pdb, winpdb/rpdb2)
Profiling (cProfile, guppy, ...)

Published in: Technology
  • Be the first to comment

Python debugging techniques

  1. 1. Python Debugging Techniques 2013-10-21
  2. 2. Who am I Tuomas Suutari Software Developer at Anders Inno 8 years Python experience Anders Inno Software Company from Turku specialized in e-commerce www.andersinno.fi 2013-10-21
  3. 3. Outline Basic techniques Debuggers Profiling 2013-10-21
  4. 4. Basic techniques Print statements Logging 2013-10-21
  5. 5. Print statements ● ● ● The simplest way to see what's happening Sometimes also the fastest and easiest, so don't overthink it Could cause problems when temporary print statements slip into production 2013-10-21 def do_stuff(): something = do_something() print 'Something:', something do_more(something) print 'Calling do_last' do_last()
  6. 6. Logging ● ● ● ● Python has powerful and flexible logging module Like print statements with more context information Used through a logger object with name Log messages have levels 2013-10-21 import logging LOG = logging.getLogger(__name__) def do_stuff(): something = do_something() LOG.debug( 'Something: %s', something) do_more(something) LOG.info('Calling do_last') do_last()
  7. 7. Logging ● Log output can be easily controlled • • ● Outputting to stdout, file, network socket, ... Filtering based on log levels or logger names Logging statements are OK in production 2013-10-21 import logging msg_format = ( '%(asctime)s %(name)s' ' %(levelname)s %(funcName)s' ' %(message)s') logging.basicConfig( format=msg_format, level=logging.DEBUG)
  8. 8. Debuggers PDB Winpdb 2013-10-21
  9. 9. PDB ● The Python Debugger ● In standard library ● Console interface ● Launch script with PDB or jump into debugger with set_trace() python -m pdb some_script.py def do_stuff(): import pdb; pdb.set_trace() something = do_something() # ... 2013-10-21
  10. 10. PDB ● Basic commands • c, cont: continue running • s, step: step into function • p, print: print value of variable/expression • b, break: set a breakpoint • u, up: move the current frame one level up • d, down: move the current frame one level down • l, list: list source code for the current file 2013-10-21
  11. 11. Winpdb ● ● ● A Platform Independent Python Debugger Run Python script from the GUI or embed to script Attaching to embedded debugger is possible from a remote host too 2013-10-21 def do_stuff(): import rpdb2 rpdb2.start_embedded_debugger( 'passwd') something = do_something() # ...
  12. 12. Profiling cProfile Guppy 2013-10-21
  13. 13. cProfile ● ● For profiling time consumption of parts of the program python -m cProfile -s cumulative Visualizers for the data could be useful python -m cProfile -o out.pyprof • Run Snake Run • KCacheGrind (with pyprof2calltree) 2013-10-21 some_script.py some_script.py runsnake out.pyprof
  14. 14. Guppy ● ● For profiling memory consumption of parts of the program # Injecting Guppy memory dumping Data structures in the program that are growing can be detected by dumping memory profile periodically with Guppy and watching the process with profile browser hp.setref() 2013-10-21 # to the program: from guppy import hpy hp = hpy() for x in a_loop_in_the_program: hp.heap().dump('memprof.hpy') # To start Profile Browser: python -c " from guppy import hpy; hpy().pb()"
  15. 15. Guppy ● Objects can be tracked down by connecting a monitor to running program # Allow monitor to connect to the # program import guppy from guppy.heapy import Remote Remote.on() # To connect with the monitor: python -c "from guppy import hpy; hpy().monitor()" <Monitor> sc 1 <Annex> int >>> hp.heap() 2013-10-21
  16. 16. Thank you! Questions? 2013-10-21

×