What the heck
went wrong?
Andy McKay
andym on #django
clearwind consulting
andy@clearwind.ca
@clearwind

debugging django

reporting errors

not a question of if...

development → production

development
use the dev. server
python manage.py runserver

development
turn on debug
DEBUG = True

development
assert False

development
a print
print "some debugging statement"

development
a print
print "some debugging statement"
print some_variable
print type(some_variable)

there's a problem

development
mod_wsgi
IOError: sys.stdout access restricted by mod_wsgi
WSGIRestrictStdout Off
http://code.google.com/p/modwsgi/wiki/ApplicationIssues

logging
http://www.flickr.com/photos/astro-dudes/1492818224/

development
http://docs.python.org/library/logging.html
import logging
logging.basicConfig(
level = logging.DEBUG,
format = '%(asctime)s %(levelname)s %(message)s',
)

development
import logging
log.debug("My brilliant debug message")

import logging
development
import logging.handlers
if getattr(settings, "LOG_FILENAME", None):
logger = logging.getLogger("django")
handler = logging.handlers.RotatingFileHandler(settings.LOG_FILENAME)
formatter = logging.Formatter("%(asctime)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
else:
# so that if you don't have LOG_FILENAME defined, don't blow up
class NullHandler(logging.Handler):
def emit(self, record):
pass
logging.getLogger("django").addHandler(NullHandler())

development
pdb
http://docs.python.org/library/pdb.html

development
import pdb; pdb.set_trace()
Django version 1.2 pre-alpha, using settings 'test_profiler.settings'
Development server is running at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
> /mnt/hgfs/sandboxes/test_profiler/profiler/middleware.py(12)
process_response()
-> url = request.get_full_path()
(Pdb)

development
l: shows lines of code at your current point
n: execute the next line
s: step into
c: continue
q: quit
locals(): see what's in your current scope

development
pdb is essential

there's a problem

development
leave pdb in and your site...

development
werkzeug debugger
easy_install Werkzeug
http://dev.pocoo.org/projects/werkzeug/wiki/UsingDebuggerWithDjango

development
#!/usr/bin/env python
from werkzeug import run_simple, DebuggedApplication
from django.core.handlers.wsgi import WSGIHandler
# This is only needed for Django versions < [7537]:
def null_technical_500_response(request, exc_type, exc_value, tb):
raise exc_type, exc_value, tb
from django.views import debug
debug.technical_500_response = null_technical_500_response
if __name__ == '__main__':
run_simple('localhost', 8080,
DebuggedApplication(WSGIHandler(), True))

development

development
unit tests

development
django-debug-toolbar
http://github.com/robhudson/django-debug-toolbar/tree/master

development
rocks!

development
MIDDLEWARE_CLASSES = (
[...snip]
'debug_toolbar.middleware.DebugToolbarMiddleware',
)
INSTALLED_APPS = (
[...snip]
'debug_toolbar'
)
INTERNAL_IPS = ('127.0.0.1',)

development

development
cheaper and easier

staging
continuous integration
e.g: Teamcity, Integrity, buildbot, Tinderbox ... others are available

staging
custom test runner
TeamCity: http://bit.ly/geAA
[..snip]
if hasTeamcity and underTeamcity():
result = TeamcityTestRunner().run(suite)
else:
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)

staging

production
varies upon server
mod_wsgi → http://code.google.com/p/modwsgi/wiki/DebuggingTechniques
mod_python → http://yenaer.com/blog/2008/jan/12/debugging-modpython-scripts-apache-and-pdb

production
errors
http://docs.djangoproject.com/en/dev/howto/error-reporting/

production
404 and 500
(has to be in root url.conf)
handler404 = 'mysite.views.custom_404'
handler500 = 'mysite.views.custom_500'
http://docs.djangoproject.com/en/dev/topics/http/views/

production
emailed
DEBUG = False
ADMIN = ( ('Andy', 'andy@clearwind.ca'), )

production
Traceback (most recent call last):
File "/usr/lib/python2.5/site-packages/django/core/handlers/base.py", line 86, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/usr/lib/python2.5/site-packages/django/contrib/admin/sites.py", line 154, in root
return shortcut(request, *url.split('/')[1:])
File "/usr/lib/python2.5/site-packages/django/contrib/contenttypes/views.py", line 57, in shortcut
object_domain = Site.objects.get_current().domain
File "/usr/lib/python2.5/site-packages/django/contrib/sites/models.py", line 22, in get_current
current_site = self.get(pk=sid)
File "/usr/lib/python2.5/site-packages/django/db/models/manager.py", line 93, in get
return self.get_query_set().get(*args, **kwargs)
File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 298, in get
num = len(clone)
File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 154, in __len__
self._result_cache = list(self.iterator())
File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 269, in iterator
for row in self.query.results_iter():
File "/usr/lib/python2.5/site-packages/django/db/models/sql/query.py", line 206, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/lib/python2.5/site-packages/django/db/models/sql/query.py", line 1723, in execute_sql
cursor.execute(sql, params)
ProgrammingError: relation "django_site" does not exist
<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{'__utma': '90285002.4337325975145378300.1232037035.1233126418.1233185854.12',
'__utmb': '90285002.2.10.1233185854',
'__utmc': '90285002',
'__utmz': '90285002.1232037035.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)',
'sessionid': 'b139e8d00ed433fb5e6dff50a3956036'},
META:{'DOCUMENT_ROOT': '/htdocs',
'GATEWAY_INTERFACE': 'CGI/1.1',
'HTTP_ACCEPT': 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5',
'HTTP_ACCEPT_ENCODING': 'gzip, deflate',
'HTTP_ACCEPT_LANGUAGE': 'en-us',
'HTTP_CONNECTION': 'keep-alive',
'HTTP_COOKIE': '__utmz=90285002.1232037035.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=90285002.4337325975145378300.1232037035.1233126418.1233185854.12; __utmc=90285002;
__utmb=90285002.2.10.1233185854; sessionid=b139e8d00ed433fb5e6dff50a3956036',
'HTTP_HOST': 'www.areciboapp.com',
'HTTP_REFERER': 'http://www.areciboapp.com/arecibo-admin/singleblog/post/7/',
'HTTP_USER_AGENT': 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1',
'PATH': '/usr/local/bin:/usr/bin:/bin',
'PATH_INFO': u'/arecibo-admin/r/18/7/',
'PATH_TRANSLATED': '/var/www/arecibo/django.wsgi/arecibo-admin/r/18/7/',
'QUERY_STRING': '',
'REMOTE_ADDR': '70.79.161.59',
'REMOTE_PORT': '63593',
'REQUEST_METHOD': 'GET',
'REQUEST_URI': '/arecibo-admin/r/18/7/',
'SCRIPT_FILENAME': '/var/www/arecibo/django.wsgi',
'SCRIPT_NAME': u'',
'SERVER_ADDR': '216.139.224.26',
'SERVER_ADMIN': '[no address given]',
'SERVER_NAME': 'www.areciboapp.com',
'SERVER_PORT': '80',
'SERVER_PROTOCOL': 'HTTP/1.1',
'SERVER_SIGNATURE': '<address>Apache/2.2.8 (Ubuntu) DAV/2 SVN/1.4.6 mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_wsgi/1.3 Server at www.areciboapp.com Port 80</address>n',
'SERVER_SOFTWARE': 'Apache/2.2.8 (Ubuntu) DAV/2 SVN/1.4.6 mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_wsgi/1.3',
'mod_wsgi.application_group': 'areciboapp.com|',
'mod_wsgi.callable_object': 'application',
'mod_wsgi.case_sensitivity': '1',
'mod_wsgi.listener_host': '',
'mod_wsgi.listener_port': '80',
'mod_wsgi.output_buffering': '0',
'mod_wsgi.process_group': 'www-data',
'mod_wsgi.reload_mechanism': '0',
'mod_wsgi.script_reloading': '1',
'wsgi.errors': <mod_wsgi.Log object at 0x8c0d620>,
'wsgi.input': <mod_wsgi.Input object at 0xb469958>,
'wsgi.multiprocess': True,
'wsgi.multithread': False,
'wsgi.run_once': False,
'wsgi.url_scheme': 'http',
'wsgi.version': (1, 0)}>

production
arecibo
http://www.areciboapp.com/
http://www.areciboapp.com/listener/docs/django/

production
get account
Install Arecibo library
ARECIBO_PUBLIC_ACCOUNT_NUMBER = "xxxx"

production
add to error
from arecibo.wrapper import post
def application_error(request):
t = loader.get_template('500.html')
uid = post(request, 500)
c = RequestContext(request, {"uid": uid})
return HttpResponse(t.render(c), status=500)

production

honourable mentions
django-db-log
http://code.google.com/p/django-db-log/
"Logs Django exceptions to your database handler."

honourable mentions
django-sql-profiler
http://code.google.com/p/django-sql-profiler
"Records every SQL query in Django in the database so that
you can find slow queries in your site."

The error message is the Truth.
The error message is God.
James Bennett, earlier today.
Andy McKay
clearwind consulting
andy@clearwind.ca
@clearwind
slides will be going on http://djangozen.com